The WelTel Trial in context and the importance of null findings

The past decade has seen major advances in HIV care. Effective treatment exists, and drugs are becoming cheaper, more effective, and easier to tolerate. Thus, although most important clinical treatment questions have been answered, questions remain about how to get people into care earlier and remain on life-long treatment. It is crucial to identify effective interventions to improve linkage to and retention in HIV care.1Fox MP Rosen S Geldsetzer P Bärnighausen T Negussie E Beanland R Interventions to improve the rate or timing of initiation of antiretroviral therapy for HIV in sub-Saharan Africa: meta-analyses of effectiveness.J Int AIDS Soc. 2016; 19: 20888Crossref PubMed Scopus (49) Google Scholar The recently WHO-endorsed test-and-treat2WHOConsolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection Recommendations for a public health approach.Second edition. World Health Organization, Geneva2016Google Scholar policy means that patients are eligible to start HIV treatment earlier than ever before.3Fox MP Rosen S A new cascade of HIV care for the era of ‘treat all’.PLoS Med. 2017; 14: e1002268Crossref PubMed Scopus (55) Google Scholar In The Lancet Public Health, Mia van der Kop and colleagues4van der Kop ML Muhula S Nagide PI et al.Effect of an interactive text-messaging service on patient retention during the first year of HIV care in Kenya (WelTel Retain): an open-label, randomised parallel-group study.Lancet Public Health. 2018; (published online Jan 17.)http://dx.doi.org/10.1016/S2468-2667(17)30239-6Summary Full Text Full Text PDF PubMed Scopus (35) Google Scholar report the results of the WelTel study, in which they sought to identify whether a weekly two-way text-message check-in with patients in Kenya who were newly positive for HIV could improve one-year retention. Although it previously worked for those on treatment,5Lester RT Ritvo P Mills EJ et al.Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial.Lancet. 2010; 376: 1838-1845Summary Full Text Full Text PDF PubMed Scopus (915) Google Scholar this randomised trial showed no benefit. Despite the absence of efficacy, the results provide crucial evidence for policy, for two reasons central to evidenced-based thinking: first, the importance of person, place, and time in interpreting results; and second, the importance of validly and precisely estimated null findings. Regarding the first point, though we hoped the intervention would be effective, it was not guaranteed because the population studied was very different from the previous trial.5Lester RT Ritvo P Mills EJ et al.Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial.Lancet. 2010; 376: 1838-1845Summary Full Text Full Text PDF PubMed Scopus (915) Google Scholar Text-messaging interventions are a popular approach in sub-Saharan Africa (and beyond) because mobile phone penetration is high and the cost of intervention is low. Although numerous trials have been done,6Mbuagbaw L van der Kop ML Lester RT et al.Mobile phone text messages for improving adherence to antiretroviral therapy (ART): an individual patient data meta-analysis of randomised trials.BMJ Open. 2013; 3: e003950Crossref PubMed Scopus (85) Google Scholar results have been mixed. This is expected since the effects of behavioural interventions are likely to vary depending on where and when they are used, to whom they are targeted, and how they are implemented. While the previous trial targeted participants already on HIV treatment, in the WelTel study participants had just tested HIV-positive and many might not have accepted the necessity of treatment. Many factors determine whether an intervention like this will succeed. Is the text-messaging service free? Is the population highly motivated to seek care? Have the participants disclosed their HIV status? Do the participants know others who have sought treatment? We don't know the answers to these questions in the WelTel study, but we do know retention was high in this population, suggesting that the population might have been more motivated than the average person who tested HIV-positive. The authors found retention rates of 79% in the control group, by contrast with a much lower rate in most sub-Saharan African programmes.7Fox MP Rosen S Retention of adult patients on antiretroviral therapy in low- and middle-income countries: systematic review and meta-analysis 2008–2013.J Acquir Immune Defic Syndr. 2015; 69: 98-108Crossref PubMed Scopus (220) Google Scholar Moreover, participants only answered texts 55% of the time, often because of problems with their phones, suggesting the intervention itself might need to be improved. Regarding the second point on the importance of valid and precise findings, we appreciate that appropriate attention is being brought to null results.8Lash TL Kaufman JS Seeking persuasively null results.Epidemiology. 2015; 26: 499-550Crossref Scopus (2) Google Scholar The authors' finding didn't simply fail to demonstrate an effect of the intervention, they effectively showed lack of an effect through a strong design that minimised confounding and entailed appropriate measurement and good follow-up. The trial was not without its limitations, including the absence of blinding and the fact that only two-thirds of the participants completed the 12-month questionnaire. The primary study finding was that there was no significant difference in 12-month retention between groups (79% for the intervention group vs 81% for the control group). Although the finding was null (risk ratio 0·98) the effect was precisely estimated (95% CI 0·91–1·05). This study therefore does not leave us wondering if a bigger study would have found meaningful effects. Instead, we have persuasive evidence that the intervention was not successful, at least as implemented. As Poole wrote nearly two decades ago,9Poole C Low p-values or narrow confidence intervals: which are more durable?.Epidemiology. 2001; 12: 291-294Crossref PubMed Scopus (261) Google Scholar we must take precise and highly informative estimates like these seriously, even if null.9Poole C Low p-values or narrow confidence intervals: which are more durable?.Epidemiology. 2001; 12: 291-294Crossref PubMed Scopus (261) Google Scholar Unfortunately, null findings are often difficult to publish or get little attention. Knowing there is no effect of an intervention is just as important as knowing there is one. Conversely, a wide confidence interval, even if statistically significant in that it excludes the null value, conveys substantially less information. Policy makers must base decisions on precise and valid estimates, meaning they are not threatened by random variation or systematic bias. The WelTel study appears to be quite informative in this respect, and will therefore have high impact for policy decisions. This approach is why the outdated model in which results were prioritised only by their statistical significance is harmful for scientific progress and has been abandoned.10American Statistical AssociationAmerican Statistical Association releases statement on statistical signficance and p-values: provides principles to improve the conduct and interpretation of quantitative science.https://www.amstat.org/asa/files/pdfs/P-ValueStatement.pdfDate: March 7, 2016Google Scholar Still, with this trial, selection bias through loss to follow-up is probably the dominant form of error, and as such, should be taken into consideration when making decisions about the study results.11Barnett LA Lewis M Mallen CD Peat G Applying quantitative bias analysis to estimate the plausible effects of selection bias in a cluster randomised controlled trial: secondary analysis of the Primary care Osteoarthritis Screening Trial (POST).Trials. 2017; 18: 585Crossref PubMed Scopus (7) Google Scholar With renewed attention being paid to patients newly testing positive for HIV under a test-and-treat strategy, it would have been exciting if the WelTel text-messaging intervention had improved retention and gotten more patients onto treatment. But knowing that it does not, at least as implemented, is an important advance as well, and one we should learn from as we seek new ways to improve retention in HIV care. We declare no competing interests. Effect of an interactive text-messaging service on patient retention during the first year of HIV care in Kenya (WelTel Retain): an open-label, randomised parallel-group studyThis weekly text-messaging service did not improve retention of people in early HIV care. The intervention might have a modest role in improving self-perceived health-related quality of life in individuals in HIV care in similar settings. Full-Text PDF Open Access

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.