Research and Implementation Lessons Learned From a Youth-Targeted Digital Health Randomized Controlled Trial (the ARMADILLO Study)
Bibliographic record
Abstract
BACKGROUND: Evidence is lacking on the efficacy of sexual and reproductive health (SRH) communication interventions for youth (aged 15-24 years), especially from low- and middle-income countries. Therefore, the World Health Organization initiated the Adolescent/Youth Reproductive Mobile Access and Delivery Initiative for Love and Life Outcomes (ARMADILLO) program, a free, menu-based, on-demand text message (SMS, short message service) platform providing validated SRH content developed in collaboration with young people. A randomized controlled trial (RCT) assessing the effect of the ARMADILLO intervention on SRH-related outcomes was implemented in Kwale County, Kenya. OBJECTIVE: This paper describes the implementation challenges related to the RCT, observed during enrollment and the intervention period, and their implications for digital health researchers and program implementers. METHODS: This was an open, three-armed RCT. Following completion of a baseline survey, participants were randomized into the ARMADILLO intervention (arm 1), a once-a-week contact SMS text message (arm 2), or usual care (arm 3, no intervention). The intervention period lasted seven weeks, after which participants completed an endline survey. RESULTS: Two study team decisions had significant implications for the success of the trial's enrollment and intervention implementation: a hands-off participant recruitment process and a design flaw in an initial language selection menu. As a result, three weeks after recruitment began, 660 participants had been randomized; however, 107 (53%) participants in arm 1 and 136 (62%) in arm 2 were "stuck" at the language menu. The research team called 231 of these nonengaging participants and successfully reached 136 to learn reasons for nonengagement. Thirty-two phone numbers were found to be either not linked to our participants (a wrong number) or not in their primary possession (a shared phone). Among eligible participants, 30 participants indicated that they had assumed the introductory message was a scam or spam. Twenty-seven participants were confused by some aspect of the system. Eleven were apathetic about engaging. Twenty-four nonengagers experienced some sort of technical issue. All participants eventually started their seven-week study period. CONCLUSIONS: The ARMADILLO study's implementation challenges provide several lessons related to both researching and implementing client-side digital health interventions, including (1) have meticulous phone data collection protocols to reduce wrong numbers, (2) train participants on the digital intervention in efficacy assessments, and (3) recognize that client-side digital health interventions have analog discontinuation challenges. Implementation lessons were (1) determine whether an intervention requires phone ownership or phone access, (2) digital health campaigns need to establish a credible presence in a busy digital space, and (3) interest in a service can be sporadic or fleeting. CLINICAL TRIAL: International Standard Randomized Controlled Trial Number (ISRCTN): 85156148; http://www.isrctn. com/ISRCTN85156148.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.337 | 0.369 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.004 | 0.005 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.002 | 0.003 |
| Scholarly communication | 0.006 | 0.006 |
| Open science | 0.004 | 0.004 |
| Research integrity | 0.008 | 0.007 |
| Insufficient payload (model declined to judge) | 0.007 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".