Methodological Challenges in Randomized Controlled Trials of mHealth Interventions: Cross-Sectional Survey Study and Consensus-Based Recommendations
Bibliographic record
Abstract
BACKGROUND: Mobile health (mHealth) refers to using mobile communication devices such as smartphones to support health, health care, and public health. mHealth interventions have their own nature and characteristics that distinguish them from traditional health care interventions, including drug interventions. Thus, randomized controlled trials (RCTs) of mHealth interventions present specific methodological challenges. Identifying and overcoming those challenges is essential to determine whether mHealth interventions improve health outcomes. OBJECTIVE: We aimed to identify specific methodological challenges in RCTs testing mHealth interventions' effects and develop consensus-based recommendations to address selected challenges. METHODS: A 2-phase participatory research project was conducted. First, we sent a web-based survey to authors of mHealth RCTs. Survey respondents rated on a 5-point scale how challenging they found 21 methodological aspects in mHealth RCTs compared to non-mHealth RCTs. Nonsystematic searches until June 2022 informed the selection of the methodological challenges listed in the survey. Second, a subset of survey respondents participated in an online workshop to discuss recommendations to address selected methodological aspects identified in the survey. Finally, consensus-based recommendations were developed based on the workshop discussion and email interaction. RESULTS: We contacted 1535 authors of mHealth intervention RCTs, of whom 80 (5.21%) completed the survey. Most respondents (74/80, 92%) identified at least one methodological aspect as more or much more challenging in mHealth RCTs. The aspects most frequently reported as more or much more challenging were those related to mHealth intervention integrity, that is, the degree to which the study intervention was implemented as intended, in particular managing low adherence to the mHealth intervention (43/77, 56%), defining adherence (39/79, 49%), measuring adherence (33/78, 42%), and determining which mHealth intervention components are used or received by the participant (31/75, 41%). Other challenges were also frequent, such as analyzing passive data (eg, data collected from smartphone sensors; 24/58, 41%) and verifying the participants' identity during recruitment (28/68, 41%). In total, 11 survey respondents participated in the subsequent workshop (n=8, 73% had been involved in at least 2 mHealth RCTs). We developed 17 consensus-based recommendations related to the following four categories: (1) how to measure adherence to the mHealth intervention (7 recommendations), (2) defining adequate adherence (2 recommendations), (3) dealing with low adherence rates (3 recommendations), and (4) addressing mHealth intervention components (5 recommendations). CONCLUSIONS: RCTs of mHealth interventions have specific methodological challenges compared to those of non-mHealth interventions, particularly those related to intervention integrity. Following our recommendations for addressing these challenges can lead to more reliable assessments of the effects of mHealth interventions on health outcomes.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.890 | 0.933 |
| Meta-epidemiology (narrow) | 0.003 | 0.007 |
| Meta-epidemiology (broad) | 0.010 | 0.012 |
| Bibliometrics | 0.019 | 0.014 |
| Science and technology studies | 0.008 | 0.015 |
| Scholarly communication | 0.017 | 0.024 |
| Open science | 0.013 | 0.015 |
| Research integrity | 0.017 | 0.012 |
| Insufficient payload (model declined to judge) | 0.005 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".