Defining and Measuring Engagement and Adherence in Digital Mental Health Interventions: Protocol for an Umbrella Review
Bibliographic record
Abstract
BACKGROUND: Digital mental health interventions (DMHIs) offer scalable solutions to address mental health needs, particularly among marginalized populations. However, engagement and adherence rates in DMHIs are often suboptimal, limiting their potential impact. Despite the growing body of literature on DMHI engagement, there is no consensus on how engagement and adherence are defined and measured across studies. Understanding these variations is crucial to improving DMHI design, evaluation, and outcomes. OBJECTIVE: Using the population, concept, context framework to frame the objectives, this umbrella review aims to synthesize existing systematic reviews, meta-analyses, and scoping reviews to identify how engagement and adherence are defined and measured in DMHIs. Additionally, this review seeks to explore factors that may influence DMHI engagement and adherence. METHODS: A systematic search of peer-reviewed literature will be conducted across major electronic databases following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Eligible studies will include systematic reviews, meta-analyses, and scoping reviews published in English in the past 10 years that examine engagement and/or adherence in DMHIs. Data will be extracted and synthesized to identify definitions, measurement methods, and influencing factors. Risk of bias will be assessed using the Joanna Briggs Institute (JBI) critical appraisal checklist for systematic reviews and research syntheses. Findings will be presented using a mixed methods convergent integrated approach, identifying and synthesizing themes across the included quantitative and qualitative study results. RESULTS: This study is expected to be conducted over a 6-month period. The search, conducted in early March 2025, initially identified 5087 papers. An additional 35 papers were found through manual handsearching of BMC Digital Health. These totals were recorded prior to the removal of duplicates. This umbrella review is expected to be conducted with screening, quality assessment, and data extraction streamlined through the Covidence platform. The screening and selection of studies will be performed in month 1, followed by data extraction and quality appraisal in months 2 and 3. Data synthesis and integration will take place in months 4 and 5, and writing conclusions and preparing the manuscript will occur in month 6. This review will provide a comprehensive summary of how engagement and adherence are operationalized across existing literature. It will highlight commonalities, inconsistencies, and gaps in definitions and measurement methods. Additionally, this review will outline the key factors that influence engagement and adherence, including individual, technological, and contextual elements. CONCLUSIONS: This umbrella review will contribute to a more nuanced understanding of engagement and adherence in DMHIs, informing future intervention design and evaluation. The findings will support the development of standardized definitions and measurement frameworks, ultimately enhancing the effectiveness and inclusivity of DMHIs. TRIAL REGISTRATION: PROSPERO CRD42025637603; https://www.crd.york.ac.uk/PROSPERO/view/CRD42025637603. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/73438.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.121 | 0.130 |
| Meta-epidemiology (narrow) | 0.005 | 0.006 |
| Meta-epidemiology (broad) | 0.011 | 0.021 |
| Bibliometrics | 0.013 | 0.014 |
| Science and technology studies | 0.006 | 0.005 |
| Scholarly communication | 0.009 | 0.009 |
| Open science | 0.006 | 0.008 |
| Research integrity | 0.008 | 0.009 |
| Insufficient payload (model declined to judge) | 0.085 | 0.020 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".