Examining Challenges to the Incorporation of End Users in the Design of Digital Health Interventions: Protocol for a Systematic Review
Bibliographic record
Abstract
BACKGROUND: The process of designing a digital health intervention (DHI)-also referred to as mobile health or eHealth-spans needs assessments, technical functionality and feasibility, user satisfaction, effectiveness, impact, and value. These interventions are causing a rapid evolution in the landscape of health care. Multiple studies have shown their propensity to extend both the quality and reach of interventions. However, failure to improve DHI design is linked to failed uptake and health outcomes. This dilemma is further conflicted by the colliding backdrops of the digital and health industries, both of which approach, understand, and involve end users differently in the framing of a DHI. OBJECTIVE: The objective of this systematic review is to assess the challenges to incorporating end users in the design stage of digital health interventions, to identify key pain points, and to identify limitations and gaps for areas of future investigation. METHODS: The PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols) checklist will be used to structure this protocol. A systematic search of the PsycINFO, PubMed (MEDLINE), Web of Science, CINAHL, Scopus, and IEEE Xplore databases will be conducted. Additionally, the PerSPEcTiF guidelines for complex interventions will be consulted. Two reviewers will independently screen the titles and abstracts of the identified references and select studies according to the eligibility criteria. Any discrepancies will then be discussed and resolved. Two reviewers will independently extract and validate data from the included studies into a standardized form and conduct quality appraisal. RESULTS: As of February 2021, we have completed a preliminary literature search examining challenges to the incorporation of end users in the design stage of DHIs. Systematic searches, data extraction and analysis, and writing of the systematic review are expected to be completed by December 2021. CONCLUSIONS: This systematic review aims to provide an effective summary of key pain points toward incorporating end users in DHIs. Results from this review will provide an evidence base for a better approach to end user involvement in the interest of improving efficacy and uptake of DHIs. TRIAL REGISTRATION: PROSPERO International Prospective Register of Systematic Reviews CRD42021238164; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=238164. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/28083.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.183 | 0.222 |
| Meta-epidemiology (narrow) | 0.008 | 0.008 |
| Meta-epidemiology (broad) | 0.019 | 0.018 |
| Bibliometrics | 0.015 | 0.015 |
| Science and technology studies | 0.005 | 0.009 |
| Scholarly communication | 0.010 | 0.011 |
| Open science | 0.006 | 0.007 |
| Research integrity | 0.013 | 0.012 |
| Insufficient payload (model declined to judge) | 0.054 | 0.012 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".