Examining Challenges to Co-Design Digital Health Interventions With End Users: Systematic Review
Bibliographic record
Abstract
BACKGROUND: Digital health interventions (DHIs) are changing the dynamic of health care by providing personalized, private, and instantaneous solutions to end users. However, the explosion of digital health has been fraught with challenges. The approach to co-design with end users varies across a diverse domain of stakeholders, often resulting in siloed approaches with no clear consensus. The concept of validating user experiences contrasts greatly between digital stakeholders (ie, user experience and retention) and health stakeholders (ie, safety and efficacy). Several methodologies and frameworks are being implemented to address this challenge to varying degrees of success. OBJECTIVE: We aimed to broadly examine the advancements and challenges to co-design DHIs with end users over the last decade. This task was undertaken to identify the key problem areas at the domain level, with the ultimate goal of creating recommendations for better approaches to co-design DHIs with end users. METHODS: We conducted a systematic search of key databases for co-design studies involving end users in DHIs. Searches were divided into 3 relevant streams: health behavior, user experience, and digital methodologies and frameworks. The eligibility criteria were guided by the PerSPEcTiF framework and the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist. In line with this framework, studies were included in this review that (1) address research on DHIs; (2) focus on interaction and co-design with end users; (3) explain results such that uptake, effectiveness, satisfaction, and health outcomes are discernible, positively or negatively; and (4) describe actionable procedures for better DHI design. The search was conducted in a diverse group of 6 bibliographical databases from January 2015 to May 2024: PsycINFO, PubMed (MEDLINE), Web of Science, CINAHL, Institute of Electrical and Electronics Engineers Xplore, and Scopus. From the 13,961 studies initially screened for titles and abstracts, 489 (3.6%) were eligible for a full-text screening, of which 171 (1.2%) studies matched the inclusion criteria and were included in a qualitative synthesis. RESULTS: Of the 171 studies analyzed across 52 journals, we found 5 different research approaches, spanning 8 different digital health solution types and 5 different design methodologies. These studies identified several core themes when co-designing with end users: advancements, which included participatory co-design; challenges, which included participatory co-design, environment and context, testing, and cost and scale; and gaps, which included a pragmatic hybridized framework and industry implementability. CONCLUSIONS: This research supports a pragmatic shift toward using mixed methods approaches at scale, methods that are primed to take advantage of the emerging big data era of digital health co-design. This organic outlook should blend the vision of digital health co-designers with the pragmatism of Agile design methodology and the rigor of health care metrics. TRIAL REGISTRATION: PROSPERO CRD42021238164; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021238164. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/28083.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.025 | 0.004 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.005 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.000 |
| Open science | 0.003 | 0.001 |
| Research integrity | 0.000 | 0.005 |
| Insufficient payload (model declined to judge) | 0.002 | 0.005 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; both teacher heads agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".