Data elicited through apps for health systems improvement
Bibliographic record
Abstract
A promising approach to meeting the need in many jurisdictions for timely, in-depth qualitative health systems experience data, is to elicit feedback through smartphone and web applications (apps). Apps offer an appealing tool to elicit data from patients and family members who may feel stigma when receiving some services and a power imbalance when providing feedback to health-care providers. In this article, we examine the effectiveness of a suite of smartphone and web apps called myExperience (myEXP) that were created to gather care experiences of youth, family members, and service providers as part of an experience-based co-design (EBCD) study in Ontario involving youth with mental disorders. We analyzed data from 12 triads of youth (aged 16–24), family members, and service providers gathered between August 2015 and December 2016. We used qualitative content analysis to understand participant feedback on the myEXP apps and identify thematic categories that emerged from experience data elicited through the myEXP apps. We found overall that the myEXP apps were more effective at eliciting experience data from youth compared with family members and service providers. Rich experience data were gathered from youth about treatment plans in real time through the apps. The apps also showed important promise as reflective tools for all participants. They may offer advantages in research that seeks to improve responsiveness in service delivery and build mutual understanding. The apps also offer choice in how data are elicited, encourage more candid feedback and help to overcome stigma, which are important considerations for some vulnerable populations. For service redesign research using approaches such as EBCD, apps offer real-time data gathering that can complement and enhance traditional approaches such as retrospective interviews and observation.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.003 | 0.001 |
| Scholarly communication | 0.000 | 0.002 |
| Open science | 0.004 | 0.002 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".