Building of a Learning Health System surrounding Hospital Discharge: A toolbox for Sustainable Metrics from Implementation to Evaluation and Emulation
Bibliographic record
Abstract
Background: Integrated healthcare delivery systems which meaningfully address patientsneeds as they transition between acute care and home/community supports can achieve the quintuple aims, leading to improved experience and health outcomes. Minimum datasets rooted in high quality, cross-sectoral and patient-centered outcomes can help direct continuous improvement and ensure sustainability of these integrated care systems. Since 209, in Toronto, Ontario, University Health Network (UHN) Integrated Care (IC) program has been enrolling and linking patients and their caregivers, post-discharge, to home and community supports through one point of contact (i.e., an IC lead) and 24/7 phone support line. As the program expands within and beyond our hospital, our aim was to create a feasible, standardized minimum dataset that addresses all ten Ontario quality standards and Alberta Home to Hospital to Home guidelines on care transitions and the quintuple aims to help inform learning health systems. ApproachOur primary objective was to create and test the feasibility of a minimum dataset (MDS) that could be used for continuous program evaluation. The construction of the MDS involved a mixed methods approach that incorporated chart-level and program-specific data and qualitative interviews with patients and providers. Our secondary objective was to test this MDS as a learning health system for all patients enrolled in the program during the first 4 years of implementation. Through the use of chart, program-specific and hospital level data collection data was harnessed for 3075 patients enrolled in the program between June , 209 and May 30, 2023. Stakeholders including patient and caregiver partners, institutional and program leaders, and provincial policy leads helped inform the selection, use, and dissemination of the metrics for continuous program refinement and sustainability of ongoing program evaluation. ImplicationsNotable strengths that served as accelerants for the program evaluation included harnessing hospital-level chart data, homecare and program specific data through shared data records. Low response rate (%) to CIHI Canadian Patient Experience- Inpatient Care survey led the team to use a modified patient experience survey along with qualitative interviews. Site-specific data needed further linkage to provincial administrative data to allow for comparison with controls, and to fully evaluate impact beyond the institution where the program was implemented. Additionally, low or incomplete response on language, gender, race and income equity and diversity measures when admitted to hospital led to manual chart review for ascertainment. Moving forward, the scalability of health equity and patient experience data along with greater information sharing across sites and teams must be addressed. Use of AI and machine learning for extrapolating chart level sociodemographic data may help capture health equity data. Systems that meaningfully engage with patients, caregivers, hospital, community and policy stakeholders to harness linkages between patient and corporate values are an essential component to building a Learning Health System, and play a significant role in building sustainable and prospective program evaluation for integrated care models surrounding hospital admissions.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.229 | 0.254 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.010 | 0.010 |
| Science and technology studies | 0.007 | 0.010 |
| Scholarly communication | 0.017 | 0.021 |
| Open science | 0.007 | 0.024 |
| Research integrity | 0.003 | 0.006 |
| Insufficient payload (model declined to judge) | 0.005 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".