Using the Quadruple Aim Framework to Measure Impact of Heath Technology Implementation: A Case Study of eConsult
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
BACKGROUND: Health technology solutions are too often implemented without a true understanding of the system-level problem they seek to address, resulting in excessive costs, poor adoption, ineffectiveness, and ultimately failure. Before implementing or adopting health care innovations, stakeholders should complete a thorough assessment to ensure effectiveness and value. In this article, we describe how to evaluate the impact of a health technology innovation through the 4 dimensions of care outlined by the Quadruple Aim Framework, using our experience with the Champlain Building Access to Specialists through eConsultation (BASE) eConsult service as a case example. METHODS: A descriptive overview of data was collected between April 1, 2011, and August 31, 2017, using 4 dimensions of care outlined by the Quadruple Aim Framework: patient experience, provider experience, costs, and population health. Findings were drawn from use data, primary care provider closeout surveys, surveys/interviews with patients and provider, and costing data. RESULTS: Overall, patients have received access to specialist advice within days and find the advice useful in 86% of cases. Provider experience is very positive, with satisfaction ratings of high/very high value in 94% of cases. The service cost a weighted average of $47.35/case, compared with $133.60/case for traditional referrals. In total, 1,299 primary care providers have enrolled in the service, completing 28,838 cases since 2011. Monthly case volumes have grown from an average of 13 cases/month in 2011 to 969 cases/month in 2016. CONCLUSIONS: The eConsult service has been widely adopted in our region and is currently expanding to new jurisdictions across Canada. However, although we successfully demonstrated eConsult's impact on patient experience, provider satisfaction, and reducing costs, we met several challenges in evaluating its impact on population health. More work is needed to evaluate eConsult's impact on key population health metrics (eg, mortality, morbidity, and system use). Efforts to conduct such evaluations are underway.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.002 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it