The evolution of a pediatric virtual care evaluation framework
Bibliographic record
Abstract
Abstract Background The use of virtual care has increased dramatically in response to the COVID-19 pandemic, yet evidence is lacking regarding the impact of virtual care on patient outcomes, particularly in pediatrics. A standardized evaluation approach is required to support the integration of virtual care into pediatric health care delivery programs. The objective of this work was to develop a comprehensive and structured framework for pediatric virtual care evaluation. This framework is intended to engage and guide care providers, health centres, and stakeholders towards the development of a standardized approach to the evaluation of pediatric virtual care. Methods We brought together a diverse multidisciplinary team, including pediatric clinicians, researchers, digital health leads and analysts, program leaders, a human factors engineer, a family advisor and our manager of health equity and diversity. The team reviewed the literature, including published evaluation frameworks, and used a consensus-based method to develop a virtual care evaluation framework applicable to a broad spectrum of pediatric virtual care programs. We iteratively developed framework components including domains and sub-domains, examples of evaluation questions, measures, and data sources. Team members provided feedback on all components of the framework until consensus was reached. The framework was then applied to an existing virtual care program. Results The resulting framework includes four domains (health outcomes, health delivery, individual experience, and program implementation) and 19 sub-domains designed to support the development and evaluation of pediatric virtual care programs. We also developed guidance on how to use the framework and illustrate its utility by applying it to an existing pediatric virtual care program. Conclusions This virtual care evaluation framework expands on previously developed frameworks by providing additional detail and a structure that supports practical application. It can be used to evaluate a wide range of pediatric virtual care programs in a standardized manner. Use of this comprehensive yet easy to use evaluation framework will inform appropriate implementation and integration of virtual care into routine practice and support its sustainability and continuous improvement.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.340 | 0.186 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.002 | 0.004 |
| Bibliometrics | 0.017 | 0.008 |
| Science and technology studies | 0.007 | 0.015 |
| Scholarly communication | 0.017 | 0.016 |
| Open science | 0.009 | 0.014 |
| Research integrity | 0.005 | 0.007 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".