The role of evaluation in iterative learning and implementation of quality of care interventions
Bibliographic record
Abstract
Background: The Lancet Global Health Commission (LGHC) has argued that quality of care (QoC) is an emergent property that requires an iterative process to learn and implement. Such iterations are required given that health systems are complex adaptive systems.Objective: This paper explores the multiple roles that evaluations need to play in order to help with iterative learning and implementation. We argue evaluation needs to shift from a summative focus toward an approach that promotes learning in complex systems. A framework is presented to help guide the iterative learning, and includes the dimensions of clinical care, person-centered care, continuum of care, and ‘more than medicine. Multiple roles of evaluation corresponding to each of the dimensions are discussed.Methods: This paper is informed by reviews of the literature on QoC and the roles of evaluation in complex systems. The proposed framework synthesizes the multiple views of QoC. The recommendations of the roles of evaluation are informed both by review and experience in evaluating multiple QoC initiatives.Results: The specific roles of different evaluation approaches, including summative, realist, developmental, and participatory, are identified in relationship to the dimensions in our proposed framework. In order to achieve the potential of LGHC, there is a need to discuss how different evaluation approaches can be combined in a coherent way to promote iterative learning and implementation of QoC initiatives.Conclusion: One of the implications of the QoC framework discussed in the paper is that time needs to be spent upfront in recognizing areas in which knowledge of a specific intervention is not complete at the outset. This, of course, implies taking stock of areas of incompleteness in knowledge of context, theory of change, support structures needed in order for the program to succeed in specific settings. The role of evaluation should not be limited to only providing an external assessment, but an important goal in building evaluation capacity should be to promote adaptive management among planners and practitioners. Such iterative learning and adaptive management are needed to achieve the goals of sustainable development goals.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".