How beneficiaries see complex health interventions: a practice review of the Most Significant Change in ten countries
Bibliographic record
Abstract
BACKGROUND: The Most Significant Change is a story-based evaluation approach used in many international development programs. This practice review summarises practical experience with the approach in complex health interventions in ten countries, with the objective of making it more accessible in evaluation of other complex health interventions. RESULTS: Participatory research practitioners and trainees discussed five themes following brief presentations by each of the seven attendees who led the exercise: (i) sampling and recruitment; (ii) phrasing the questions to elicit stories; (iii) story collection strategies; (iv) quality assurance; and (v) analysis. Notes taken during the meeting provided the framework for this article. Recruitment strategies in small studies included universal engagement and, in larger studies, a purposive, systematic or random sampling. Meeting attendees recommended careful phrasing and piloting of the question(s) as this affects the quality and focus of the stories generated. They stressed the importance of careful training and monitoring of fieldworkers collecting stories to ensure full stories are elicited and recorded. For recording, in most settings they preferred note taking with back-checking or self-writing of stories by story tellers, rather than audio-recording. Analysis can combine participatory selection of a small number of stories, deductive or inductive thematic analysis and discourse analysis. Meeting attendees noted that involvement in collection of the stories and their analysis and discussion had a positive impact for research team members. CONCLUSIONS: Our review confirms the plasticity, feasibility and acceptability of the Most Significant Change technique across different sociopolitical, cultural and environmental contexts of complex interventions. Although the approach can surface unexpected impacts, it is not a 360-degree evaluation. Its strength lies in characterising the changes, where these happen, in the words of the beneficiaries. We hope this distillation of our practice makes the technique more readily available to health sector researchers.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.020 | 0.022 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.004 | 0.001 |
| Bibliometrics | 0.001 | 0.003 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.002 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".