Realist evaluations in low- and middle-income countries: reflections and recommendations from the experiences of a foreign researcher
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Realist evaluation, a methodology for exploring generative causation within complex health interventions to understand 'how, why and for whom' programmes work, is experiencing a surge of interest. Trends indicate that the proliferation in the use of this methodology also applies to research in low- and middle-income countries (LMICs). The value of using realist evaluation for project evaluation is also being noticed by non-governmental organisations (NGOs) and other programme implementers within such contexts. Yet, there is limited exploration of the use of realist evaluations in LMICs, especially their use by foreign researchers. This paper draws on the author's experience of conducting two realist evaluations across three different sub-Saharan African settings: Mundemu, Tanzania; Kabale, Uganda and Marsabit, Kenya. The realist evaluations were used as an operations research methodology to study two NGO community health programmes. This paper highlights four main challenges experienced by the author throughout the methodological process: (1) power imbalances prevalent during realist interviews, (2) working through translation and what this means for identfying Context-Mechanism-Outcome Configurations, (3) limited contextual familiarity and being an 'engaged researcher' and (4) the use or dependence on 'WEIRD' theories (i.e. theories based on the study of Western, Educated, Industrialized, Rich, Democratic people) within testing and refinement. Realist evaluation's enticing and straightforward slogan of finding 'what works, for whom and why' is in contrast to the complexity of the methodology used to generate these results (and often to the results themselves). Striking a balance between theory and pragmatism, while adhering to realist ontological underpinnings of generative causation and retroduction, is no easy task. This paper concludes by providing concrete recommendations for those who want to undertake a realist evaluation, with particular attention to cross-cultural settings, in light of the aforementioned challenges. In doing so, it aims to foster improved methodological rigour and help those engaging in this research methodology to work towards more appropriate and contextually relevant findings.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.002 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it