Exploring the perspectives of International Nongovernmental Organizations (INGOs) on the use of program evaluation and impact assessment in their work
Bibliographic record
Abstract
In the twenty-first century, the call for International Non-governmental Organizations (INGOs) to demonstrate their effectiveness has become popularized. This has given rise to scholarly attention examining the roles of program evaluation and impact assessment in assisting INGOs in demonstrating their effectiveness. While previous studies suggest that INGOs actively conduct program evaluation and impact assessment, this article explores the perspectives of two Canadian INGOs on how they understand, use, and experience evaluation and assessment as it relates to their work. Our study uncovers three continuing challenges: evaluation and assessment are largely descriptive and lack more sophisticated analyses; efforts to conduct evaluation and assessment are consolidated within organizations’ head offices, while staff members and volunteers are largely excluded; and evaluation and assessment remain rooted in the paradigm of quantifiable results, which do not truly reflect the nature of work being conducted on the ground. Au vingt-et-unième siècle, on veut de plus en plus que les organisations non gouvernementales internationales (ONGI) démontrent leur efficacité. Ce désir a motivé les chercheurs à se pencher sur les évaluations de programme et les études d’impact pour voir dans quelle mesure celles-ci peuvent aider les ONGI à montrer qu’elles sont efficaces. Des études antérieures suggèrent que les ONGI mènent de manière concertée des évaluations de programme et des études d’impact. Cet article explore comment aujourd’hui deux ONGI canadiens comprennent, utilisent et vivent l’évaluation et la mesure de leur travail. Notre étude relève trois défis actuels : l’évaluation et la mesure tendent à être descriptives sans offrir d’analyses plus poussées; ce sont les sièges sociaux des organismes qui gèrent l’évaluation et la mesure en excluant ainsi bon nombre de fonctionnaires et volontaires; l’évaluation et la mesure se limitent au paradigme des résultats mesurables et par conséquent elles ne reflètent pas nécessairement le véritable travail mené sur le terrain.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.086 | 0.068 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.005 | 0.006 |
| Science and technology studies | 0.034 | 0.057 |
| Scholarly communication | 0.027 | 0.009 |
| Open science | 0.003 | 0.018 |
| Research integrity | 0.005 | 0.014 |
| Insufficient payload (model declined to judge) | 0.003 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".