Reimagining statistical analysis for evidenced-based policy making: Early experiences using Stats Report
Bibliographic record
Abstract
o help overcome capacity-building challenges in the NEP, we began using an online web application called Stats Report which allows us to spend more time analyzing and interpreting data with workshop participants.Our experiences suggest that Stats Report is able to aid global health practitioners by allowing them to run their analyses without knowing a statistical programming language, the same way one can drive a car without knowing how the engine works.Global health decision-makers need tools to more easily obtain, analyze, and use data.In the National Evaluation Platform (NEP), we worked with mid-and high-level government staff to build local capacity for data analysis.With these experiences came challenges in statistical capacity-building and collaboration that are likely to be similar in other contexts.To address these challenges in the NEP, we used an online web application called Stats Report.Built on the R statistical package, Stats Report allows complex data analysis to be undertaken more easily and collaboratively.We used Stats Report in data analysis workshops with government staff in Malawi, Mali, Mozambique, and Tanzania.Statistical experts from our team prepared analysis code, and in-country workshop participants ran the analyses themselves, without having to adjust code, manipulate files, or download software.Using Stats Report, participants generated a variety of analytical outputs more quickly and more reliably than had been possible in previous workshops.We report the way in which participants used and responded to Stats Report.Our early experiences suggest that Stats Report is easy to use, increases efficiency for data analysis, and enhances transparency and scientific replicability, which may be useful beyond the NEP.A unique strength is the ability to foster collaboration while encouraging users who first run existing analyses to later develop independent skills and the potential to teach others.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.414 | 0.607 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.003 |
| Bibliometrics | 0.007 | 0.012 |
| Science and technology studies | 0.001 | 0.006 |
| Scholarly communication | 0.012 | 0.010 |
| Open science | 0.007 | 0.008 |
| Research integrity | 0.003 | 0.016 |
| Insufficient payload (model declined to judge) | 0.005 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".