Decision-makers’ experiences with rapid evidence summaries to support real-time evidence informed decision-making in crises: a mixed methods study
Bibliographic record
Abstract
BACKGROUND: There is a clear need for research evidence to drive policymaking and emergency responses so that lives are saved and resources are not wasted. The need for evidence support for health and humanitarian crisis is even more pertinent because of the time and practical constraints that decision-makers in these settings face. To improve the use of research evidence in policy and practice, it is important to provide evidence resources tailored to the target audience. This study aims to gain real-world insights from decision-makers about how they use evidence summaries to inform real-time decision-making in crisis-settings, and to use our findings to improve the format of evidence summaries. METHODS: This study used an explanatory sequential mixed method study design. First, we used a survey to identify the views and experiences of those who were directly involved in crisis response in different contexts, and who may or may not have used evidence summaries. Second, we used the insights generated from the survey to help inform qualitative interviews with decision-makers in crisis-settings to derive an in-depth understanding of how they use evidence summaries and their desired format for evidence summaries. RESULTS: We interviewed 26 decision-makers working in health and humanitarian emergencies. The study identified challenges decision-makers face when trying to find and use research evidence in crises, including insufficient time and increased burden of responsibilities during crises, limited access to reliable internet connection, large volume of data not translated into user friendly summaries, and little information available on preparedness and response measures. Decision-makers preferred the following components in evidence summaries: title, target audience, presentation of key findings in an actionable checklist or infographic format, implementation considerations, assessment of the quality of evidence presented, citation and hyperlink to the full review, funding sources, language of full review, and other sources of information on the topic. Our study developed an evidence summary template with accompanying training material to inform real-time decision-making in crisis-settings. CONCLUSIONS: Our study provided a deeper understanding of the preferences of decision-makers working in health and humanitarian emergencies about the format of evidence summaries to enable real-time evidence informed decision-making.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.266 | 0.399 |
| Meta-epidemiology (narrow) | 0.001 | 0.002 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.005 | 0.005 |
| Science and technology studies | 0.011 | 0.007 |
| Scholarly communication | 0.019 | 0.016 |
| Open science | 0.005 | 0.019 |
| Research integrity | 0.006 | 0.007 |
| Insufficient payload (model declined to judge) | 0.005 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".