Strengthening Evidence-support Systems in a Challenged World
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
The COVID-19 pandemic stimulated new innovations in evidence support and exacerbated long-standing challenges confronting those providing evidence support. To build stronger and more sustainable evidence-support systems, two key issues are important: 1) matching a decision maker’s need to the right combination of forms of evidence; and 2) ensuring updated summaries of the evidence are available when decision-makers need it. This dissertation aims to address both issues by (1) creating a taxonomy of demand-driven types of question for which research could provide insight; (2) building lists of study designs that optimally address each type of question; and (3) producing a theoretical framework to better understand what constitutes a living evidence synthesis, when and how to update them, and their role in the decision-making process. The first study is a cross-sectional survey targeting units providing evidence support to decision makers to create a demand-driven taxonomy of types of question. The second study is an online Delphi process asking methodological experts to create a list of study designs to answer these questions. Finally, study 3 is a critical interpretive synthesis to create a theoretical framework to understand living evidence syntheses and their role in decision-making processes. In chapter 2, 29 participants responded the cross-sectional survey, and a taxonomy of 40 demand-driven types of questions structured in the four main decision-making stages was created. In chapter 3, 29 methodological experts participated in the online Delphi process, and consensus was reached for 28 out of the 40 types of questions. Finally, in chapter 4, 152 publications were included, and six thematic categories were found to produce a conceptual framework. Together, the first two studies provide a way to facilitate the alignment between evidence demand and supply, while the third study helps to clarify the role of living evidence syntheses in decision-making processes.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.001 | 0.003 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.002 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it