A taxonomy of demand-driven questions for use by evidence producers, intermediaries and decision-makers: results from a cross-sectional survey
Bibliographic record
Abstract
BACKGROUND: Globally, a growing number of calls to formalize and strengthen evidence-support systems have been released, all of which emphasize the importance of evidence-informed decision making. To achieve this, it is critical that evidence producers and decision-makers interact, and that decision-makers' evidence needs can be efficiently translated into questions to which evidence producers can respond. This paper aims to create a taxonomy of demand-driven questions for use by evidence producers, intermediaries (i.e., people working in between researchers and decision-makers) and decision-makers. METHODS: We conducted a global cross-sectional survey of units providing some type of evidence support at the explicit request of decision-makers. Unit representatives were invited to answer an online questionnaire where they were asked to provide a list of the questions that they have addressed through their evidence-support mechanism. Descriptive analyses were used to analyze the survey responses, while the questions collected from each unit were iteratively analyzed to create a mutually exclusive and collectively exhaustive list of types of questions that can be answered with some form of evidence. RESULTS: Twenty-nine individuals completed the questionnaire, and more than 250 submitted questions were analysed to create a taxonomy of 41 different types of demand-driven questions. These 41 questions were organized by the goal to be achieved, and the goals were grouped in the four decision-making stages (i) clarifying a societal problem, its causes and potential impacts; (ii) finding and selecting options to address a problem; (iii) implementing or scaling-up an option; and (iv) monitoring implementation and evaluating impacts. CONCLUSION: The mutually exclusive and collectively exhaustive list of demand-driven questions will help decision-makers (to ask and prioritize questions), evidence producers (to organize and present their work), and evidence-intermediaries (to connect evidence needs with evidence supply).
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.100 | 0.217 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.005 | 0.005 |
| Science and technology studies | 0.002 | 0.003 |
| Scholarly communication | 0.004 | 0.005 |
| Open science | 0.002 | 0.006 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".