Identifying priority questions regarding rapid systematic reviews’ methods: protocol for an eDelphi study
Bibliographic record
Abstract
INTRODUCTION: Rapid systematic reviews (RRs) have the potential to provide timely information to decision-makers, thus directly impacting healthcare. However, consensus regarding the most efficient approaches to performing RRs and the presence of several unaddressed methodological issues pose challenges. With such a large potential research agenda for RRs, it is unclear what should be prioritised. OBJECTIVE: To elicit a consensus from RR experts and interested parties on what are the most important methodological questions (from the generation of the question to the writing of the report) for the field to address in order to guide the effective and efficient development of RRs. METHODS AND ANALYSIS: An eDelphi study will be conducted. Researchers with experience in evidence synthesis and other interested parties (eg, knowledge users, patients, community members, policymaker, industry, journal editors and healthcare providers) will be invited to participate. The following steps will be taken: (1) a core group of experts in evidence synthesis will generate the first list of items based on the available literature; (2) using LimeSurvey, participants will be invited to rate and rank the importance of suggested RR methodological questions. Questions with open format responses will allow for modifications to the wording of items or the addition of new items; (3) three survey rounds will be performed asking participants to re-rate items, with items deemed of low importance being removed at each round; (4) a list of items will be generated with items believed to be of high importance by ≥75% of participants being included and (5) this list will be discussed at an online consensus meeting that will generate a summary document containing the final priority list. Data analysis will be performed using raw numbers, means and frequencies. ETHICS AND DISSEMINATION: This study was approved by the Concordia University Human Research Ethics Committee (#30015229). Both traditional, for example, scientific conference presentations and publication in scientific journals, and non-traditional, for example, lay summaries and infographics, knowledge translation products will be created.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.290 | 0.309 |
| Meta-epidemiology (narrow) | 0.004 | 0.005 |
| Meta-epidemiology (broad) | 0.005 | 0.006 |
| Bibliometrics | 0.009 | 0.008 |
| Science and technology studies | 0.005 | 0.005 |
| Scholarly communication | 0.006 | 0.008 |
| Open science | 0.005 | 0.007 |
| Research integrity | 0.010 | 0.011 |
| Insufficient payload (model declined to judge) | 0.075 | 0.022 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".