Reviews: Rapid! Rapid! Rapid! …and systematic
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
“Although time is a reality, lack of time is not lack ofreality” - HOJESSystematic reviewsSystematic reviews are scientific investigations, with pre-planned methods and an assembly of original studies astheir “subjects” [1,2]. They synthesize the results of mul-tiple primary investigations by using strategies that limitbias and random error [1,2]. Systematic reviews aretransparent about how studies were identified and whichwere included or excluded, the risk of bias assessment,and the methods to summarize data and assess the cer-tainty in the evidence. Standards for the conduct of sys-tematic reviews have been made available by theCochrane Collaboration and other organizations. If sys-tematic reviews are done well, e.g., by adhering to con-duct (e.g., Cochrane Handbook for Systematic Reviewsof Interventions) and reporting (e.g., PRISMA State-ment) best practice standards, it is not sensible to ques-tion the value of systematic reviews as a source ofinformation for shaping decision making [3,4]. Thismethodology of systematic reviews—although laid outthree or more decades ago—is continuously and rapidlyupdated by scientists specializing in research synthesis.Now, Systematic Reviews is publishing a series of articlesincluding methods and examples of accelerating ap-proaches to conducting literature reviews. As a rule ofthumb, rapid systematic reviews should be conducted inless than 8 weeks, including protocol publication. Onthe whole, this is a saving of about 75% in terms of timecompared to what most researchers would propose asstandard timeline for systematic reviews. Examples willhighlight how health policy decisions can be influencedwhen a rapid review methodology is used.The challenge of traditional systematic reviewsFindings from a single randomized trial are often rapidlychallenged by succeeding studies, and rigorous systematicreviews help approximate “true evidence” and estimatesin effects [5]. High-quality systematic reviews are usedmore often and are considered more trustworthy byhealth professionals in terms of relevance to clinical prac-tice than other types of designs [6]. However, conductingand adhering to the standards of traditional systematic re-views can be time consuming. The reason for that lies inthe rigorous approach to methods ensuring that the bestavailable evidence is identified, assessed, and synthesized.But those demanding evidence syntheses for decisionmaking are increasingly living in faster paced times, influ-enced by innovative interventions and technology that ac-celerate communication and interaction. Decision makersoften do not appreciate the intricacies of researchmethods and the time needed to comply with the task.The argument that transparency is ensured by completinga traditional systematic reviews is often not convincingenough. But there are other reasons.The rationale for rapid —systematic reviewsThe concern regarding a timely decision on health care andpolicies is the driving force for rapid reviews. In fact, deci-sion making should not be delayed in most situations andcannot be delayed in some. In the face of a tragic Ebola epi-demic, we are reminded of how rapidly answers are re-quired. To base answers on the best available evidence, thisevidence must be synthesized without undue delays. Whiletypical systematic reviews can take years to complete (oneof the author was involved in a systematic review that took12 years to complete), rapid reviews are required when fa-cing such dramatic situations. Prior to the Ebola epidemic,the fear of avian influenza prompted the World HealthOrganization to offer rapid guidelines that were supportedby a rapid review methodology. From guideline panel for-mation to completion of the recommendations, only about12 weeks passed [7]. Another recent rapid systematic re-view was commissioned to inform decision making with re-gard to the safety of two drugs, bevacizumab (Avastin) andranibizumab (Lucentis), widely used to stabilize vision in
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.712 | 0.731 |
| Meta-epidemiology (narrow) | 0.005 | 0.002 |
| Meta-epidemiology (broad) | 0.100 | 0.016 |
| Bibliometrics | 0.002 | 0.005 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.007 | 0.001 |
| Open science | 0.013 | 0.001 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.011 | 0.100 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it