Assessing and presenting summaries of evidence in Cochrane Reviews
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Cochrane Reviews are intended to help providers, practitioners and patients make informed decisions about health care. The goal of the Cochrane Applicability and Recommendation Methods Group (ARMG) is to develop approaches, strategies and guidance that facilitate the uptake of information from Cochrane Reviews and their use by a wide audience with specific focus on developers of recommendations and on healthcare decision makers. This paper is part of a series highlighting developments in systematic review methodology in the 20 years since the establishment of The Cochrane Collaboration, and its aim is to present current work and highlight future developments in assessing and presenting summaries of evidence, with special focus on Summary of Findings (SoF) tables and Plain Language Summaries.A SoF table provides a concise and transparent summary of the key findings of a review in a tabular format. Several studies have shown that SoF tables improve accessibility and understanding of Cochrane Reviews.The ARMG and GRADE Working Group are working on further development of the SoF tables, for example by evaluating the degree of acceptable flexibility beyond standard presentation of SoF tables, developing SoF tables for diagnostic test accuracy reviews and interactive SoF tables (iSoF).The plain language summary (PLS) is the other main building block for dissemination of review results to end-users. The PLS aims to summarize the results of a review in such a way that health care consumers can readily understand them. Current efforts include the development of a standardized language to describe statistical results, based on effect size and quality of supporting evidence.Producing high quality PLS and SoF tables and making them compatible and linked would make it easier to produce dissemination products targeting different audiences (for example, providers, health policy makers, guideline developers).Current issues of debate include optimal presentation formats of SoF tables, the training required to produce SoF tables, and the extent to which the authors of Cochrane Reviews should provide explicit guidance to target audiences of patients, clinicians and policy-makers.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.348 | 0.382 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.015 | 0.002 |
| Bibliometrics | 0.000 | 0.002 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.002 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.004 | 0.005 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it