Going from evidence to recommendations: Can GRADE get us there?
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
The evidence based medicine movement has championed the need for objective and transparent methods of clinical guideline development. The Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) framework was developed for that purpose. Central to this framework is criteria for assessing the quality of evidence from clinical studies and the impact that body of evidence should have on our confidence in the clinical effectiveness of a therapy under examination. Grades of Recommendation, Assessment, Development, and Evaluation has been adopted by a number of professional medical societies and organizations as a means for orienting the development of clinical guidelines. As a result, the method of GRADE has implications on how health care is delivered and patient outcomes. In this paper, we reveal several issues with the underlying logic of GRADE that warrant further discussion. First, the definitions of the "grades of evidence" provided by GRADE, while explicit, are functionally vague. Second, the "criteria for assigning grade of evidence" is seemingly arbitrary and arguably logically incoherent. Finally, the GRADE method is unclear on how to integrate evidence grades with other important factors, such as patient preferences, and trade-offs between costs, benefits, and harms when proposing a clinical practice recommendation. Much of the GRADE method requires judgement on the part of the user, making it unclear as to how the framework reduces bias in recommendations or makes them more transparent-both goals of the programme. It is our view that the issues presented in this paper undermine GRADE's justificatory scheme, thereby limiting the usefulness of GRADE as a tool for developing clinical recommendations.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.160 | 0.394 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.002 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it