GRADE Guidance: Using Thresholds for Judgments on Health Benefits and Harms in Decision Making (GRADE Guidance 42)
Bibliographic record
Abstract
Users of GRADE (Grading of Recommendations Assessment, Development and Evaluation) make judgments about the size of intervention effects on desirable and undesirable people-important health outcomes or on benefits and harms. Benchmarking effect sizes by using decision thresholds (DTs) can help to facilitate these judgments and the process. This article provides GRADE guidance for use of DTs for judgments about the magnitude of desirable and undesirable health effects, such as in a health guideline or health technology assessment. Through iterative discussions and refinement in in-person and online meetings of a GRADE project group and through e-mail communication, the authors developed guidance for using DTs in Evidence-to-Decision (EtD) frameworks. The authors applied the approach and used these examples from guidelines and the results of a randomized methodological study to develop official GRADE guidance. Several alternatives for determining and using DTs are presented. In the first main approach, outcome-specific DTs for trivial, small, moderate, and large effects are determined through a calculation using empirically derived generic coefficients and the outcome's utility value and are compared with the effect estimate obtained from an evidence synthesis. In the second main approach, outcome-specific DTs are also determined, but through direct surveying of decision makers to explicitly assign thresholds for the prioritized health outcomes. The article also describes how these approaches can be combined. The suggested approaches provide transparency for judgments in EtD frameworks that are based on findings from evidence syntheses.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.243 | 0.627 |
| Meta-epidemiology (narrow) | 0.004 | 0.006 |
| Meta-epidemiology (broad) | 0.008 | 0.016 |
| Bibliometrics | 0.018 | 0.013 |
| Science and technology studies | 0.003 | 0.006 |
| Scholarly communication | 0.014 | 0.008 |
| Open science | 0.016 | 0.011 |
| Research integrity | 0.019 | 0.022 |
| Insufficient payload (model declined to judge) | 0.034 | 0.020 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".