075 The Use of GRADE Methods in the World Health Organization (Who) Public Health Guidelines (PHG): Distribution of Strength of Recommendations and Confidence in Estimates of Effect
Bibliographic record
Abstract
Background A perception exists that expert guideline panellists are sometimes reluctant to offer weak/conditional/contingent recommendations. GRADE guidance warns against strong recommendations when confidence in estimates of effect is low or very low (low or very low quality evidence), suggesting that such recommendations may seldom be justified. Objectives To characterise the distribution of strength of recommendations and confidence in estimates of effect in WHO guidelines that have used the GRADE approach and graded strength of recommendations and confidence in effect estimates. Methods We reviewed guidelines listed in the WHO guidelines database as of November 2012. We identified those that use GRADE and, in these guidelines, examined the distributions of strong and weak and associated confidence in estimates of effect (high, moderate, low, and very low). Results We identified 116 WHO guidelines; 48 (41.3%) referred to GRADE methods, and 43 (37%) utilised GRADE and provided both a strength of recommendation and confidence in estimates grading. These 43 guidelines included 456 recommendations, of which 290 (63.6%) were strong and 166 (36.4%) were conditional/weak. Of the 290 strong recommendations, 97 (33.4%) were based on evidence warranting low confidence in estimates of effect and 63 (21.7%) on evidence warranting only very low confidence. Discussion Strong recommendations based on low or very low confidence in effect estimates are very frequently made in WHO guidelines. Further study to determine the reasons for such recommendations is warranted. Implications for Guideline Developers/Users Guideline authors should provide a clear, compelling rationale for any strong recommendations based on low confidence estimates.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.166 | 0.587 |
| Meta-epidemiology (narrow) | 0.001 | 0.002 |
| Meta-epidemiology (broad) | 0.003 | 0.005 |
| Bibliometrics | 0.021 | 0.016 |
| Science and technology studies | 0.001 | 0.003 |
| Scholarly communication | 0.008 | 0.005 |
| Open science | 0.005 | 0.004 |
| Research integrity | 0.004 | 0.004 |
| Insufficient payload (model declined to judge) | 0.009 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".