What is the quality of drug therapy clinical practice guidelines in Canada?
Bibliographic record
Abstract
BACKGROUND: The Canadian Medical Association maintains a national online database of clinical practice guidelines developed, endorsed or reviewed by Canadian organizations within 5 years of the current date. This study was designed to identify and describe guidelines in the database that make recommendations related to the use of drug therapy, and to assess their quality using a standardized guideline appraisal instrument. METHODS: Drug therapy guidelines in the database were identified with the use of search terms and hand searching. Descriptive information about the developers, endorsement by other organizations, publication status, disease and drug focus was abstracted. Each guideline was independently assessed by 3 appraisers (a physician, a pharmacist and a methodologist) with the use of the Appraisal Instrument for Clinical Guidelines. Conditions were classified according to the tenth revision of the International Statistical Classification of Diseases and Related Health Problems. RESULTS: We identified 217 drug therapy guidelines produced or reviewed from 1994 to 1998. Guideline developers included national organizations (47.0%), paragovernment organizations (39.6%) and professional associations (30.9%); 31.3% of the guidelines were published, and 10.6% stated drug company sponsorship. The most common conditions addressed by the guidelines were infections and parasitic diseases (39.6%), neoplasms (11.5%) and diseases of the circulatory system (11.5%). Drugs most commonly cited were anti-infective agents (42.9%), antiviral agents (15.2%) and cardiovascular drugs (16.1%). Eleven organizations produced 176 (81.1%) of the guidelines. In all, 14.7% of the guidelines met half or more of the 20 items assessing rigour of guideline development on the appraisal instrument (mean quality score 30.0% [95% confidence interval (CI) 27.5%-32.6%]), 61.8% met half or more of the 12 items assessing guideline context and content (mean score 57.0% [95% CI 54.6%-59.3%]), and none met half or more of the 5 items assessing guideline application (mean score 5.6% [95% CI 4.7%-6.5%]). Overall, 64.6% of the guidelines were recommended with modification by at least 2 of the 3 appraisers, 9.2% were recommended without change, and 26.3% were not recommended. The quality of the guidelines assessed varied significantly by developer, publication status and drug company sponsorship. No substantial improvement in guideline quality was observed over the 5-year study period. INTERPRETATION: Developers of Canadian drug therapy guidelines are producing guidelines that are often perceived to be clinically useful to physicians and pharmacists, although the methods (or the description of the methods) by which they are developed need to be more rigorous and thorough.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.076 | 0.405 |
| Meta-epidemiology (narrow) | 0.000 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.017 | 0.034 |
| Science and technology studies | 0.005 | 0.004 |
| Scholarly communication | 0.012 | 0.004 |
| Open science | 0.005 | 0.003 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".