MétaCan
Menu
Back to cohort

Practice guidelines by specialist societies are surprisingly deficient

2007· letter· en· W1677059730 on OpenAlexaff
James M Wright

Bibliographic record

VenueInternational Journal of Clinical Practice · 2007
Typeletter
Languageen
FieldMedicine
TopicClinical practice guidelines implementation
Canadian institutionsUniversity of British Columbia
Fundersnot available
KeywordsRigourMedicineGuidelineEvidence-based medicineQuality (philosophy)Alternative medicineClinical PracticeEvidence-based practiceInterpretation (philosophy)Medical educationFamily medicineEngineering ethicsEpistemologyPathology

Abstract

fetched live from OpenAlex

The term evidence-based implies that it is supported by data from randomised controlled trials (RCTs). RCTs, if conducted and reported correctly, are accepted as the best way to confidently know the benefits and harms of an intervention. When purportedly evidence-based guidelines and recommendations stray from this principle, they should no longer be able to call them evidence-based. So much of the evidence-based paradigm is missing from the Joint British Societies Guidelines on Prevention of Cardiovascular Disease in Clinical Practice (JBS-2) that the author critiquing them has entitled his article ‘Eminence Based Guideline’ (1). The use of the word ‘eminence’ seems particularly appropriate, as it connotes the ‘arrogance’ that seems to be an invariable part of guidelines based on expert opinion (2). The JBS-2 guidelines score low on most of the main quality criteria according to the AGREE instrument (3): stakeholders’ involvement, rigour of development, applicability and editorial independence. This is not the first time that specialist societies’ clinical practice guidelines have been found deficient; however, it surprising that guideline rigour and quality is not improving. Clinical practice guidelines came about when specialists were asked to give their opinions and guidance as to best practice in an attempt to decrease unacceptable variability and cost inefficiencies in clinical practice. Not surprisingly, this proved inadequate and often irrelevant to the problems encountered in primary care. The evidence-based medicine movement arose to provide some evidential basis behind guideline recommendations. The result has been some improvement in the validity and reliability of most guidelines, but unfortunately this has not had much impact on clinical practice (4). Why is that the case? In my opinion one of the reasons is because guideline writers are unable to overcome their own ‘arrogance’ that they can provide ‘aggressively assertive’ guidance despite lack of evidence (2). The attempt by guideline writers to make the rationale behind the decision process more transparent by grading the recommendations has not worked. Readers of guidelines do not distinguish between grade A, level 1, grade B, level 2 recommendations and grade C, level 4 recommendations. In my experience, when doctors find any recommendations in guidelines inconsistent with their own clinical practice, they become sceptical about all of them. Furthermore as long as guidelines continue to be funded by companies and written by individuals with competing interests, they lack credibility (5). This sad situation is certainly not helped when guidelines, such as JBS-2, are produced that after evaluation are judged as in Minhas’ article: ‘of low quality and should not be recommended for clinical practice’ (1). How can guidelines be improved? I suggest that the best way forward is to return to the first principle. Guidelines should limit their recommendations to the interventions and patient populations that are supported by high quality evidence from RCTs. This approach has many advantages: (i) recommendations would be supported by RCTs and systematic reviews; (ii) there would be no necessity to grade recommendations; (iii) recommendations would be fewer and less subject to bias; (iv) recommendations would be more likely to be followed, because they are fewer and better supported; (v) time and effort to produce guidelines would be less and (vi) clinical settings with no recommendations would be identified as requiring RCTs. Following this first principle, cardiovascular guidelines would not make recommendations regarding lipid targets or blood pressure targets as was done by JBS-2; RCTs are badly needed to determine optimal treatment targets for both lipids and blood pressure. I am not suggesting that it is always easy to determine when RCT evidence is clear enough to make a strong recommendation. There will still be recommendations that are open to debate, and this would be healthy. I am also aware that the methods of evaluating guidelines, including the AGREE instrument (3) used by Minhas could be improved. I am suggesting that physicians and patients deserve truly evidence-based guidelines and not ‘eminence’-based guidelines, and I am hopeful that by putting forward this provocative proposal that I can help to discourage guideline recommendations based on expert (eminent) opinion. None.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame distilled prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.

metaresearch head score (Codex)0.029
metaresearch head score (Gemma)0.575
Version: codex-gemma-dda1882f352aValidation status: machine_predicted_unvalidated
Candidate categoriesMetaresearch, Meta-epidemiology (narrow), Research integrity
Consensus categoriesResearch integrity
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: Not applicable
GenreCandidate signal: Commentary · Consensus signal: Commentary
Teacher disagreement score0.547
Threshold uncertainty score1.000

Codex and Gemma teacher scores by category

CategoryCodexGemma
Metaresearch0.0290.575
Meta-epidemiology (narrow)0.0010.001
Meta-epidemiology (broad)0.0020.002
Bibliometrics0.0000.000
Science and technology studies0.0000.000
Scholarly communication0.0000.002
Open science0.0010.000
Research integrity0.0020.010
Insufficient payload (model declined to judge)0.0010.000

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.416
GPT teacher head0.619
Teacher spread0.204 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; both teacher heads agree on what is shown here.

Study designNot applicable
Domainnot available
GenreCommentary

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations13
Published2007
Admission routes1
Has abstractyes

Explore more

Same venueInternational Journal of Clinical PracticeSame topicClinical practice guidelines implementationFrench-language works237,207