How Do ASH Guidelines Panels Make Decisions? Association between Decision Making Factors and the Strength of Recommendations
Notice bibliographique
Résumé
Abstract Background: Clinical practice guidelines (CPGs) represent a key mechanism for optimizing health care decision making. CPGs are a product of discussion by a group of, typically, 10-20 individuals with varying expertise. Little is known how CPG panels actually make their recommendations. Objective: In a study of real-life decision making, we investigated the factors considered by members of panels convened by the American Society of Hematology (ASH) to develop guidelines, when using the widely accepted formal GRADE (Grading of Recommendations Assessment, Development, and Evaluation) system. Methods: To account for panel level factors and individual level factors, we employed two level hierarchical, random-effect, multivariate logistic and ordered logistic regression analysis. Results: 101 participants taking part in 8 CPGs panels issued 1,289 recommendations. Association of GRADE (normative) factors with the strength of recommendations (SOR) dominated the findings over the non-GRADE (descriptive) factors. In the main analysis certainty in evidence (regardless of direction for or against intervention) [OR=1.83 (95CI% 1.45 to 2.31;p<0.0001)], balance of benefits and harms [OR=1.49 (95CI% 1.30 to 1.69;p<0.0001)] and variability or uncertainty in the patients' values and preferences [OR=1.47 (95CI% 1.15 to 1.88;p<0.002)] were the strongest predictors of SOR coded as "neither for nor against" , "weak for or against" or "strong for or against" health intervention. Greater judgment of certainty of evidence proved highly associated with a strong recommendation [OR=3.60 (95% CI 2.16 to 6.00;p<0.00001] when panel members were issuing recommendation "for" interventions. When, however, panels made recommendations "against" intervention, certainty in evidence was not associated with probability of issuing strong recommendation [OR=0.98 (95%CI: 0.57 to 1.8; p=0.94)]. Two panelist characteristics were associated with strong recommendations: age (per decade) [OR=1.79 (95 CI% 1.2 to 2.84; p<0.005)], and greater intolerance of uncertainties [OR=0.57 (95 CI% 0.37 to 0.86; p<0.008)]. Agreement between individual panel members and the group ranged from very poor (average kappa of -0.01 in one panel) to moderate (kappa=0.64 in another panel), with most panels in an intermediate range. We also found that the panel members who were asked by the ASH to recuse themselves from voting due to high risk of conflict of interest (COI) would have voted differently if they were allowed to do so. Conclusion: Factors associated with GRADE's conceptual framework proved, in general, highly associated with strong versus weak recommendations and with the direction of recommendation. However, some non-GRADE factors of importance for decision-making were also identified. Findings that panel members with high risk of COI made different judgments than those without COI provide empirical support for the importance of managing conflict of interest. The low agreement between individual panel members and group consensus, and failure of certainty of evidence to be associated with strength of recommendations against an intervention, suggest the need for improvements in the process. Disclosures Cuker: Synergy: Consultancy; Genzyme: Consultancy; Kedrion: Membership on an entity's Board of Directors or advisory committees; Spark Therapeutics: Research Funding.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,133 | 0,561 |
| Méta-épidémiologie (sens strict) | 0,000 | 0,001 |
| Méta-épidémiologie (sens large) | 0,001 | 0,002 |
| Bibliométrie | 0,003 | 0,003 |
| Études des sciences et des technologies | 0,001 | 0,002 |
| Communication savante | 0,005 | 0,004 |
| Science ouverte | 0,002 | 0,002 |
| Intégrité de la recherche | 0,002 | 0,003 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,003 | 0,001 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».