Bibliographic record
Abstract
Background and objectives: Incorporating patient values and preferences as an essential input for decision-making has its potential merits in respecting the autonomy of patients, improving adherence and clinical outcomes. The Grading of Recommendations Assessment, Development and Evaluation (short GRADE) working group conceptualizes patient values and preferences as “the relative importance patient place on the main outcomes”. The objectives of this thesis include: 1) to provide an overview of a process for systematically incorporating values and preferences in guideline development; 2) to conduct a systematic review on outcome importance studies, using chronic obstructive pulmonary disease (COPD) as an example; 3) to provide guidance on how to assess certainty of evidence describing outcome importance using the GRADE criteria. Methods: We performed systematic reviews, asked clinical experts to provide feedback according to their clinical experience, and consulted patient representatives to obtain information about relative importance of outcomes in a new national guideline program. We conducted a systematic review to summarize the COPD related relative importance of outcome studies. We used a multi-pronged approach to develop the guidance for assessing certainty of evidence about relative importance of outcome and values and preferences. We applied the developed GRADE approach to relative importance of outcome systematic review examples and consulted the stakeholders in the GRADE working group for feedback. Results and conclusion: We provided an empirical strategy to find and incorporate values and preferences in guidelines by performing systematic reviews and eliciting information from guideline panel members and patient representatives. However, we identified the need for researches on how to assess the certainty of this evidence, and best summarize and present the findings. In our comprehensive systematic review project on COPD patient values and preferences we demonstrated the utility of rating evidence in systematic reviews of outcome importance. We describe the rationale for considering GRADE domains for the evidence about the importance of outcomes. We propose the assessment of the body of evidence starts at “high certainty”, and rate down for serious problems in GRADE domains including risk of bias, indirectness, inconsistency, imprecision and publication bias. Specific to risk of bias domain, we propose a preliminary consideration for risk of bias. The sources of indirectness for relative importance of outcome evidence include indirectness from PICO (population, intervention, comparison, and outcome) elements, and methodological indirectness. As meta-analyses are uncommon when summarizing the evidence about relative importance of outcome, inconsistency and imprecision assessments are challenging. Inconsistency arises from PICO and methodological elements that should be explored. The width of the confidence interval and sample size should inform judgments about imprecision. We also provide suggestions on how to detect publication bias based on empirical information. Finally, we also discuss the applicability of domains to rate up the certainty. We develop the GRADE approach for rating risk of bias, indirectness, inconsistency, imprecision and other domains when evaluating a body of evidence describing the relative importance of outcomes. Our examples should guide users and provide a basis for discussion and further development of the GRADE system.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.002 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.006 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".