Critical Appraisal of Systematic Reviews With Costs and Cost-Effectiveness Outcomes: An ISPOR Good Practices Task Force Report
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
A systematic review (SR) can provide rigorous and complete evidence to support decision makers who consider both the effectiveness and cost-effectiveness of health interventions. A dramatic increase in published health economic (HE) studies, more specifically cost and cost-effectiveness studies, has resulted in the consequent proliferation of systematic reviews with cost and cost-effectiveness outcomes (SR-CCEO).First, such reviews help to indentify strenghts and weaknesses in HE studies, modelling methodologies, and data for modelling inputs. Second, SR-CCEOs may be informative for decisionmakers in resource allocation decisions for health interventions, especially in countries with limited capacity for health technology assessment (HTA). For the purpose of this article, cost studies are defined as studies analyzing the costs of healthcare interventions, includingcost descriptions and cost-of-illness (economic burden of disease) studies. By cost-effectivenessstudies we mean full economic evaluations, including cost-minimization, cost-effectiveness analysis, cost-utility analysis, cost-benefit analysis, and cost-consequence analysis. Sometimes cost studies might be based on an explicit comparison of alternatives.However, it is challenging to appropriately interpret SR-CCEOs owing to their heterogeneity in applied methods and reporting, and furthermore, owing to variability in clinical and health settings in the original studies they include. Methodologic guidance and checklists that improve the quality of SRs on clinical evidence or decrease risk of bias in their interpretation or synthesis have limited applicability for SR-CCEOs. There is little specific methodologicguidance for SR-CCEOs.Although Chapter 20 of the Cochrane Handbook for Systematic Reviews of Interventions of the Cochrane Collaboration 12 and 3 articles related to informing clinical practice guidelines provide guidance, their recommendations do not focus on evaluating the quality of conduct or the risk of bias in SR-CCEOs. A critical analysis of guidelines on conducting and reporting SR-CCEOs identified multiple disagreements in these recommendations, suggesting that a standardized approach to conducting SR-CCEOs is needed.Making universal recommendations for SR-CCEOs is difficult because they differ in several important aspects, in particular, with regard to their search and inclusion criteria, such as the types of studies included (trial or model-based, cost, or cost-effectiveness), or in reporting solely economic characteristics or economic data alongside clinical outcomes. They also have different objectives (eg, to assess variability in outcomes and synthesize the findings) to identify the evidence gaps, or to assess the methods used.Overall, SR-CCEO reliability and usefulness will improve with good practice guidance for SR-CCEOs with different objectives. Thus, ISPOR (The Professional Society for Health Economics and Outcomes Research) established a global, multistakeholder, multidisciplinary expert task force to address this need (Appendix 1 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.01.002). Although general recommendations on conducting SR-CCEOs are provided, the main goal is guidance on critical appraisal of SR-CCEOs regarding their quality and risk of bias. This report, which includes the ISPOR Criteria for Cost(-Effectiveness) Review Outcomes (CiCERO) Checklist, will assist researchers, producers of health technologies, and evidence users (decision makers/commissioners). The task force categorized the recommendations according to the 6 stages of conducting an SR-CCEO (Table 1).
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.055 | 0.079 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.003 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it