Do different clinical evidence bases lead to discordant health-technology assessment decisions? An in-depth case series across three jurisdictions
Bibliographic record
Abstract
BACKGROUND: Health-technology assessment (HTA) plays an important role in informing drug-reimbursement decision-making in many countries. HTA processes for the Pharmaceutical Benefits Advisory Committee (PBAC) in Australia, the Common Drug Review (CDR) in Canada, and the National Institute for Health and Clinical Excellence (NICE) in England and Wales are among the most established in the world. In this study, we performed nine in-depth case studies to assess whether different clinical evidence bases may have influenced listing recommendations made by PBAC, CDR, and NICE. METHODS: Nine drugs were selected for which the three agencies had provided listing recommendations for the same indication between 2007 and 2010. We reviewed the evidence considered for each listing recommendation, identified the similarities and differences among the clinical evidence bases considered, and evaluated the extent to which different clinical evidence bases could have contributed to different decisions based on HTA body comments and public assessment of the evidence. RESULTS: HTA agencies reached the same recommendation for reimbursement (recommended for listing) for four drugs and different recommendations for five drugs. In all cases, each agency used different evidence bases in their recommendations. The agencies considered overlapping sets of clinical comparators and trials when evaluating the same drug. While PBAC and NICE considered indirect and/or mixed-treatment comparisons, CDR did not. In some cases, CDR and/or NICE excluded trials from review if the drug and/or the comparator were not administered according to the relevant marketing authorization. CONCLUSIONS: In the listing recommendations reviewed, considerable variability exists in the clinical evidence considered by PBAC, CDR, and NICE for drug-listing recommendations. Differences in evidence resulted from differences in the consideration of indirect and mixed-treatment comparison data and differences in medical practice in each jurisdiction.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.047 | 0.199 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.005 | 0.006 |
| Science and technology studies | 0.007 | 0.004 |
| Scholarly communication | 0.005 | 0.003 |
| Open science | 0.002 | 0.006 |
| Research integrity | 0.007 | 0.005 |
| Insufficient payload (model declined to judge) | 0.003 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".