Stability of additive treatment effects in multiple treatment comparison meta-analysis: a simulation study
Bibliographic record
Abstract
Kristian Thorlund1, Edward Mills1,21Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, 2Faculty of Health Sciences, University of Ottawa, Ottawa, ON, CanadaBackground: Many medical interventions are administered in the form of treatment combinations involving two or more individual drugs (eg, drug A + drug B). When the individual drugs and drug combinations have been compared in a number of randomized clinical trials, it is possible to quantify the comparative effectiveness of all drugs simultaneously in a multiple treatment comparison (MTC) meta-analysis. However, current MTC models ignore the dependence between drug combinations (eg, A + B) and the individual drugs that are part of the combination. In particular, current models ignore the possibility that drug effects may be additive, ie, the property that the effect of A and B combined is equal to the sum of the individual effects of A and B. Current MTC models may thus be suboptimal for analyzing data including drug combinations when their effects are additive or approximately additive. However, the extent to which the additivity assumption can be violated before the conventional model becomes the more optimal approach is unknown. The objective of this study was to evaluate the comparative statistical performance of the conventional MTC model and the additive effects MTC model in MTC scenarios where additivity holds true, is mildly violated, or is strongly violated.Methods: We simulated MTC scenarios in which additivity held true, was mildly violated, or was strongly violated. For each scenario we simulated 500 MTC data sets and applied the conventional and additive effects MTC models in a Bayesian framework. Under each scenario we estimated the proportion of treatment effect estimates that were 20% larger than ‘the truth' (ie, % overestimates), the proportion that were 20% smaller than ‘the truth' (ie, % underestimates),The coverage of the 95% credible intervals, and the statistical power. We did this for all the comparisons under both models. Results: Under true additivity, the additive effects model is superior to the conventional model. Under mildly violated additivity, the additive model generally yields more overestimates or underestimates for a subset of treatment comparisons, but comparable coverage and greater power. Under strongly violated additivity, the proportion of overestimates or underestimates and coverage is considerably worse with the additive effects model.Conclusion: The additive MTC model is statistically superior when additivity holds true. The two models are comparably advantageous in terms of a bias-precision trade-off when additivity is only mildly violated. When additivity is strongly violated, the additive effects model is statistically inferior.Keywords: multiple treatment, meta-analysis, additive effects, statistical modeling
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Direct model labels (unvalidated)
Per-model category and study-design labels from the labeling rounds. They are machine output, unvalidated, and the disagreement between models ships as data. No study design here is MEDLINE-validated yet.
| Model arm | Categories | Study design | Confidence |
|---|---|---|---|
| gemma | Metaresearch Domain: Methods · Genre: Empirical About the Canadian research system: no · About a Canadian topic: no | Simulation or modeling | low |
| gpt | no category Domain: not available · Genre: Methods About the Canadian research system: no · About a Canadian topic: no | Simulation or modeling | high |
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.263 | 0.402 |
| Meta-epidemiology (narrow) | 0.003 | 0.002 |
| Meta-epidemiology (broad) | 0.006 | 0.018 |
| Bibliometrics | 0.003 | 0.004 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.004 | 0.004 |
| Open science | 0.004 | 0.004 |
| Research integrity | 0.004 | 0.007 |
| Insufficient payload (model declined to judge) | 0.005 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedLabeled directly by 2 models reading the full record.
The models disagree on parts of this classification; every voice is preserved in the section at the end of the page.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".