Do machine learning methods make better predictions than conventional ones in pharmacoepidemiology? A systematic review, meta-analysis, and network meta-analysis
Bibliographic record
Abstract
OBJECTIVE: To synthesize existing evidence and compare the predictive performance of conventional statistical (CS) models versus machine learning (ML) methods in pharmacoepidemiology. METHODS: Medline, Embase, PsycINFO, CINAHL and Web of Science databases were systematically searched for predictive pharmacoepidemiologic studies published between January 2018 and September 2025. Independent reviewers extracted predictive metrics and other data from each study and assessed the quality of the comparison between methods. The relative performance of ML compared to CS was estimated for each prediction objective. Performance metrics were pooled in meta-analyses and Bayesian network meta-analyses (NMA). RESULTS: Among 9106 records identified, 65 studies met inclusion criteria, encompassing 83 prediction objectives. For 84 % of these objectives, CS was outperformed by at least one ML method. The median sample size across these studies was 2691 subjects (50 to 1,807,159), and, for binary outcomes, the median number of events per candidate predictor was 17.9 (range 0.28 to 24,260). We observed a risk of bias in the comparison according to at least one of eight major criteria for 39 prediction objectives (47 %). The pooled area under the receiver-operator curve (AUC) ratio for the highest-performing ML method in studies with low risk of bias was estimated as 1.07 (95 % confidence interval 1.03-1.12) in favor of ML, but with very high heterogeneity. NMA of 197 comparisons estimated an AUC ratio of 1.07 (95 % credible interval 1.04-1.12) for boosted methods compared with logistic regression and ranked Gradient Boosting Machine and XGBoost consistently among the best-performing methods. CONCLUSION: Machine learning methods applied to structured pharmacoepidemiologic data demonstrated a consistent yet modest advantage in discriminative performance relative to conventional statistical models. This advantage was most evident for boosted methods such as GBM and XGBoost. However, greater rigor in reporting methodological details is recommended to improve the comprehension, transparency, and reproducibility of studies. REGISTRATION: PROSPERO 2023 registration number: CRD42023426986.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.327 | 0.083 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.027 | 0.011 |
| Bibliometrics | 0.004 | 0.021 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.002 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.026 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; both teacher heads agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".