An empirical comparison of univariate and multivariate meta‐analyses for categorical outcomes
Bibliographic record
Abstract
Treatment effects for multiple outcomes can be meta-analyzed separately or jointly, but no systematic empirical comparison of the two approaches exists. From the Cochrane Library of Systematic Reviews, we identified 45 reviews, including 1473 trials and 258,675 patients, that contained two or three univariate meta-analyses of categorical outcomes for the same interventions that could also be analyzed jointly. Eligible were meta-analyses with at least seven trials reporting all outcomes for which the cross-classification tables were exactly recoverable (e.g., outcomes were mutually exclusive, or one was a subset of the other). This ensured known correlation structures. Outcomes in 40 reviews had an is-subset-of relationship, and those in 5 were mutually exclusive. We analyzed these data with univariate and multivariate models based on discrete and approximate likelihoods. Discrete models were fit in the Bayesian framework using slightly informative priors. The summary effects for each outcome were similar with univariate and multivariate meta-analyses (both using the approximate and discrete likelihoods); however, the multivariate model with the discrete likelihood gave smaller between-study variance estimates, and narrower predictive intervals for new studies. When differences in the summary treatment effects were examined, the multivariate models gave similar summary estimates but considerably longer (shorter) uncertainty intervals because of positive (negative) correlation between outcome treatment effects. It is unclear whether any of the examined reviews would change their overall conclusions based on multivariate versus univariate meta-analyses, because extra-analytical and context-specific considerations contribute to conclusions and, secondarily, because numerical differences were often modest.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.032 | 0.050 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.006 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.004 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; both teacher heads agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".