Individual-Participant Data Meta-Analysis Methodological Guidance: A Systematic Review
Bibliographic record
Abstract
Edith Otalike,1 Mike Clarke,2 Ngianga-Bakwin Kandala,1 Joel J. Gagnier1,3 Objective Individual-participant data meta-analysis (IPD-MA) is regarded as the criterion standard in evidence synthesis, but it is resource intensive. While there is consensus on reporting items, a consensus-based tool for the critical appraisal of the methodological quality of IPD-MA does not currently exist. We undertook a systematic methodology review as the initial phase in the development of a critical appraisal checklist. This review collated and summarized the available methodological guidance on IPD-MA of randomized and observational studies. Design We followed the guidelines for Cochrane Methodology Reviews and reported following the PRISMA 2020 guidance. We performed an electronic search of MEDLINE, Embase, CINAHL, Web of Science Scopus, Cochrane Methodology Registry, CONSORT Database of Methodological Papers, Health Technology Assessment Review Database, Research Synthesis Methods, and Journal of the Royal Statistical Society covering publications from 1946 to June 2024. We included studies published in English that addressed any methodological guidance and essential statistical and software requirements for IPD-MA. Data extraction focused on study characteristics, domain of the review process, and the specific recommendations. Risk of bias was assessed using resources relevant to the study design. A thematic synthesis was performed to group recurring themes into domains. For each domain, signalling questions were generated to develop a preliminary checklist for assessment and refinement in a 2-round e-Delphi survey involving international IPD-MA experts. Results The literature search yielded 13,589 citations. After screening 9436 unique abstracts and reviewing 286 full texts, we included 130 articles that met our inclusion criteria. These articles consisted of narrative reviews, handbooks, critical reviews, empirical studies, and statistical method articles. They were published between 1995 and 2024, with most originating from the UK (62 [48%]), the US (20 [15%]), and the Netherlands (16 [12%]) and 50 (38%) originating from 10 other countries. Most of these studies had a low risk of bias. We identified 14 domains of guidance for conducting and reporting of IPD-MA, and we categorized them into 5 sections (Table 25-0999). This finding informed the initial version of the checklist to be evaluated in the e-Delphi survey. https://assets.underline.io/markdown_image/1/image/ae40b29d7f5b93606f06cb13df104504.png Conclusions There are many recommendations in the literature on the general conduct of IPD-MA and on specific aspects of this research, which would benefit from consensus recommendations for all aspects of IPD-MA and critical appraisal of reports. This review provides many suggestions for these recommendations, and our e-Delphi survey will seek consensus on items to include in the critical appraisal tool, which we expect to be completed in 2025. 1Department of Epidemiology & Biostatistics, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada, eotalike@uwo.ca; 2Northern Ireland Methodology Hub, Queen’s University Belfast, Northern Ireland, UK; 3Department of Surgery, Schulich School of Medicine & Dentistry, Western University, London, Ontario, Canada. Conflict of Interest Disclosures Edith Otalike receives internal funding from Western University and the Dean’s Research Scholarship award. No other disclosures were reported. Additional Information Joel J. Gagnier is a co–corresponding author (jgagnie4@uwo.ca).
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.308 | 0.627 |
| Meta-epidemiology (narrow) | 0.004 | 0.006 |
| Meta-epidemiology (broad) | 0.018 | 0.025 |
| Bibliometrics | 0.027 | 0.022 |
| Science and technology studies | 0.002 | 0.003 |
| Scholarly communication | 0.008 | 0.010 |
| Open science | 0.008 | 0.006 |
| Research integrity | 0.007 | 0.006 |
| Insufficient payload (model declined to judge) | 0.020 | 0.004 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".