Immortal Time Bias Prevalence and Influence on Estimates in Systematic Reviews and Meta-Analyses

Jae Il Shin,1,2,3 Min Seo Kim,4,5,6,7 Dong Keon Yon,8 Seung Won Lee,9 Masoud Rahmati,10 Marco Solmi,11,12,13,14,15 Andre F. Carvalho,16 Ai Koyanagi,17,18,19 Lee Smith,20 John P. A. Ioannidis21,22,23,24,25 Objective Immortal time bias (ITB) is the error in estimating the association between the exposure and the outcome that results from misclassification or exclusion of time intervals. The aim of our study was to estimate the prevalence of ITB in systematic reviews and meta-analyses and to assess the degree to which it contributes to effect size estimates and evidence reversal. Design We performed a systematic review of systematic reviews with meta-analyses (SRMA) that only underwent detailed analysis based on ITB. We only included SRMA of observational studies including cohort and case-control studies and excluded those without meta-analysis or SRMA of randomized control trials. We searched PubMed/MEDLINE, Embase, and Cochrane Database of Systematic Reviews from database inception to July 31, 2024. Two authors independently extracted data and evaluated the methodological quality of the systematic reviews. Information on ITB judgment and effect sizes with 95% CIs for individual studies in forest plots were extracted to run reanalysis using generic inverse variance fixed- and random-effects methods. After extracting data, we conducted subgroup analyses by the presence of ITB for all available topics and assessed the influence of ITB on the heterogeneity (I2), changes of evidence, statistical significance of the finding, and altering effect size in favor of intervention/exposure. Results Among the 12 systematic reviews with relevant data, there were 25 eligible topics (only 21 could be divided by ITB, as 4 included only studies without ITB). The median (IQR) number of studies included for a topic was 6 (4-10). Among 182 studies among 25 topics, 44.0% (80 studies) were affected by ITB. Among the 21 topics where both studies with ITB and studies without ITB were available, 57.1% (12/21) demonstrated discordant results between ITB subgroups (Figure 25-0976). Evidence reversal occurred in 23.8% (5/21), where overall summary results changed from statistically significant to non–statistically significant or vice versa after excluding studies with ITB. The ratio of effect size (effect sizes pooled from studies with ITB relative to those pooled from studies without ITB) was 0.71 (95% CI, 0.66-0.78), suggesting that the effect sizes from studies with ITB were exaggerated by an average of 29% in favor of the intervention/exposure. Excluding studies involving ITB reduced the heterogeneity (I2) of overall pooled results by 21.4% on average. https://assets.underline.io/markdown_image/1/image/f4e85eb7af8587aaffcff8858c101115.png https://assets.underline.io/markdown_image/1/image/1a8b75ac4280c10ed9e7883e7ed371a3.png Conclusions We quantitatively captured how far ITB has influenced our knowledge and clinical practices. Given the projected high prevalence and nontrivial influence of ITB, ITB should be considered in studies with survival analyses, and improving reporting standards by researchers as well as collective surveillance from readers, reviewers, and editors is warranted. Future studies should address how ITS may also interact with other trial characteristics and biases in affecting treatment effect estimates. Affiliations 1Department of Pediatrics, Yonsei University College of Medicine, Seoul, Republic of Korea, shinji@yuhs.ac; 2Severance Underwood Meta-Research Center, Institute of Convergence Science, Yonsei University, Seoul, Republic of Korea; 3Affiliate in Meta-Research Innovation Center at Stanford, Stanford University, Stanford, CA, US; 4Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Samsung Medical Center, Seoul, Republic of Korea; 5Medical and Population Genetics and Cardiovascular Disease Initiative, Broad Institute, Cambridge, MA, US; 6Cardiovascular Research Center, Massachusetts General Hospital, Boston, MA, US; 7Department of Medicine, Harvard Medical School, Boston, MA, US; 8Center for Digital Health, Medical Science Research Institute, Kyung Hee University College of Medicine, Seoul, Republic of Korea; 9Department of Data Science, Sejong University College of Software Convergence, Seoul, Republic of Korea; 10Department of Physical Education and Sport Sciences, Faculty of Literature and Human Sciences, Lorestan University, Khoramabad, Iran; 11Department of Psychiatry, University of Ottawa, Ottawa, Ontario, Canada; 12Department of Mental Health, The Ottawa Hospital, Ottawa, Ontario, Canada; 13Ottawa Hospital Research Institute (OHRI) Clinical Epidemiology Program University of Ottawa, Ottawa, Ontario, Canada; 14School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada; 15Department of Child and Adolescent Psychiatry, Charité Universitätsmedizin, Berlin, Germany; 16Innovation in Mental and Physical Health and Clinical Treatment, Strategic Research Centre, School of Medicine, Barwon Health, Deakin University, Geelong, VIC, Australia; 17Research and Development Unit, Parc Sanitari Sant Joan de Déu, Universitat de Barcelona, Fundació Sant Joan de Déu, CIBERSAM, Barcelona, Spain; 18ICREA, Pg. Lluis Companys 23, Barcelona, Spain; 19Instituto de Salud Carlos III, Centro de Investigación Biomédica en Red de Salud Mental, CIBERSAM, Madrid, Spain; 20The Cambridge Centre for Sport and Exercise Sciences, Anglia Ruskin University, Cambridge, United Kingdom; 21Department of Medicine, Stanford University, Stanford, CA, US; 22Department of Epidemiology and Population Health, Stanford University, Stanford, CA, US; 23Department of Biomedical Data Science, Stanford University, Stanford, CA, US; 24Department of Statistics, Stanford University, Stanford, CA, US; 25Meta-Research Innovation Center at Stanford, Stanford University, Stanford, CA, US. Conflict of Interest Disclosures John P. A. Ioannidis is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. Funding/Support The work of John P. A. Ioannidis is supported by an unrestricted gift from Sue and Bob O’Donnell to Stanford University. Jae Il Shin is supported by the Yonsei Faculty Fellowship, funded by Lee Youn Jae. Role of the Funder/Sponsor The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Additional Information John P. A. Ioannidis is a cocorresponding author (jioannid@stanford.edu).

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.