[V] Estimation of an Upper Limit on the Maximum Effect That Can Be Detected in Randomized Trials of Cancer Therapeutics

Benjamin Djulbegovic,¹ Iztok Hozo,² Renata Iskander,⁵ Austin J. Parish,^3,4 Jonathan Kimmelman,⁵ John P. A. Ioannidis⁴ <h4>Objective</h4> Randomized clinical trials (RCTs) are commonly considered important for detecting small treatment benefits. However, previous studies have shown they are equally likely to detect large (“dramatic”) treatment effects.^1-3 Forecasting the likelihood of future large treatment effects is critical for informing policies on resource allocation in conducting human RCTs, including cancer trials. <h4>Design </h4> We conducted a systematic review to inform generalized Pareto distribution (GPD) under extreme value theory to predict future maximum treatment effects based on data from the past 65 years. We included consecutive cancer RCTs (“cohorts”) identified by funders or trial registries designed to minimize publication bias and analyzed all trials regardless of publication status. Five such cohorts have been described in the literature to date. <h4>Results </h4> Between 1955 and 2018, a total of 716 RCTs testing 984 experimental vs standard treatments in 349,947 patients were conducted and published by 2022, averaging approximately 20 RCTs per year. The shape parameter of the GPD had positive values, indicating no upper limit on maximum treatment effects. We found that the treatment with the largest effect in the past had an odds ratio (OR) of 45 (95% CI, 2-1008). If effect patterns remain the same and a similar pace of performing RCTs continues, the largest predicted future effect over the next 50 years would be an OR of 23 (95% CI, 4-43) (<span class="CharOverride-4"><b>Figure 25-0942</b></span>). We estimated a 20% probability of detecting new treatment effects with an OR greater than 50 in the next 50 years. We also found that conducting more RCTs (40 or 60 per year) would double or triple the probability of detecting breakthrough treatments with dramatic effects. https://assets.underline.io/markdown_image/1/image/e757b092f8aa49cda20b7b719367bd63.png <h4>Conclusions</h4> Our analysis indicates there may be no upper bound on the maximum discoverable treatment effects in cancer RCTs, but effect estimates will likely remain within the range of those observed between 1955 and 2022. Conducting more RCTs would accelerate the detection of treatments with large effects. <h4>References</h4> 1. Djulbegovic B, Kumar A, Glasziou P, Miladinovic B, Chalmers I. Medical research: Trial unpredictability yields predictable therapy gains. <i>Nature</i>. 2013;500(7463):395-396. doi:10.1038/500395a 2. Hozo I, Djulbegovic B, Parish AJ, Ioannidis JPA. Identification of threshold for large (dramatic) effects that would obviate randomized trials is not possible. <i>J Clin Epidemiol</i>. 2022;145:101-111. doi:10.1016/j.jclinepi.2022.01.016 3. Djulbegovic B, Kumar A, Soares HP, et al. Treatment success in cancer: new cancer treatment successes identified in phase 3 randomized controlled trials conducted by the National Cancer Institute-sponsored cooperative oncology groups, 1955 to 2006. <i>Arch Intern Med</i>. 2008;168(6):632-642. doi:10.1001/archinte.168.6.632 ¹Medical University of South Carolina, Division of Medical Hematology and Oncology, Department of Medicine, Charleston, SC, US, djulbegov@musc.edu; ²Department of Mathematics, Indiana University Northwest, Gary, IN, US; ³Department of Emergency Medicine, Lincoln Medical Center, Bronx, NY; ⁴Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine; Department of Epidemiology and Population Health, Stanford University School of Medicine; Department of Biomedical Data Science, Stanford University School of Medicine; Department of Statistics, Stanford University School of Humanities and Sciences, Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, US; ⁵ Department of Equity, Ethics and Policy School of Population and Global Health, McGill University, Montreal, QC, Canada. <h4>Conflict of Interest Disclosures </h4> John P. A. Ioannidis is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. No other disclosures reported. <h4>Additional Information </h4> Datasets for this work were obtained with support of grants from the US National Institute of Health: R01CA140408, R01NS044417, R01NS052956, and R01CA133594 (Benjamin Djulbegovic).

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.