Study

A substantial number of negative randomized controlled trials presented at ASCO Annual Meetings are not powered with enough participants to detect potentially important clinical findings. That was the conclusion of a study led by Ian F. Tannock, MD, PhD, Professor of Medical Oncology at Princess Margaret Hospital and the University of Toronto, published in the Society's Journal of Clinical Oncology (2007;25:3488–3494). The review of all negative two-arm Phase III randomized controlled trials presented at ASCO Annual Meetings from 1995 to 2003 showed that 55% of the studies didn't have enough participants to detect even a medium-size treatment effect. “Essentially, we found that many trials that have been done were not big enough to detect or rule out clinical differences that might be expected to occur,” Dr. Tannock, Chair of ASCO's Ethics Committee, said in an interview. The power of a study is its probability of detecting a clinically important effect of the experimental treatment, compared with the control arm, if a difference actually exists, he explained. Studies were considered to be underpowered if the total number of assessable participants was less than the sample size needed to detect a prespecified difference in outcome with 80% power and a p level of 0.05. Predictors of Inadequate Size Although the authors of the studies assessed did not specify why the trials were underpowered, 35 of the studies were halted prematurely—37% due to poor accrual, and 25% to what was characterized as a lack of efficacy of the experimental treatment. In multivariable analysis, several predictors of inadequate sample size emerged: ▪ Studies presented at poster sessions or studies that were only published rather than being presented were significantly more likely to lack adequate sample size compared with studies presented orally. ▪ Single-center studies and multicenter studies not sponsored by a cooperative group were significantly more likely to lack adequate sample size than multicenter studies supported by cooperative groups were. ▪ Studies reporting a proportion as a primary endpoint were significantly more likely to lack adequate sample size than those using time-to-event as a primary endpoint were. Dr. Tannock noted that trials that had time-to-event endpoints had larger sample sizes than studies that had a proportion or mean variable as a primary endpoint. Moreover, time-to-event studies were more likely to involve multiple centers or be part of a cooperative group and to be presented at oral sessions. What this suggests, Dr. Tannock said, is that trials that use survival rates or other time-to-event primary endpoints may be more heavily funded and may be more likely to involve a statistician in the research design phase. The type of cancer and whether or not there was pharmaceutical company sponsorship had no significant effect on whether a trial had adequate sample size. Recommendations The findings point to a need to improve the design and reporting of abstracts presented at the annual meeting, he said. “If a clinical trial fails to show a statistically significant benefit in favor of the experimental treatment, an investigator may erroneously conclude that the experimental treatment is of no benefit, even if the trial did not include enough participants to demonstrate reliably a clinically meaningful effect.” Among his group's recommendations are that abstracts that report clinical trials in oncology should explicitly identify a primary endpoint; provide a brief summary of the sample-size calculation; and indicate the statistical power, p value, and anticipated treatment effect size. Additionally, he said, authors should provide a clear explanation as to why a trial is underpowered if it fails to reach its target sample size so that the findings of negative clinical trials can be interpreted appropriately.Figure: Ian F. Tannock MD, PhD: “Many trials that have been done were not big enough to detect or rule out clinical differences that might be expected to occur.”Guidelines Dr. Tannock said he works closely with ASCO to improve the reporting of trials at annual meetings, and in fact, his team's previously published guidelines for reporting of clinical trials have already been adopted by the Society (JCO 2004;22:1993–1999). That analysis showed that a brief description of the intervention, explicit identification of the primary endpoint, and presentation of results accompanied by statistical tests were regarded by experts as the most important items to include in an abstract. But aside from offering guidelines, “there is not much ASCO can do but encourage well-powered studies,” Dr. Tannock said. “ASCO can use guidelines to decide whether to publish an abstract or whether it should be an oral or poster presentation, but it can't actually tell researchers whether to do a trial.” Biostatistics Review James L. Abbruzzese, MD, Scientific Program Committee Chair of the most recent ASCO Annual Meeting, said that a new biostatistical review of the abstracts effectively addresses many of the concerns raised by Dr. Tannock's study. “Each track at the meeting now has an assigned biostatistician to specifically look at many of these factors, such as whether there is an explicit endpoint or how well the study is powered. Then, each abstract to be included in a plenary presentation goes under a second review,” said Dr. Abbruzzese, Professor of Gastrointestinal Medical Oncology and Associate Medical Director of the Gastrointestinal Center at the University of Texas M. D. Anderson Cancer Center. “The 2007 Call for Abstracts had a fairly explicit description of what we want to see in abstract design. If upon review, the biostatistician determines that important elements are missing, he can flag the abstract, and while the abstract wouldn't be rejected outright, it could be downgraded to being a poster or publish-only abstract.” Dr. Abbruzzese added that in certain settings, he thought that it might be better to design studies that detect only large treatment effects. “In the adjuvant setting, progress is often incremental, so, yes, we do want trials that detect even a small effect as that small difference could cure thousands of patients, such as with some of the breast cancer agents,” he said. “But in the advanced disease setting, I am not as sure. If we designed studies that detect a small effect, we may end up with small differences in outcomes that add only a few weeks to the lifespan of patients but expose them to a treatment that is heavily toxic. In this setting, large differences in outcomes are more likely to be important clinically and to be cost-effective.”

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.