Commentary

Over the last three decades, 26 prescription drugs (3.5% of all those approved in this period) have been withdrawn from the US market for safety reasons.1 On average, these drugs were withdrawn 6 years after their approval and, often, after many millions of people had been exposed. Terfenadine, an antihistamine linked to ventricular arrhythmias, was withdrawn from the US market in 1990 after an estimated 100 million people were exposed worldwide.2 In 1996, >18 million prescriptions were dispensed in the United States for the weight-loss drug fenfluramine. The next year, fenfluramine was withdrawn from the market for causing heart valve damage.3 Today, data on hundreds of millions of patients who use prescription medications and other medical products are routinely collected in electronic healthcare databases. These data typically include adjudicated insurance claims for prescription drugs dispensed at pharmacies and transactional records of diagnoses made and procedures performed in various care settings.4 However, as we learned with rofecoxib, despite the ubiquity of these data, their use has not been optimized to quickly identify and quantify potential harms associated with new medical products. A meta-analysis of 11 observational studies demonstrated the ability of well-conducted pharmacoepidemiologic studies to provide valid documentation of the adverse cardiovascular effects of rofecoxib.5 Yet, only two of these 11 studies were completed and published before rofecoxib’s market withdrawal, a withdrawal that followed exposure to rofecoxib by 80 million people worldwide.6 Yogi Berra7 once said, “You can observe a lot by watching.” With rofecoxib, we could have observed its association with myocardial infarction much sooner had we been actively watching patients’ experiences with the drug as they were accruing.8 Using routinely collected electronic healthcare data to more quickly characterize serious adverse effects of medical products requires three fundamental changes to the ways in which we currently assess the safety of these products once they reach the market. It requires analyses that are distributed, sequential, and semiautomated. Distributed analyses allow investigators to leverage multiple databases without the need for pooling the data into a central repository. Analyses in large networks of distributed databases are increasingly common in epidemiology, particularly for evaluating rare outcomes related to medical product exposure.9 Existing and emerging networks, including the Canadian Network for Observational Drug Effect Studies,10 the HMO Research Network,11 and the US Food and Drug Administration’s Mini-Sentinel system,12 provide an unprecedented opportunity to evaluate medical product use and associated outcomes in up to hundreds of millions of patients. Once a new medical product is authorized for marketing, each of the databases within these networks prospectively captures data describing the use of the product. To characterize associated outcomes as quickly as possible, analyses must also be performed prospectively, in a sequential manner, as the data accrue. The US CDC’s Vaccine Safety Datalink serves as a nice model for distributed, prospective safety surveillance, from which much can be learned.13 Extending the application of active monitoring from vaccines, which relies mainly on self-controlled approaches, to drugs, devices, and other medical products, requires additional methods for addressing confounding in cohort-based design approaches and raises interesting questions about implementing these methods in both distributed and sequential settings.14–16 Semiautomated programs, in which standardized code is sequentially deployed across a distributed network, are required to rapidly perform each analysis as data accrue in each database. Using few inputs, these modularized programs can perform all steps in a typical study, from cohort identification to confounding adjustment to effect estimation and testing. Such programs ensure that methods are applied uniformly across databases and sequential analyses. These programs also enable simultaneous observation of many exposure-outcome pairs across distributed data networks. Semiautomated, distributed, sequential analysis will improve public health only if it leads to timelier decision making. Just as the data infrastructure and semiautomated analysis programs must be developed in advance, making the optimal use of information that comes out of an active monitoring system requires upfront planning about how the results will support decision making. One important consideration is in which database(s) a given assessment should be conducted. To address this, Maro and colleagues17 in this issue of EPIDEMIOLOGY propose a four-step planning process for optimal database configuration selection for active monitoring within a distributed data network. This process involves: (1) sample size calculation to determine how many patients must be watched to detect a certain effect with specified statistical power; (2) estimation of utilization functions in each candidate database; (3) aggregation of exposure information across database configurations to estimate calendar times at which statistically significant results or target numbers of exposed patients would be reached; and (4) selection of the optimal database configuration. Perhaps the most useful component of this framework is a figure that enables investigators to easily visualize the trade-offs among all the inputs in a sample size calculation for a sequential analysis. The configuration that uses all available databases will always minimize time-to-detection. However, Maro and colleagues17 remind us that active monitoring as a public health activity must consider public health costs. In particular, the authors point out that using all available databases may be associated with higher costs in terms of public health dollars if, for example, there is a fixed cost associated with including each database in a particular monitoring activity. On the contrary, not every aspect of routine active surveillance can be automated, and longer surveillance time requires more resources to review, interpret, and double-check results of each sequential analysis. Thus, a key consideration in deciding among various database configurations will be the cost of longer surveillance among a set of data partners versus adding one or more additional databases and conducting surveillance for a shorter duration. Maro and colleagues17 recommend decision analysis with uncertainty to consider the monetary costs of each configuration along with the public health consequences of incurring avoidable adverse events. Quantifying the costs and consequences of all inputs into a decision analytic model will be a difficult, but necessary, part of making efficient use of information that arises from an active monitoring system. In addition to the costs of the databases and the public health consequences of delayed decision making about an unsafe medical product, the costs to follow-up on false-positive signals are a critical consideration that could affect the selection of parameters to minimize type 1 errors. False positives can have potentially profound implications for patients (who might discontinue a needed medication based on concern that it may be unsafe), manufacturers (who stand to lose financially if patients or healthcare provider avoid using their products), and other stakeholders vested in an active monitoring program because false positives could undermine the credibility of the system. To enhance the effective specificity of an active monitoring program beyond statistical alphas and betas, stakeholders can use alerting thresholds that also integrate the magnitude of observed increased risk. Such thresholds will help ensure that alerts are not only statistically significant but also reflect findings that are large enough to be actionable on clinical or regulatory grounds. Thresholds might depend on the severity of the outcome being monitored, the availability of alternative treatments, and the relative effectiveness of the product being monitored to those treatments. Additional work is required to develop frameworks for selecting such alerting thresholds. As these thresholds share some similarity with margins used in noninferiority trials, this could be a useful place to start. Ultimately, for a successful active medical product monitoring system, epidemiology must interdigitate with decision analysis. The framework of Maro and colleagues17 for selecting an optimal database configuration makes this explicit. While the data infrastructure and epidemiologic methods for performing rapid, sequential analyses across databases in distributed networks are easily scalable, innovative approaches to planning and scaling the decision-making processes are needed. With a well-planned decision-making process, we will not only “observe a lot by watching” but also be able to efficiently act on what we observe. As Yogi Berra7 also said, “The future ain’t what it used to be”—true especially for postmarketing drug safety surveillance. ABOUT THE AUTHOR JOSHUA J. GAGNE is an Assistant Professor of Medicine in the Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School. His research focuses on methods for generating postmarketing comparative safety and effectiveness evidence for new medical products. He has developed a propensity score-based semiautomated active monitoring program for sequential monitoring in distributed databases.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.