Design, Conduct, and Analysis of Externally Controlled Trials
Bibliographic record
Abstract
Importance: Externally controlled trials (ECTs) can serve as an alternative in settings where randomized clinical trials (RCTs) are unfeasible. However, the methodological rigor of ECTs, particularly with regard to bias control, is often inadequately assessed, which can compromise the validity of studies and lead to incorrect decisions. Objective: To examine the design, conduct, and analysis characteristics of current ECTs and to assess whether appropriate methods were used to control bias. Design, Setting, and Participants: This cross-sectional study searched PubMed for ECTs published between January 1, 2010, and December 31, 2023. Eligible ECTs included single-arm trials with an external control or that used a treatment group from an RCT compared with an external control and evaluated the efficacy and/or safety of a drug or medical device. Data analysis was conducted from March 5 to 20, 2025. Main Outcomes and Measures: Extracted information included design characteristics, external control data sources, transparency in covariate selection, statistical methods, and the use of sensitivity and quantitative bias analyses. The characteristics of included ECTs were compared between journals in the top 25% in their Journal Citation Reports category (Q1) and non-Q1. Results: This study included 180 ECTs, of which 85 (47.2%) focused on oncology. Only 64 (35.6%) provided reasons for using external controls, and 29 (16.1%) were prespecified to use external controls. The main sources of external controls were clinical (also termed real-world) data (98 [54.4%]) and trial-derived controls (67 [37.2%]), while concurrent data collection with the treatment arm was relatively infrequent (18 [10.0%]). Only 14 studies (7.8%) conducted feasibility assessments to evaluate the adequacy of data sources, and 13 (7.2%) specified how to handle missing data in external control datasets. Covariate selection procedures were described in 37 of the 164 studies (22.6%) that reported important covariates. Sixty studies (33.3%) used statistical methods to adjust for important covariates when generating the external control, with the propensity score method being the most common (35 of 60 [58.3%]). Among 120 ECTs that generated external controls without statistical methods, 91 (75.8%) used univariate analysis to estimate treatment effects, and only 18 (15.0%) used multivariable regression analysis. Sensitivity analyses for primary outcomes were performed in 32 studies (17.8%), and quantitative bias analyses (2 [1.1%]) were nearly absent. ECTs in Q1 journals were more likely to prespecify the use of external controls (χ21 = 9.86; P = .002) and provided rationales for using external controls (χ21 = 4.33; P = .04). Thirteen recommendations for the careful practice of ECTs are proposed. Conclusions and Relevance: In this cross-sectional study of ECTs, current practices in the design, conduct, and analysis were suboptimal, limiting their reliability and credibility. The study identified several critical methodological issues, such as the lack of justification for using external controls, failure to prespecify external controls in the protocol, insufficient use of confounding adjustment techniques, inadequate sensitivity analyses, and almost complete absence of quantitative bias analyses. Therefore, actionable suggestions for future ECT practices are proposed.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.562 | 0.785 |
| Meta-epidemiology (narrow) | 0.006 | 0.005 |
| Meta-epidemiology (broad) | 0.017 | 0.010 |
| Bibliometrics | 0.017 | 0.019 |
| Science and technology studies | 0.004 | 0.014 |
| Scholarly communication | 0.013 | 0.011 |
| Open science | 0.006 | 0.006 |
| Research integrity | 0.013 | 0.011 |
| Insufficient payload (model declined to judge) | 0.016 | 0.008 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".