Use of Real‐World Data to Emulate a Clinical Trial and Support Regulatory Decision Making: Assessing the Impact of Temporality, Comparator Choice, and Method of Adjustment
Bibliographic record
Abstract
External controls have been primarily used in the setting of single-arm trials of rare diseases; their use in common diseases has not been readily investigated, nor is there guidance on how to best select comparators. Thus, the objective of this study was to emulate a large cardiovascular outcome trial of type 2 diabetes to compare associations of effectiveness with different comparator groups to those reported in the trial. Using the Liraglutide Effect and Action in Diabetes: Evaluation of Cardiovascular Outcome Results (LEADER) trial, we investigated six comparator groups using three calendar time periods (Early: 1999-2003; Later: 2004-2008, and Contemporaneous: 2009-2013) and two comparators (sulfonylureas and other second-to-third-line antidiabetic drugs). Hazard ratios (HRs) of the three-point composite cardiovascular outcome were estimated using four variations of the propensity score (adjustment, stratification, fine stratification, and matching) and compared with the LEADER trial (HR, 0.87; 95% confidence interval, 0.78-0.97). When comparing users of liraglutide with users of sulfonylureas, the HRs ranged from 0.57 to 1.03, with estimates in the early period most closely reflecting the LEADER trial (HR, 0.57-0.88). In contrast, the HRs ranged from 0.73 to 0.97 when comparing liraglutide users with users of any second-to-third-line antidiabetic drugs, although the later period generated estimates closest to the LEADER trial (HR, 0.77-0.84). Different methods of adjustment led to generally consistent HRs, aside from the fine stratification in the early period. This study highlights the complex interplay between comparator, temporality, and method of adjustment when selecting comparators using real-word data. These design choices must be considered in the design of trial emulation studies.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Direct model labels (unvalidated)
Per-model category and study-design labels from the labeling rounds. They are machine output, unvalidated, and the disagreement between models ships as data. No study design here is MEDLINE-validated yet.
| Model arm | Categories | Study design | Confidence |
|---|---|---|---|
| gemma | no category Domain: not available · Genre: Empirical About the Canadian research system: no · About a Canadian topic: no | Simulation or modeling | low |
| gpt | no category Domain: not available · Genre: Methods About the Canadian research system: no · About a Canadian topic: no | Other design | low |
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.685 | 0.777 |
| Meta-epidemiology (narrow) | 0.003 | 0.002 |
| Meta-epidemiology (broad) | 0.005 | 0.009 |
| Bibliometrics | 0.004 | 0.006 |
| Science and technology studies | 0.001 | 0.005 |
| Scholarly communication | 0.008 | 0.009 |
| Open science | 0.007 | 0.006 |
| Research integrity | 0.006 | 0.006 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedLabeled directly by 2 models reading the full record.
The models disagree on parts of this classification; every voice is preserved in the section at the end of the page.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".