Use of Real‐World Data to Emulate a Clinical Trial and Support Regulatory Decision Making: Assessing the Impact of Temporality, Comparator Choice, and Method of Adjustment
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
External controls have been primarily used in the setting of single-arm trials of rare diseases; their use in common diseases has not been readily investigated, nor is there guidance on how to best select comparators. Thus, the objective of this study was to emulate a large cardiovascular outcome trial of type 2 diabetes to compare associations of effectiveness with different comparator groups to those reported in the trial. Using the Liraglutide Effect and Action in Diabetes: Evaluation of Cardiovascular Outcome Results (LEADER) trial, we investigated six comparator groups using three calendar time periods (Early: 1999-2003; Later: 2004-2008, and Contemporaneous: 2009-2013) and two comparators (sulfonylureas and other second-to-third-line antidiabetic drugs). Hazard ratios (HRs) of the three-point composite cardiovascular outcome were estimated using four variations of the propensity score (adjustment, stratification, fine stratification, and matching) and compared with the LEADER trial (HR, 0.87; 95% confidence interval, 0.78-0.97). When comparing users of liraglutide with users of sulfonylureas, the HRs ranged from 0.57 to 1.03, with estimates in the early period most closely reflecting the LEADER trial (HR, 0.57-0.88). In contrast, the HRs ranged from 0.73 to 0.97 when comparing liraglutide users with users of any second-to-third-line antidiabetic drugs, although the later period generated estimates closest to the LEADER trial (HR, 0.77-0.84). Different methods of adjustment led to generally consistent HRs, aside from the fine stratification in the early period. This study highlights the complex interplay between comparator, temporality, and method of adjustment when selecting comparators using real-word data. These design choices must be considered in the design of trial emulation studies.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Direct model labels (unvalidated)
Per-model category and study-design labels from the labeling rounds. They are machine output, unvalidated, and the disagreement between models ships as data. No study design here is MEDLINE-validated yet.
| Model arm | Categories | Study design | Confidence |
|---|---|---|---|
| gemma | no category Domain: not available · Genre: Empirical About the Canadian research system: no · About a Canadian topic: no | Simulation or modeling | low |
| gpt | no category Domain: not available · Genre: Methods About the Canadian research system: no · About a Canadian topic: no | Other design | low |
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.018 | 0.024 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.003 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.002 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it