[V] Outcome Switching in Observational Studies of Interventions: Comparison of Registration Records and Published Articles
Bibliographic record
Abstract
Zexing Song,<sup>1,2</sup> Cecilie Jespersen,<sup>3,4</sup> Asbjørn Hróbjartsson,<sup>3,4</sup> S. Joseph Kim,<sup>1,5,6</sup> Rob Fowler,<sup>1,5</sup> Peter C. Austin,<sup>1,6</sup> An-Wen Chan<sup>1,2,5</sup> <h4>Objective</h4> Outcome switching between study design and reporting is a potential source of bias in observational studies, but there is a paucity of evidence as to its frequency. We aimed to estimate the prevalence of outcome switching in observational studies of interventions (defined as controlled cohort studies investigating the causal effects of interventions on health-related outcomes). Secondary aims included assessing the completeness of prespecification of primary outcomes and factors associated with outcome switching. <h4>Design</h4> This meta-epidemiological study involved longitudinal analyses of observational studies of interventions prospectively registered on ClinicalTrials.gov within 1 month of their study start date between 2014 and 2016 that had results published in a peer-reviewed journal. We screened registry records from January through December 2024 to create the study sample and completed outcome data extraction and analysis from January through April 2025. Complete outcome prespecification required explicit definition in the registry of the measurement variable, analysis metric, method of aggregation (the statistic to summarize the outcome within each group), and time point of the outcome. We evaluated outcome switching by identifying discrepancies in the primary outcomes between the registry and published articles, including omission (prespecified primary outcomes not reported), downgrading (prespecified primary outcomes reported as nonprimary), upgrading (prespecified nonprimary outcomes reported as primary), and introduction of new primary outcomes not listed in the registry. We considered outcome switching to favor statistically significant results if a new statistically significant primary outcome was introduced or upgraded or a nonsignificant one was downgraded. We performed multivariable logistic regression to estimate the association between study characteristics and outcome switching. <h4>Results</h4> We screened 9965 registry records labelled as observational studies and included 127 eligible studies with results published between January 2015 and October 2024. Only 23 studies (18%) completely prespecified their primary outcome in the registry, and the method of aggregation was the least commonly defined element (33 [26%]). Outcome switching was found in 60 studies (47%), and only 1 of these studies (2%) provided a rationale for the changes. The most common discrepancy was omission (32 [25%]) followed by downgrading (30 [24%]), introduction of new primary outcomes (23 studies with 29 new primary outcomes [18%]), and upgrading (2 [2%]). New primary outcomes differed most commonly between the registry and published articles in the measurement variable (21 of 29 [72%]) and time point (15 of 29 [52%]). Among 54 studies that had discrepancies not limited to omitted primary outcomes, statistically significant results were favored in 80% (43 of 54). No study characteristics were significantly associated with outcome switching (<span class="CharOverride-4"><b>Table 25-0874</b></span>). https://assets.underline.io/markdown_image/1/image/d12bf58a0e21db28d130f25b903c1bba.png <h4>Conclusions</h4> Unreported outcome switching and inadequate outcome prespecification were common in observational studies of interventions. These findings underscore the need for improved registration practices and greater transparency to better understand the risk of bias in observational research. <sup>1</sup>Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada, zexing.song@mail.utoronto.ca; <sup>2</sup>Division of Dermatology, Women’s College Research Institute, Women’s College Hospital, Toronto, Ontario, Canada; <sup>3</sup>Cochrane Denmark & Centre for Evidence-Based Medicine Odense (CEBMO), University of Southern Denmark, Odense, Denmark; <sup>4</sup>Open Patient data Explorative Network (OPEN), Odense University Hospital, Odense, Denmark; <sup>5</sup>Department of Medicine, University of Toronto, Toronto, Ontario, Canada; <sup>6</sup>ICES, Toronto, Ontario, Canada. <h4>Conflict of Interest Disclosures</h4> An-Wen Chan is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. No other disclosures were reported.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.009 | 0.015 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.000 |
| Bibliometrics | 0.004 | 0.005 |
| Science and technology studies | 0.000 | 0.004 |
| Scholarly communication | 0.000 | 0.002 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".