Spin Reporting Is Common in Pilot Randomized Controlled Trials in Surgery: A Methodological Survey
Bibliographic record
Abstract
INTRODUCTION: Spin reporting has been studied across a variety of study types and domains; however, it has yet to be studied in the context of pilot and feasibility trials. We designed this methodological survey to evaluate spin reporting in surgical pilot and feasibility trials. METHODS: Medline, Embase, and Cochrane Central Register of Controlled Trials were searched from January 1, 2011, to December 31, 2011, and January 1, 2021, to December 31, 2021. Studies were included if they were pilot or feasibility randomized trials evaluating a surgical intervention. The primary objective was to determine the proportion of pilot and feasibility trials utilizing spin reporting defined as primary focus on efficacy as opposed to feasibility, focus on statistically significant findings as opposed to feasibility, and/or presentation of results as feasible despite not actually being feasible. Secondary objectives included determining the type of spin reporting and exploring the association between study characteristics and spin reporting. RESULTS: After screening 1991 citations, 38 studies from 2011 to 34 studies from 2021 were included. Overall, 59 of the included pilot and feasibility trials (81.9%: 59/72, 95% confidence interval [CI] 71.4-89.3%) utilized spin reporting. Fifty-eight trials (80.6%, 95% CI 69.8-88.2%) primarily focused on efficacy as opposed to feasibility, 34 trials (47.2%, 95% CI 36.1-58.6%) focused on statistically significant findings as opposed to feasibility, and four trials (5.6%, 95% CI 1.8-13.8%) suggested feasibility objectives were met when they were not. Spin was identified in 94.7% (95% CI 81.8-99.5%) and 67.6% (95% CI 50.7-81.0%) of studies published in 2011 and 2021, respectively. CONCLUSIONS: Most pilot and feasibility trials in surgery inappropriately focus on clinical outcomes and statistical significance as opposed to feasibility outcomes for the main future trial. This practice is concerning given that pilot trials are not adequately powered and are intended to serve as exploratory study to increase the likelihood of a successful definitive trial.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Direct model labels (unvalidated)
Per-model category and study-design labels from the labeling rounds. They are machine output, unvalidated, and the disagreement between models ships as data. No study design here is MEDLINE-validated yet.
| Model arm | Categories | Study design | Confidence |
|---|---|---|---|
| gemma | Metaresearch Domain: Reporting · Genre: Review About the Canadian research system: no · About a Canadian topic: no | Observational | low |
| gpt | Metaresearch Domain: Reporting · Genre: Methods About the Canadian research system: no · About a Canadian topic: no | Observational | medium |
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.802 | 0.906 |
| Meta-epidemiology (narrow) | 0.002 | 0.003 |
| Meta-epidemiology (broad) | 0.007 | 0.012 |
| Bibliometrics | 0.023 | 0.027 |
| Science and technology studies | 0.003 | 0.011 |
| Scholarly communication | 0.011 | 0.017 |
| Open science | 0.006 | 0.009 |
| Research integrity | 0.005 | 0.004 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedLabeled directly by 2 models reading the full record.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".