PUBLICATION BIAS AND PROTOCOL DEVIATIONS IN REGISTERED ORTHOPAEDIC TRIALS
Bibliographic record
Abstract
The number randomized controlled trials (RCTs) published in orthopaedic has increased substantially since the early 20th century, though these studies still comprise a small proportion of the overall orthopaedic literature. Not all RCTs are conducted according to the original methodology (i.e. protocol deviation), and not all completed trials are eventually published (i.e. publication bias). At the macro level, publication bias and protocol deviations can greatly impact the overall quality and trustworthiness of the literature available to orthopaedic surgeons and their patients. Thus, the aim of this study was to evaluate publicly registered orthopaedic clinical trials to determine: 1) the rate of uncompleted/unpublished studies, and 2) the frequency and types of protocol deviations in these studies. Using the ClinicalTrials.gov (CTG) database, all orthopaedic RCTs were included if they met the following criteria: 1) interventional RCTs, 2) registered on the CTG registry, 3) the trial was completed between January 1, 2010 and December 31, 2020, and 4) the trial has status “completed.” Ongoing randomized control trials were excluded from the inquiry. Data on trial characteristics was extracted. The proportion of completed studies was calculated and tracked over time. Among published studies, the frequency and types of protocol deviations were recorded. A total of 105 completed RCTs were included. Industry and non-industry funded trials were present in similar proportions (48.6% and 49.5%, respectively). Overall, 35% of trials marked as ‘complete’ were not published, at a minimum of two years following the end of the inclusion period. The proportion of unpublished trials in a given year ranged from 0% (0/3, 2011) to 62.5% (5/8, 2016), with no clear temporal trend. Of the published trials, 63.2% (43/68) did not have a major protocol deviation. Industry and non-industry funded studies had similar rates of protocol deviations, at 37.9% and 36.8%, respectively. The most common type of protocol deviation was the use of a different primary outcome than originally planned, including some studies which did not report on the originally planned primary outcome at all. The most important finding of this study is that despite rising numbers of registered orthopaedic RCTs, publication bias and protocol deviations remain common, with no discernible trend towards improvement over time. Publication bias should be addressed at both the researcher and journal levels, with a commitment to the ongoing application of the scientific method, including thoughtful and intentional replication of previously published RCTs. Protocol deviations must always be tracked and reported with a clear rationale, and a priori outcome selection is an important step towards minimizing risk of bias in RCTs. Ongoing tracking and auditing of protocol adherence and publication efforts are an important part of the research process which should be carefully monitored after RCTs are funded and approved.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.227 | 0.316 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.003 | 0.001 |
| Bibliometrics | 0.002 | 0.006 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; both teacher heads agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".