Does the medical literature remain inadequately described despite having reporting guidelines for 21 years? – A systematic review of reviews: an update
Bibliographic record
Abstract
Yanling Jin,1,* Nitika Sanger,2,* Ieta Shams,3,* Candice Luo,4,* Hamnah Shahid,5,* Guowei Li,1,* Meha Bhatt,1 Laura Zielinski,6 Bianca Bantoto,7 Mei Wang,1 Luciana PF Abbade,8 Ikunna Nwosu,4 Alvin Leenus,1 Lawrence Mbuagbaw,1 Muhammad Maaz,1 Yaping Chang,1 Guangwen Sun,1 Mitchell AH Levine,1,9 Jonathan D Adachi1,9 Lehana Thabane,1,9 Zainab Samaan1,10 1Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada; 2Department of Medical Science, Medical Sciences Graduate Program, McMaster University, Hamilton, ON, Canada; 3Department of Psychology, Neuroscience and Behaviour, McMaster University, Hamilton, ON, Canada; 4Faculty of Health Sciences, Bachelors of Health Sciences, McMaster University, Hamilton, ON, Canada; 5Department of Arts and Science, McMaster University, Hamilton, ON, Canada; 6Department of Neuroscience, McMaster Integrative Neuroscience Discovery and Study, McMaster University, Hamilton, ON, Canada; 7Department of Science, Honours Integrated Sciences Program, McMaster University, Hamilton, ON, Canada; 8Department of Dermatology and Radiotherapy, Botucatu Medical School, Universidade Estadual Paulista, UNESP, São Paulo, Brazil; 9St. Joseph’s Healthcare Hamilton, Hamilton, ON, Canada; 10Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada *These authors contributed equally to this work Purpose: Reporting guidelines (eg, Consolidated Standards of Reporting Trials [CONSORT] statement) are intended to improve reporting standards and enhance the transparency and reproducibility of research findings. Despite accessibility of such guidelines, researchers are not required to adhere to them. Our goal was to determine the current status of reporting quality in the medical literature and examine whether adherence of reporting guidelines has improved since the inception of reporting guidelines. Materials and methods: Eight reporting guidelines, such as CONSORT, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), STrengthening the Reporting of OBservational studies in Epidemiology (STROBE), Quality of Reporting of Meta-analysis (QUOROM), STAndards for Reporting of Diagnostic accuracy (STARD), Animal Research: Reporting In Vivo Experiments (ARRIVE), Consolidated Health Economic Evaluation Reporting Standards (CHEERS), and Meta-analysis of Observational Studies in Epidemiology (MOOSE) were examined. Our inclusion criteria included reviews published between January 1996 to September 2016 which investigated the adherence to reporting guidelines in the literature that addressed clinical trials, systematic reviews, observational studies, meta-analysis, diagnostic accuracy, economic evaluations, and preclinical animal studies that were in English. All reviews were found on Web of Science, Excerpta Medical Database (EMBASE), MEDLINE, and Cumulative Index to Nursing and Allied Health Literature (CINAHL). Results: Among the general searching of 26,819 studies by using the designed searching method, 124 studies were included post screening. We found that 87.9% of the included studies reported suboptimal adherence to reporting guidelines. Factors associated with poor adherence included non-pharmacological interventions, year of publication, and trials concluding with significant results. Improved adherence was associated with better study designs such as allocation concealment, random sequence, large sample sizes, adequately powered studies, multiple authorships, and being published in journals endorsing guidelines. Conclusion: We conclude that the level of adherence to reporting guidelines remains suboptimal. Endorsement of reporting guidelines by journals is important and recommended. Keywords: guidelines, adherence, review, CONSORT
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.188 | 0.525 |
| Meta-epidemiology (narrow) | 0.003 | 0.005 |
| Meta-epidemiology (broad) | 0.014 | 0.013 |
| Bibliometrics | 0.041 | 0.030 |
| Science and technology studies | 0.002 | 0.006 |
| Scholarly communication | 0.013 | 0.021 |
| Open science | 0.007 | 0.009 |
| Research integrity | 0.008 | 0.005 |
| Insufficient payload (model declined to judge) | 0.005 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".