Fairness and transparency in medical journals
Bibliographic record
Abstract
Medical journals serve the public through vetting, dissemination, and archiving of scientific information. This letter will focus on some controversial trends, including prepublication, open reviews, and double-blind review. Prepublication allows open access to data before vetting has occurred. Obvious risks include wide dissemination of biased data, but the human genome project served as a model for the legitimate use of prepublication, allowing more rapid progress of research beyond that which the producers of the data could have accomplished on their own.1Birney E. Hudson T.J. et al.Toronto International Data Release Workshop AuthorsPrepublication data sharing.Nature. 2009; 461: 168-170Crossref PubMed Scopus (197) Google Scholar At its best, prepublication allows for early dissemination of vital data. At its worst, unvetted data can cause patient harm, or important data may be lost or remain hidden if no journal accepts responsibility for its vetting, dissemination, and archiving. Standards for prepublication are being developed, including how to conform to ethical standards, how data are made available, how investigators should cite the original source, and how data should be archived. Funding agencies may help determine which data sets have broad utility that mandate rapid prepublication release. Peer review remains the criterion standard among medical journals, but it can be difficult to balance anonymity and transparency. Most journals do everything possible to protect reviewer confidentiality to ensure candid reviews, but some journals have adopted an open process where reviewer identity is known and the reviews themselves are widely available to public scrutiny or published with the article. Open reviews have been promoted as a means of allowing open discourse about limitations in study design and giving credit to reviewers for an important but often thankless job. The negative effects of open reviews include hesitation to disagree with prominent authors and difficulty in finding reviewers who are willing to be candid without protection of their identity.2Vercellini P. Buggio L. Viganò P. Somigliana E. Peer review in medical journals: beyond quality of reports towards transparency and public scrutiny of the process.Eur J Intern Med. 2016; 31: 15-19Abstract Full Text Full Text PDF PubMed Scopus (18) Google Scholar Although some favor open reviews, other journals have gone in the opposite direction with double-blinded reviews. The major advantage of double-blinded review is avoidance of a perception of bias. Disadvantages include reduced ability to determine author expertise, conflict of interest, duplicate publication, or salami-slicing of data sets. Published evidence suggests that when selecting presenters at national meetings, single-blinding favors prestigious speakers,3Tomkins A. Zhang M. Heavlin W.D. Single- vs. double-blind reviewing at WSDM 2017.Proc Natl Acad Sci U S A. 2017; 114: 12708-12713Crossref PubMed Scopus (218) Google Scholar but there is less evidence to suggest a benefit to double-blinding of journal reviews. Double-blind review is often not truly blinded, because reviewers can commonly identify the authors by other means.4Saini J.R. Sonthalia N.R. Dodiya K.A. Identification of author and reviewer from single and double blind paper.World Acad Sci Eng Technol. 2014; 8: 143-147Google Scholar In the case of one dermatology journal, blinding during peer review did not appear to affect the disposition of the manuscript, and there was no difference in word count between blinded and unblinded reviews.5Alam M. Kim N.A. Havey J. et al.Blinded vs. unblinded peer review of manuscripts submitted to a dermatology journal: a randomized multi-rater study.Br J Dermatol. 2011; 165: 563-567Crossref PubMed Scopus (40) Google Scholar Data also suggest that double-blind peer reviews do not result in higher rates of female authorship. On the contrary, although female authorship has increased across all journals, it decreased in double-blind while increasing in single-blind journals.6Cox A.R. Montgomerie R. The cases for and against double-blind reviews.PeerJ. 2019; 7: e6702Crossref PubMed Scopus (16) Google Scholar Acceptance rates are lower and reviews are more critical with double-blind review, and these patterns are the same for female and male authors.7Blank R.M. The effects of double-blind versus single-blind reviewing: experimental evidence from The American Economic Review.Am Econ Rev. 1991; 81: 1041-1067Google Scholar Given these factors, the majority of journals have retained single-blinded review. Journal of the American Academy of Dermatology allows authors to recommend experts in the field as possible reviewers and enumerate reviewers whom they believe could be biased against their work and therefore should be avoided as reviewers for a particular article. We have always honored the latter request. We consider requests for double-blinded review on a case-by-case basis when there is a high likelihood of bias and publish commentary when reviews suggest important limitations in research methods.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.011 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.003 | 0.000 |
| Bibliometrics | 0.001 | 0.002 |
| Science and technology studies | 0.000 | 0.002 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.009 | 0.001 |
| Research integrity | 0.001 | 0.012 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".