Medical publishing under review

Is it time for an overhaul? The surge of science in response to the novel severe acute respiratory syndrome coronavirus 2 (COVID-19) pandemic has placed an unprecedented strain on journal editorial teams. The recent retraction of several high-profile medical publications has highlighted gaps in peer review and provides us with an opportunity to address them. Peer review, intended to filter research and improve the quality of manuscripts, is not without limitations. The process from submission to final publication can be arduous, and timelines can range from a few months to more than a year. Formatting requirements vary between journals. Following rejection, resubmissions are delayed by 2 weeks to 3 months due to reformatting.1 Globally, authors spend ∼23.8 million hours formatting articles, with estimated annual costs exceeding $1.1 billion dollars.1 Peer-reviewed research generates profit for journals and can be expensive for researchers, especially if open access. In 2019, Elsevier—one of the largest global publishers of scholarly journals—made a profit of $1.2 billion USD and based on net margins was more profitable than bank, pharmaceutical, tobacco, and oil sectors.2 In the current system, scientists are forced to pay thrice for research: first to apply for funding that will cover the costs of designing and conducting the research; second for publication fees, especially if open access; and third to volunteer time, often with lost revenue, as a peer reviewer. The quality of the peer review itself, can vary as there are no benchmark qualifications required for peer review and journals typically do not offer reviewers standard tools to assess study quality or validity. The current peer review system is not designed to detect scientific misconduct. Driven largely by the publish or perish scientific culture, a small number of researchers go as far as to fabricate data. Two high-impact medical journals retracted publications on drug treatments for COVID-19 after post-publication concerns regarding fraudulent data were raised by readers. To test the peer review system, a deliberately flawed manuscript was sent to 304 journals and was accepted by 157, highlighting the challenges in data authentication.3 Scientific misconduct can extend to the review process itself due to conflicts of interest. Authors can recommend allies as reviewers to receive favourable reviews; author-recommended reviewers are more likely to recommend acceptance than editor-selected reviewers.4 Even legitimate reviewers can be biased when there are competing interests, providing delayed or harsh reviews to stall publication of a manuscript. Editors may be biased by relationships with the authors or with industry. More than half of 713 editors of high-impact journals received payments from industry, which may influence editorial decisions.5 The limited diversity on editorial teams has meant that journal research priorities and publications do not always reflect those of diverse researchers and populations. In an analysis of thousands of submissions to a biosciences journal between 2012 and 2017, women and authors from continents outside North America and Europe were under-represented as editors, reviewers and senior authors. Editors and peer reviewers were more likely to accept manuscripts from authors of the same gender and country as themselves, especially when the team of editors and reviewers were all men.6 The peer review process can contribute to publication bias when editorial teams favour positive trial outcomes. In an analysis of Food and Drug Administration reviews of antidepressant agents, 36/37 positive studies and only 3/36 negative studies were published.7 In an analysis of 4600 publications, more than 85% reported positive results.7 The pressure to publish positive findings can influence research design, outcome selection, post hoc analyses, reporting, and marketing. Major journals, while in competition with each other, can implement standard, agreed-upon formatting that adheres to International Committee of Medical Journal Editors reporting standards. As a first step, all journals should permit a standard format for initial submission (Figure 1). Such an initiative can streamline workflow and minimize both cost and time drain on the part of researchers. Recommendations to Improve Medical Publishing. All journals should set benchmark timelines for submission to initial decision and meet these by improving efficiencies. Authors could be encouraged to submit reviewers’ comments from prior unsuccessful submissions, along with rebuttals or responses to reviewers’ comments. Revenue-generating journals could remunerate peer reviewers, a strategy associated with a 22% reduction in peer review time.8 Free access to journal content, acknowledgment on the journal websites, and automatic submission of reviewed studies into platforms such as Publons may also incentivize reviewers. Formal reviewer training programs may increase the expertise of reviewers. A randomized controlled trial investigated the effects of short training sessions on peer reviewer’s review quality, and observed small but beneficial effects in error detection.9 The trial recommended longer training sessions for peer reviewers, but such training has yet to be tested. Transparent peer review, with publication of reviewers' commments in a supplement may increase the quality of feedback provided to authors. The potential of sham or biased reviews can be minimized by having journal editors select appropriate reviewers without a recommendation from authors. Journals and editors can create a shared bank or registry of skilled reviewers who can be drawn from a central database to improve the efficiency of reviewer selection. Qualified researchers can be identified and matched to manuscripts based on expertise using artificial intelligence-based recruiting systems. Technology can be used to authenticate datasets and detect data fabrication, which can allow reviewers to focus entirely on critiquing the quality and content of the submissions. Requesting statistical code from authors along with access to the primary data set—if required—may also help confirm data authenticity. Artificial intelligence technologies can be used to assess completeness, consistency and validity of statistical tests in papers. Specialized software can be used to detect plagiarism. Journals should support well-trained statisticians to provide statistical reviews prior to (or contemporaneously) with the manuscript being sent for peer review. As much as possible, the quality and integrity of large datasets should be independently verified, and statistical reviews should be forwarded with the manuscript to the rest of the peer review team. Standardized checklists could improve the quality and consistency of reviews across reviewers and journals. In a randomized trial, the quality of reviews significantly improved when peer reviewers used checklists.9 Checklists could be implemented according to the research methodology. For example, systematic reviews and meta-analyses could follow Preferred Reporting Items for Systematic Reviews criteria and use the Risk of Bias in Systematic Reviews tool; randomized controlled trials could be evaluated using the Revised Cochrane Risk of Bias Tool for Randomized Trials; non-randomized studies could be evaluated using the Risk of Bias in Non-randomized Studies of Interventions tool; and diagnostic studies could be evaluated using the Quality Assessment Tool for Diagnostic Accuracy Studies.9 Double-blind peer review can minimize bias by allowing manuscripts to be evaluated on quality and content, without the identity and reputation of the authors serving as a distraction. In a study of 500 papers, single-blinded reviewers privy to the identity of submitting authors were more likely to recommend acceptance of manuscripts from well-known authors and reputable universities than double-blinded reviewers, highlighting the role of bias and the importance of blinding reviewers.10 Journals should implement a transparent, publicly accessible editorial conflict of interest policy, and exclude individuals with significant industry relations from editorial positions. Conflicts of interest of reviewers should be considered prior to allocating a manuscript for review. Publishers and editorial boards should take the lead in using objective criteria rather than relying on informal networks to select editorial board members and reviewers, an intervention that may facilitate diversity. Researchers across career stage, geographic region, ethnicity, and sex should be encouraged to participate in the editorial and peer review process. Acknowledging that all methodologically rigorous work is important, regardless of results, can minimize bias and improve the scientific evidence base. Publishers should commit to publishing neutral results. Some journals publish high-quality study protocols and commit to publishing the final results regardless of the study outcome, as long as the protocol is followed. Online repositories and pre-print servers are rapidly growing forms of scientific communication, providing researchers the opportunity to disseminate work and receive informal reviews by self-selected reviewers. The Massachusetts Institute of Technology press has announced that it will utilize a pool of 1600 potential reviewers to publish open reviews of pre-prints related COVID-19. Journals could capitalize on the online pre-print repositories and scan articles by subspecialty, research question, and methodology to solicit articles for publication; this would shift the power balance in favour of authors, who are currently limited to submitting manuscripts for publication to one journal at a time. Post-publication approaches to quality assurance, either through post-print servers or real time online commenting might allow journals to be more responsive to post-publication concerns. Peer review is an important aspect of scientific publication. Reviews shaped by content knowledge and methodological expertise are irreplaceable. However, the surge of manuscript submissions and retractions during COVID-19 have highlighted the need to revamp the current medical publishing system. Journals must cooperate to harness technologies, implement efficient processes, limit entrenched biases, and engage self-identified reviewers in open forums to complement double-blinded peer reviewers and improve the quality of scientific publications. Conflict of interest: The authors have no pertinent conflicts of interest. Dr Van Spall's opinions are her own and do not reflect the positions of the journals that she serves as peer reviewer, associate editor, or editorial board member.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.