MétaCan
Menu
Back to cohort
Record W4385295156 · doi:10.1111/ajo.13727

Improving trustworthiness in research in women's health: A collective effort by OBGYN Editors

2023· editorial· en· W4385295156 on OpenAlexafffund
Vincenzo Berghella, Amir Aviram, Nancy C. Chescheir, Caroline de Costa, Patrick Dicker, A Goggins, Janesh Gupta, Thomas D’Hooghe, Anthony Odibo, Aris T. Papageorghiou, George R. Saade, Michael Geary

Bibliographic record

VenueAustralian and New Zealand Journal of Obstetrics and Gynaecology · 2023
Typeeditorial
Languageen
FieldDecision Sciences
TopicMeta-analysis and systematic reviews
Canadian institutionsHealth Sciences CentreSunnybrook Health Science Centre
FundersKU LeuvenDepartment of Obstetrics, Gynecology and Reproductive Sciences, University of PittsburghUniversity of TorontoJames Cook UniversityThomas Jefferson UniversityRoyal College of Surgeons in IrelandUniversity of North Carolina at Chapel HillWashington University in St. LouisYale University
KeywordsTrustworthinessPsychologyComputer scienceSocial psychology

Abstract

fetched live from OpenAlex

Overall, more than 1.7 million entries were added to PubMed in 2022. Over 35 000 Obstetrics articles alone have been added per year since 2020, with that number reaching over 40 000 in 2021 and 2022.1 As the volume of published medical research articles continues to expand, so too do the occurrences of errors and outright research and publication misconduct. It is important for all stakeholders in medical research to take an active role in identifying possible errors, misconduct, and manipulation of the publication process. An ad hoc group of editors of journals related to obstetrics, gynaecology, women's health, and related topics are collectively focusing on the problem of manuscripts with flawed or fabricated data, or other types of misconduct. Although our initial focus is on randomized controlled trials and systematic reviews with or without meta-analysis, the principles apply across methodologies. We aim to improve the trustworthiness of published data in research in women's health with the processes we describe here. Current clinical guidance relies heavily on randomized controlled trials (RCTs) and meta-analyses of RCTs, often considered the pinnacle of evidence-based medicine. These studies are used to determine treatment and follow-up protocols, and they are frequently included in national and international guidelines.2, 3 However, if the data in these RCTs and meta-analyses of RCTs are untrustworthy, medical decisions based on them can lead to patient harm. Costs to the research enterprise include wasted effort to replicate results; chasing flawed hypotheses; closing lines of inquiry if the answer is considered “known;” an individual's career aspirations; and journal resources.4 To improve the reliability of data in RCTs and meta-analyses of RCTs, two paths can be taken: prevention of publication of flawed studies in the first place, and retrospective identification of untrustworthy data with post-publication correction of the literature. There have been several separate strategies proposed to prevent the publication of flawed RCTs and systematic reviews.5-16 We have agreed to implement a bundle of strategies17 which will be required upon submission to our journals (Table 1). Non-adherence to these criteria will result in rejection. Those who perform systematic reviews and meta-analyses of RCTs are also called upon to ensure that the included RCTs are free of falsified data by asking the investigators for documents or datasets that can verify the authenticity of the data. The global view of protecting the obstetrics and gynaecology literature from publishing flawed research requires communication among editors. This must be balanced by respect for the authors and an “innocent until proven guilty” mindset. Even so, we know that authors, once rejected by one journal, will usually submit to another. Present general practice prevents editors from communicating with each other about suspected misconduct in a manuscript. However, COPE guidelines support communication between editors when it is necessary for the Editor-in-Chief to fulfil their obligation to “respond to suspected research misconduct.” Such communication will need to be transparent, factual, and respectful. Our group is investigating methods to allow for such communications, compliant with COPE guidelines. We will continue to require that potentially publishable submissions be screened using software to identify evidence of plagiarism and text-recycling. To identify untrustworthy data in studies that have previously been published in our journals, editors will consider all post-publication allegations of misconduct. Credible allegations relating to articles published within the previous 5 years will always be fully investigated. Investigations of this type are best done by examining the primary data sets on which the paper is based. It is at the discretion of the editor whether to investigate allegations about papers published before this timeframe. The steps agreed upon are outlined in Box 1, and are based upon the COPE guidance for handling cases of suspected data fabrication.18 They involve contacting the authors and their institutions to ask for clarifications, with suggested deadlines for each step, to facilitate a timely investigation process (Fig. 1). While COPE's guidelines are non-specific regarding the timeliness of these investigations and editorial actions, we have agreed upon a schedule for completion of these steps. Authors/institution response time: The following timeframes are suggested: Notes aCo-authors will be contacted where their contact details have been provided, or where these are readily available (e.g. through Internet search). Apart from these steps, several journals have appointed research integrity or trustworthiness editors, or committees, who work to improve the submission, review, and editorial processes so that only studies with true and reliable data are published. All journals will continue to update their author guidelines to reflect the agreed-upon changes. Journals are invited to join this initiative, and feedback from authors and readers is welcome. Most authors and institutions perform honest, reliable, and reproducible research. The concern of this editorial is focused on intentional falsification of data, and not on honest mistakes or errors and other types of misconduct. In 2019, the editors of the European Journal of Clinical Investigation stated “As a linchpin in maintaining the quality of the scientific literature, those involved in the editorial process have a duty to redouble their commitment for fulfilling the ideals of science as a self-correcting process.”19 With this editorial, we state our aspiration to provide the scientific community with dependable, trustworthy data to base clinical decision-making on. We are united in our efforts in this battle and welcome your input. American Journal of Obstetrics & Gynecology MFM, International Journal of Gynecology and Obstetrics, BJOG: An International Journal of Obstetrics and Gynaecology. Reproductive, Female and Child Health, Ultrasound in Obstetrics and Gynecology, European Journal of Obstetrics and Gynaecology, Australian and New Zealand Journal of Obstetrics and Gynaecology and American Journal of Perinatology.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame distilled prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.

metaresearch head score (Codex)0.107
metaresearch head score (Gemma)0.133
Version: codex-gemma-dda1882f352aValidation status: machine_predicted_unvalidated
Candidate categoriesMetaresearch, Meta-epidemiology (narrow), Research integrity
Consensus categoriesMetaresearch
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: Not applicable
GenreCandidate signal: Editorial · Consensus signal: Editorial
Teacher disagreement score0.375
Threshold uncertainty score1.000

Codex and Gemma teacher scores by category

CategoryCodexGemma
Metaresearch0.1070.133
Meta-epidemiology (narrow)0.0000.000
Meta-epidemiology (broad)0.0050.000
Bibliometrics0.0040.005
Science and technology studies0.0000.000
Scholarly communication0.0010.000
Open science0.0010.000
Research integrity0.0010.002
Insufficient payload (model declined to judge)0.0010.000

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.450
GPT teacher head0.498
Teacher spread0.049 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; both teacher heads agree on what is shown here.

Study designNot applicable
Domainnot available
GenreEditorial

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations1
Published2023
Admission routes2
Has abstractyes

Explore more

Same venueAustralian and New Zealand Journal of Obstetrics and GynaecologySame topicMeta-analysis and systematic reviewsFrench-language works237,207