Addressing concerns around the veracity of scientific research and publication
Bibliographic record
Abstract
any readers of this issue of the CUAJ will be looking forward to attending the 71 st annual meeting of the CUA, being held this month in Vancouver, BC.The academic offerings at the meeting appear to be outstanding and we look forward to celebrating the multidisciplinary collegiality, as well as the research/ education advancements that have defined the meeting over the years.However, if you have been following the science sections of several popular news outlets over the last few months, you will recognize that not everyone has been similarly rejoicing in the aura of such scientific inquiry.As an example, in April 2016, the CBC ran a piece decrying the apparently increasing issue of academic dishonesty in scientific/medical research and, in particular, the phenomenon of manuscript retraction in scientific journals for various offenses.Headlines such as, "I think we have to call it what it is.It is the corruption of the scientific process," certainly garnered our attention.1 This particular CBC exposé highlighted the work of the current, and somewhat controversial, BMJ editor Dr. Fiona Godlee, as well as ongoing campaigns to re-envision how scientific information is reviewed, judged, and reported.There has indeed been burgeoning evidence and increasing focus on scientific misconduct, including issues around plagiarism, falsified data and, occasionally, simple honest mistakes leading to manuscript retraction from journals of all different stripes."Medicine and science are run by human beings, so there will always be crooks," Godlee is reported to say in the CBC piece. 1 Without doubt, there are more and more papers retracted from scientific/biomedical journals (up to 400-500 a year), with some estimating that the majority of these are due to some degree of academic dishonesty.For interesting, and somewhat disturbing, reading we would suggest a quick perusal of the blog, "Retraction Watch."The blog was generated by Ivan Oransky and Adam Marcus in 2010 to shed some light on journal retractions, many of which are often not sufficiently announced or publicized.As of the writing of this editorial, at least one urological article was on the leader board of top 10 referenced articles that had been subsequently retracted.Although article retractions could and should be viewed as a positive mechanism for scientific self-adjustment, the blog also highlights the more nefarious cases of academic misconduct -a symptom of the extraordinary commercial and professional pressures of scientific inquiry and biomedical research.At CUAJ specifically, we have endorsed a code of conduct and best practice guidelines of the Committee on Publication Ethics, a link for which can be found on our website.This editorial was run through several online plagiarism checkers!Concern over retractions of articles for either legitimate mistakes/misinterpretation or more fraudulent motives are only one component of more widespread apprehension over the direction and implementation of biomedical research.Many have voiced alarm around funding and justification of certain drug trials, issues of publication bias, and lack of fulsome reporting of clinical trial results, all of which could potentially lead to inappropriate clinical decision-making and increased healthcare costs, and can create real harm to patients.2 In a recent attempt to address at least some of these real issues, the International Committee of Medical Journal Editors (ICMJE) published a proposal in JAMA that would require authors publishing in their networked journals to automatically share the de-identified individual patient data that make up the results presented in an article within six months of publication.The rationale for such a mandate would be to theoretically allow early independent analysis and confirmation of results, allowing increased "confidence and trust in the conclusions drawn from clinical trials."As stated in their Addressing concerns around the veracity of scientific research and publication
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.065 | 0.248 |
| Meta-epidemiology (narrow) | 0.004 | 0.002 |
| Meta-epidemiology (broad) | 0.005 | 0.004 |
| Bibliometrics | 0.009 | 0.005 |
| Science and technology studies | 0.009 | 0.013 |
| Scholarly communication | 0.028 | 0.013 |
| Open science | 0.008 | 0.005 |
| Research integrity | 0.050 | 0.055 |
| Insufficient payload (model declined to judge) | 0.010 | 0.008 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".