Will technology trample peer review in ecology? Ongoing issues and potential solutions
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
The classical view of peer review is that it is our primary process for assessing and judging whether research results should be published in a scholarly journal. However, the increased pressure to publish and technological developments are transforming peer review such that it is becoming a system that judges where work is published rather than whether the research is publishable (a ‘where rather than if’ process). Ecology is a field in which publication numbers puts a particular pressure on the review system. In this forum piece, I summarize the issues with the current publication system and discuss how technology is changing it, while suggesting solutions for important prior and ongoing issues with the peer review system. The view explored here is that technological developments (e.g. ease of creating journals, internet sites, storage, data generation, sharing of data and analytical code) will not eliminate peer review per se but will allow for a new set of parameters in which ethics and the optimal use of public funding will play a vital role in the evolution of the review process. Synthesis The number of papers and journals in Ecology has increased dramatically in the past decade. I present a critical overview of our review system and proposes that pressure to publish and technological developments have transformed peer review into a system that decides “where rather than if” papers are publishable. While reviewing the current pressures and factors playing a vital role in the evolution of the review and publication systems, I propose potential solutions to deal with current and future challenges to the peer review and publication systems.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Direct model labels (unvalidated)
Per-model category and study-design labels from the labeling rounds. They are machine output, unvalidated, and the disagreement between models ships as data. No study design here is MEDLINE-validated yet.
| Model arm | Categories | Study design | Confidence |
|---|---|---|---|
| gemma | Metaresearch Domain: Evaluation · Genre: Commentary About the Canadian research system: no · About a Canadian topic: no | Theoretical or conceptual | low |
| gpt | MetaresearchScholarly communication Domain: Evaluation · Genre: Commentary About the Canadian research system: no · About a Canadian topic: no | Not applicable | high |
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.011 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it