An Empirical Study on Roles and Their Information Correlations for Interdisciplinary Peer Reviewer
Bibliographic record
Abstract
Peer review has always been known as the gatekeeper for the academic quality of papers. Traditional peer review is mostly carried out in a singular discipline, while peer review for interdisciplinary scientific research must go across the boundaries of multiple disciplines, and there is a real opposition between disciplines ‘differentiation’ and ‘de-differentiation.’ Due to this opposition, the practice of interdisciplinary peer review has more research value. Through an empirical study using Publons data, this article introduces the role theory from sociology as a theoretical support, studies role characteristic information of interdisciplinary peer reviewers, extracts their roles, measures the gaps between the roles according to their correlations, and proposes countermeasures to narrow the gaps through role construction to improve the quality of peer review in general. Specifically, this study adopts the empirical analysis method to obtain the behavioural characteristics data of the reviewers in the top 1 per cent in Cross-Field of the Global Peer Review Awards in 2018 and 2019 from the Publons platform and uses the exploratory factor analysis method to extract the roles of the reviewers. Using structural equation modelling to fit the role information association model, the authors compare and analyse the data models of 2018 and 2019 by using the comparative analysis method. The results found are as follows: The ‘researcher’ role, ‘reviewer’ role, and ‘editor’ role are all positively correlated, and the correlations are getting higher. It indicates that the correlations between the three are getting stronger, which will make transitioning between roles less difficult.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.037 | 0.217 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.004 | 0.004 |
| Science and technology studies | 0.002 | 0.002 |
| Scholarly communication | 0.003 | 0.004 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".