Engagement of early career researchers in collaborative assessments of IPCC reports: achievements and insights
Bibliographic record
Abstract
The participation of a diverse –in terms of geography, discipline and gender– group of Early Career Researchers (ECRs) in the peer review process can help alleviate the workload of senior researchers and counteract the perceptual biases that the latter tend to show. Moreover, ECRs can benefit from developing skills that are often not included in educational programs. From 2018 to 2021, the Association of Polar Early Career Scientists, in collaboration with other associations, organized six group reviews of the Intergovernmental Panel on Climate Change (IPCC) reports by a total of more than 600 ECRs from over 70 different countries. This study aims to evaluate this group review in terms of its contribution to the production of scientific knowledge, and as a career development opportunity for ECRs. The data analyzed consists of application forms, review comments, and feedback surveys that were collected during each review process. The results of this study show that, overall, the group reviews were a success in terms of the experience of ECRs and their contribution to the peer review of the IPCC reports. Most survey respondents considered the general organization of the group reviews satisfactory and expressed interest in participating in future group reviews. However, most participants did not engage in discussions with their peers, which constitutes a missed opportunity to engage in active learning and the shared production of knowledge. ECRs made a significant contribution to the review of the IPCC reports by producing an average of 2,422 ± 532 comments per group review, 36% of which were substantive. PhD students were shown to be as proficient reviewers as postdoctoral researchers and faculty reviewers. More importantly, the diversity of reviewers in terms of geography and discipline, together with the fact that they are ECRs, can help produce more balanced scientific reports since they bring new perspectives, thus counteracting the biases that senior researchers have. These group reviews could be improved by providing more comprehensive training and facilitating communication among reviewers so that they can engage in meaningful exchanges. We conclude that the IPCC should formalize the inclusion of ECRs in future reviews of the IPCC reports.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.098 | 0.201 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.003 | 0.003 |
| Science and technology studies | 0.005 | 0.004 |
| Scholarly communication | 0.013 | 0.008 |
| Open science | 0.002 | 0.010 |
| Research integrity | 0.002 | 0.003 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".