Bibliographic record
Abstract
We thank Dr. Phillips and colleagues for their letter detailing their experience with peer review on their blog. The process they describe sounds similar to ours, and we are excited to see other blogs taking on scholarly approaches to fostering future authors. We read with interest the findings of increased readership attributable to the new peer review process. While our viewership has also increased (CanadiEM.org now receives > 40,000 page views per month), our article described the results of a survey which investigated the acceptability and feasibility of the coached peer review innovation. We did not focus on increased readership/followership metrics because we did not feel that it would be possible to remove the effect of other confounders. However, we agree that growing numbers of followers are a likely result of increased quality. In fact, we have previously worked on an initiative that uses followership to infer the impact and quality of blogs.1 The Social Media Index has been met with skepticism within the online community associated with the Free Open Access Medical education (FOAM) movement,2 which led us to embark on a longer research program to develop quality evaluation tools for blog posts.3–6 Moving forward, we believe that it is imperative for the scholars within our field to consider outcome measures more explicitly tied to quality when investigating initiatives like Dr. Phillips and colleagues’ and ours. Unfortunately, the quality tools that we have developed have not yet been formally validated, but in the future it may be possible to audit our initiatives using these tools. By rating the quality of our posts before and after the implementation of coached peer review, we could provide further validity evidence that the coached peer review process improves the writing of junior blog contributors. We look forward to hearing more about Dr. Phillips and colleagues’ work and possibly collaborating in the future. Teresa M. Chan, MD, FRCPC, MHPEAssistant professor of emergency medicine, McMaster University, Hamilton, Ontario, Canada, and senior editor, CanadiEM.org; [email protected] Brent Thoma, MA, MD, MSc, FRCPC Assistant professor of emergency medicine, University of Saskatchewan, Saskatoon, Saskatchewan, Canada, and editor-in-chief, CanadiEM.org.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.006 | 0.057 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.004 | 0.004 |
| Scholarly communication | 0.006 | 0.009 |
| Open science | 0.004 | 0.004 |
| Research integrity | 0.046 | 0.053 |
| Insufficient payload (model declined to judge) | 0.025 | 0.023 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".