Publishing history does not correlate with clinical performance among internal medicine residents
Bibliographic record
Abstract
OBJECTIVES Selection criteria for applicants to the internal medicine programme at the University of Toronto have included the number and quality of scholarly items published. We sought to determine whether previous publishing record correlated with resident performance as measured by in-training evaluation reports (ITERs) and global impressions of clinical competency by site programme directors and senior educators (global impression). METHODS Data on the total number, quality and type of items published, as well as the timing of publishing with regard to pre-MD training, were abstracted from the curricula vitae of individuals who applied for residency during 2001-2005. These were correlated with overall, Expert and Scholar role ITER scores, and with global impression, using Spearman rank correlation scores. RESULTS We gathered publishing history data on 181 residents, for 162 of whom ITER data were available. Overall, 68.5% of residents had published, but only 14.9% had published during medical school. There was a weak correlation of borderline significance (rho = 0.15, P = 0.055) between overall ITER score and number of items published. No such correlation was found with CanMEDS Medical Expert and Scholar role scores. Global impression classified 33.9% of residents as top-rated. More top-rated residents had published (76.7% versus 65.1%; P = 0.07), but the number of items published during medical school were similar between top-rated and non-top-rated residents (16.1% versus 12.3%; P = 0.46). CONCLUSIONS Our results do not support publishing record as a predictor of residents' clinical performance. Surprisingly, the correlation between publishing record and Scholar role scores was also weak, possibly indicating an inability of the ITER to capture this competency. Further research is needed to identify predictors and measures of performance in scholarly activities.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.025 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.001 | 0.003 |
| Insufficient payload (model declined to judge) | 0.006 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".