Assessor Personality Traits Are Not Educationally Important Drivers of Assessor Stringency/Leniency
Bibliographic record
Abstract
ABSTRACT Introduction Assessor stringency/leniency (ASL), the tendency for an assessor to consistently provide low or high scores, has been shown to have educationally important effects on learner assessment scores in multiple settings. To date, there is no clear understanding of the underlying drivers of ASL in the context of medical education. Some authors have hypothesized a link between personality traits and ASL, but there is currently insufficient data to reach any conclusions. This study seeks to determine whether there is a significant association between physician assessors' personality traits and ASL. Methods This prospective cohort study was conducted at an academic emergency department in Ottawa, Canada. Participating assessors volunteered to complete the IPIP‐Neo 120, a personality questionnaire based on the five‐factor model. All end‐of‐shift assessments completed between July 1, 2021, and June 30, 2022, were collected, and ASL was quantified for each assessor using the mean delta method. Linear regression was used to assess the correlation between personality scores and ASL. Results A total of 2127 assessments, representing 184 learners, were analyzed. Twenty‐five assessors were enrolled, with a wide distribution of assessor personality scores for each trait. While there was a trend toward leniency with increasing assessor extraversion, this did not reach statistical significance ( p = 0.07, R 2 = 0.13). There was no significant link between other personality traits and ASL. Conclusion Integrating our findings with the existing literature, we conclude that personality traits are likely not educationally important drivers of ASL in medicine. Future research should examine other possible contributors to ASL in medical education.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.003 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".