Misbehavior or misalignment? Examining the drift towards bureaucratic box-ticking in Competency-Based Medical Education
Bibliographic record
Abstract
Within competency-based medical education (CBME) residency programs, Entrustable Professional Activity (EPA) assessments endeavor to both bolster learning and inform promotion decisions. Recent implementation studies describe successes but also adverse effects, including residents and preceptors drifting towards bureaucratic / purely administrative behaviors and attitudes, although the drivers behind this tendency are not adequately understood. This study sought to examine resident and faculty experiences with implemented EPA processes to elucidate what leads them toward a 'tick-box' approach that has been described in the literature. The internal medicine residency program at the University of Alberta implemented a CBME pilot in 2016. From March to June 2018, a research assistant interviewed 16 residents and 27 preceptors shortly after they completed an EPA assessment. They described their goals, judgements, and actions during a recent EPA observation. Three researchers analyzed the data to identify themes following qualitative description methodology. The requirement to accrue EPA assessments turned them into currency exchanged by preceptors and residents to acknowledge clinical work. Predicaments arose when the prescriptive EPA process felt misaligned with the assessment context. The selected encounter sometimes suited formative but not summative purposes. Preceptors variably prioritized the dual formative and summative purposes and framed the message for either the resident's or the program's benefit. The drift toward bureaucracy in workplace-based assessments is becoming a predictable implementation pattern. Instead of solely attributing this pattern to residents and preceptors misusing the assessment process, viewing their actions as workarounds suggests that users make rational choices to overcome obstacles in the assessment system. Obstacles identified by workarounds could be targeted by design modifications.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.043 | 0.119 |
| Meta-epidemiology (narrow) | 0.000 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.007 | 0.009 |
| Scholarly communication | 0.006 | 0.006 |
| Open science | 0.002 | 0.006 |
| Research integrity | 0.001 | 0.004 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".