How do Emergency Medicine Attending Physicians Evaluate their Trainees? A Multicenter Focus Group Study
Bibliographic record
Abstract
Background: In-training evaluations have an invaluable role in assessing the clinical competency of the trainee. In this study, we explore which trainees’ characteristics have the strongest impact on their evaluation and whether these characteristics fit in the Royal College of Physicians and Surgeons of Canada's CanMEDS Physician Competency Framework. Based on the seven roles that physicians need to have, the framework describes the capabilities that physicians need to produce better patient outcomes. Methods: Emergency medicine attending physicians involved in supervising residents at the four main emergency medicine residency training sites in Riyadh, Saudi Arabia participated in focus group sessions to identify resident characteristics most frequently noted and their impact on the overall evaluation. The interview process followed a standard format. All interviews were audiotaped, and field notes were taken. Two independent coders coded the interviews using CanMEDS competencies as a framework. The frequency of each mention of a particular characteristic was recorded. Following the interviews, participants were also asked to complete a questionnaire about the CanMEDS competencies they routinely or rarely assess. Results are presented in a descriptive fashion. Results: A total of six focus groups sessions were held with 19 participants. The focus group sessions yielded a total of 145 features, or characteristics. Characteristics relating to medical expertise competencies had the strongest impact, followed by professionalism-related competencies, while characteristics relating to health advocacy and managerial skills had the weakest impact on the evaluation. Conclusion: Our results are consistent with previous literature in showing that evaluators tend to base their evaluations on certain competencies and fail to evaluate competencies across the entire CanMEDS spectrum.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.019 | 0.034 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.003 | 0.002 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.002 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".