Evaluating pre‐clerkship professionalism in longitudinal small groups
Bibliographic record
Abstract
Context and setting This report focuses on evaluation of professionalism within a new subcomponent of McMaster University's redesigned (2005) curriculum, specifically exploring the use of a rating tool to evaluate and provide feedback on 4 domains of professionalism identified as critical to a Year 1 medical student's performance in a competency-based, integrated professional skills curriculum. The ‘Professional Competency’ curriculum domains include: ethics and moral reasoning; communication skills; self-awareness; self-care; professionalism; clinical skills; lifelong learning, and social and community contexts of health care. Groups of 10 students and 2 facilitators (1 doctor and 1 from a non-medical clinical discipline) meet for 3 hours per week; the group remains intact until clerkship. The format of each week's session varies according to the identified domain(s) and may include practice sessions with standardised patients, case-based ethics or epidemiology problems, large-group sessions, personal reflections and critical incident reports. Why the idea was necessary New tools were necessary to monitor and evaluate performance within the new curriculum addressing pre-clerkship professionalism behaviours identified as essential for the foundations of collaborative practice. What was done Both facilitators completed weekly 10-point observational scales on each student. Students were informed that their performance on the following 4 behaviours would provide the basis of their interim professionalism evaluation (satisfactory/provisional satisfactory): accountability: showing up on time, informing group members of absences; respectful listening: making good eye contact, mirroring non-verbal cues, not interrupting, allowing others to complete thoughts; balancing inquiry and advocacy: respectfully holding dissenting opinion, exploring difference in others, generating/considering alternative perspectives; taking experiential education seriously: being honest about both interest and disengagement, being willing to provide/receive constructive feedback, making links with prior experience, being open to exploring personal impact on others. All students are required to achieve a satisfactory rating on all 4 domains in order to enter clerkship. Evaluation of results and impact Curriculum co-directors reviewed all 150 written interim reports in month 5 of the new curriculum. We were interested to know whether any students received ‘provisional satisfactories’ (and why) and whether students rated as ‘satisfactory’ also received specific ‘educational prescriptions’ that might guide ongoing development. A total of 6 students received ‘provisional satisfactory’ ratings. In this group, most were identified as having problems with difference, engaging with different opinions and managing strong emotions (especially anger). Two were identified as having problems in engaging with experiential education seriously. Thirty students received a ‘satisfactory’ rating, with specific suggestions for improvement. The category ‘taking experiential learning seriously’ received by far the most comments, specifically concerns about not being prepared for tutorials, needing to take more risks in contributing to group discussions and learning how to give constructive feedback. Other comments included concerns about punctuality and professional attire. The 7 comments under ‘balancing inquiry and advocacy’ were evenly divided between students who dominated process and others who exhibited problems in expressing their opinions. Our first use of this tool provided meaningful ratings and educational feedback about professional performance in a group-based curriculum designed to teach professional skills. Rating descriptors will need modification for performance review in clinical settings.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.028 | 0.042 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.003 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".