Improving Narrative Feedback for Resident-Led Academic Rounds: The Effects of Assessment Form Design Changes
Bibliographic record
Abstract
ABSTRACT Background Learners benefit more from narrative feedback than numerical scores on formative assessments, yet they often report that feedback is lacking in quality and quantity. Changes to the formatting of assessment forms is a practical intervention with limited literature regarding its impact on feedback. Objective This study explores the effect of a formatting change (ie, relocating the comment section from the bottom of a form to the top) on residents' oral presentation assessment forms and if this affects the quality of narrative feedback. Methods We used a feedback scoring system based on the theory of deliberate practice to evaluate the quality of written feedback provided to psychiatry residents on assessment forms from January to December 2017 before and after a form design change. Word count and presence of narrative comments were also assessed. Results Ninety-three assessment forms with the comment section at bottom and 133 forms with the comment section at the top were evaluated. When the comment section was placed at the top of the evaluation form, there were significantly more comment sections with any number of words than left blank (X2(1)=6.54, P=.011) as well as a significant increase in the specificity related to the task component, or what was done well (X2(3)=20.12, P≤.0001). Conclusions More prominent placement of the feedback section on assessment forms increased the number of sections filled as well as the specificity related to the task component.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.051 | 0.370 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.002 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".