Supervisor–trainee continuity and the quality of work‐based assessments
Bibliographic record
Abstract
Context Work‐based assessments ( WBA s) represent an increasingly important means of reporting expert judgements of trainee competence in clinical practice. However, the quality of WBA s completed by clinical supervisors is of concern. The episodic and fragmented interaction that often occurs between supervisors and trainees has been proposed as a barrier to the completion of high‐quality WBA s. Objectives The primary purpose of this study was to determine the effect of supervisor–trainee continuity on the quality of assessments documented on daily encounter cards ( DEC s), a common form of WBA . The relationship between trainee performance and DEC quality was also examined. Methods Daily encounter cards representing three differing degrees of supervisor–trainee continuity (low, intermediate, high) were scored by two raters using the Completed Clinical Evaluation Report Rating ( CCERR ), a previously published nine‐item quantitative measure of DEC quality. An analysis of variance ( anova ) was performed to compare mean CCERR scores among the three groups. Linear regression analysis was conducted to examine the relationship between resident performance and DEC quality. Results Differences in mean CCERR scores were observed between the three continuity groups (p = 0.02); however, the magnitude of the absolute differences was small (partial eta‐squared = 0.03) and not educationally meaningful. Linear regression analysis demonstrated a significant inverse relationship between resident performance and CCERR score (p < 0.001, r 2 = 0.18). This inverse relationship was observed in both groups representing on‐service residents (p = 0.001, r 2 = 0.25; p = 0.04, r 2 = 0.19), but not in the Off‐service group (p = 0.62, r 2 = 0.05). Conclusions Supervisor–trainee continuity did not have an educationally meaningful influence on the quality of assessments documented on DEC s. However, resident performance was found to affect assessor behaviours in the On‐service group, whereas DEC quality remained poor regardless of performance in the Off‐service group. The findings suggest that greater attention should be given to determining ways of improving the quality of assessments reported for off‐service residents, as well as for those residents demonstrating appropriate clinical competence progression.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.024 | 0.132 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.001 | 0.002 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".