Prescribing competency assessment for Canadian medical students: a pilot evaluation
Bibliographic record
Abstract
Background: The knowledge and ability to prescribe safely and effectively is a core competency for every graduating medical student. Our previous research suggested concerns about medical student prescribing abilities, and interest in a standardized assessment process. Methods: A multi-year cross-sectional study evaluating the feasibility, acceptability, and discriminative ability of an online prescribing competency assessment for final year Canadian medical students was conducted. Students at nine sites of four Ontario medical schools were invited to participate in an online one-hour exam of eight domains related to prescribing safely. Student feedback on perceived fairness, clarity, and ease of use formed the primary outcome. Exam performance and parity between schools were the secondary outcome. Results: A total of 714 students completed the assessment during spring final review courses between 2016 and 2018. Student feedback was more favourable than not for appropriateness of content (53.5% agreement vs 18.3% disagreement), clarity of questions (65.5% agreement vs 11.6% disagreement), question layout and presentation (70.8% agreement vs 12.2% disagreement), and ease of use of online interface (67.1% agreement vs 13.6% disagreement). Few (23.6% believed their course work had prepared them for the assessment. Mean total exam score was 70.0% overall (SD 10.4%), with 47.6% scoring at or above the pass threshold of 70%. Conclusion: Our prescribing competency assessment proved feasible, acceptable, and discriminative, and indicated a need for better medical school training to improve prescribing competency. Further evaluation in a larger sample of medical schools is warranted.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.011 | 0.021 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.001 | 0.000 |
| Open science | 0.002 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".