THE MINI MILESTONES ASSESSMENT (MINI-MAS), A DIRECT OBSERVATIONAL TOOL TO ASSESS CLINICAL MILESTONES IN THE ERA OF COMPETENCY-BASED EDUCATION
Bibliographic record
Abstract
Background: Competency based medical education will move training programs away from a time based standard to a model of competence and milestones. Assessment, observation and feedback are key ingredients to successful implementation. Work based assessment tools will be an important part of a multi-modal programmatic assessment for learners. Purpose: The purpose of the study was: 1) to facilitate the development and implement a pilot competency-based curriculum into the general pediatric component of the PGY 1 year at McMaster University, pediatric residency program, 2) to develop a tool, the Mini Milestones Assessment (Mini-MAS) to assess six medical competencies and progression through milestones using the Dreyfus Developmental Model and to pilot the tool in the general pediatric rotations for the PGY 1 and 4 residents and 3) to test the reliability, validity, acceptability and feasibility of the Mini-MAS tool. Method: Twelve PGY 1 residents at McMaster Children’s Hospital were required to complete 40 observations (10 history taking, 10 physical exam, 5 clinical reasoning, 5 communication with families, 5 communication with staff and 5 collaboration) during their general pediatric component of the 2013-2014 academic year. These same competencies were also observed for 9 PGY 4 residents over the same competencies over the same time period although this group was required to complete only 15-20 encounters. Following the study period, a survey was administered to the residents and faculty to assess acceptability and feasibility of the Mini-MAS tool. Kane’s validity framework, which is divided into four components (scoring, generalization, extrapolation and decision), was used to evaluate the Mini-MAS tool. Results: PGY 1 and PGY 4 residents had an average of 36 and 16 observations completed, respectively, across a wide variety of settings and clinical problems with multiple assessors. The scale of the Mini-MAS tool was used appropriately. The tool was able to differentiate between the PGY 1 and PGY 4 learners and showed progress of the PGY 1 learners through the academic year. The G coefficient overall for the Mini-MAS tool was 0.8 for the PGY1 residents and 0.5 for the PGY 4 residents. Correlation between the six competencies assessed was low, achieved by only one competency being observed and having grounded anchors. Learners and faculty were satisfied with the tool. The tool allowed learners to be observed more frequently and receive timely valuable feedback. Conclusion: A pilot competency based curriculum for PGY 1 residents was successfully implemented. The Mini-MAS tool added, as a formative assessment mode to a multi-modal assessment program will benefit the trainee, by increasing their observations and providing residents with valuable feedback. The assessment will inform residents where they stand with respect to their level of training, what competencies they can improve on and how they can make such improvements.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.002 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".