Are Simulation Learning Objectives Educationally Sound? A Single-Center Cross-Sectional Study
Bibliographic record
Abstract
INTRODUCTION: Accreditation standards of simulation stress the importance of educationally sound learning objectives. We aimed to assess whether learning objectives adhered to theoretical frameworks outlined by accreditation standards, lending themselves to maximal learning outcomes. METHODS: A retrospective study was conducted at the Centre for Simulation-Based Learning at McMaster University. Raters coded 848 faculty-designed learning objectives from 722 sessions based on Bloom's Taxonomy, SMART (Specific, Measurable, Attainable, Realistic, and Timely) criteria, and the presence of inappropriate verbs. Learning objective categorization was compared with student evaluations. RESULTS: Using Bloom's Taxonomy, learning objectives were mostly focused on application 53%, followed by smaller percentages focused on knowledge 21.4% and comprehension 12.2%. Few learning objectives focused on higher levels of analysis 7.2%, synthesis 2.3%, and evaluation 3.7%. By SMART criteria, learning objectives were 49.6% specific, 60.8% measurable, 88.8% attainable, 85.0% realistic, and 9.1% timely. Approximately 1 in 5 objectives used inappropriate verbs. No correlations were observed between categorization by Bloom's Taxonomy or inappropriate verbs to student ratings. However, those containing attainable and timely goals were associated with lower levels of perceived achievement by students. CONCLUSIONS: There was a disconnect between simulation accreditation standards and current practices at McMaster University's simulation center. Most objectives were classified at lower stages of Bloom's Taxonomy. The majority followed SMART guidelines, with the exception of specificity and mention of time frames. A minority of learning objectives contained inappropriate verbs. Given the costs associated with simulation-based education, educators should focus simulation learning objectives on higher levels of Bloom's Taxonomy and include references to time frames.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.009 |
| Meta-epidemiology (narrow) | 0.000 | 0.001 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".