Simulation-Based Assessment to Measure Proficiency in Mechanical Ventilation among Residents
Bibliographic record
Abstract
Abstract Background Mechanical ventilation (MV) skills are essential for clinicians caring for critically ill patients, yet few training programs use structured curricula and appropriate assessments. Objective structured clinical exams (OSCEs) have been used to assess clinical competency in many areas, but there are no OSCE models focused on MV. Objective To develop and validate a simulation-based assessment (SBA) with an OSCE structure to assess baseline MV competence among residents and identify knowledge gaps. Methods We developed an SBA using a lung simulator and a mechanical ventilator, and an OSCE structure, with six clinical scenarios in MV. We included internal medicine residents at the beginning of their rotation in the respiratory intensive care unit (ICU) of a university-affiliated hospital. A subset of residents was also evaluated with a validated multiple-choice exam (MCE) at the beginning and at the end of the ICU rotation. Scores on both assessments were normalized to range from 0 to 10. We used Cronbach’s α coefficient to assess reliability and Spearman correlation to estimate the correlation between the SBA and the MCE. Results We included 80 residents, of whom 42 also completed the MCE examinations. The final version of the SBA had 32 items, and the Cronbach’s α coefficient was 0.72 (95% confidence interval [CI], 0.64–0.81). The average SBA score was 6.2 ± 1.3, and performance was variable across items, with 80% correctly adjusting initial ventilatory settings and only 12% correctly identifying asynchrony. The MCE had 24 questions, and the average score was 7.6 ± 2.4 at the beginning of the rotation and 8.2 ± 2.3 at the end of the rotation (increase of 0.6 points; 95% CI, 0.30–0.90; P < 0.001). There was moderate correlation between the SBA and the MCE (rho = 0.41; P = 0.002). Conclusion We developed and validated an objective structured assessment on MV using a pulmonary simulator and a mechanical ventilator addressing the main competencies in MV. The performance of residents in the SBA at the beginning of an ICU rotation was lower than the performance in MCE, highlighting the need for greater emphasis on practical skills in MV during residency.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.018 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.001 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".