Board 352 - Research Abstract Simulation and the Maintenance of Competence in the Acute Medical Specialties - A Systematic Review (Submission #473)

Introduction/Background In an era of growing public demand for the provision of high quality care and accountability, guidelines for continuing medical education and maintenance of competence are under constant review.1-3 The main objective of our systematic review was to determine whether simulation-based teaching, in isolation or combined with alternative educational strategies, is effective when aimed at the level of the independently practicing physician in the acute medical specialties. For the purpose of this review we define effectiveness relative to Kirkpatrick’s four-level model of educational outcomes.4 Our secondary goal was to describe the current use of simulation-based performance assessment within this population. Methods A systematic search of online databases (MEDLINE, EMBASE, PubMed, CENTRAL, and CDSR) was conducted to include studies published from 1948 (MEDLINE) until January 31, 2013. All original research describing simulation-based educational interventions for attending physicians in anesthesiology, critical care and emergency medicine were considered eligible for inclusion. Data analysis was carried out independently by two authors with further review by a third author in cases of disagreement until consensus was reached. Data extraction was again performed in duplicates and focused primarily on the type of educational intervention, target group, instructional design and effectiveness according to Kirkpatrick’s adapted criteria. For simulation-based performance assessment, tool characteristics and sources of validity evidence were also collated. Results Of 1963 studies screened, 39 studies met our inclusion criteria. Thirty studies described the effectiveness of a simulation-based educational intervention. The number of participants ranged from 2 to 272, with anesthesiologists forming the overwhelming majority (73%). Only seven studies intentionally recruited independent practitioners as their sole target group. The majority of studies were quasi-experimental in design and examined a wide variety of clinical areas, as outlined in Table 1. Educational outcomes targeted the lower levels of Kirkpatrick’s adapted hierarchy (level 1 and 2a) in 13 of the studies identified. With the exception of studies employing survey-based methodology, the effectiveness of the simulation-based education intervention was evaluated in the simulated setting in 14 instances with the aim of determining a change in the learner’s knowledge and/or skills. Over two thirds of studies identified reported a positive effect on learning. All surveys describing the learner’s reaction and self-perceived impact on subsequent clinical practice reported high satisfaction scores with associated improved preparedness and/or performance. We found nine studies that reported the psychometric properties of simulation-based performance assessment as their sole objective. These predominantly recruited independent practitioners as a convenience sample in order to establish discriminative validity and concentrated on a single aspect of validity evidence. Conclusion Acute care physician perceive simulation as a positive experience with self-reported impact on knowledge, skills and attitudes in a wide variety of clinical settings. Limited evidence also suggests that simulation-based educational interventions are associated with improved learning outcomes both immediately and in the longer-term when compared to no other intervention. Future research in this area should focus not only on the optimal modality and frequency of exposure, but also the quality of assessment tools and the impact of simulation-based educational interventions beyond the individual towards improved patient safety. Within the realms of assessment, further dedicated work is needed with efforts to gather multiple sources of validity evidence for a given tool. References 1. Gallagher CJ, Tan JM. The current status of simulation in the maintenance of certification in anesthesia. Int Anesthesiol Clin 2010;48(3):83-99. 2. Bashook PG, Parboosingh J. Recertification and the maintenance of competence. BMJ 1998;316(7130):545-548. 3. McGaghie WC, Siddall VJ, Mazmanian PE, Myers J. Lessons for continuing medical education from simulation research in undergraduate and graduate medical education: effectiveness of continuing medical education: American College of Chest Physicians Evidence-Based Educational Guidelines. Chest 2009;135(3 Suppl):62S-68S. 4. Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 2005;27(1):10-28. Disclosures None.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.