A Method for Designing Symmetrical Simulation Scenarios for Evaluation of Behavioral Skills

The increasing use of simulation technology in medical education necessitates a method to formulate carefully planned scenarios to assess performance. Much of simulation in healthcare endeavors to teach technical skills such as the drugs and dosages needed during Advanced Cardiac Life Support (ACLS)1 or procedural skills such as those needed for laparoscopic surgery2–8 or bronchoscopy.9–11 In contrast, scenarios can be designed specifically to evaluate behavioral skills such as communication, leadership, and situation awareness. These are skills typically emphasized in crisis resource management (CRM) courses.12–15 This manuscript describes a systematic framework for developing interchangeable, “symmetrical” simulation scenarios that can be randomly ordered to repeatedly test crisis behavioral skills. The mechanism given in this methodologic description is reproducible and can generate a series of scenarios that are similar in flow, basic content, and difficulty. This standardizes the expertise necessary to solve each crisis and avoids the wide range of complexity commonly seen in simulation scenarios. Symmetrical scenarios provide a context for assessment using behavioral skills scoring systems by providing clear opportunities within each scenario for participants to demonstrate elements of each skill. To compare “before and after” performance and behavioral skills retention using such a rating system, it is essential to have comparable simulation scenarios. Many institutions derive their CRM scenarios from a database of clinical scenarios already in existence. Scenarios developed from actual cases may be the most realistic and can avoid a potential complaint that “this would never happen in real life.” Some experts would argue that all scenarios should be based on actual clinical events for this reason, and it is reasonable to assume that purely formative sessions might best be grounded in specific clinical experiences. However, restricting simulations to known clinical events may lead to widely disparate scenarios, making comparisons of technical and behavioral skills difficult. A recent study by Yee and coauthors assessed and reevaluated crisis behavioral skills in anesthesiology residents using clinical scenarios from an existing repertoire.16 Using a variety of clinical scenarios with a diverse range of complexity, they found a measurable benefit from CRM training but could not show additional benefit from further simulator sessions. When assessing behavioral skills with complex, catastrophic crises, it may be difficult to distinguish poor performance from flawed scenarios.17 Indeed, an early investigation by Gaba et al. in 1998 stated a need for “more tightly controlled scenarios.”17 This proposed method of symmetrical scenario development relies on actual clinical experiences adapted in a realistic but specific manner to elicit particular responses from team members, thus facilitating assessment of behavioral skills for controlled trials. Evaluating the performances of simulation laboratory participants has prompted the development of many scoring systems. One such tool is the Anesthesia Nontechnical Skills (ANTS) scoring system, whose categories are shown in Table 1.18 ANTS is a validated system created by researchers at the University of Aberdeen.19 It is recognized for its practical ease of use, capability to assess both nurses and physicians, and applicability to multiple clinical settings and specialties.19,20 Other evaluation systems that facilitate the evaluation of crisis behavioral skills include the older NOTECHS system, borrowed from aviation literature,21 the Crisis Management Behaviors rating instrument (Table 2),17 and the Ottawa Crisis Resource Management Global Rating Scale (Table 3).22 The method described in this paper is based on the principles outlined in the ANTS construct; however, scenarios can be developed to use any scoring system that assesses teamwork and interpersonal skills.Table 1: Anesthesia Nontechnical Skills (ANTS)Table 2: Crisis Management BehaviorsTable 3: Ottawa Crisis Resource Management Global Rating ScaleDESCRIPTION OF METHODOLOGY (WITH EXAMPLES) This methodology creates a series of two or more CRM scenarios that can be used to assess crisis behavioral skills (Table 4). Each series of scenarios has common characteristics, such as the type of human simulator and the nature of the “crisis event.” Once these determinations are made, each individual scenario in the series needs unique identifying characteristics, such as event location and the roles of the human actors who interact with study participants. Finally, it is useful to organize the details of the resulting scenarios (Table 5) and match each element to a specific CRM behavior (Table 6).Table 4: Crisis Scenario Development MethodologyTable 5: Series of Four Crisis Scenarios Created from Development MethodologyTable 6: Scenario Circumstances and Corresponding Behavioral Skills Matched to CRM Scoring ScalesTable 6: (Continued)STEPS COMMON TO ALL SCENARIOS 1. Assemble a Team of Scenario Developers Members of the scenario development team should include those with clinical expertise, knowledge of the CRM principles being assessed, and familiarity with the measurement system being used. For example, the researchers in our planning team included a multidisciplinary group of nurses and physicians well-versed in the principles of CRM. Several members were proficient with the ANTS scale. 2. Consider Study Participants When Selecting Patient Characteristics Next, consider the levels of clinical expertise of the participants as well as their medical disciplines. The selection of patient characteristics can be tailored to the experience of study participants, such as pairing pediatric nurses with pediatric patients. In this example, the physiologic parameters and basic traits of the simulator patient (such as age and weight) were identical for each scenario. This standardization contributed to symmetry among the scenarios in the series, acting as a control to eliminate the differing clinical issues associated with different patients and different clinical signs. Also, participants can focus on CRM principles rather than on technical details given that a standard patient weight simplifies the calculation of drug dosages. 3. Choose a “Crisis Event” To further deemphasize clinical and technical issues, we avoided highly complex medical problems and presented circumstances where mastery of behavioral skills would promote smooth handling of the crisis. Additionally, based on the clinical expertise of the design team, a straightforward, typical, single pathology was chosen as the triggering event for the crisis. For example, each of the four scenarios involves a medication error leading to a hypoxic event. Medication errors are a daily risk encountered by all health care providers and provide a logical clinical basis for hypoxia, making the resulting scenarios realistic. This crisis event can be recognized by multiple levels of clinicians from medical students to nurses to attending physicians, and therefore was appropriate for participants with different skill sets. 4. Develop Scenario Flow The goal of this step is to determine how and when the study participants will appear in the scenario and how varying the makeup of the crisis team can elicit CRM behaviors. For example, the scenario begins when pediatric nurses assume care of the simulated patient from an actor. These first responders may then call for help from a more specialized pool of health care providers, such as pediatric residents, who may eventually involve a participating anesthesia resident. The basic flow of this scenario algorithm tests the ability of the team to recognize the crisis early, identify a leader, declare an emergency, exchange information, and support the other team members. Because exchange of information among the group is a key element of team working, it is essential to provide opportunities for several handovers so that this action can be assessed. If appropriate, a new leader may be chosen, who is challenged to display new authority. This person must assess the capabilities of the evolving group and assign tasks appropriately. In this example, the hypoxic event begins to unfold relatively quickly. If it is not resolved, the simulated patient becomes bradycardic, requiring the participants to initiate Pediatric Advanced Life Support (PALS) protocols. The scenario ends when the patient is stabilized, either by securing the airway or by reversing the medication error (Fig. 1).Figure 1.: Sample simulator algorithm for all scenarios.Because the makeup of the team members present at the crisis varies, much like real-life emergencies, these scenarios require the group to support each other in achieving each goal. As the flow of the scenarios demand that the roles of the players change by taking on new roles and needing to continuously exchange old and new information, it becomes apparent which teams are supportive and work cohesively during the crisis. Additionally, the varying makeup of the teams demands good task allocation and distribution of work by the leader.23 Steps Unique to Each Scenario Until this point, every scenario in this series has the same framework to create parallel scenarios. But by changing enough variables, such as location of the crisis event, the particular cause of the problem (slow incipient onset of hypoxia, followed by apnea), and the roles of the actors, the scenarios appear different, despite using identical pathophysiology. 5. Develop Events Leading up to Crisis The first step in developing individual scenarios is to create varying background stories leading up to the crisis event. For example, each of the patients has a different specific reason for becoming hypoxic, such as accidentally receiving a narcotic overdose for a procedure. The sequence of events before the first responders arrive must be brief but adequately convoluted so that the participants will need to work together to ascertain the correct history. Here they have opportunities to use situational awareness skills, identify resources, gather details, and exchange information. 6. Select a Setting for the Crisis The location and setting of each scenario is varied to create the sense of a new scenario and preserve suspension of disbelief. The participants are put in generalized, recognizable settings instead of locations unfamiliar to them. For example, in two of the scenarios, the nurses are asked to transport a patient to his room from the MRI scanner or postanesthesia recovery room. These are typical, reasonable tasks for this group of trainees. 7. Define the Distracters and Distractions The next step is to define the distractions and interrupters that are present during the crisis. Task management is challenged by altering the position of the patient or the bed and adding a noise disturbance (e.g., a loud radio or loud TV). Live actors who divert the attention of the participants challenge team working and situation awareness. Two types of actors were chosen: a “clueless” personality who offers no help and constantly draws attention away from the patient, and a “potentially helpful” individual who possesses some relevant information regarding the pathogenesis of the problem and may be used to perform certain tasks during the crisis if requested. Proper planning and preparing can help the team to prioritize current and possible future events. As equipment malfunctions are a common issue faced by multiple levels of health care providers, two types of equipment issues are represented: equipment failures and absent equipment. These failures can be managed by successful teamwork and create opportunities for decision making. For example, the nature of the crisis event requires the team members to assess the airway and provide oxygen for the hypoxic patient. Equipment failures and distractions rapidly create new problems and trigger new crisis events as the initial problem increases in severity.23 A faulty bag-valve-mask and empty or missing oxygen tank connector are smaller problems that can distract from the main issue (proper patient care) and add up to create a sequence of events more problematic than the initial crisis.23 If the participants never realize that there is an equipment failure, the crisis continues to worsen and challenges the participants to set new priorities and find the origin of new problems. In this way, the scenario regenerates opportunities to demonstrate effective task management. This method creates symmetrical scenarios that require groups to deal with other types of distractions: predisposing factors and latent errors. These are the underlying aspects of the crisis that are present before the event.23 They are usually hidden factors and pose no distraction until the adverse consequences declare themselves in the midst of the crisis. The team is challenged by a clueless nursing student to assess his limited capabilities and use assertiveness to remove him from the crisis if he becomes a liability. His annoying presence is not important and he can be ignored initially until the nurses become preoccupied with the ensuing emergency. Then, his persistence challenges the leader's vigilance and may cause mismanagement of the crisis. He can thwart the team's ease in exchanging information and coordinating activities. Alternatively, an efficient team can make this person a nonmedical resource, and turn the most annoying distracter into a valuable helper or messenger. Equipment failures also provide opportunities for team members to exchange information about missing or faulty equipment. The team leader is confronted with issues in coordinating activities, relegating distractions, and reducing fixation errors. The identification of some equipment issues may be limited to those with more in-depth knowledge of the tools; an anesthesiologist may recognize a faulty bag-valve-mask more readily than a pediatric nurse and will need to report this information to the leader. Since there are several failures in each scenario, there is a higher chance that the team will recognize and address this type of issue. 8. Make Resources Available Despite introducing multiple distractions to challenge the subjects' CRM skills, multiple resources should be available during the crisis to assist them when they seek help and information. Replacement equipment and additional patient documentation, such as a history and physical, anesthesia record, or other pertinent medical records should be available. Actors can positively contribute to the efforts of the group; they should be instructed to give information when specifically questioned by the participants but refrain from volunteering any supplemental information. Incorporating teams of providers with different areas of expertise force them to turn towards each other as resources. 9. Write a Script for the Actors Scripts of key phrases and comments for each scenario were written and distributed to the actors to serve as a guide for their dynamic portrayal as the human distracters. Specific cues such as “Wow, the patient looks really blue,” were mixed with unhelpful comments similar to, “This is my first day working in MRI,” and more distracting statements like, “You need to move this patient because we have a full schedule of scans to do.” Upon completion and finalization of the scenarios in our series, we provided wireless headsets to the actors and practiced scenario run-throughs to facilitate smooth and standardized performances. RESULTS The product of our practical example is the series of scenarios shown in Table 5. Organizing the data in this way not only helps to identify the symmetrical glitches among these four scenarios, but also aids in setting up the details of the actual simulation. Because we assessed the CRM skills of participants using the ANTS system, we were required to ensure that this scenario development plan includes adequate testing of the nontechnical skills described in this particular assessment tool. Taking each separate component of the scenario and matching it to a particular element in a scoring system produces a graphical representation like Table 6. Many elements are assessed more than once by each of the circumstances. This duplication is intentional and increases the probability that the participants will have opportunities to demonstrate these behaviors. DISCUSSION This is the first methodological description of the development of a series of interchangeable simulation scenarios for the purpose of teamwork skills evaluation. It allows researchers to create a series of symmetrical scenarios to test crisis behavioral skills such as communication, avoiding fixations, team working, leadership, and situation awareness. This method deemphasizes the importance of clinical knowledge and generates conditions within the scenario to reward effective task management and use of resources. These scenarios are likely amenable to evaluation by various behavioral rating scales, including ANTS (Table 6). The scenarios created by using this method make it possible to assess a group's and of more problems. For example, the problem of is enough to require the team to further problems. They must for the of the As they these scenarios require team members to either avoid or They must make the decision to the medication error with before the patient becomes and needs a Gaba and coauthors to this as the of future and that also the available for future If a leader is early in each scenario, then the ability of the team to risk evaluation and decision making is The leader should assume the of decision and not be in the task Gaba this person the who must on identify and make The leader should be and constantly his situation the the and determine which are This scenario development opportunities to gather information from the patient, and actors, who can be resources as well as distracters. As the crisis and becomes more new data is and it becomes the of the leader and the team members to recognize and the new information. This method was also developed to assess effective and as it is of the most important of a good The exchange of involves patient resource and decision making. The noise distractions clear and specific but they are by the participants who can the by the to equipment errors requires to the and report to the leader.23 also involves specific tasks to specific and exchange of information avoiding of the on CRM in the aviation and anesthesiology This paper those by multiple types of to assign pediatric nurses to the of first as nurses typically a crisis. However, the in which groups arrive to the is interchangeable and creates a different dynamic with each new of providers is possible within this from nonmedical and to nurses and This can the sequence of crisis events. One of our first challenges was to make scenarios that are similar in design the suspension of disbelief. This is the ability of the participants to themselves in a real-life crisis and put on the knowledge that they are in a simulated with an patient. each new patient, it may be more difficult for participants to themselves in a new crisis. it is possible that they may a patient with as they each new scenario in the ability to the circumstances common to all scenarios may certain elements of crisis resource such as situation awareness and decision making. The nurses may first oxygen on the patient because they that particular from a not because they have assessed the situation and have recognized the not specifically our participants for regarding our series of scenarios and any This may be of the first in future of this a certain of among the scenarios may be to preserve their design The of this system is the ability to create many different scenarios based on the same framework and the same sequence of simulation events. The of comparable scenarios that can be is and their can be for use in controlled trials. the same basic equipment and of required to in each scenario, multiple CRM sessions can be given to a group of participants within a of with As other scoring systems for assessing CRM are this method can be adapted for evaluation with those new evaluation during CRM is also if no validated scoring system is used to evaluate performance. may involve the method for the development of scenarios for a variety of health care providers in receiving CRM The method can be to test clinical for clinical problems for any of It may also useful for developing crisis management scenarios that assess technical skills and procedural for such as and design of CRM scenarios is essential for assessment and future of teamwork skills to move This method can guide the development of simulation scenarios that assess crisis behavioral skills. As of the first described for scenario more testing and use by other researchers is necessary to further define its However, this of symmetrical design for a series of simulation scenarios an important new for the scenario

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.