The Journal of Continuing Education in Nursing

Original Article 

The Effect of Simulation-Based Crew Resource Management Training on Measurable Teamwork and Communication Among Interprofessional Teams Caring for Postoperative Patients

Douglas E. Paull, MD; Lori D. DeLeeuw, RN, MSN; Seth Wolk, MD; John T. Paige, MD; Julia Neily, RN, MS, MPH; Peter D. Mills, PhD

Abstract

Background: Many adverse events in health care are caused by teamwork and communication breakdown. This study was conducted to investigate the effect of a point-of-care simulation-based team training curriculum on measurable teamwork and communication skills in staff caring for postoperative patients.

Methods: Twelve facilities involving 334 perioperative surgical staff underwent simulation-based training. Pretest and posttest self-report data included the Self-Efficacy of Teamwork Competencies Scale. Observational data were captured with the Clinical Teamwork Scale.

Results: Teamwork scores (measured on a five-point Likert scale) improved for all eight survey questions by an average of 18% (3.7 to 4.4, p < .05). The observed communication rating (scale of 1 to 10) increased by 16% (5.6 to 6.4, p < .05).

Conclusion: Simulation-based team training for staff caring for perioperative patients is associated with measurable improvements in teamwork and communication.

J Contin Educ Nurs 2013;44(11):516–524.

Dr. Paull is Director of Patient Safety Curriculum and Ms. DeLeeuw is Nurse Educator, VA National Center for Patient Safety, Ann Arbor, Michigan. Dr. Wolk is Department Chief of Surgical Specialties, Spectrum Health, Grand Rapids, Michigan. Dr. Paige is Assistant Professor of Clinical Surgery, Louisiana State University School of Medicine, New Orleans, Louisiana. Ms. Neily is Associate Director and Dr. Mills is Director, Field Office, National Center for Patient Safety, White River Junction, Vermont.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.

Address correspondence to Lori D. DeLeeuw, RN, MSN, National Center for Patient Safety, 24 Frank Lloyd Wright Drive, Lobby M, Ann Arbor, MI 48106. E-mail: Lori.DeLeeuw@va.gov.

Received: November 13, 2012
Accepted: July 30, 2013
Posted Online: September 10, 2013

Abstract

Background: Many adverse events in health care are caused by teamwork and communication breakdown. This study was conducted to investigate the effect of a point-of-care simulation-based team training curriculum on measurable teamwork and communication skills in staff caring for postoperative patients.

Methods: Twelve facilities involving 334 perioperative surgical staff underwent simulation-based training. Pretest and posttest self-report data included the Self-Efficacy of Teamwork Competencies Scale. Observational data were captured with the Clinical Teamwork Scale.

Results: Teamwork scores (measured on a five-point Likert scale) improved for all eight survey questions by an average of 18% (3.7 to 4.4, p < .05). The observed communication rating (scale of 1 to 10) increased by 16% (5.6 to 6.4, p < .05).

Conclusion: Simulation-based team training for staff caring for perioperative patients is associated with measurable improvements in teamwork and communication.

J Contin Educ Nurs 2013;44(11):516–524.

Dr. Paull is Director of Patient Safety Curriculum and Ms. DeLeeuw is Nurse Educator, VA National Center for Patient Safety, Ann Arbor, Michigan. Dr. Wolk is Department Chief of Surgical Specialties, Spectrum Health, Grand Rapids, Michigan. Dr. Paige is Assistant Professor of Clinical Surgery, Louisiana State University School of Medicine, New Orleans, Louisiana. Ms. Neily is Associate Director and Dr. Mills is Director, Field Office, National Center for Patient Safety, White River Junction, Vermont.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.

Address correspondence to Lori D. DeLeeuw, RN, MSN, National Center for Patient Safety, 24 Frank Lloyd Wright Drive, Lobby M, Ann Arbor, MI 48106. E-mail: Lori.DeLeeuw@va.gov.

Received: November 13, 2012
Accepted: July 30, 2013
Posted Online: September 10, 2013

Adverse events occur in 2.3% of hospital admissions (Agency for Healthcare Research and Quality, 2010). Communication failure is the underlying cause of 70% of these adverse events (Leonard, Graham, & Bonacum, 2004). Many of these adverse events involve surgical patients (de Vries, Hollmann, Smorenburg, Gouma, & Boermeester, 2009). Understandably, health care organizations have focused on improving teamwork and communication in the operating room. Patient safety initiatives, including checklist-guided preoperative briefings and postoperative debriefings, have been associated with lower surgical morbidity and mortality rates (Haynes et al., 2009; Neily, Mills, Young-Xu et al., 2010). Simulation-based training has been associated with improvements in teamwork and communication skills among operating room interprofessional teams (Paige et al., 2008). However, in many cases, adverse events and miscommunication affecting surgical patients occur in the postoperative period (de Vries et al., 2010).

Medical team training (MTT) is based on aviation-style crew resource management (CRM) tools and techniques (Dunn et al., 2007). Given the success of MTT, health care organizations have increasingly committed resources to CRM training for staff outside of the operating room (Pruitt & Liebelt, 2010). What constitutes threshold CRM curriculum and competencies? A Delphi study of expert CRM educators showed 100% concurrence on several knowledge, skill, and attitude competencies, including “articulating a concern about a course of action”; “establishing an atmosphere to encourage open communication”; “communicating critical information in a succinct manner”; “speaking up when concerned”; “describing how checklists can minimize error”; “acknowledging that safety is everyone’s business”; “describing the concept of situational awareness”; “using strategies to promote situational awareness”; “communicating a plan of action”; “is willing to review decisions in light of new information”; “lists common characteristics of effective leadership and followership”; and “cross-monitors actions of other team members” (Clay-Williams & Braithwaite, 2009, pp. 436–438). These same CRM competencies have been the cornerstone of the MTT Program. Before affordable portable high-fidelity simulators were available, MTT educators relied on role playing, docudramas, and exercises to practice CRM techniques (Kirkegaard & Fish, 2004).

The use of high-fidelity simulators allows deliberate practice of CRM tools and techniques with immediate feedback for participants (Rosen et al., 2008). To capitalize on these advantages, the authors incorporated an in situ, point-of-care simulation training model into the MTT Program to teach and practice CRM competencies. This study investigated the effect of a point-of-care simulation-based CRM training curriculum on measurable teamwork skills among interprofessional teams caring for postoperative patients.

Literature Review

The conceptual framework guiding the study was the high-reliability organization (Carroll & Rudolph, 2006). Attributes of high-reliability organizations include a preoccupation with failure, deference to expertise, flattening of the hierarchy, and a culture of inquiry and advocacy. CRM is the teamwork and communication model used to achieve high reliability. CRM is defined as using all available resources, including personnel, information, and equipment, to conduct safe operations (Heard, 2005). It consists of a body of basic patient safety behaviors, including leadership, assertiveness with respect, and closed-loop communication, as well as the use of tools such as checklists, briefings, and debriefings (Dunn et al., 2007). Both high-reliability organizations and CRM are grounded by a deeper understanding of the influence of human factors, such as communication, fatigue, distractions, mental and task loads, and the human–computer interface, on determining human performance.

For example, to maintain situational awareness, teams practice “stepping back,” or pausing, when there is conflicting information or confusion. In an effort to overcome the power gradients in health care, team members practice calling a physician with a concern and increasing their level of assertiveness to address a problem. These CRM skills are associated with improved teamwork and communication, fewer medical errors, and better patient outcomes (McCulloch et al., 2009).

Research Questions

  1. Are there significant differences in confidence in using CRM teamwork and communication techniques, as measured by the Self-Efficacy of Teamwork Competencies Scale, among interprofessional teams caring for postoperative patients before and after a simulation-based curriculum?

  2. Are there significant differences in observable CRM teamwork and communication techniques, as measured by the Clinical Teamwork Scale (CTS), among interprofessional teams caring for postoperative patients before and after a simulation-based curriculum?

Methods

Curriculum

From 2003 to 2010, the Veterans Health Administration (VHA) MTT Program has conducted 210 learning sessions involving 14,896 health care providers. Initially, the MTT Program targeted operating room personnel, using didactic and group interactive methods to teach CRM competencies to targeted facilities. Simulation-based training has been incorporated into MTT curricular activities incrementally since January 2010. Between January 14, 2010, and August 25, 2010, MTT faculty conducted training at 26 VHA facilities. This study focused on the 12 facilities that had simulation-based training for interprofessional staff caring for postoperative patients. The study proposal was approved by the VHA Ann Arbor Research and Development Committee as meeting the requirements for institutional review board exemption.

The VHA MTT Program has been described previously (Dunn et al., 2007; Paull et al., 2009). The curriculum is based on aviation-style CRM communication techniques. Tools and techniques discussed and practiced in the didactic learning session include closed-loop communication (“read back and repeat back”); transparent communication (“call out”); direct communication (use of names and eye contact); “step back” (to prevent “fixation error”); graded assertiveness (“two-attempt rule”); assuming nothing; workload distribution; situational awareness; SBAR (Situation–Background– Assessment–Recommendation) for handoffs; checklists; and briefings and debriefings. A learning session agenda is provided in Table A (available in the online version of this article).

Simulation-based training allowed participants to practice and demonstrate nontechnical skills related to the CRM content of the MTT Program. It also provided an opportunity for experiential learning in a safe environment. Groups of 5 to 10 participants were trained in simulation sessions that were facilitated by experienced MTT physicians and nurse educators. Each 2-hour simulation session followed a standard pattern. Two distinct clinical scenarios were each followed by a structured debriefing on teamwork and communication.

The VHA National Center for Patient Safety (NCPS) library of patient safety simulation scenarios was used. Scenarios included serious postoperative complications and crises, such as postoperative sepsis, pulmonary embolus, bile leak, diabetic ketoacidosis, atrial fibrillation with rapid ventricular response, gastrointestinal bleeding, and myocardial ischemia. The learning objectives for each simulation scenario were based on demonstration of CRM teamwork and communication skills. They included calling for help, patient assessment, situational awareness, assertiveness, patient advocacy, leadership, closed-loop communication, and “stepping back” (Dunn et al., 2007). An example of a patient safety simulation scenario with learning objectives, props, and scripts is provided in Table B (available in the online version of this article).

Study Design

The study used a before-and-after design (Brown et al., 2008b). In part, this study design was chosen because of its simplicity and avoidance of the ethical issues associated with randomization. The intervention was a CRM-based didactic workshop supplemented with simulation-based training in which participants could practice teamwork and communication skills with the CRM tools. Preintervention evaluation included simulation-based training scenarios that were based on actual, de-identified, root cause analysis cases involving the care of postoperative patients (Brown et al., 2008a). The overarching concept was to improve leadership and communication and promote a greater understanding of human factors in an effort to improve teamwork during a crisis, such as the unexpected clinical deterioration of a patient.

For each of the 12 study sites, two to six simulation sessions were scheduled, depending on the total number of participants to be trained. Each session included a first simulation followed by a debriefing, a break, and a second simulation experience with a debriefing. One or more instructors facilitated each simulation scenario.

Study Setting

The setting included surgical care floors at 12 VA Medical Centers across the United States. Didactic training occurred in conference rooms. Simulation training occurred in a patient care room (seven facilities), in a conference room (four facilities), or at an adjoining simulation center (one facility). In all instances, the simulation environment used standardized simulators, monitors, and props. The 12 facilities were located in 10 states, representing 9 of the 21 total Veterans Integrated Service Networks (Veterans Administration, 2010). The complexity of the surgical care delivered at each facility participating in the point-of-care simulation-based training was rated on a three-point scale (complex, intermediate, or standard), based on previously defined criteria (Veterans Administration, 2010).

Sample

The 12 locations were among 26 consecutive sites that received scheduled MTT during an 8-month period and received simulation-based training as a pilot implementation program. The 14 excluded sites did not have simulation-based training. The investigators chose to focus on teamwork and the postoperative patient for two reasons: (1) the previous reported success of CRM training, including briefings and checklists, in reducing harm intraoperatively; and (2) the fact that preventable harm often occurs to patients in the postoperative period (Neily, Mills, Young-Xu et al., 2010). Previously, MTT had focused on operating room and intensive care unit teams. The study sample included physicians and nurses staffing postoperative care areas at these 12 facilities.

Each group of 5 to 10 participants experienced two simulation scenarios that focused on a postoperative surgical patient. Each participant was on the “hot seat,” or on the team caring for the simulated patient, for at least one scenario, but may have been an observer during the second scenario. All group members participated in the debriefings after each of the two simulation scenarios, regardless of whether they were active participants or observers.

Instruments

Self-Efficacy of Teamwork Competencies Scale. Participant confidence in CRM techniques was measured before and after simulation-based training with the Self-Efficacy of Teamwork Competencies Scale. This scale was developed by Paige et al. (2008) to assess learning after in situ simulation-based team training of operating room personnel. Paige et al. (2008) showed a statistically significant increase in scores (up to 40% from baseline) with the use of the tool in a variety of teamwork domains, such as role clarity and team interactions. The current study shared many features with the work of Paige et al. (2008). These included the use of an interdisciplinary team, surgical patient care, and in situ simulation. Also included were eight identical pretest and posttest questions on teamwork and communication. Two additional items about the learner’s simulation experience were included on the posttest only: “Patient scenarios reflected realistic situations that teams might face in real life” and “Scenarios were effective for practicing teamwork and patient safety.”

The validity of the instrument was supported by its measurement of confidence in the CRM behaviors taught in the MTT Program, including building a shared mental model of the patient’s plan of care and using specific strategies to promote clear communication.

Clinical Teamwork Scale. The CTS is a global assessment of teamwork and communication that uses a 10-point Likert-type scale. This scale was initially developed to evaluate teamwork during in situ simulation-based training in obstetrics. A companion worksheet described specific patient safety behaviors for each level of scoring. Defining behaviors more specifically changed the CTS from a subjective global assessment tool to a more objective, behaviorally anchored tool.

The patient safety behaviors that were chosen for rating from the CTS map closely to the unanimously agreed on core CRM competencies of the Delphi project. These include overall teamwork (item 1); overall communication (item 2); SBAR (item 3); transparency (item 4); directness (item 5); closed-loop communication (item 6); overall situational awareness (item 7); resource allocation and workload distribution (item 8); “stepping back” to prevent fixation (item 9); overall decision making (item 10); prioritization (item 11); leadership (item 12); followership and assertiveness (items 13 and 14); and patient-centeredness (item 15).

In preparation for the incorporation of simulation-based training into the MTT Program, all MTT faculty attended the Crisis Resource Management Advanced Instructor Workshop at the Stanford School of Medicine Center for Immersive and Simulation-Based Learning over a 3-day period (August 17, 2009, to August 20, 2009). This course is a comprehensive program in simulation training and debriefing directed by Dr. David Gaba (Gaba et al., 1998). In addition, NCPS faculty participated in a hands-on short course on the measurement of teamwork and communication competencies. This course included several well-established global, behaviorally anchored, and event-based tools, including the Anesthetists’ Non-Technical Skills (ANTS) instrument, the Observational Teamwork Assessment for Surgery (OTAS), the Non-Technical Skills (NOTECHS) scale, the Mayo High Performance Teamwork Scale, and the CTS (Flin & Patey, 2011; Hull, Arora, Kassab, Kneebone, & Sevdalis, 2011; Malec et al., 2007; Sevdalis et al., 2008).

During the course, the MTT faculty was trained in the use of the CTS in rating teamwork and communication by evaluating and scoring multiple recorded videos of poor and good clinical teamwork. Mean faculty ratings for overall teamwork and communication items were 3.43 and 4.14, respectively, in the video showing poor teamwork and 7.17 and 6.50, respectively, in the video dedicated to good teamwork, demonstrating good discriminative agreement among raters. Inter-rater reliability among the MTT faculty during the video course was 0.71, comparable to the value of 0.78 reported in the original CTS study (Guise et al., 2008). Multiple MTT faculty observed and rated teamwork and communication during the simulation scenarios that were conducted at 11 of the 12 facilities during the study period, allowing for monitoring and calculation of inter-rater reliability of 0.67. One MTT faculty member observed simulation scenarios at the 12th facility.

Table A (available in the online version of this article) shows a de-identified agenda from one of the facilities. The curriculum includes an overview of communication errors as a cause of adverse events in health care; an introduction to CRM; and practice using teamwork and communication tools and techniques, including briefings, handoffs, leadership, and graded assertiveness. The training is facilitated by a physician and a nurse. A representative simulation scenario is highlighted in Table B (available in the online version of this article). The scenario depicts a postoperative patient with sepsis. The team is challenged to use the CRM skills that they learned previously during the didactic portion of the MTT Program.

Procedures

The Self-Efficacy of Teamwork Competencies Scale was administered before the beginning of the didactic portion of the MTT Program. Participation was anonymous and confidential. Learners were instructed not to include their names on the survey forms. The same survey was readministered at the conclusion of the simulation portion of the MTT Program, with the addition of the two questions on simulation fidelity and effectiveness that were described earlier. Surveys were securely transported back to the NCPS, and data were transcribed to a spreadsheet for statistical analysis. Pretest and posttest surveys included eight identical questions on teamwork and communication.

Faculty completed CTS observations for each simulation scenario. The unit for observation and measurement was the team, not the individual. No identifying information was recorded on the CTS tool or worksheet by the faculty with the exception of whether the simulation was the first or second (last) simulation of the program for a given team. Completed CTS worksheets were returned to the NCPS, and the data were entered on a separate spreadsheet. No personal identifying data were recorded on the survey, the CTS worksheet, or the spreadsheet that was maintained at the NCPS. Neither the simulation sessions nor the debriefings were recorded.

Data Analysis

Descriptive data were collected, analyzed, and reported as mean plus or minus standard error. Pretest and posttest Self-Efficacy of Teamwork Competencies Scale survey data and first and second (last) scenario CTS observational data were compared with a two-tailed unpaired Student’s t test. Inter-rater reliability for the observational tool was determined by Ebel’s algorithm, as previously described (Solomon, 2004). A p value of less than .05 was considered significant. Missing data occurred 0.2% of the time with each instrument, and these instances were managed by elimination.

Results

Facility demographics are shown in Table 1. The size of the facility is represented by the number of authorized beds. One of the 12 facilities performed only out-patient ambulatory surgical care. Ten of the 11 inpatient facilities were classified as either complex or intermediate-level institutions. The volume of surgical procedures ranged from 1,200 to 6,800 cases per year. Between 12 and 52 providers per facility participated in simulation-based training.

Facility Demographics

Table 1:

Facility Demographics

Confidence

A total of 313 Self-Efficacy of Teamwork Competencies Scale surveys were returned from 334 participants, for a baseline response rate of 94%. Everyone who attended the training participated in the simulations and debriefings. After the didactic session, two simulation experiences, and debriefings, 325 participants completed the same survey, for a postintervention response rate of 98%.

The mean pretest and posttest Self-Efficacy of Team-work Competencies Scale scores, based on a five-point Likert scale, are shown in Table 2. Scores for each of the eight communication and teamwork items showed a significant increase from baseline. Combining all items, scores increased from 3.7 to 4.4, representing a net improvement of 18.5%. After training, team members were more confident in their ability to build shared mental models of a patient’s condition and plan of care; use structured communication tools (e.g., closed-loop communication) during patient care; and use a team approach to an unexpected crisis or challenge.

Self-Efficacy of Teamwork Competencies Scale Findings

Table 2:

Self-Efficacy of Teamwork Competencies Scale Findings

Participants found the scenarios true to life, with mean scores for the statement “Patient scenarios reflected realistic situations that teams might face in real life” of 4.6 ± 0.03 on a five-point Likert scale. The scores for agreement with the statement “Scenarios were effective for practicing teamwork and patient safety” were 4.7 ± 0.03.

Performance

Teamwork skills were observed by trained instructors and assessed with the CTS. First and second scenario scores, based on a 10-point scale, are shown in Table 3. The 245 instructor-completed CTS worksheets from an observed first simulation scenario were compared with the 233 CTS worksheets submitted for a second simulation scenario. Fourteen of the 15 observed skills showed significant improvement that ranged from 15% to 23%. Only teams’ “resource allocation” skills (item 8) did not show a significant increase. Team members participated in robust debriefings after their first simulation experience, discussing “What went well?” and “What could be improved?” These deliberations were led by MTT instructors and focused on CRM teamwork and communication skills, including examples of team successes in applying those skills as well as any missed opportunities.

Clinical Teamwork Scale Observational Tool Findings

Table 3:

Clinical Teamwork Scale Observational Tool Findings

Team members were more likely to use clear, concise communication. When confronted with a challenge, team members were more likely to assert themselves respectfully. CRM tools, such as “stepping back,” led to improvements in team situational awareness and better decision making, as captured on the CTS. With training, teams began to engage in specific teamwork and communication strategies when faced with a simulated patient whose condition was deteriorating postoperatively, improving overall team performance. The only CRM behavior that did not improve during simulation training was workload distribution (item 8).

Discussion

Health care has specific characteristics that make it vulnerable to errors. These latent conditions include high acuity and volume, interruptions, uncertainty, time constraints, multitasking, shift work, and little opportunity for feedback and learning (Pruitt & Liebelt, 2010). As recommended by the Institute of Medicine, CRM teamwork and communication techniques borrowed and adapted from the aviation industry can address many of these hazards (Kohn, Corrigan, & Donaldson, 2000). Established team training programs, such as the Department of Defense TeamSTEPPS® and the VA’s MTT Programs, are based on CRM principles (Clark, 2009; Dunn et al., 2007). A consensus of CRM knowledge, skills, and attitudes has emerged: “speaking up” with concerns; communication of critical information (e.g., SBAR); a systems approach to human error; the use of checklists; workload management; situational awareness and strategies to maintain it (briefings); “stepping back,” or pausing in light of new information; leadership; followership, including assertiveness; and cross-monitoring (Clay-Williams & Braithwaite, 2009, pp. 436–438). Providers from operating rooms, intensive care units, and emergency rooms have reported higher levels of teamwork, better patient outcomes, lower costs of care, increased reporting of “near misses” and unsafe conditions, and reduced litigation risk after team training (Neily, Mills, Young-Xu et al., 2010; Pruitt & Liebelt, 2010; Sax et al., 2009).

Confidence

The most effective educational format for delivering CRM training is unknown. Simulation is an attractive training technique for a number of reasons. As pointed out by Gaba (2004), simulation allows teams to train together and practice improving their teamwork skills while rehearsing for crises. This vision is in keeping with a systems approach to errors and is a departure from the development of individual knowledge and skills. Simulation-based continued training is fundamental to safety in the aviation industry. Best practices have been established for this type of training (Rosen et al., 2008). The current study adhered to best practices by capturing teamwork competencies, focusing on observable behaviors, measuring multiple levels of performance, employing trained observers, and using structured, guided, and nonjudgmental debriefings (Fanning & Gaba, 2007; Rosen et al., 2008).

Delivery of postoperative surgical care is a complex task, with many individuals responsible for providing care. Traditional training and education of physicians, nurses, and allied health staff has focused on individual technical skills to achieve proficiency in specific tasks. However, analysis of adverse events in surgery has shown that in many cases the underlying cause involves communication and teamwork rather than technical expertise (Rogers et al., 2006). Therefore, technical skills appear necessary but are not sufficient to ensure safety in the care of surgical patients. MTT curricula use a variety of instructional formats. Incorporating simulation-based training methods creates a realistic and safe learning environment for practicing CRM skills, managing infrequent events, and showing the consequences of team actions (Beaubien & Baker, 2004).

Participants in this study reported enhanced confidence in using strategies to maintain situational awareness, improve communication skills, and use a team approach to solve clinical problems. Significant increases (range, 13% to 26%) were noted among the domains of the Self-Efficacy of Teamwork Competency Scale.

Performance

Improved observable safety behaviors were noted within multiple domains, including teamwork, communication, situational awareness, decision making, and leadership. Improved confidence in teamwork and communication was reflected in better team performance in 14 of 15 skills observed after simulation-based training with the CTS. The study findings are consistent with previous work evaluating the potential benefit of simulation and teamwork training in health care settings and demonstrate Kirkpatrick-Miller level 2b to 3 effectiveness in knowledge and skills (Issenberg, McGaghie, Petrusa, Gordon, & Scalese, 2005; Kirkpatrick, 1998; Salas et al., 2008).

The only CRM behavior that did not improve in this study was resource allocation and workload distribution (item 8) on the CTS. Workload distribution was covered during the didactic portion of the course. However, both the didactic and the simulation portions of the program focused on “speaking up” (assertiveness) and countermeasures to prevent loss of situational awareness. In part, this emphasis was understandable, given attempts to address the large role that “silence” and losing sight of the big picture play in adverse events in health care (Gosbee, 2010; Johnson & Kimsey, 2012). The investigators may not have installed enough “triggers” in the simulation scenarios to promote demonstration of workload distribution. Future iterations of the curriculum will include more opportunities to practice workload distribution.

Limitations

Although statistically significant improvements were documented within survey and observational data on teamwork and communication, it is not possible to determine fully whether these enhancements are clinically relevant. This study did not record any clinical outcomes. However, other studies have shown that CRM teamwork and communication skills, learned and practiced, lead to improvements in clinical care (McCulloch et al., 2009).

The use of Likert scales may be associated with acquiescence bias and social desirability bias (Bertram, 2006). For example, participants may have given their confidence higher ratings after training because they perceived that this outcome was expected. Furthermore, the nonrandomized sample and the pretest–posttest design further increased the potential for selection bias and confounding (Higgins et al., 2013).

Because the study survey was anonymous and confidential and the observational tool was used to measure team performance, information was not collected on individual participants, such as the number of nurses and physicians, level of experience, and so on. Most of the participants were nurses caring for postoperative patients. However, the evaluation of possible differences in the perception of teamwork and communication training between physicians and nurses would have been valuable to obtain, given its prevalence in the literature (Carney, West, Neily, Mills, & Bagian, 2010).

This study also did not address the durability of the measured increases in knowledge and skills in teamwork and communication. Previous studies have shown the durability of simulation-based acquired clinical skills in health care (Barsuk, Cohen, McGaghie, & Wayne, 2010). These studies suggest that the enhanced CRM knowledge and skills demonstrated with simulation-based team training in the current study may be sustained with refresher training.

Conclusion

The results of this study suggest that point-of-care simulation-based training improves teamwork and communication skills among interprofessional staff caring for postoperative patients. Interestingly, the degree of improvement in observed CRM skills and self-reported confidence in CRM competencies after MTT with simulation-based training is of the same magnitude as the percentage of improvement found in the team-work and safety domains of culture surveys after MTT (Neily, Mills, Lee et al., 2010). Further studies are needed to establish the durability of these enhanced team-based skills and determine whether these observations translate into enhanced patient outcomes.

References

  • Agency for Healthcare Research and Quality. (2010). HealthGrades seventh annual patient safety in American hospitals study. Retrieved from http://psnet.ahrq.gov/resource.aspx?resourceID=17898
  • Barsuk, J. H., Cohen, E. R., McGaghie, W. C. & Wayne, D. B. (2010). Long-term retention of central venous catheter insertion skills after simulation-based mastery learning. Academic Medicine, 85(10 Suppl.), S9–S12. doi:10.1097/ACM.0b013e3181ed436c [CrossRef]
  • Beaubien, J. M. & Baker, D. P. (2004). The use of simulation for training teamwork skills in healthcare: How low can you go?Quality and Safety in Health Care, 13(Suppl. 1), i51–i56. doi:10.1136/qshc.2004.009845 [CrossRef]
  • Bertram, D. (2006). Likert scales. Retrieved from http://poincare.matf.bg.ac.rs/~kristina/topic-dane-likert.pdf
  • Brown, C., Hofer, T., Johal, A., Thomson, R., Nicholl, J., Franklin, B. D. & Lilford, R. J. (2008a). An epistemology of patient safety research: A framework for study design and interpretation. Part 1. Conceptualising and developing interventions. Quality and Safety in Health Care, 17(3), 158–162. doi:10.1136/qshc.2007.023630 [CrossRef]
  • Brown, C., Hofer, T., Johal, A., Thomson, R., Nicholl, J., Franklin, B. D. & Lilford, R. J. (2008b). An epistemology of patient safety research: A framework for study design and interpretation. Part 2. Study design. Quality and Safety in Health Care, 17(3), 163–169. doi:10.1136/qshc.2007.023648 [CrossRef]
  • Carney, B. T., West, P., Neily, J., Mills, P. D. & Bagian, J. P. (2010). Differences in nurse and surgeon perceptions of teamwork: Implications for use of a briefing checklist in the OR. AORN Journal, 91(6), 722–729. doi:10.1016/j.aorn.2009.11.066 [CrossRef]
  • Carroll, J. S. & Rudolph, J. W. (2006). Design of high reliability organizations in health care. Quality and Safety in Health Care, 15(Suppl. 1), i4–i9. doi:10.1136/qshc.2005.015867 [CrossRef]
  • Clark, P. R. (2009). Teamwork: Building healthier workplaces and providing safer patient care. Critical Care Nursing Quarterly, 32(3), 221–231. doi:10.1097/CNQ.0b013e3181ab923f [CrossRef]
  • Clay-Williams, R. & Braithwaite, J. (2009). Determination of health-care teamwork training competencies: A Delphi study. International Journal for Quality in Health Care, 21(6), 433–440. doi:10.1093/intqhc/mzp042 [CrossRef]
  • de Vries, E. N., Hollmann, M. W., Smorenburg, S. M., Gouma, D. J. & Boermeester, M. A. (2009). Development and validation of the SURgical PAtient Safety System (SURPASS) checklist. Quality and Safety in Health Care, 18(2), 121–126. doi:10.1136/qshc.2008.027524 [CrossRef]
  • de Vries, E. N., Prins, H. A., Crolla, R. M., den Outer, A. J., van Andel, G., van Helden, S. H. & Boermeester, M. A. (2010). Effect of a comprehensive surgical safety system on patient outcomes. New England Journal of Medicine, 363(20), 1928–1937. doi:10.1056/NEJMsa0911535 [CrossRef]
  • Dunn, E. J., Mills, P. D., Neily, J., Crittenden, M. D., Carmack, A. L. & Bagian, J. P. (2007). Medical team training: Applying crew resource management in the Veterans Health Administration. Joint Commission Journal on Quality and Patient Safety, 33(6), 317–325.
  • Fanning, R. M. & Gaba, D. M. (2007). The role of debriefing in simulation-based learning. Simulation in Healthcare, 2, 115–125. doi:10.1097/SIH.0b013e3180315539 [CrossRef]
  • Flin, R. & Patey, R. (2011). Non-technical skills for anaesthetists: Developing and applying ANTS. Best Practice & Research: Clinical Anaesthesiology, 25(2), 215–227. doi:10.1016/j.bpa.2011.02.005 [CrossRef]
  • Gaba, D. (2004). The future vision of simulation in health care. Quality and Safety in Health Care, 13(Suppl.), i2–i10. doi:10.1136/qshc.2004.009878 [CrossRef]
  • Gaba, D. M., Howard, S. K., Flanagan, B., Smith, B. E., Fish, K. J. & Botney, R. (1998). Assessment of clinical performance during simulated crises using both technical and behavioral ratings. Anesthesiology, 89(1), 8–18. doi:10.1097/00000542-199807000-00005 [CrossRef]
  • Gosbee, J. (2010). Handoffs and communication: The underappreciated roles of situational awareness and inattentional blindness. Clinical Obstetrics and Gynecology, 53(3), 545–558. doi:10.1097/GRF.0b013e3181ec1ac7 [CrossRef]
  • Guise, J.-M., Deering, S. H., Kanki, B. G., Osterweil, P., Li, H., Mori, M. & Lowe, N. K. (2008). Validation of a tool to measure and promote clinical teamwork. Simulation in Healthcare, 3(4), 217–223. doi:10.1097/SIH.0b013e31816fdd0a [CrossRef]
  • Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R., Breizat, A. H., Dellinger, E. P. & Gawande, A. A. (2009). A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of Medicine, 360(5), 491–499. doi:10.1056/NEJMsa0810119 [CrossRef]
  • Heard, G. (2005). Errors in medicine: A human factors perspective. Australasian Anaesthesia, 1–12.
  • Higgins, J. P. T., Ramsay, C., Reeves, B. C., Deeks, J. J., Shea, B., Valentine, J. C. & Wells, G. (2013). Issues relating to study design and risk of bias when including non-randomized studies in systematic reviews on the effects of interventions. Research Synthesis Methods, 4(1), 12–25. doi:10.1002/jrsm.1056 [CrossRef]
  • Hull, L., Arora, S., Kassab, E., Kneebone, R. & Sevdalis, N. (2011). Observational teamwork assessment for surgery: Content validation and tool refinement. Journal of the American College of Surgeons, 212(2), 234–243. doi:10.1016/j.jamcollsurg.2010.11.001 [CrossRef]
  • Issenberg, S. B., McGaghie, W. C., Petrusa, E. R., Gordon, D. L. & Scalese, R. J. (2005). Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Medical Teacher, 27(2), 10–28. doi:10.1080/01421590500046924 [CrossRef]
  • Johnson, H. L. & Kimsey, D. (2012). Patient safety: Break the silence. AORN Journal, 95(5), 591–601. doi:10.1016/j.aorn.2012.03.002 [CrossRef]
  • Kirkegaard, M. & Fish, J. (2004). Doc-U-drama: Using drama to teach about patient safety. Family Medicine, 36(9), 628–630.
  • Kirkpatrick, D. (1998). Evaluating training programs: The four levels(2nd ed.). San Francisco, CA: Berrett-Koehler.
  • Kohn, L. T., Corrigan, J. M. & Donaldson, M. S. (2000). To err is human: Building a safer health system. Washington, DC: National Academy Press.
  • Leonard, M., Graham, S. & Bonacum, D. (2004). The human factor: The critical importance of effective teamwork and communication in providing safe care. Quality and Safety in Health Care, 13(Suppl. 1), i85–i90. doi:10.1136/qshc.2004.010033 [CrossRef]
  • Malec, J. F., Torsher, L. C., Dunn, W. F., Wiegmann, D. A., Arnold, J. J., Brown, D. A. & Phatak, V. (2007). The Mayo High Performance Scale: Reliability and validity for evaluating key crew resource management skills. Simulation in Healthcare, 2(1), 4–10. doi:10.1097/SIH.0b013e31802b68ee [CrossRef]
  • McCulloch, P., Mishra, A., Handa, A., Dale, T., Hirst, G. & Catchpole, K. (2009). The effects of aviation-style non-technical skills training on technical performance and outcome in the operating theatre. Quality and Safety in Health Care, 18(2), 109–115. doi:10.1136/qshc.2008.032045 [CrossRef]
  • Neily, J., Mills, P. D., Lee, P., Carney, B., West, P., Percarpio, K. & Bagian, J. P. (2010). Medical team training and coaching in the Veterans Health Administration: Assessment and impact on the first 32 facilities in the program. Quality and Safety in Health Care, 19(4), 360–364. doi:10.1136/qshc.2008.031005 [CrossRef]
  • Neily, J., Mills, P. D., Young-Xu, Y., Carney, B. T., West, P., Berger, D. H. & Bagian, J. P. (2010). Association between implementation of a medical team training program and surgical mortality. Journal of the American Medical Association, 304(15), 1693–1700. doi:10.1001/jama.2010.1506 [CrossRef]
  • Paige, J. T., Kozmenko, V., Yang, T., Paragi Gururaja, R., Hilton, C. W., Cohn, I. Jr.. & Chauvin, S. W. (2008). High fidelity, simulation-based training at the point-of-care improves teamwork in the operating room. Journal of the American College of Surgeons, 207(3, Suppl.), S87–S88. doi:10.1016/j.jamcollsurg.2008.06.219 [CrossRef]
  • Paull, D. E., Mazzia, L. M., Izu, B. S., Neily, J., Mills, P. D. & Bagian, J. P. (2009). Predictors of successful implementation of preoperative briefings and postoperative debriefings after medical team training. American Journal of Surgery, 198(5), 675–678. doi:10.1016/j.amjsurg.2009.07.008 [CrossRef]
  • Pruitt, C. M. & Liebelt, E. L. (2010). Enhancing patient safety in the pediatric emergency department: Teams, communication, and lessons from crew resource management. Pediatric Emergency Care, 26(12), 942–948. doi:10.1097/PEC.0b013e3181fec9cf [CrossRef]
  • Rogers, S. O. Jr.. , Gawande, A. A., Kwaan, M., Puopolo, A. L., Yoon, C., Brennan, T. A. & Studdert, D. M. (2006). Analysis of surgical errors in closed malpractice claims at 4 liability insurers. Surgery, 140(1), 25–33. doi:10.1016/j.surg.2006.01.008 [CrossRef]
  • Rosen, M. A., Salas, E., Wilson, K. A., King, H. B., Salisbury, M., Augenstein, J. S. & Birnbach, D. J. (2008). Measuring team performance in simulation-based training: Adopting best practices for healthcare. Simulation in Healthcare, 3(1), 33–41. doi:10.1097/SIH.0b013e3181626276 [CrossRef]
  • Salas, E., DiazGranados, D., Klein, C., Burke, C. S., Stagl, K. C., Goodwin, G. F. & Halpin, S. M. (2008). Does team training improve team performance? A meta-analysis. Human Factors, 50(6), 903–933. doi:10.1518/001872008X375009 [CrossRef]
  • Sax, H. C., Browne, P., Mayewski, R. J., Panzer, R. J., Hittner, K. C., Burke, R. L. & Coletta, S. (2009). Can aviation-based team training elicit sustainable behavioral change?Archives of Surgery, 144(12), 1133–1137. doi:10.1001/archsurg.2009.207 [CrossRef]
  • Sevdalis, N., Davis, R., Koutantji, M., Undre, S., Darzi, A. & Vincent, C. A. (2008). Reliability of a revised NOTECHS scale for use in surgical teams. American Journal of Surgery, 196(2), 184–190. doi:10.1016/j.amjsurg.2007.08.070 [CrossRef]
  • Solomon, D. J. (2004). The rating reliability calculator. BMC Medical Research Methodology, 4(11), 1–3. doi:10.1186/1471-2288-4-11 [CrossRef]
  • U.S. Department of Veterans Affairs. (2010). Interactive U.S. map. Retrieved from www2.va.gov/directory/guide/map.asp?dnum=1
  • Veterans Administration. (2010). VA medical centers designated for surgical complexity. Retrieved from www.patientcare.va.gov/20100518a1.asp

Table A:

Medical Team Training

Table B:

Early Sepsis on Surgical Floor

Key Points

Simulation

Paull, D. E., DeLeeuw, L. D., Wolk, S., Paige, J. T., Neily, J. & Mills, P. D. (2013). The Effect of Simulation-Based Crew Resource Management Training on Measurable Teamwork and Communication Among Interprofessional Teams Caring for Postoperative Patients. The Journal of Continuing Education in Nursing, 44(11), 516–524.

  1. Postoperative patient care is complex and involves many health care professionals. Many adverse events in surgical patients result from teamwork and communication breakdown.

  2. Simulation-based training, built on the tools and techniques of aviation-based crew resource management (CRM), addresses vulnerabilities in teamwork and communication.

  3. A standardized simulation-based training curriculum delivered in an interprofessional framework to nurses and physicians caring for postoperative patients was associated with improved confidence and demonstrated skills in applying CRM tools and techniques.

10.3928/00220124-20130903-38

Sign up to receive

Journal E-contents