The recent estimate of 400,000 patient deaths per year from medical errors (James, 2013) is significantly increased from that reported in the landmark study To Err Is Human (Institute of Medicine [IOM], 1999) of 98,000 deaths from medical errors per year. The dollar cost of these errors is staggering, but the real tragedy is the impact on patients, their families, and health care professionals whose desire is to help and serve others. Most errors resulting in patient harm are the result of communication breakdowns (Cavanaugh & Konrad, 2012; The Joint Commission, 2014; Sutcliffe, Lewton, & Rosenthal, 2004). To decrease these kinds of errors, health care professionals at all levels must be educated using strategies that improve the structure and quality of information exchange through innovative interdisciplinary team training programs (Bhutta et al., 2010; IOM, 2003, 2010; Mochan & Nash, 2015).
In 2006, the Agency for Healthcare Research and Quality (AHRQ), in collaboration with the Department of Defense, released TeamSTEPPS® (Strategies and Tools to Enhance Performance and Patient Safety) as the national standard for team training in health care (AHRQ, 2014). The program has demonstrated effectiveness in improving team behaviors and patient safety outcomes (Baker & Durham, 2013; Capella et al., 2010; Robertson et al., 2010; Thomas & Galla, 2013; Weaver et al., 2010). Further, the TeamSTEPPS curriculum is an adaptable educational tool that combines didactic coverage of the principles of teamwork with simulation exercises (King et al., 2008; Salas, DiazGranados, Weaver, & King, 2008).
Virtual Learning Environments
On-site simulations with interprofessional groups of health providers are difficult to conduct in educational settings due to varying learner schedules (Fewster-Thuente, 2014; Robertson, et al., 2010; Solomon et al., 2010). However, use of three-dimensional Virtual Learning Environments (VLEs) allow for flexibility of scheduling and location. VLEs have been used for developing communication skills for medical and nursing students (Fors, Muntean, Botezatu, & Zary, 2009; Sweigart & Hodson Carlton, 2013; Sweigart, Burden, Hodson-Carlton, & Fillwalk, 2014; Sweigart, Hodson Carlton, Campbell, & Lutz, 2010) and other interactive health care professions training programs (Andrade, 2010; Dev, Youngblood, Heinrichs, & Kusumoto, 2007; Heinrichs, Youngblood, Harter, & Dev, 2008; King et al., 2012; Lim, Moriarty, & Huthwaite, 2011; Wiecha, Heyden, Sternthal, & Merialdi, 2010). VLEs are particularly applicable to distance learning, and they can facilitate national and international collaborations among health professionals (Umoren et al., 2014). As a learning tool, VLEs have been effective in the development of leadership and conflict resolution skills, thus improving collaboration and information (Djukic, Fulmer, Adams, Lee, & Triola, 2012).
Experiential Learning Theory
Stimulating emotional engagement among participants requires a holistic approach to designing team-based, simulated patient care. Kolb's model of experiential learning theorizes that learning involves four phases: (a) concrete experience, where the learner is doing or having an experience; (b) observation and reflection, where the learner reviews and reflects on the experience; (c) abstract conceptualization, where the learner forms abstract concepts; and (d) active experimentation, where the learner plans or tries out what has been learned (Kolb, 1984; Kolb, Boyatzis, & Mainemelis, 2001). Much of Kolb's theory is concerned with the learner's internal cognitive processes. Kolb proposed that learning encompasses the acquisition of abstract concepts that can be applied flexibly in a range of situations. Thus, the stimulus for the development of new perceptions is provided by new experiences. In VLE, the environment provides the context and intrinsic feedback for the experience, and the scenarios are the events that challenge, teach, and test the learners to facilitate a more holistic learning cycle (Alston & Schatz, 2013).
To date, the use of VLE in individual and group activities for training health professional students located at remote campuses has not been well studied. The current cross-sectional feasibility study had two objectives:
- To test the utility and acceptability of VLE immersive training using the TeamSTEPPS curriculum.
- To examine the change in teamwork attitudes regarding interprofessional communication.
A pretest–posttest design was used to measure change in teamwork attitudes in a group of interprofessional learners who participated in virtual scenarios. The TeamSTEPPS Teamwork Attitudes Questionnaire (T-TAQ) assessed the change in attitudes toward teamwork before and after working through the scenarios. The institutional review boards of both Indiana State University and Ball State University certified the study as exempt.
A total of 109 professional students from the disciplines of nursing, medicine, occupational therapy, and social work were recruited via e-mail, announcements, and personal invitations from the research team on four campuses of the two universities. Access was provided to the virtual platform and the T-TAQ in a computer laboratory on each campus. All participants logged into their sessions anonymously with a randomly assigned number used for the pretest–posttest comparisons. These numbers were stratified to allow for assignment to the respective four professions.
Three scenarios were developed from TeamSTEPPS case studies by the project team leader (R.A.U.), with review and modifications by an interprofessional team with experience as health professionals and in both health professional education and the TeamSTEPPS program. These simulated patient care situations used scripted nonplayer characters in the roles of professionals, including nursing, medicine, and radiology. Students participated in these 5-minute scenarios via their own avatar. Each team member demonstrated the use of teamwork communication tools for student participants who were immersed in their role as a student observing and learning from the virtual team. The students, prompted by virtual team professionals, were required to identify TeamSTEPPS strategies (SBAR [Situation, Background, Assessment, Recommendation] questions, check-back, call out, two-challenge rule, CUS [Concerned, Uncomfortable, Safety issue], handoff) and conflict resolution (DESC script [Describe, Express, Suggest, Consequences]) that (a) were used by the virtual team member or (b) should be used as a next step to prevent adverse outcomes.
Unity 3D, which is a three-dimensional game development platform, was used due to its enhanced graphics, ease of use for learners and instructors, successful use in distance education settings, and adaptability to a variety of devices. A virtual simulation of a hospital environment, along with other educational areas, was developed by digital intermedia designers at Ball State University for use in various nursing simulations and experiences. (Tour the environment at http://idialab.org).
The TeamSTEPPS T-TAQ was used to measure the effectiveness of the TeamSTEPPS curriculum presented in a VLE in changing attitudes toward interprofessional teamwork. The T-TAQ is one of the validated tools created as part of the AHRQ TeamSTEPPS curriculum (AHRQ, 2008; Baker, Amodeo, Krokos, Slonim, & Herrera, 2010). Complete information on the construction, testing, and validation of the tool is available in the Agency for Healthcare Research and Quality TeamSTEPPS Teamwork Attitudes Questionnaire Manual (2008). The questionnaire includes six Likert-type items for each of the five constructs: Team Structure, Leadership, Situation Monitoring, Mutual Support, and Communication. For the current study's sample, the Cronbach's alpha coefficients for the scales were found to be .71, .82, .89, .75, and .57, respectively. These scale coefficients were similar to those reported by the Manual, except for the Communication construct, which had been given as .74 (AHRQ, 2008).
Demographic information enabled the authors to examine differences between professional groups, and an activity evaluation on a Likert-type scale included questions such as “I felt immersed (as if I was actually there)” and “I would recommend this activity to a colleague,” to assess the learner's subjective experience and acceptability. Utility was reflected by the time required to complete the scenarios and participants' responses to questions embedded within the scenarios.
Preparatory material was provided within and outside the VLE on teamwork and TeamSTEPPS. Participants were introduced to the relevance of teamwork and communication skills to patient safety using AHRQ TeamSTEPPS materials (AHRQ, 2014) and were oriented to the VLE by instructors and technical staff. Study participants then took a baseline survey using the TeamSTEPPS T-TAQ and completed three scenarios. When participants incorrectly identified communication strategies, they were given immediate feedback and directed to additional TeamSTEPPS resources, then they were redirected back through the scenario to review and select the appropriate strategy or strategies. Identification of appropriate strategies was met with positive feedback and advancement of the scenario. After the scenarios, participants completed the post-T-TAQ and provided feedback.
In addition to descriptive statistics, the five constructs of the T-TAQ were compared using paired-samples t tests to assess pretest-to-posttest changes. A p value of < .05 was considered statistically significant. Given the sample size of 109, this was sufficient to detect differences with effect sizes of d = .27 or larger, with 80% power (Cohen, 1988).
A total of 109 students completed the three scenarios. Complete data were available for 92 of these students; four nursing and 13 occupational therapy students had incomplete data. Of the 92 matched pretest–posttest measures, nearly half (n = 45, 48.9%) were completed by nursing students; occupational therapy students were the next largest group (n = 27, 29.3%). The remainder of the completed survey measures derived from medical students (n = 13, 14.1%) and social work students (n = 7, 7.6%). The sample was predominantly female (80.4%) and Caucasian (91.3%). The nursing students were college seniors, whereas the students from the other disciplines were in graduate programs. Although some students were older, most students were in the 18 to 24 years (72.8%) or 25 to 30 years (17.4%) age range categories.
Using the built-in instructions, participants were easily able to master the basic skills needed to use the VLE. The average length of time required to complete three scenarios ranged from 15 to 18 minutes. Student feedback on the ease of use and perceived effectiveness for teaching communication and professionalism to a diverse group of health professional students was highly positive. However, students' comments did reflect a desire for more control over the rate of scenario progression, more animated facial expressions on the professional avatars, and the inclusion of more professions in each scenario. These items are certainly possible, given sufficient resources for development.
Participants' scores on the T-TAQ revealed significant positive changes in four of the five categories of teamwork attitudes. The categories of Leadership, Situation Monitoring, Mutual Support, and Communication were significant, with p values of .001 to .002. Although the category of Team Structure did not reach a statistically significant level as a whole, three of the six individual attitude items did show differences (p < .05). Fourteen of the 30 individual items on the attitudes questionnaire showed changes (p < .05), and four of the six indicator attitudes in the communication section were significant at the p < .001 level.
Although most research on the TeamSTEPPS program has focused on practicing health professionals, Baker and Durham (2013) reported that nursing, medical, and pharmacy students perceived an improvement in their competency in collaboration from an in-person interprofessional course based on Team-STEPPS materials. The use of nonplayer characters as virtual team members for interprofessional training in health professional students located at remote campuses using individual and group activities has not been well studied, thus no published results are available for comparison.
Conclusion and Future Directions
VLE activities provide opportunities for interprofessional learning that would not otherwise exist. The initial experience of using the conceptual framework of TeamSTEPPS in a VLE for interprofessional team training in health professional students was positive. The study subjectively and objectively evaluated students' recognition and application of communication skills as they observed and interacted with health care team members. A comparison of pretest and posttest responses on the T-TAQ showed statistically significant differences in all domains except for Team Structure. This is may be related to the limited number of disciplines represented and the focus of the scenarios on communication strategies. These preliminary findings support that reflective observation, in keeping with Kolb's (1984) theory, was occurring with the experience. Further, the researchers postulate that there is potential to move into the levels of abstract conceptualization and active experimentation as students select communication strategies used by various health professionals in the VLE scenarios that would potentially prevent harm and improve patient outcomes.
Of importance, these experiences were valuable for disciplines not directly represented in the scenario, which clearly establishes the opportunity for inclusion of the variety of health professionals who come into contact with a patient. As additional students participate, the data will be stratified according to profession for comparison. Expanded scenarios will feature more health professionals, with a broader focus on team structure, and include other health care settings. With this expansion, the focus is on incorporating more of the physical, psychosocial, and post-acute care needs of patients as part of the health care team situations. These scenarios will also include a debriefing segment. During the debriefing session, the health care professionals will discuss the use of TeamSTEPPS strategies and omissions or weaknesses in strategy selection while student participants observe and provide input as solicited by the professionals.
The authors' long-term goal is to enhance teamwork training for patient safety through greater educational efficiencies and effectiveness. Additional research is needed to compare the immersive experiences with other teaching strategies and to measure the application of communication strategies in immersive scripted and nonscripted interprofessional case studies. The preliminary data presented in this article are supportive that TeamSTEPPS virtual teams offer a methodology that is suitable for impacting teamwork attitudes in learners across professions.
- Agency for Healthcare Research and Quality. (2014). TeamSTEPPS national implementation. Retrieved from http://www.ahrq.gov/professionals/education/curriculum-tools/teamstepps/about-teamstepps/index.html
- Agency for Healthcare Research and Quality, Center for Quality Improvement and Patient Safety. (2008). TeamSTEPPS Teamwork Attitudes Questionnaire manual. Washington, DC: American Institutes for Research. Retrieved from http://www.ahrq.gov/professionals/education/curriculum-tools/teamstepps/instructor/reference/teamattitude.html
- Alston, A.L. & Schatz, S. ( 2013, December. ). Beyond scenarios: Designing holistic experiences for simulation-based training. Proceedings from the Interservice/Industry Training, Simulation & Education Conference. , Orlando, FL. .
- Andrade, A. ( 2010, October. ). Tomorrow's technologies have arrived: Avatars. Proceedings from the Kentucky Nursing Convention. , Louisville, KY. .
- Andrade, A.D., Bagri, A., Zaw, K., Roos, B.A. & Ruiz, J.G. (2010). Avatar-mediated training in the delivery of bad news in a virtual world. Journal of Palliative Medicine, 13, 1415–1419. doi:10.1089/jpm.2010.0108 [CrossRef]
- Baker, D.P., Amodeo, A.M., Krokos, K.J., Slonim, A. & Herrera, H. (2010). Assessing teamwork attitudes in healthcare: Development of the TeamSTEPPS Teamwork Attitudes Questionnaire. Quality and Safety in Health Care, 19(6), e49.
- Baker, M.J. & Durham, C.F. (2013). Interprofessional education: A survey of students' collaborative competency outcomes. Journal of Nursing Education, 52, 713–718 http://dx.doi.org/10.3928/01484834-20131118-04 doi:10.3928/01484834-20131118-04 [CrossRef]
- Bhutta, Z.A., Chen, L., Cohen, J., Crisp, N., Evans, T., Fineberg, H. & Zurayk, H. (2010). Education of health professionals for the 21st century: A global independent commission. The Lancet, 375, 1137–1138. doi:10.1016/S0140-6736(10)60450-3 [CrossRef]
- Capella, J., Smith, S., Philp, A., Putnam, T., Gilbert, C., Fry, W. & Baker, D. (2010). Teamwork training improves the clinical care of trauma patients. Journal of Surgical Education, 67, 439–443. doi:10.1016/j.jsurg.2010.06.006 [CrossRef]
- Cavanaugh, J.T. & Konrad, S. (2012). Fostering the development of effective person-centered healthcare communication skills: An interprofessional shared learning model. Work, 41, 293–301.
- Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
- Dev, P., Youngblood, P., Heinrichs, W.L. & Kusumoto, L. (2007). Virtual worlds and team training. Anesthesiology Clinics, 25, 321–336 http://dx.doi.org/10.1016/j.anclin.2007.03.001 doi:10.1016/j.anclin.2007.03.001 [CrossRef]
- Djukic, M., Fulmer, T., Adams, J.G., Lee, S. & Triola, M.M. (2012). NYU3T: Teaching, technology, teamwork: A model for interprofessional education scalability and sustainability. Nursing Clinics of North America, 47, 333–346. doi:10.1016/j.cnur.2012.05.003 [CrossRef]
- Fewster-Thuente, L. (2014). A contemporary method to teach collaboration to students. Journal of Nursing Education, 53, 641–645. doi:10.3928/01484834-20141027-02 [CrossRef]
- Fors, U.G., Muntean, V., Botezatu, M. & Zary, N. (2009). Cross-cultural use and development of virtual patients. Medical Teacher, 31, 732–738. doi:10.1080/01421590903124724 [CrossRef]
- Heinrichs, W.L., Youngblood, P., Harter, P.M. & Dev, P. (2008). Simulation for team training and assessment: Case studies of online training with virtual worlds. World Journal of Surgery, 32, 161–170. doi:10.1007/s00268-007-9354-2 [CrossRef]
- Institute of Medicine. (1999). To err is human: Building a safer health system. Washington, DC: National Academies Press.
- Institute of Medicine. (2003). Health professions education: A bridge to quality. Washington, DC: National Academies Press.
- Institute of Medicine. (2010). The healthcare imperative: Lowering cost and improving outcomes. Washington, DC: National Academies Press.
- James, J.T. (2013). A new, evidence-based estimate of patient harms associated with hospital care. Journal of Patient Safety, 9, 122–128. doi:10.1097/PTS.0b013e3182948a69 [CrossRef]
- The Joint Commission. (2014). Root causes by event type. Retrieved from http://www.jointcommission.org/assets/1/18/Root_Causes_by_Event_Type_2004-2Q_2014.pdf
- King, H.B., Battles, J., Baker, D.P., Alonso, A., Salas, E., Webster, J. & Salisbury, M. (2008). TeamSTEPPS™: Team strategies and tools to enhance performance and patient safety. In Henriksen, K. (Eds.), Advances in patient safety: New directions and alternative approaches (Vol. 3: Performance and tools). Washington, DC: Agency for Healthcare Research and Quality.
- King, S., Chodos, D., Stroulia, E., Carbonaro, M., MacKenzie, M., Reid, A. & Greidanus, E. (2012). Developing interprofessional health competencies in a virtual world. Medical Education Online, 17, 1–11. doi:10.3402/meo.v17i0.11213 [CrossRef]
- Kolb, D.A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall.
- Kolb, D.A., Boyatzis, R.E. & Mainemelis, C. (2001). Experiential learning theory: Previous research and new directions. In Sternberg, R.J. & Zhang, L.-F. (Eds.), Perspectives on thinking, learning, and cognitive styles (pp. 227–247). Mahwah, NJ: Lawrence Erlbaum.
- Lim, B.T., Moriarty, H. & Huthwaite, M. (2011). “Being-in-role”: A teaching innovation to enhance empathic communication skills in medical students. Medical Teacher, 33, e663–e669. doi:10.3109/0142159X.2011.611193 [CrossRef]
- Mochan, E. & Nash, D.B. (2015). Weaving quality improvement and patient safety skills into all levels of medical training: An annotated bibliography. American Journal of Medical Quality, 30, 232–247. doi:10.1177/1062860614528568 [CrossRef]
- Robertson, B., Kaplan, B., Atallah, H., Higgins, M., Lewitt, M.J. & Ander, D.S. (2010). The use of simulation and a modified TeamSTEPPS curriculum for medical and nursing student team training. Simulation in Healthcare, 5, 332–337. doi:10.1097/SIH.0b013e3181f008ad [CrossRef]
- Salas, E., DiazGranados, D., Weaver, S.J. & King, H. (2008). Does team training work? Principles for health care. Academic Emergency Medicine, 15, 1002–1009. doi:10.1111/j.1553-2712.2008.00254.x [CrossRef]
- Solomon, P., Baptiste, S., Hall, P., Luke, R., Orchard, C., Rukholm, E. & Damiani-Taraba, G. (2010). Students' perceptions of interprofessional learning through facilitated online learning modules. Medical Teacher, 32, e384–e391. doi:10.3109/0142159X.2010.495760 [CrossRef]
- Sutcliffe, K.M., Lewton, E. & Rosenthal, M.M. (2004). Communication failures: An insidious contributor to medical mishaps. Academic Medicine, 79, 186–194. doi:10.1097/00001888-200402000-00019 [CrossRef]
- Sweigart, L. & Hodson Carlton, K. (2013). Improving student interview skills: The virtual avatar as patient. Nurse Educator, 38, 11–15. doi:10.1097/NNE.0b013e318276df2d [CrossRef]
- Sweigart, L., Hodson Carlton, K., Campbell, B.R. & Lutz, D.R. (2010). Second Life environment: A venue for interview skill development. CIN: Computers, Informatics, Nursing, 28, 258–263.
- Sweigart, L.I., Burden, M.L., Hodson-Carlton, K.E. & Fillwalk, J. (2014). Virtual world simulations across curriculum prepare students for patient interviews. Clinical Simulation in Nursing, 10, e139–e145. doi:10.1016/j.ecns.2013.10.003 [CrossRef]
- Thomas, L. & Galla, C. (2013). Republished: Building a culture of safety through team training and engagement. Postgraduate Medical Journal, 89 (1053), 394–401. doi:10.1136/postgradmedj-2012-001011rep [CrossRef]
- Umoren, R.A., Stadler, D.J., Gasior, S.L., Al-Sheikhly, D., Truman, B. & Lowe, C. (2014). Global collaboration and team-building through 3D virtual environments. Innovations in Global Medical and Health Education, 2014(1), 1. doi:10.5339/igmhe.2014.1 [CrossRef]
- Weaver, S.J., Rosen, M.A., DiazGranados, D., Lazzara, E.H., Lyons, R., Salas, E. & Barker, M. (2010). Does teamwork improve performance in the operating room? A multilevel evaluation. Joint Commission Journal on Quality and Patient Safety, 36, 133–142.
- Wiecha, J., Heyden, R., Sternthal, E. & Merialdi, M. (2010). Learning in a virtual world: Experience with using Second Life for medical education. Journal of Medical Internet Research, 12(1), e1. doi:10.2196/jmir.1337 [CrossRef]