Journal of Nursing Education

Major Article 

Realism and Presence in Simulation: Nursing Student Perceptions and Learning Outcomes

Sharon MacLean, RM, RN; Fiona Geddes, PhD; Michelle Kelly, PhD, RN; Phillip Della, PhD, RN

Abstract

Background:

Research examining how perceived realism and presence affects participants' learning experiences and outcomes is limited.

Method:

A convergent mixed-methods design was used, with quantitative data assigned as the primary method. After engaging in a communication training simulation, 141 undergraduate nursing students completed the Concept of Presence, Simulation Design, and the Quality of Discharge Teaching scales. A subsample of 12 participants were interviewed to provide qualitative data, as the secondary method, on their learning experience. Hierarchical multiple regression analysis was performed on the quantitative data and thematic analysis for qualitative data.

Results:

Differences in participants' perceived realism and level of presence were not affected by the communication-based learning interventions. A positive, fully mediated relationship between realism, presence, and learning outcomes in discharge communication skills was found. The quality of the simulation experience gave participants the opportunity to reflect on their knowledge and capacity to transfer skills into clinical practice.

Conclusion:

The convergence of findings supports the theory that perceived realism and presence positively affected learning outcomes. [J Nurs Educ. 2019;58(6):330–338.]

Abstract

Background:

Research examining how perceived realism and presence affects participants' learning experiences and outcomes is limited.

Method:

A convergent mixed-methods design was used, with quantitative data assigned as the primary method. After engaging in a communication training simulation, 141 undergraduate nursing students completed the Concept of Presence, Simulation Design, and the Quality of Discharge Teaching scales. A subsample of 12 participants were interviewed to provide qualitative data, as the secondary method, on their learning experience. Hierarchical multiple regression analysis was performed on the quantitative data and thematic analysis for qualitative data.

Results:

Differences in participants' perceived realism and level of presence were not affected by the communication-based learning interventions. A positive, fully mediated relationship between realism, presence, and learning outcomes in discharge communication skills was found. The quality of the simulation experience gave participants the opportunity to reflect on their knowledge and capacity to transfer skills into clinical practice.

Conclusion:

The convergence of findings supports the theory that perceived realism and presence positively affected learning outcomes. [J Nurs Educ. 2019;58(6):330–338.]

Simulation has become popular in contemporary education because of its ability to provide a “real clinical experience.” The use of simulated patients (SPs) is increasing because it offers a realistic and affordable model of teaching (Nestel, Morrison, & Pritchard, 2014). Although several studies have refuted the idea that high levels of realism are required for effective training in simulation (Grossman & Salas, 2011; Stevens & Kincaid, 2015), Dunnington (2015) argued that students should perceive the simulation as realistic to induce a sense of presence. The question of what constitutes an adequate or optimal level of realism and the nature of the relationship between simulation fidelity and students' subjective perceptions of realism continues to be explored. The aim of this study is to investigate whether students' subjective perceptions of realism influence their level of presence in a simulation experience and correspondingly have any direct or indirect influence on their learning outcomes.

Research Questions

Research questions included the following:

  • Does modifying simulation scenario design characteristics significantly increase participants' perceptions of realism and presence?
  • Do participants' individual perceptions of realism and their presence during nursing communication simulations influence their learning outcomes?
  • In what ways do the quantitative and qualitative findings converge?

Realism

The term realism is defined as “the ability to impart the suspension of disbelief to the learner by creating an environment that mimics that of the learners work environment. Realism includes the environment, simulated patients, and activities of the educators, assessors, and/or facilitators” (Lopreiato, 2016, p. 30). The International Association for Clinical Simulation in Learning (INACSL Standards Committee, 2016) outline 11 criteria for effective implementation of simulation in education including fidelity/realism (criterion five). To promote realism, the guidelines identify three aspects of fidelity—physical, conceptual, and psychological—that should inform the simulation design (INACSL Standards Committee, 2016). All the contextual elements are intended to help participants suspend their disbelief, become immersed in the learning (Brackney & Priode, 2017), and, as Dunnington (2015) described, be present within the simulation. The level of fidelity is often linked to realism, simulation design, and learning outcomes (Choi et al., 2017). A recent meta-analysis (Shin, Park, & Kim, 2015) reported on the impact of fidelity and learning outcomes. The results indicated that a high amount of realism and fidelity had benefits for cognitive, affective, and psychomotor skills. By providing a more realistic experience, simulation provides participants with an education modality that could improve learning outcomes, compared with traditional education (Shin et al., 2015).

Presence

The concept of presence is not new in the field of nursing. Presence has long been reported as an important notion in the context of nurse–patient care and is derived from the Latin term praesetia, meaning “To be present—as with others” (Welch & Wellard, 2005, p. 5). In nursing, presence is defined as a sense of being with a patient or family member for the benefit of a therapeutic relationship (Covington, 2003; Welch & Wellard 2005). Although presence in the nursing context suggests the process of being with a patient, presence in the virtual simulation world implies the sense of “being there” (Minsky, 1980, p. 45).

In the virtual reality (VR) domain, presence is defined as the subjective interaction of a person with the virtual environment or, as Witmer and Singer (1998) described, how well a participant is involved or immersed in virtual simulation. As a multidimensional construct, presence has been studied extensively in VR contexts (Lee, 2004; Sanchez-Vives & Slater, 2005) where it has been shown to have a positive impact on task performance, learning, and the transfer of skills (De Leo, Diggs, Radici, & Mastaglio, 2014; Witmer & Singer, 1998). More recently, presence has been evaluated in nursing simulation studies using VR modalities (Dang, Palicte, Valdez, & O'Leary-Kelley, 2018; Dubovi, Levy, & Dagan, 2017). Dubovi et al. (2017) found a positive correlation between students' sense of presence during VR skills training and their conceptual–procedural learning of medication administration. Students found that their sense of presence was greater in the virtual simulation with application to medication administration compared to studying via lecture-based curriculum. Dubovi et al. (2017) recommend further research be performed to compare learning outcomes achieved when using traditional lecture, VR, and high-fidelity patient simulations. Dang et al. (2018) found that the sense of presence experienced by nursing students engaged in VR simulations closely mirrored that of students actively involved in the simulation scenario. Dang et al. (2018) recommended that future simulation studies should modify terminology and items used in VR presence scales to accommodate non-VR modalities and avoid potential variability in simulation research data. Both of the studies indicated that simulation-based education could benefit from exploring the influence of presence on learning outcomes.

Theoretical Underpinnings

A review of the literature identified three key frameworks (Centricity of Presence, National League for Nursing/Jeffries simulation framework [NLN/JSF], and Kirkpatrick's Model of Evaluation) for use in this research. The principal author (S.M.) integrated core elements of these frameworks to produce an integrated model of presence in nursing simulation (Figure 1). The model underpins the simulation design, methodology, model testing, and analysis of this research.

Integrated model of presence in nursing simulation.

Figure 1.

Integrated model of presence in nursing simulation.

Centricity of Presence

In relation to nursing education, Dunnington (2015) defined presence as a state of being between the primary domains of endocentricity and exocentricity. These domains refer to the centricity or focus of a participant's state of being and involves the participant having sensory, perceptual, and actional dominance from the simulation either internally (endocentricity) or externally (exocentricity) to the situation. More specifically, an endocentric presence is the dominant perception of the person interacting in a real patient care situation represented by the simulation. Dunnington (2015) proposed that a goal of simulation is to induce a sense of endocentric presence where the student believes he or she is in the actual clinical environment. To achieve this, simulation educators have the challenge of creating learning environments that generate a sense of presence to promote real-life learning opportunities.

The linkage between simulation design and presence has been reported in Dunnington's (2015) research comparing different clinical problems in high-fidelity mannikin-based scenarios. Dunnington found no difference in presence based on the type of clinical problem represented in the simulation. However, a change in patient cues indicating deterioration in the scenario invoked a more endocentric presence in participants. Dunnington noted that although participants often started off standing on the periphery of the simulation in a more disconnected and observational mode, as the scenario unfolded to include more stimuli the participants were pulled into the clinical situation—into a state of being endocentrically present in the simulation. This finding supports the idea that the stimuli experienced during simulation, which is determined by the scenario design, is critical to evoking presence.

The NLN-JSF

The NLN-JSF synthesizes learner-centered, sociocultural, and constructivist learning theories in developing five conceptual components of simulation: design characteristics, educational practices, teacher characteristics, participant characteristics, and learning outcomes. The NLN-JSF is relevant to this study because it explores the relationship between simulation design characteristics and learning outcomes into which presence can be embedded (Jeffries, 2016).

Kirkpatrick's Model of Evaluation

When seeking to measure participants' learning outcomes and the transfer of skills into practice, Kirkpatrick's (1994) model of evaluation is recognized as a core framework. The four levels of evaluation that are integrated into the model proposed in this study are reaction, learning, behavior, and results.

The objective of this study was to improve the communication skills of nurses in a simulation of discharging a patient from a hospital setting. To date, no research has investigated the association between students' perceptions of realism, their experience of presence, and the subsequent impact of these two factors on their learning outcomes when using simulated patients (SPs) in simulation.

Method

Study Design

A convergent mixed-methods design was used to allow both model testing with quantitative data and more in-depth examination of participants' perceptions of the concepts in question using qualitative data. This convergent design is where quantitative and qualitative data are collected and analyzed at the same time (quantitative + qualitative = convergence) (Morse & Niehaus, 2009). The purpose of convergent design is to more accurately define relationships among variables of interest. In convergent designs, both qualitative and quantitative data are collected during the same stage, although one form of data is often given more weight over the other (Creswell & Plano Clark, 2018). This research design triangulated and validated quantitative-dominant findings with significant qualitative data.

Three groups were established based on the patient discharge communication intervention: T1 = control group (no intervention); T2 = information group (readmission risk factor information provided); and T3 = information and interaction group (readmission risk factor information and teach-back provided). Full details of the study groups have been reported elsewhere (MacLean, Kelly, Geddes, & Della, 2018). The simulation scenarios took place in a purpose-built simulation laboratory at the university during nonteaching periods.

Sample (Participants and SPs)

A convenience sample of 141 participants in a Bachelor of Science Nursing voluntarily enrolled in the study. Participants met the following criteria: (a) second- or third-year nursing students who had completed discharge education in course curriculum, (b) attended a clinical placement in an acute care setting and agreed to the study procedure and purpose. A total of 137 female (97%) and four male participants, with a mean age of 27.67 years (SD = 8.54) completed the simulation. A purposive sample of 12 participants completed the semistructured interviews at the completion of the scenario (four participants from each group). Eight SPs (six women and two men) were recruited through the university's SP database. Consistent with the depiction of a patient older than 65 years in the scenario, the mean age of the SPs was 65.67 years (SD = 2.50).

Ethics

The study was approved by the university's human research ethics committee as part of the first author's doctoral research. Participation did not affect students' course outcomes. The participants completed the simulation during semester breaks and were not engaged in curricular activities with the researcher. Participation in the study was voluntary, and students were free to withdraw without prejudice at any stage. All videos and outcome data were electronically encoded and stored according to university requirements. Author permission was obtained for the use and modification of all surveys.

Data Collection and Management

Quantitative. Participants completed the Simulation Design Scale (SDS student version; Jeffries, 2016). The SDS is a 20-item instrument using a 5-point agreement scale, designed to evaluate the five design features of simulation: objectives/information, support, problem solving, feedback, and fidelity/realism. Full-scale reliability of the instrument was found to be .92 (Jeffries, 2016). To assess the participants' perceptions of realism, two questions from the fidelity subscale were used in combination with one question from Dinh's Concept of Presence (COP) tool (Dinh, Walker, Song, Kobayashi, & Hodges, 1999).

To assess the participants' sense of presence, a modified version of the COP tool was used. The original questionnaire, used in virtual simulation, consists of 14 items with a 5-point response scale. For this study, nine items were modified for use in an SP context. The COP tool had not been previously validated in the simulation with SPs, and no Cronbach's alpha has been reported in the virtual context.

The Quality Discharge Teaching Scale (QDTS; Maloney & Weiss, 2008) was used to measure participants' ability to deliver discharge-related information. The full QDTS comprises 24 items and applies a 10-point Likert format with anchors from none to a great deal. One subscale of six items relating to patients' needs was deemed not relevant to the SP scenario, and four timing-based items (item 6, 13, 14, and 15) were removed. The modified content subscale (QDTSC) consisted of six items, representing the type of information the patient receives. The eight-item delivery subscale (QDTS-D) measures nonclinical skills of the nurse, as a patient educator, in presenting discharge information. The delivery subscale includes items about listening, expressing concerns, expressing sensitivity to personal beliefs and values, and teaching in a way that the patient can understand. Further items in this subscale focus on providing consistent information, promoting patient confidence in their ability to care for themselves and knowing what to do in an emergency, and decreasing anxiety about going home. Maloney and Weiss (2008) reported Cronbach's alpha coefficients for the original full scale (.92), content subscale (.85), and delivery subscale (.89). An exploratory factor analysis was previously conducted to investigate the underlying structure of the scales in this study population; results, including internal reliabilities, have been reported in MacLean, Geddes, Kelly, and Della (2018).

Qualitative. Qualitative data were collected to add a deeper understanding of participants' perceptions of realism and presence. To address the qualitative research questions data were obtained using predetermined, open-ended questions based on the study's objectives and Dunnington's (2015) previous work in this field. Sample questions for the interviews included:

  • Did you think the scenario was a real to life?
  • Please describe your focus during the simulation. What were you aware of during the simulation? (Please describe.)
  • Were there times that you were totally engaged and did not notice anything else in the room? What you were engaged with? (Please describe.)
  • How did the simulation situation compare to a real patient care situation? What do you think you achieved by doing the simulation?
  • How present did you feel in the scenario? What aspects of the scenario made you fully engaged in the simulation?

Procedure

The study involved several phases: simulation design, SP training, creating participant groups, providing prebriefing, simulation sessions, video reflection/self-evaluation, group debriefing, and individual interviews.

Simulation Design. The National Health Education and Training in Simulation (NHET-Sim) model of simulation (Nestel et al., 2014) provided the framework for the simulation scenario design. One scenario of an individual with diabetes who required a new medication on discharge from hospital was developed. The scenario was validated by an educator with diabetes, two simulation experts, and three RNs with acute care experience.

SP Training. The next phase consisted of preparing SPs for the patient role using an established evidenced-based framework (NHET-Sim; Nestel et al., 2014). Full details of the SP training and reliability evaluation have been reported elsewhere (MacLean, Geddes, et al., 2018).

Participant Groups and Prebriefings. Six participants were randomly allocated to each of the 24 scheduled simulation sessions held over a 12-month period. All participants had previously undergone course-approved education in communication and transitions of care (discharge) skills in the first year of the nursing course. A group briefing was conducted prior to participants individually completing the simulation task. Details of procedures for standardized briefing and groups (control, information, and interaction) can be found in the study by Maclean, Kelly, Geddes, and Della (2018).

Simulation Sessions and Video Reflection. Simulations were run concurrently in two fully equipped simulation laboratories in the university's school of nursing. With permission, audiovisual recordings were taken of each participant using laboratory cameras and a GoPro camera. At the conclusion of each simulation, participants were invited individually to watch and self-reflect on their own audiovisual recording in a private space. Participants completed the study questionnaires at this time. Full details of this procedure can be found in the study by MacLean, Kelly, Geddes, and Della (2019).

Group Debriefing. The Gather, Analyze, and Summarize (GAS) debriefing tool was used during the debriefing. This tool allowed the researcher to rapidly debrief in a safe learning environment and supported the overall research aims (Phrampus & O'Donnell, 2013).

Individual Interviews. At the end of session, individual semi-structured interviews took place with randomly selected participants. The interviews were intended to develop a greater understanding of the factors affecting the participants' perception of presence in the simulation. An experienced faculty member (S.M.) facilitated each interview using an interview guide with open-ended questions. All interviews were audio recorded and transcribed.

Analysis

Quantitative: Factor Analysis. All quantitative data analysis was performed using SPSS® version 24.0. Initially, factor analysis using principal component extraction with Varimax rotation was conducted to explore the underlying structure of the scales. Factors with Eigenvalues above one were retained and a minimum criterion of .60 was set for factor loadings. The internal reliabilities of the resultant subscales were calculated using Cronbach's alpha.

Quantitative: Main Analysis. All variables were examined at the item and scale level for outliers, missing data, and potential violations of assumptions for the relevant analysis. Means, standard deviations, and bivariate correlations were estimated for the continuous variables.

To answer research question one, group differences in perceived realism, presence, and learning outcomes were tested using three one-way between-groups analysis of variance (ANOVA). Any significant differences among the three groups would need to be statistically controlled in the main analysis. To answer the second research question, a three-stage hierarchical multiple regression was conducted with realism as the independent variable, presence as a mediating variable, and QDTS score as the dependent variable. When conducting hierarchical multiple regression, Baron and Kenny (1986) recommend testing three criteria for mediation: first, that the independent variable (Realism) is significantly associated with the mediator variable (Presence); second, that the independent variable (Realism) is significantly correlated with the dependent variable (QDTS); finally, that the relationship between the mediator variable and the dependent variable is significant when the impact of the independent variable is statistically accounted for. If the effect of the independent variable is reduced or made nonsignificant in the last of these criterion equations, partial or complete mediation is inferred. As the impact of the education intervention on learning outcomes has been previously established (MacLean, Kelly, et al., 2018), the categorical variable (Groups) was dummy coded and entered in the first stage of the regression to account for the influence of the condition on learning outcomes.

Statistical significance was set at an alpha level of .05. R2 and SR2 (squared semiparital correlation coefficient) were used to measure the overall model and individual variable effect sizes and were interpreted using Cohen's (1988) conventions. In line with Baron's and Kenny's (1986) recommendations, the regression entry procedures were: Step 1, control variables (dummy coded 3 × Groups − Control, − Informational, − Interactional); Step 2, Group + Realism entered; and Step 3, Group + Realism + Presence entered.

Qualitative. To strengthen the overall findings of this study, qualitative interview data were used to substantiate the quantitative survey data. This process provided a rich set of data with which to describe the concept of realism and presence with SP encounters. Open-ended interview questions were transcribed verbatim into Microsoft® Word documents. The first author (S.M.) listened to the audio recordings lasting 14 to 27 minutes to become familiar with the data and to make sure that the transcripts were accurate. Responses to the open-ended items were organized in a manner that permitted codifying and categorizing of the qualitative data by three of the authors (S.M., F.G., M.K.). Data were coded using NVivo 10 and analyzed using thematic analysis.

A five-step process of data familiarization, generating initial codes, searching for themes, reviewing themes, and naming themes to produce the final results was applied. Data were continually explored until saturation was reached. The validity and trustworthiness of the data was ensured through the components of the mixed-methods framework by Creswell and Plano Clark (2018). Credibility was achieved by iterative verification across the research team (S.M., F.G., M.K.) then reaching consensus on the coded themes. Each interview was analyzed by at least two coders. Dependability was achieved by maintaining a clear and comprehensive audit trail documenting coding decisions made by the research team to ensure consistency of the data. As the sample (final year students) had left the university at the time of analysis, participants were not able to verify the interpretation of their responses.

Results

As this study used a convergent mixed-methods design, the results for each phase are presented separately with a summary and synthesis of findings into a unitary account provided in the discussion and results (Creswell & Plano Clark, 2018).

Quantitative Findings

The first phase of the research provided psychometric properties for the COP tool. The principal components analysis for the COP items identified a two-factor solution, accounting for 59.63% of the total variance. As shown in Table 1, the two factors represented presence–sense and presence–affect. The remaining four items did not load cleanly within the two factors. One item overlapped with the realism scale items and was included in the 3-item Realism scale. Additionally, as participants' level of satisfaction with the simulation experience corresponds with Kirkpatrick's level 1 learning outcome and did not fit with the definition of presence in this study, only the 3-item sense of presence subscale was used in the hierarchal regression. All scale psychometric statistics are shown in Table 2. The internal consistency of the presence subscale was considered adequate (Cronbach's alpha = .864).

Results of Exploratory Factor Analysis and Internal Consistency of the Presence Subscales (N = 141)

Table 1:

Results of Exploratory Factor Analysis and Internal Consistency of the Presence Subscales (N = 141)

Means, Standard Deviations, and Bivariate Correlations for Groups and Combined Sample (N = 141)a

Table 2:

Means, Standard Deviations, and Bivariate Correlations for Groups and Combined Sample (N = 141)

As shown in Table 2, participants found the simulation experience to be highly realistic (M = 4.59, SD = 0.40) and reported a strong sense of presence (M = 4.11, SD = 0.68). Pearson's correlation coefficients (r) between study variables were significant. In answer to research question 1, the ANOVAs for both realism and presence were not statistically significant (p = .089, p = .952, respectively), indicating that participants given extra information (e.g., patient risk factors, groups 2 and 3) or interactional strategies (e.g., teach-back, group 3) did not perceive the scenario as more realistic or report a greater level of presence than those in the control condition. This finding indicates that variation in perceived realism and presence was operating at an individual level rather than group level. In contrast, the ANOVA for QDTS was significant F(2,138) = 8.41, p = .000, indicating that participants in the intervention groups had improved learning outcomes, which supported the need for the effects of the group to be controlled in the regression analysis.

Following Baron's and Kenny's (1986) recommendations, the bivariate correlation between realism and presence was positive and strong, r(139) = .515, p < .001. The coefficients for each predictor in the three-stage hierarchical regression with learning outcomes (QDTS) as the dependent variable are presented in Table 3. In step one, to control for the influence of the intervention on learning outcomes, dummy-coded variables for groups were entered and found to be statistically significant R2 = .11, F(2,138) = 8.41, p < .001. The condition accounted for 11% of the variance in the model. In step two, realism was added to the regression model and contributed significantly ΔR2 = .08, ΔF(1,137) = 12.88, p < .001, accounting for a further 8% of the variation in the QDTS. In step three, presence accounted for a further significant change ΔR2 = .04, ΔF(1,136) = 6.59, p < .01. In combination, all three variables accounted for 22.3% of the variation in the QDTS, R2 = .22, F(4,136) = 9.76, p < .001. By Cohen's (1988) convention, this is considered a medium effect (f2 = .29). In support of the proposed mediation model, when both realism and presence were included in stage three, presence was a significant predictor (sr2 = .04), whereas realism was no longer a significant predictor of learning outcomes. Therefore, it was found that presence fully mediated the relationship between realism and positive learning outcomes (QDTS). In combination, these quantitative findings demonstrate that despite exposure to the same high-fidelity simulation environment, when the impact of the communication intervention was controlled for, the participants who perceived the simulation as comparatively less realistic also found it more difficult to immerse themselves in the experience, which subsequently negatively affected their self-ratings of task performance. Supplementary analysis at the group level indicated that the strength of the relationships was greatest in the information group, R2 = .25, F(2,43) = 6.98, p = .002 and was suppressed by ceiling effects and reduced variability in learning outcomes in the interactional group R2 = .04, F(2,47) = 1.06, p = .356.

Hierarchal Regression Analysis Coefficients and Effect Sizes (N = 141)a

Table 3:

Hierarchal Regression Analysis Coefficients and Effect Sizes (N = 141)

To complement the quantitative findings, theme analysis of responses about the simulation experience revealed three main themes: Realism, Presence, and the Learning Experience.

Realism

Realism was manifested in all responses (N = 12) and included subthemes relating to the physical environment, the scenario or task, and the simulated patient. The majority of comments were positive, recognizing the appropriateness of the scenario and the value of working in the simulation laboratory using SPs to facilitate the authenticity of the simulation. Comments included:

  • Yeah, the scenario was real to life, my patient was not compliant…. It was a good scenario because that is the type of patient [who] would get discharged without much thought into the process. I think because they haven't had surgery nurses might dismiss their concerns.
  • Apart from having the business of a ward, it was similar. Having a real person as a patient made it real and the way they would ask you questions and so on, what I am supposed to do about this, it made it real. It's the most real-life setting I have worked in at university.

The following examples highlight how participants perceived limitations to realism in the simulation:

  • I think maybe it was a little controlled, but you are always going to get that with a simulation. I feel as though in a hospital setting, the patient's concerns are going to be different.
  • It would have been more realistic if the MET [medical emergency team] call emergency button went off, or I feel like I should have been interrupted with more nurses coming in and out of the room.

Presence

Presence was also a significant theme across the three groups. Two subthemes of Presence were identified: immersion and engagement. Immersion relates to the intensity of the participant's involvement in the physical environment and task. Engagement relates to the quality of the connection and interaction with the patients. Comments relating to immersion and engagement both supported the value of the simulation and highlighted how participants were able to suspend their disbelief during the learning experience:

  • I forgot I was in a simulation—for me it was about trying to convince my patient to be compliant with his medication and how to use the medication.
  • She was such a good actor that you just completely forgot you were even being watched by anyone or being filmed…. I completely forgot about it [the camera] even though it was right in front of me.
  • I did feel very present in the scenario. The patient asked me what they will do if they have problems once they are home, and I felt lost in that conversation. I felt very focused and engaged with the patient, you get a person-to-person relationship going.

Learning Outcomes

When describing the value of the learning experience, three subthemes emerged: personal reflection, self-efficacy, and transfer of learning. Participants reflected on how the simulation scenario would better prepare them for clinical practice and had increased their confidence to transfer their learning during clinical placements. Several participants also reflected on deficits in their knowledge and capabilities:

  • Next time on clinical placement, I will be more comfortable talking to patients when they are going home.
  • As I was working through the discharge, I thought about when I go on clinical placement and how I will feel confident doing this again.
  • It does instill more confidence when you are working with the simulated patients as you get good feedback. So, it does make you focus, it does make you feel like you need to know your stuff in the clinical area.

In examining for convergence of the quantitative and qualitative results, findings for the three key areas of interest consistently aligned. Emergent themes in the quantitative findings and results of the initial factor analysis provided validation of the decision to clearly distinguish realism from presence when operationalizing these highly correlated constructs. According to both data sources, participants indicated that the simulation was realistic. Importantly, the qualitative data showed that the majority of students recognized the simulation environment as being appropriate and comparable to the clinical setting.

Results for both sets of data showed that participants had a strong sense of presence within the simulation experience. Those who were present within the simulation reported feeling engaged in, immersed in, and focused on patients' emotional and physical aspects of nursing care. The convergence of data confirmed the positive relationship between realism and presence, highlighting how participants who experienced the simulation as realistic were able to immerse themselves into the learning environment. Importantly, the interviews provided insights into why some participants may have experienced a reduced sense of realism and presence, with findings indicating how the lack of background noise and interruptions potentially detracted from the experience. Finally, the convergence of learning outcomes was supported in both sets of data. The QDTS measures nonclinical skills, such as evaluating the nurse as an educator, promoting confidence, and displaying skills in listening and being sensitive to patients' needs. Positive learning outcomes on the QDTS were observed in the quantitative data. The qualitative findings also highlighted participants' increased confidence and capacity to reflect on their learning experience and apply their newly gained skills in clinical practice.

Discussion

In this study, the nine-item COP tool was assessed for validity and reliability using a sample of 141 undergraduate nursing students. The tool was designed to measure the participant's perceptions of presence in virtual reality simulations but has not been validated previously in VR or health care simulation. Therefore, a requirement of this study was to modify the item terminology and validate the tool for use with SPs. Factor analysis revealed a two-factor model representing a sense of presence and affect. Other items in the original COP tool overlapped with the realism construct. The authors found that although the COP tool demonstrated applicability to nursing simulation, inclusion of items relating to engagement with SPs may be warranted.

As would be expected in a study using SPs in a simulation laboratory, most participants across the three groups reported high realism scores. Our results indicated that the incorporation of additional information and interactional elements in the simulation context did not significantly affect the perception of realism. However, for those students who rated realism lower across all groups, there was a significant negative effect on both their sense of presence and learning outcomes.

Evidence from the qualitative analysis supported the quantitative findings that the simulation was realistic, with students recognizing the authenticity of the scenario, SP, and physical environment. Several participants commented on the lack of background noise and need for distractions such as a call bell, monitor alarms, or another nurse coming into the room to more closely replicate the hospital environment and enhance the realism. In this instance, the educator's attempts to remove potential distractions from the simulation diminished some students' sense of realism. This finding may have been mitigated by discussing the potential for or lack of hospital-based interruptions in the simulation briefing and highlighting the importance of providing consistent and comprehensive briefings for managing student's expectations leading into the simulation.

In 1998, Witmer and Singer claimed that an increase in presence would increase learning and learning outcomes; however, to date there has been little research regarding presence in health care simulation using SPs. Our results show a positive correlation between realism, presence, and learning outcomes, which is comparable with studies in nursing VR simulation (Dang et al., 2018; Dubovi et al., 2017). Our results provide nursing educators with evidence that working with SPs in scenarios that represent realistic clinical experiences can optimize the student's sense of presence and learning outcomes in communication-based simulations.

Important details relating to the concept of presence also emerged from the qualitative data. In asking participants to reflect on their learning experience, it was clear that they distinguished between being both nursing task focused and patient focused. The use of SPs in the scenario provided the students with an endocentric presence, which enabled them to immerse themselves in their role and engage in realist patient–nurse interactions.

Quantitative and qualitative data converged around the finding that nursing students achieved level two (learning) and level three (behavior) in Kirkpatrick's model of evaluation (Kirkpatrick, 1994). Learning can be said to have taken place when attitudes change, knowledge is increased, or skill is improved as a result of the experience. The results showed both an increase in confidence and performance across the groups, as assessed by the QDTS questionnaire and qualitative evidence.

Strengths and Limitations

Strengths of this research include the testing of the conceptual model (Figure 1) and the study design incorporating both quantitative and qualitative data to explore the concepts of realism and presence. The interviews provided greater detail of the students' perceptions of their experience and learning during the simulation, which supported the operationalization and measurement of presence in the study. The three-item sense of presence subscale within the COP tool demonstrated validity in the SP context and is an instrument that can be used for further research in simulation.

Regarding limitations, the consideration and measurement of realism was an important component of this study, but characteristics of participants that could potentially affect their perceptions of realism were not considered. Although designers of simulation cannot account for all of the related experiences an individual brings to a simulation, further investigation of how personal characteristics, such as motivations and prior experiences, contribute to the perception of realism and presence is recommended. A further limitation was the reliance on self-report measures. Although the subjective nature of realism and presence necessitated self-report ratings, future research could examine the effects using independent ratings of learning outcomes from SPs or educators.

Conclusion

Our results indicate that realism and presence should be treated as independent but highly correlated concepts. The finding that ratings of realism did not increase systematically across the three study groups, regardless of the introduction of communication-based interventions, but rather varied within the groups indicates that perceived realism is sensitive to characteristics of the individual participants, as well as the simulation design. Furthermore, in designing simulations, there may be an optimal level of fidelity beyond which perceived realism is not impacted. Presence was found to be a function of the individual's perception of and reaction to the simulation experience and a mediating influence on learning outcomes. In sum, a student's capacity to suspend their sense of disbelief and perceive the simulation as realistic was found to induce a higher sense of presence, which in turn affected learning. Given that the goal of simulation-based education is to optimize learning and increase students' capacity to transfer their knowledge into the clinical setting, this finding is significant. This study shows promising results and recommends that future research focuses on examining the personality traits, characteristics, and clinical experiences of participants to understand further how these factors influence the concepts of realism and presence in health care simulation.

References

  • Baron, R.M. & Kenny, D.A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182. doi:10.1037/0022-3514.51.6.1173 [CrossRef]
  • Brackney, D.E. & Priode, K. (2017). Back to reality: The use of the presence questionnaire for measurement of fidelity in simulation. Journal of Nursing Measurement, 25, 66–73. doi:10.1891/1061-3749.25.2.E66 [CrossRef]
  • Choi, W., Dyens, O., Chan, T., Schijven, M., Lajoie, S., Mancini, M. & Aggarwal, R. (2017). Engagement and learning in simulation: Recommendations of the Simnovate Engaged Learning Domain Group. BMJ Simulation and Technology Enhanced Learning, 3, S23–S32. doi:10.1136/bmjstel-2016-000177 [CrossRef]
  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: L. Erlbaum Associates.
  • Covington, H. (2003). Caring presence: Delineation of a concept for holistic nursing. Journal of Holistic Nursing, 21, 301–317. doi:10.1177/0898010103254915 [CrossRef]
  • Creswell, J. & Plano Clark, V. (2018). Designing and conducting mixed methods research (3rd ed.). Thousand Oaks, CA: Sage.
  • Dang, B.K., Palicte, J.S., Valdez, A. & O'Leary-Kelley, C. (2018). Assessing simulation, virtual reality, and television modalities in clinical training. Clinical Simulation in Nursing, 19, 30–37. doi:10.1016/j.ecns.2018.03.001 [CrossRef]
  • De Leo, G., Diggs, L.A., Radici, E. & Mastaglio, T.W. (2014). Measuring sense of presence and user characteristics to predict effective training in an online simulated virtual environment. Simulation in Healthcare, 9, 1–6. doi:10.1097/SIH.0b013e3182a99dd9 [CrossRef]
  • Dinh, H.Q., Walker, N., Song, C., Kobayashi, A. & Hodges, L.F. (1999). Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. Proceedings of the IEEE Virtual Reality 1999 Conference. , Houston. , 222–228.
  • Dubovi, I., Levy, S.T. & Dagan, E. (2017). Now I know how! The learning process of medication administration among nursing students with non-immersive desktop virtual reality simulation. Computers & Education, 113, 16–27. doi:10.1016/j.compedu.2017.05.009 [CrossRef]
  • Dunnington, R.M. (2015) The centricity of presence in scenario-based high fidelity human patient simulation: A model. Nursing Science Quarterly, 28, 64–73. doi:10.1177/0894318414558605 [CrossRef]
  • Grossman, R. & Salas, E. (2011). The transfer of training: What really matters. International Journal of Training and Development, 15, 103–120. doi:10.1111/j.1468-2419.2011.00373.x [CrossRef]
  • INACSL Standards Committee. (2016). INACSL standards of best practice: SimulationSM simulation glossary [Supplemental material]. Clinical Simulation in Nursing, 12, S39–S47. doi:10.1016/j.ecns.2016.09.012 [CrossRef]
  • Jeffries, P.R. (Ed.). (2016). The NLN Jeffries simulation theory. Philadelphia, PA: Wolters Kluwer.
  • Kirkpatrick, D.L. (1994). Evaluating training programs: The four levels. San Francisco, CA: Berrett-Koehler.
  • Lee, K.M. (2004). Presence, explicated. Communication Theory, 14, 27–50. https://doi.org/10.1111/j.1468-2885.2004.tb00302.x doi:10.1111/j.1468-2885.2004.tb00302.x [CrossRef]
  • Lopreiato, J.O. (2016). Healthcare simulation dictionary. Rockville, MD: Agency for Healthcare Research and Quality.
  • MacLean, S., Geddes, F., Kelly, M. & Della, P. (2018). Simulated patient training: Using inter-rater reliability to evaluate simulated patient consistency in nursing education. Nurse Education Today, 62, 85–90. doi:10.1016/j.nedt.2017.12.024 [CrossRef]
  • MacLean, S., Geddes, F., Kelly, M. & Della, P. (2019). Video reflection in discharge communication skills training with simulated patients: A qualitative study of nursing students' perceptions. Clinical Simulation in Nursing, 28, 15–24. doi:10.1016/j.ecns.2018.12.006 [CrossRef]
  • MacLean, S., Kelly, M., Geddes, F. & Della, P. (2018). Evaluating the use of teach-back in simulation to improve discharge communication practices of undergraduate nursing students. Clinical Simulation in Nursing, 22, 13–21. doi:10.1016/j.ecns.2018.06.005 [CrossRef]
  • Maloney, L.R. & Weiss, M.E. (2008). Patients' perceptions of hospital discharge informational content. Clinical Nursing Research, 17, 200–219. doi:10.1177/1054773808320406 [CrossRef]
  • Minsky, M. (1980). Telepresence. Omni, 2, 45–51.
  • Morse, J.M. & Niehaus, L. (2009). Mixed method design: Principles and procedures. New York, NY: Routledge.
  • Nestel, D., Morrison, T. & Pritchard, S. (2014). Scope of contemporary simulated patient methodology. In Nestel, D. & Bearman, M. (Eds.), Simulated patient methodology: Theory, evidence and practice (pp. 7–15). Chichester, United Kingdom: John Wiley & Sons.
  • Phrampus, P.E. & O'Donnell, J.M. (2013). Debriefing using a structured and supported approach. In Levine, A.I., DeMaria, S.Jr.., , Schwartz, A.D. & Sim, A.J. (Eds.), The comprehensive textbook of healthcare simulation (pp. 73–84). New York, NY: Springer. doi:10.1007/978-1-4614-5993-4_6 [CrossRef]
  • Sanchez-Vives, M.V. & Slater, M. (2005). From presence to consciousness through virtual reality. Nature Reviews Neuroscience, 6, 332–339. https://doi.org/10.1038/nrn1651 doi:10.1038/nrn1651 [CrossRef]
  • Shin, S., Park, J.-H. & Kim, J.-H. (2015). Effectiveness of patient simulation in nursing education: Meta-analysis. Nurse Education Today, 35, 176–182. https://doi.org/http://dx.doi.org/10.1016/j.nedt.2014.09.009 doi:10.1016/j.nedt.2014.09.009 [CrossRef]
  • Stevens, J.A. & Kincaid, J.P. (2015). The relationship between presence and performance in virtual simulation training. Open Journal of Modelling and Simulation, 3, 41–48. doi:10.4236/ojmsi.2015.32005 [CrossRef]
  • Welch, D. & Wellard, S. (2005). The phenomenon of presence within contemporary nursing practice: A literature review. Australian Journal of Holistic Nursing, 12, 4–10.
  • Witmer, B.G. & Singer, M.J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence, Teleoperators and Virtual Environments, 7, 225–240. doi:10.1162/105474698565686 [CrossRef]

Results of Exploratory Factor Analysis and Internal Consistency of the Presence Subscales (N = 141)

ItemFactor 1Factor 2Scaleα
In the simulation, how was:
  Your sense of presence?0.750.28Presence: Sense.86
  Your sense of “being there?”0.840.29
  Your sense of inclusion?0.710.29
  Your awareness of the real-world surroundings?0.180.28
  Your sense of realism?0.540.29
  Your overall comfort level?0.210.74Presence: Affect.76
  Your overall enjoyment?0.340.64
  The quality of the scenario?0.350.32
  The degree to which you could move around the environment?0.460.49

Means, Standard Deviations, and Bivariate Correlations for Groups and Combined Sample (N = 141)a

Variable and GroupMeanSDαNo. of Items12
Step 1: Realism4.590.40.543
  Control4.500.45
  Information4.680.31
  Interaction4.590.42
Step 2: Presence4.110.68.8630.52**
  Control4.110.70
  Information4.130.70
  Interaction4.090.64
Step 3: Quality of Discharge Teaching Scale7.681.13.86140.28**0.30**
  Control7.361.16
  Information7.461.19
  Interaction8.180.86

Hierarchal Regression Analysis Coefficients and Effect Sizes (N = 141)a

VariableB [95% CI]βsr2R2
Step 1
  Group.11**
Step 2
  Realism0.79 [0.36 to 1.23] **.28.08.18**
Step 3
  Realism0.45 [−0.50 to 0.95].16.02
  Presence0.38 [0.09 to 0.67] *.23.04.22**
Authors

Ms. MacLean is Registered Nursing Lecturer, Dr. Geddes is Research Associate, Dr. Kelly is Associate Professor and Director of Community of Practice, and Dr. Della is Head of School of Nursing, Midwifery and Para-medicine, School of Nursing, Midwifery and Paramedicine, Curtin University, Perth, Western Australia, Australia.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

The authors thank the Australian Government Research Training Program Scholarship and the ARC Linkage Grant LP110100035.

Address correspondence to Sharon MacLean, RM, RN, PhD Candidate, Registered Nursing Lecturer, School of Nursing, Midwifery and Paramedicine, Curtin University, GPO Box U1987, Perth, Western Australia 6845, Australia; e-mail: Sharon.maclean@curtin.edu.au.

Received: November 18, 2018
Accepted: February 20, 2019

10.3928/01484834-20190521-03

Sign up to receive

Journal E-contents