Patient safety is an important issue that is receiving global attention in health care. The Institute of Medicine presented the core competencies of patient safety and quality as a new vision of education for health care professionals (Greiner & Knebel, 2003). All health care professionals, as interdisciplinary team members, should be prepared to provide patient-centered care and use evidence-based practice, quality improvement, and health care informatics in accordance with the rapidly changing health care environment (Greiner & Knebel, 2003). However, in reality, health care professionals are not currently prepared to provide high quality of care to meet the needs of changing health care systems and patient demands. The competencies of patient safety and quality are enhanced by integrating these competencies into the education of health care professionals (Greiner & Knebel, 2003).
Interest in patient safety in health care has subsequently initiated changes in nursing education as well. Nurse educators have emphasized the importance of the association between nursing education and competencies of safety and quality of care (American Association of Colleges of Nursing [AACN], 2006, 2013). As a result, the Quality and Safety Education for Nurses (QSEN) project was initiated in the United States. QSEN outlines six core competencies of knowledge, skills, and attitudes for quality and safety. In addition, QSEN has made various efforts to incorporate the competencies of quality and safety in nursing education (AACN, 2013; Barton, Armstrong, Preheim, Gelmon, & Andrus, 2009; Cronenwett et al., 2007; Cronenwett, Sherwood, & Gelmon, 2009; Jarzemsky, McCarthy, & Ellis, 2010; McKeon, Norris, Cardell, & Britt, 2009).
It is critical for nursing students to be educated in classroom settings and trained in clinical settings about patient safety and quality competencies (Cronenwett et al., 2007). Therefore, it is necessary to develop an educational program that integrates all of the components of patient safety competencies to enhance nursing students’ understanding and competencies of patient safety and quality of care (Cronenwett et al., 2007). The competencies refer to the ability to integrate knowledge, skills, and attitudes into practice for nurses’ clinical performance (Campbell & Mackay, 2001), which is considered as an education outcome for health care professionals (Clements & Mackenzie, 2005; Vleuten, 1996). Therefore, for assessing the competencies accurately, it is essential for educators to identify students’ degree of competencies and improve their competencies through the implementation of educational changes (World Health Organization [WHO], 2009).
Other countries have developed and applied the patient safety competency frameworks (AACN, 2006; Australian Council on Safety and Quality in Health Care, 2005; Cronenwett et al., 2007; Cronenwett, Sherwood, Pohl, et al., 2009; Frank & Brien, 2008; Frank & Danoff, 2007; WHO, 2009, 2011) and competency measurement tools (Chenot & Daniel, 2010; Dycus & McKeon, 2009; Madigosky, Headrick, Nelson, Cox, & Anderson, 2006; Piscotty, Grobbel, & Abele, 2013; Schnall et al., 2008; Smith, Cronenwett, & Sherwood, 2007; Sullivan, Hirst, & Cronenwett, 2009; Wolf, Altmiller, & Bicknell, 2011). However, the literature has reported that no sound tool can comprehensively evaluate the competencies based on the patient safety framework (Okuyama, Martowirono, & Bijnen, 2011). Most of the tools were developed within a specific context, such as anesthesia practice (Ardizzone, Enlow, Evanina, Schnall, & Currie, 2009) and pediatric oncology practice (Dycus & McKeon, 2009); therefore, they are limited in their application to general practice (Ginsburg, Castel, Tregunno, & Norton, 2012; Okuyama et al., 2011). In addition, only a few tools exist that evaluate all of the three aspects of patient safety competencies, including knowledge, skills, and attitudes (Ginsburg et al., 2012; Okuyama et al., 2011), or examine the evidence of validity and reliability (Ginsburg et al., 2012; Madigosky et al., 2006; Okuyama et al., 2011; Schnall et al., 2008). In particular, tools for patient safety competency of nursing students are limited in the literature (Okuyama et al., 2011; Piscotty et al., 2013).
In Korea, a lack of agreement exists on the concept of and related tools for patient safety competency. To measure the patient safety competency of nursing students, Lee (2012) proposed the Patient Safety Competency Framework (PSCF; Figure 1), which is based on the patient safety competency frameworks in the literature (Australian Council on Safety and Quality in Health Care, 2005; Cronenwett et al., 2007; Frank & Brien, 2008; WHO, 2009, 2011). In addition, Lee developed and proposed the Patient Safety Competency Self-Evaluation (PSCSE) measurement tool, reflecting the Korean health care system and nursing educational environments.
Patient Safety Competency Framework.
For the initial PSCSE, which was based on the PSCF, the current study reviewed the tools reported in the literature (Frank & Brien, 2008; Madigosky et al., 2006; Schnall et al., 2008; Sullivan et al., 2009; Wolf et al., 2011) and selected items that were suitable for nursing students in Korea. These tools were translated into Korean and were modified and supplemented in accordance with the current culture of health care in Korea (Lee, 2012). Considering that patient safety standards of most Korean hospitals are currently adjusted to the accreditation standard after the implementation of the Korean health care accreditation program in 2010 (Chang & Lee, 2012), preliminary items were supplemented in the tool by referring to the health care accreditation standards. In the PSCSE, the preliminary items were classified into the categories of knowledge, skills, and attitudes, which allowed for the comprehensive evaluation of patient safety competency. The initial items were validated through two rounds of content validity testing by four clinical experts in patient safety and another four nursing professors. Finally, the 45-item PSCSE was developed to assess nursing students’ perspectives regarding patient safety competency, including knowledge, skills, and attitudes.
The purpose of the current study was to evaluate the validity and reliability of the PSCSE for measuring nursing students’ self-reported patient safety competency. To identify factors composing the tool, the composition was investigated, and the structure was schematized. The final confirming model would provide scientific evidence of a valid and reliable instrument that measures student self-perception of patient safety competency. Ultimately, the results of this study may contribute to research and practice in the area of patient safety.
The current study used a cross-sectional survey design, with a convenience sample to describe psychometrics of the PSCSE.
The study included a convenience sample of 394 junior and senior students attending six nursing colleges selected from eleven 4-year nursing schools located in Seoul, Korea. Finally, 354 questionnaires (40 were uncollected due to students not returning their questionnaires) were included in the analyses. The students were registered for the 2011 fall semester and agreed to participate in the study. Data were collected from November 1 to December 31, 2011.
Other researchers recommended that sample size for structural equation modeling should be at least 200 participants with 10 to 15 indicators (Anderson & Gerbing, 1988; Kline, 2010; Tabachnick & Fidell, 2007; Weston & Gore, 2006). In addition, Hoelter (1983) argued that a “critical N” (p. 330) for the adequate sample size of 200 or more indicates a satisfactory model fit. The SPSS Amos version 20.0 software clearly shows Hoelter’s “critical N” for significance levels of 0.05 and 0.01. Therefore, a sample size of 354 was adequate for the current study using structural equation modeling.
The PSCSE is a student self-evaluation tool to assess patient safety competencies, consisting of 45 items with a 5-point Likert scale (six items for knowledge, 21 for skills, and 18 for attitude). The attitude items measure the level of agreement (5 = strongly agree, 1 = strongly disagree) regarding recognitions and behaviors associated with patient safety. The knowledge items determine the level of knowledge (5 = very knowledgeable, 1 = not knowledgeable) related to patient safety. The skills items measure the level of comfort (5 = very comfortable, 1 = very uncomfortable) in performing the tasks relevant to patient safety (Table 1). Summated mean scores of the tool and each of the three categories of knowledge, skills, and attitudes were calculated. Higher mean scores indicate greater competency regarding patient safety.
Naming of Factors and Questions for Patient Safety Competency Self-Evaluation
The current study performed an exploratory factor analysis (EFA) to investigate compositions of the PSCSE developed by Lee (2012). The structure of the tool was schematized to models of confirmatory factor analysis (CFA) using structural equation modeling, which enabled a multilevel evaluation of the validity of the tool. In addition, to confirm internal structure of the tool, two-dimensional models and multidimensional hierarchical models were compared. For testing reliability of the PSCSE, Cronbach’s alpha coefficient was used to test the internal consistency of the whole PSCSE and each factor of the tool. The SPSS Amos version 20.0 software was used for statistical analyses for this study.
Exploratory Factor Analysis. EFA, which is an analysis that extracts factors from a number of measured variables (Nunnally & Bernstein, 1994), was conducted to verify whether the three categories extracted from theoretical backgrounds of the reviewed literature regarding 45 items of the PSCSE would be categorized into the same factors. EFA is used for theory generation, which favors the results of the data over theoretical backgrounds of previous studies (Brown, 2006). For factor extraction, a principle component analysis (PCA) was performed. Factor rotation was conducted using varimax rotation. The factors were extracted if the eigenvalue was greater than 1.0. In the analysis, goodness-of-fit indices were confirmed by the Bartlett test for sphericity and the Kaiser-Meyer-Olkin measure of sampling adequacy, which tests whether the partial correlations among variables are small.
Confirmatory Factor Analysis. To assess whether the factors were theoretically extracted by EFA, CFA was used, which analyzes relationships between latent and observed variables. Missing values for the analysis were processed with the full information maximum likelihood (FIML), which considers the characteristics of the missing data by using the observed variables, and was applied to potential growth models. It has been reported that the FIML is capable of confirming unbiased estimates when missing at-random assumptions are not strictly satisfied (Marcoulides & Schumacker, 1996; Muthén, Kaplan, & Hollis, 1987).
Model Selection. Two-dimensional and multidimensional models were developed to investigate the internal structures of the PSCSE. The following factor names of each model are presented in Table 1:
- Model 1: A model composed of 45 items.
- Model 2: A model composed of 12 factors that explain the relationships among the 45 items.
- Model 3: A model composed of the three categories constructed from the 12 factors.
- Model 4: A final model under the assumption that the three categories can be categorized into one single measurement of the PSCSE.
Model Comparison. Goodness-of-fit indices can be evaluated by chi-square tests; however, it is limited in that values are sensitive to sample size. In other words, models may be unfit with a large sample size but may fit with a small sample size. Therefore, the current study used incremental fit indices, such as a normed fit index (NFI), comparative fit index (CFI), and Tucker-Lewis index (TLI), as well as absolute fit indices, such as root-mean-square error of approximation (RMSEA). Generally, when incremental fit indices, including the CFI, are larger than 0.9, goodness-of-fit indices of the model are considered to be good (Hu & Bentler, 1999). The RMSEA is a kind of goodness-of-fit index developed to compensate for the drawbacks of chi-square statistics. In general, goodness-of-fit indices are considered to be very good when RMSEA is less than 0.05, good when between 0.5 and 0.8, and not good when greater than 0.10 (Bollen & Long, 1993). For model estimation of the structural equation model, maximum likelihood was performed.
The current study was approved by the institutional review board of one researcher’s (N.-J.L.) institution. In collecting data, research assistants explained the purpose of the current study to the participants, and written consent was obtained from each participant. Self-reported data were collected using surveys. Survey questionnaires were kept confidential.
General characteristics of the participants are shown in Table 2. A total of 354 students participated; 94.9% (n = 336) were women and 59.9% (n = 212) were senior nursing students.
Characteristics of Study Participants (N = 354)
Exploratory Factor Analysis. The validity of the PSCSE was examined using EFA, which tests all of the items of the tool on the basis of theoretical and empirical evidence from previous studies (AACN, 2006; Australian Council on Safety and Quality in Health Care, 2005; Cronenwett et al., 2007; Cronenwett, Sherwood, Pohl, et al., 2009; Frank & Brien, 2008; Frank & Danoff, 2007; WHO, 2009, 2011). The Kaiser-Meyer-Olkin measure of sampling adequacy for all 45 items was 0.861. In Bartlett’s test of sphericity, the Kaiser-Meyer-Olkin measure of sampling adequacy showed a statistical significance among the variables (p < 0.001), indicating that factor analysis was suitable for this analysis. Communalities of each variable were in the range of 0.456 to 0.785 for all 45 items, so the items could be included in the analysis (Nunnally & Bernstein, 1994).
As a result, the PSCSE were put into the three categories (knowledge, skills, and attitudes) as initially developed (Table 3). From the EFA, 11 factors (one factor in knowledge, four factors in skills, and six factors in attitudes) were extracted. Considering the 11 factors initially extracted, explanatory values of each factor manifested 2.837% to 12.241%, with a total variance explained of 62.734% (Table 3). The factors extracted, based on theoretical and conceptual evidence reviewed in the literature and discussed with professionals, are shown in Table 1.
Item Loading From the Exploratory Factor Analysis (N = 354)
Confirmatory Factor Analysis. A CFA was performed for validating the three categories (knowledge, skills, and attitudes) based on the results of the EFA.
Knowledge. The knowledge category was divided into two factors. Because the knowledge category did not show any item with partial discriminant validity, two factors comprising six items were extracted (Table 4). Due to the low fit of the primary CFA, the secondary CFA was conducted by setting a covariance regarding errors of observed variables with high correlation and a modification index greater than 10, and satisfactory outcomes were obtained (χ2 = 11.346, df = 7, NFI = 0.988, TLI = 0.990, CFI = 0.995, RMSEA = 0.042).
Confirmatory Factor Analysis (CFA) of Patient Safety Competency Self-Evaluation
Skills. The skill category was categorized into six factors. No item had partial discriminant validity of the factor. Finally, the six factors, comprising 21 items, were extracted as shown in Table 4. Due to the low fit of the primary CFA, a covariance regarding errors of observed variables with high correlation and a modification index greater than 10 was set again for the secondary CFA. The results showed satisfactory outcomes (χ2 = 511.570, df = 169, NFI = 0.871, TLI = 0.887, CFI = 0.909, RMSEA = 0.076).
Attitudes. The attitude category was classified into four factors. After excluding the four items with partial discriminant validity, CFA was performed sequentially. Table 4 presents the results with four factors comprising 14 items. The four items that were excluded from the attitude factor are as follows: a1, Making errors in health care is inevitable; a5, The culture of health care makes it easy for health care professionals to deal constructively with errors; a10, Reporting systems do little to reduce future errors; and a12, Effective responses to errors focus primarily on the health care professional involved. Due to the low fit of CFA for the factors, a modification index was applied. After setting a covariance regarding errors of observed variables with high correlation and selecting a modification index greater than 10, CFA was again performed. As a result, goodness-of-fit indices were satisfactory (χ2 = 164.189, df = 68, NFI = 0.860, TLI = 0.881, CFI = 0.911, RMSEA = 0.063).
Model Comparison. To confirm the internal structure of the PSCSE that composes the three categories and 41 items extracted through EFA and CFA on each factor, competitive models of the two-dimensional and multidimensional hierarchical models were constructed (Figure 2). Each model was compared as follows:
- Model 1: A model composed of 41 items for constructing the types of the PSCSE.
- Model 2: A model composed of 12 factors that explain the relationships among the 41 items of the PSCSE.
- Model 3: A hierarchical model under the assumption that the 12 factors of the PSCSE can be categorized into three categories (knowledge, skills, and attitudes).
- Model 4: A final hierarchical model under the assumption that the three categories consisting of the PSCSE can be categorized into one single measurement of the PSCSE.
Final model of Patient Safety Competency Self-Evaluation (PSCSE) tool.
To explain the hierarchical relationships, each model’s goodness-of-fit indices and factor loadings were compared. The goodness of fit of Model 1 showed that it was unfit to data (χ2 = 3295.47, df = 760, NFI = 0.529, TLI = 0.558, CFI = 0.590, RMSEA = 0.097). Factor loadings of 41 items manifested as 0.049 to 0.667; however, factor loadings of 15 items among them were observed to be remarkably low at less than 0.4.
As for Model 2, the goodness of fit was found to be satisfactory (χ2 = 1248.95, df = 698, NFI = 0.822, TLI = 0.895, CFI = 0.911, RMSEA = 0.047). Factor loadings of the 12 factors composing the 41 items ranged between 0.363 and 0.907. Only one factor loading of the attitude factor 1 (A1) was less than 0.4.
The goodness of fit of Model 3 showed satisfactory outcomes (χ2 = 1345.81, df = 742, NFI = 0.808, TLI = 0.892, CFI = 0.902, RMSEA = 0.048). The 41 factor loadings of the 12 factors were observed between the range of 0.345 and 0.911. Two factor loadings of attitude factor (A3) were found to be less than 0.4. In addition, factor loadings of the three categories were between 0.432 and 0.991.
Regarding the goodness of fit of Model 4, satisfactory fit indices were shown (χ2 = 1347.00, df = 743, NFI = 0.808, TLI = 0.892, CFI = 0.902, RMSEA = 0.048). Factor loadings of the 41 items of the 12 factors were between 0.345 and 0.911. Two factor loadings of attitude factor A3 were found to be less than 0.4. In addition, the 12 factor loadings of the three categories ranged from 0.435 to 0.992. Finally, the three factor loadings of the single measurement of the PSCSE were between 0.194 and 0.980. In this final model, factor loading of the attitude factor was less than 0.4.
In the comparison of goodness of fit of Models 1 and 2, chi-square values showed statistically significant differences. Overall, Model 2 showed a good fit, indicating that the PSCSE consists of multidimensional structures of the 12 factors, not two-dimensional structures of the 41 items. When comparing the goodness of fit of Models 2 and 3, a significant difference was noted between the two models; however, goodness of fit of the models showed minimal difference. This means that the 12 factors aptly explain the three categories. In comparison of Models 3 and 4, chi-square values showed no statistically significant difference, resulting in no difference in goodness of fit of the models. Therefore, this result indicates that the 12 factors can be explained by the three categories, which can be further explained by the single measurement of the PSCSE. As a result, it was validated that the PSCSE was presented in multidimensional structures, including the 12 factors and three categories (Figure 2).
In the current study, the overall reliability of the PSCSE tool showed Cronbach’s alpha of 0.899 in EFA. The value of Cronbach’s alpha was elevated to 0.907 after performing CFA. The reliabilities of each category examined by CFA were 0.859 for knowledge, 0.912 for skills, and 0.794 for attitudes. Cronbach’s alphas for knowledge, K1 and K2, were 0.804 and 0.823, respectively. Cronbach’s alphas for skills, S1, S2, S3, S4, S5, and S6, were 0.838, 0.792, 0.767, 0.806, 0.762, and 0.801, respectively. For attitudes, Cronbach’s alphas of A1, A2, A3, and A4 were 0.738, 0.665, 0.570, and 0.404, respectively.
The PSCSE measurement tool consisted of three categories (knowledge, skills, and attitudes) and 12 factors with goodness-of-fit indices, which were verified based on the results of CFA— two factors for knowledge, six factors for skills, and four factors for attitudes. As a result of EFA, it was confirmed that the structures of the tool were identical with the structures verified by the CFA. Overall, the goodness-of-fit indices of the models studied were verified.
Regarding the knowledge category, the first factor, Concept of the Components of Patient Safety Culture, consisted of items confirming the basic knowledge for patient safety, such as the complexity of the health care system, human factors, information technology, electronic health care systems, leadership, communication, and teamwork. An understanding on the culture of patient safety is necessary to provide safe nursing care and contributes to a safe environment in nursing (Australian Council on Safety and Quality in Health Care, 2005; Frank & Brien, 2008; WHO, 2011).
The second factor, Concept of Errors and Cause Analysis, consisted of items confirming the knowledge of error types and cause analysis. Nursing students should have competency in conducting a series of systematical analysis on error types and causes and the mechanism of error occurrence. Nursing students will then be able to generate countermeasures and strategies to prevent errors in clinical settings (Australian Council on Safety and Quality in Health Care, 2005; Frank & Brien, 2008; WHO, 2011).
The skills category of the PSCSE confirmed how nursing students can comfortably perform practical skills for enhancing patient safety. The first factor, Error Reporting and Response to an Error, consisted of competencies reporting an error with an error-reporting system, systematically analyzing the causes of the errors, and supporting colleagues involved by handling the error appropriately. Such support also contributes to establishing a positive organizational culture regarding patient safety (Australian Council on Safety and Quality in Health Care, 2005; Frank & Brien, 2008; WHO, 2011). These competencies are consistent with the factor of Error Reporting in the adapted version of the Patient Safety/Medical Fallibility Assessment (Chenot & Daniel, 2010). An absence of error-reporting systems and subsequent feedback, as well as lack of information and formal education regarding error reporting methods, interrupts error reporting in clinical settings (Braithwaite, Westbrook, Travaglia, & Hughes, 2010; Handler et al., 2007; Kingston, Evans, Smith, & Berry, 2004).
The second and sixth factors of skills category were related to communication. The second factor, Communication Related to Error, confirmed whether nursing students can communicate effectively with patients and their families, nurses, mangers, and other health care professionals. The sixth factor, Precise Communications During Hand Offs, consisted of items confirming the preparation of communication skills for transferring accurate information and documenting it in accordance with the institutional rules. The importance of communication in patient safety has been emphasized as a separate competency in almost all of the patient safety frame-works (Australian Council on Safety and Quality in Health Care, 2005; Frank & Brien, 2008; WHO, 2011). In QSEN, the competencies regarding hand-off communication were included in Teamwork and Collaboration. In addition, communication-related concern of hazards or errors were included in the Safety competency (Cronenwett et al., 2007). During hand offs, poorly conveyed communication may result in harm to patients; therefore, competencies of communicating and recording accurate information in the hand offs are essential to reduce errors (Frank & Brien, 2008; WHO, 2011). Inter-professional education and the importance of effective team work and communication has been recently emphasized in the literature (Liaw, Zhou, Lau, Siau, & Chan, 2014; Tella et al., 2014). Although the communication competency of Recognition Reporting and Disclosing Errors is important, many tools that were previously developed did not include this area (Okuyama et al., 2011).
The third factor, Resource Utilization/Evidence-Based Practice, consisted of items confirming whether nursing students can utilize evidence-based clinical data, information technology, and electronic health care systems. Theses competencies were included in the Evidence-Based Practice competency and the Informatics competency of QSEN (Cronenwett, Sherwood, & Gelmon, 2009). Health care professionals should understand the effects of environmental, as well as personal, factors on their work. The effects of human computer interaction on patient safety also should be considered as important. In addition, health care professionals should be able to use the resources with critical thinking skills (Frank & Brien, 2008).
The fourth factor, Safe Nursing Practice, and the fifth factor, Infection Prevention, consisted of items confirming whether nursing students could safely perform their nursing practice, focusing on best practice guidelines. Because errors can occur at every step of patient care, risk management should be closely associated with health care services at each level (WHO, 2011).
The attitudes category of the PSCSE confirmed the recognition of patient safety. The first factor, Patient Safety Promotion/Prevention Strategy, consisted of four items confirming whether strategies for preventing errors and improving safety, which were based on the concept of patient safety, were aptly recognized. The second factor, Responsibility of Health Care Professionals for Patient Safety Culture, included items related to the duties of health care professionals, such as forming patient safety culture. Because the behaviors of health care professionals can directly affect patient safety, it has been recommended that they should monitor ethical issues, take responsibility for their practice, and maintain their clinical capabilities through consistent learning (Australian Council on Safety and Quality in Health Care, 2005; Frank & Brien, 2008; WHO, 2011). Safety cannot be obtained by a single or short-term training, rather, by continuous education in undergraduate courses.
The third factor, Error Reporting and Disclosing, excluded two items in the final model. These two items confirmed the recognition of error reporting and handling; however, they were not directly relevant to error-reporting behaviors (e.g., ignore the situation, inform patients or family, report) at the time of the error. As a result, the third factor was more consistent with the key competencies in domain six of the Canadian Patient Safety Institute (Frank & Brien, 2008). Error reporting and disclosure is one of patient safety competencies that nurses perceive to be difficult (Wagner, Harkness, Hébert, & Gallagher, 2012). Therefore, it is essential that students are consistently educated about error reporting and disclosing in their undergraduate education programs so they can prevent potential errors and become competent gatekeepers for patient safety (WHO, 2011). Open disclosure requires professionalism of health care providers and partnership with patients and their family members (WHO, 2011). The participation of patients and their families in safety plays an important role in reducing adverse events (Davis, Jacklin, Sevdalis, & Vincent, 2007; WHO, 2011).
The fourth factor, Components of Patient Safety Culture, consisted of essential items for forming the culture of patient safety. In this factor, two items were excluded in the final model. Although the two excluded items could theoretically be included in this factor, by excluding these items, the statistical values, such as goodness of fit and Cronbach’s alpha, were increased. Therefore, the current study presumed that the two items were ambiguously expressed regarding this factor. Continuous learning with questioning attitudes that is based on an understanding of patient safety components promotes patient safety (Frank & Brien, 2008).
For the current study, junior and senior nursing students attending six schools of nursing in Seoul, South Korea, were included. The differences in their characteristics may affect study results; therefore, caution should be exercised when generalizing the results. Second, the PSCSE is a student self-evaluation tool. Therefore, differences between students’ self-evaluated degree of competency and their actual levels of competency may exist.
The PSCSE is a self-evaluation tool that was developed to measure nursing students’ competencies focusing on patient safety. In the current study, this tool was verified to be valid and statistically reliable. Currently, the PSCSE is the only tool emphasizing the patient safety competencies of nursing students and reflects the current culture of health care in South Korea. On the basis of the study results, the PSCSE can be used for future research of patient safety in nursing education and can be used for:
- Identifying nursing students’ vulnerabilities.
- Integrating patient safety competencies into nursing curriculums.
- Comparing pre- and posteducational interventions.
A self-evaluation tool is an efficient and effective method for assessing and improving professional competency (Campbell & Mackay, 2001). However, considering that most of the existing patient safety tools were developed on the basis of self-evaluation, the current study suggests that future research on how to objectively measure students’ patient safety competencies should be conducted.
- American Association of Colleges of Nursing. (2006). Hallmarks of quality and patient safety: Recommended baccalaureate competencies and curricular guidelines to ensure high-quality and safe patient care. Journal of Professional Nursing, 22, 329–330. doi:10.1016/j.profnurs.2006.10.005 [CrossRef]
- American Association of Colleges of Nursing. (2013). AACN extends national nurse faculty development initiative with new funding from the Robert Wood Johnson Foundation. Retrieved from http://www.aacn.nche.edu/news/articles/2012/qsen
- Anderson, J.C. & Gerbing, D.W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411–423. doi:10.1037/0033-2909.103.3.411 [CrossRef]
- Ardizzone, L.L., Enlow, W.M., Evanina, E.Y., Schnall, R. & Currie, L. (2009). Impact of a patient safety curriculum for nurse anesthesia students. Journal of Nursing Education, 48, 706–710. doi:10.3928/01484834-20091113-01 [CrossRef]
- Australian Council on Safety and Quality in Health Care. (2005). National patient safety education framework. Canberra, Australia: Common-wealth of Australia. Retrieved from http://www.safetyandquality.gov.au/wp-content/uploads/2012/06/National-Patient-Safety-Education-Framework-2005.pdf
- Barton, A.J., Armstrong, G., Preheim, G., Gelmon, S.B. & Andrus, L.C. (2009). A national Delphi to determine developmental progression of quality and safety competencies in nursing education. Nursing Outlook, 57, 313–322. doi:10.1016/j.outlook.2009.08.003 [CrossRef]
- Bollen, K.A. & Long, J.S. (1993). Testing structural equation models. Newbury Park, CA: Sage.
- Braithwaite, J., Westbrook, M.T., Travaglia, J.F. & Hughes, C. (2010). Cultural and associated enablers of, and barriers to, adverse incident reporting. Quality & Safety in Health Care, 19, 229–233. doi:10.1136/qshc.2008.030213 [CrossRef]
- Brown, T.A. (2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.
- Campbell, B. & Mackay, G. (2001). Continuing competence: An Ontario nursing regulatory program that supports nurses and employers. Nursing Administration Quarterly, 25(2), 22–30. doi:10.1097/00006216-200101000-00006 [CrossRef]
- Chang, H.S. & Lee, S.H. (2012). Establishment, present condition, and developmental direction of the new Korean healthcare accreditation system. Journal of Korean Medical Science, 27(Suppl.), S61–S69. doi:10.3346/jkms.2012.27.S.S61 [CrossRef]
- Chenot, T.M. & Daniel, L.G. (2010). Frameworks for patient safety in the nursing curriculum. Journal of Nursing Education, 49, 559–568. doi:10.3928/01484834-20100730-02 [CrossRef]
- Clements, R. & Mackenzie, R. (2005). Competence in prehospital care: Evolving concepts. Emergency Medicine Journal, 22, 516–519. doi:10.1136/emj.2005.026237 [CrossRef]
- Cronenwett, L., Sherwood, G., Barnsteiner, J., Disch, J., Johnson, J., Mitchell, P. & Warren, J. (2007). Quality and safety education for nurses. Nursing Outlook, 55, 122–131. doi:10.1016/j.outlook.2007.02.006 [CrossRef]
- Cronenwett, L., Sherwood, G. & Gelmon, S.B. (2009). Improving quality and safety education: The QSEN learning collaborative. Nursing Outlook, 57, 304–312. doi:10.1016/j.outlook.2009.09.004 [CrossRef]
- Cronenwett, L., Sherwood, G., Pohl, J., Barnsteiner, J., Moore, S., Sullivan, D.T. & Warren, J. (2009). Quality and safety education for advanced nursing practice. Nursing Outlook, 57, 338–348. doi:10.1016/j.outlook.2009.07.009 [CrossRef]
- Davis, R.E., Jacklin, R., Sevdalis, N. & Vincent, C.A. (2007). Patient involvement in patient safety: What factors influence patient participation and engagement?Health Expectations, 10, 259–267. doi:10.1111/j.1369-7625.2007.00450.x [CrossRef]
- Dycus, P. & McKeon, L. (2009). Using QSEN to measure quality and safety knowledge, skills, and attitudes of experienced pediatric oncology nurses: An international study. Quality Management in Health Care, 18, 202–208. doi:10.1097/QMH.0b013e3181aea256 [CrossRef]
- Frank, J.R. & Brien, S. (Eds.). (2008). The safety competencies: Enhancing patient safety across the health professions (1st ed.). Ottawa, Ontario, Canada: Canadian Patient Safety Institute. Retrieved from http://www.patientsafetyinstitute.ca/English/toolsResources/safetyCompetencies/Documents/Safety%20Competencies.pdf
- Frank, J.R. & Danoff, D. (2007). The CanMEDS initiative: Implementing an outcomes-based framework of physician competencies. Medical Teacher, 29, 642–647. doi:10.1080/01421590701746983 [CrossRef]
- Ginsburg, L., Castel, E., Tregunno, D. & Norton, P.G. (2012). The H-PEPSS: An instrument to measure health professionals’ perceptions of patient safety competence at entry into practice. BMJ Quality & Safety, 21, 676–684. doi:10.1136/bmjqs-2011-000601 [CrossRef]
- Greiner, A.C. & Knebel, E. (Eds.). (2003). Health professions education: A bridge to quality. Washington, DC: National Academies Press.
- Handler, S.M., Perera, S., Olshansky, E.F., Studenski, S.A., Nace, D.A., Fridsma, D.B. & Hanlon, J.T. (2007). Identifying modifiable barriers to medication error reporting in the nursing home setting. Journal of the American Medical Directors Association, 8, 568–574. doi:10.1016/j.jamda.2007.06.009 [CrossRef]
- Hoelter, J.W. (1983). The analysis of covariance structures: Goodness-of-fit indices. Sociological Methods & Research, 11, 325–344. doi:10.1177/0049124183011003003 [CrossRef]
- Hu, L.-T. & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118 [CrossRef]
- Jarzemsky, P., McCarthy, J. & Ellis, N. (2010). Incorporating quality and safety education for nurses competencies in simulation scenario design. Nurse Educator, 35, 90–92. doi:10.1097/NNE.0b013e3181d52f6e [CrossRef]
- Kingston, M.J., Evans, S.M., Smith, B.J. & Berry, J.G. (2004). Attitudes of doctors and nurses towards incident reporting: A qualitative analysis. The Medical Journal of Australia, 181, 36–39.
- Kline, R.B. (2010). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press.
- Lee, N.J. (2012, June). Development of questionnaires to measure baccalaureate nursing students’ patient safety competencies. Paper presented at the 11th International Congress on Nursing Informatics. , Montreal, Quebec, Canada. .
- Liaw, S.Y., Zhou, W.T., Lau, T.C., Siau, C. & Chan, S.W. (2014). An inter-professional communication training using simulation to enhance safe care for a deteriorating patient. Nurse Education Today, 34, 259–264. doi:10.1016/j.nedt.2013.02.019 [CrossRef]
- Madigosky, W.S., Headrick, L.A., Nelson, K., Cox, K.R. & Anderson, T. (2006). Changing and sustaining medical students’ knowledge, skills, and attitudes about patient safety and medical fallibility. Academic Medicine, 81, 94–101. doi:10.1097/00001888-200601000-00022 [CrossRef]
- Marcoulides, G.A. & Schumacker, R.E. (Eds.). (1996). Advanced structural equation modeling: Issues and techniques. Mahwah, NJ: Lawrence Erlbaum.
- McKeon, L.M., Norris, T., Cardell, B. & Britt, T. (2009). Developing patient-centered care competencies among prelicensure nursing students using simulation. Journal of Nursing Education, 48, 711–715. doi:10.3928/01484834-20091113-06 [CrossRef]
- Muthén, B., Kaplan, D. & Hollis, M. (1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 52, 431–462. doi:10.1007/BF02294365 [CrossRef]
- Nunnally, J.C. & Bernstein, I.H. (1994). Psychometric theory. New York, NY: McGraw-Hill.
- Okuyama, A., Martowirono, K. & Bijnen, B. (2011). Assessing the patient safety competencies of healthcare professionals: A systematic review. BMJ Quality & Safety, 20, 991–1000. doi:10.1136/bmjqs-2011-000148 [CrossRef]
- Piscotty, R., Grobbel, C. & Abele, C. (2013). Initial psychometric evaluation of the nursing quality and safety self-inventory. Journal of Nursing Education, 52, 269–274. doi:10.3928/01484834-20130412-03 [CrossRef]
- Schnall, R., Stone, P., Currie, L., Desjardins, K., John, R.M. & Bakken, S. (2008). Development of a self-report instrument to measure patient safety attitudes, skills, and knowledge. Journal of Nursing Scholarship, 40, 391–394. doi:10.1111/j.1547-5069.2008.00256.x [CrossRef]
- Smith, E.L., Cronenwett, L. & Sherwood, G. (2007). Current assessments of quality and safety education in nursing. Nursing Outlook, 55, 132–137. doi:10.1016/j.outlook.2007.02.005 [CrossRef]
- Sullivan, D.T., Hirst, D. & Cronenwett, L. (2009). Assessing quality and safety competencies of graduating prelicensure nursing students. Nursing Outlook, 57, 323–331. doi:10.1016/j.outlook.2009.08.004 [CrossRef]
- Tabachnick, B.G. & Fidell, L.S. (2007). Using multivariate statistics (5th ed.). Boston, MA: Allyn & Bacon.
- Tella, S., Liukka, M., Jamookeeah, D., Smith, N.J., Partanen, P. & Turunen, H. (2014). What do nursing students learn about patient safety? An integrative literature review. Journal of Nursing Education, 53, 7–13. doi:10.3928/01484834-20131209-04 [CrossRef]
- Vleuten, C.P.M. (1996). The assessment of professional competence: Developments, research and practical implications. Advances in Health Sciences Education, 1, 41–67. doi:10.1007/BF00596229 [CrossRef]
- Wagner, L.M., Harkness, K., Hébert, P.C. & Gallagher, T.H. (2012). Nurses’ perceptions of error reporting and disclosure in nursing homes. Journal of Nursing Care Quality, 27, 63–69. doi:10.1097/NCQ.0b013e318232c0bc [CrossRef]
- Weston, R. & Gore, P.A. (2006). A brief guide to structural equation modeling. The Counseling Psychologist, 34, 719–751. doi:10.1177/0011000006286345 [CrossRef]
- Wolf, Z.R., Altmiller, G. & Bicknell, P. (2011). Development and testing of the patient safety test: Current concepts. Nurse Educator, 36, 187–191. doi:10.1097/NNE.0b013e3182297ca9 [CrossRef]
- World Health Organization. (2009). WHO patient safety curriculum guide for medical schools. Geneva, Switzerland: Author.
- World Health Organization. (2011). Patient safety curriculum guide: Multi-professional edition. Geneva, Switzerland: Author.
Naming of Factors and Questions for Patient Safety Competency Self-Evaluation
| Factor K1. Concept of the components of patient safety culture|
k1 Describe factors that create a culture of safety (e.g., teamwork, leadership, effective communication).
k2 Describe role of human factors in assuring safety (e.g., physical, psychological limitations of human, interactions between human and instrument).
k5 Describe the impact (benefits and limitations) of technology and information management care (e.g., bar codes, electronic medical record, medication pumps, automatic alerts and alarms).
k6 Explain how authority gradients (horizontal, vertical) influence teamwork and patient safety.
| Factor K2. Concept of error and cause analysis|
k3 Distinguish among errors, adverse events, near misses, and hazards.
k4 Describe processes used in analyzing causes of error (e.g., root cause analysis).
| Factor S1. Error reporting and response to an error|
s1 Report errors using an organizational error reporting system.
s2 Accurately enter an error report.
s3 Analyze a case to find the causes of an error.
s4 Support and advise a peer who must decide how to respond to an error.
| Factor S2. Communication related to error|
s5 Disclose an error to a faculty member.
s6 Communicate observations or concerns related to hazards or errors with health care professionals.
s7 Communicate observations or concerns related to hazards or errors with an affected patient and his or her family.
| Factor S3. Resource utilization/evidence-based practice|
s8 Locate evidence reports related to clinical practice topics and guidelines to define uncertainty in nursing care.
s9 Use high quality electronic sources of health care information (e.g., online medical database).
s10 Use technology and information management tools (e.g., bar codes, electronic medical record, automatic alerts and alarms) to support safe processes of care.
| Factor S4. Safe nursing practice|
s11 Prevent and manage pressure ulcers.
s13 Use falls risk assessment tool to prevent falls.
s14 Give a blood transfusion according to transfusion policies for safe care.
s15 Administer drug to patient according to medication policies for safe care.
s21 Check patient’s identity accurately (e.g., a registration number, birth date, name).
|Factor S5. Infection prevention|
s12 Practice hand hygiene to prevent infection.
s18 Use standard infection control precautions for all patient encounters and other transmission precautions as appropriate.
s19 Use appropriate personal protective equipment (e.g., mask, goggles, gloves).
s20 Apply aseptic technique when inserting invasive devices as appropriate for patient care procedures (e.g., foley catheter insertion, intravenous catheter insertion, dressing).
| Factor S6. Precise communications during hand offs|
s16 Follow communication practices that minimize risks associated with hand offs between and among providers and across transitions in care.
s17 Document hand-off communication according to institutional policies.
| Factor A1. Patient safety promotion/prevention strategy|
a15 Technology and information management tools (e.g., bar codes, electronic medical record, automatic alerts and alarms) should be used appropriately to support safe processes of care.
a16 Value own role in preventing errors.
a17 Value nurses’ involvement in design, selection, implementation, and evaluation of information technologies to support patient care.
a18 A standardized procedure minimizes risks associated with hand off (e.g., transfer, shifts) within disciplines and across transitions in care.
| Factor A2. Responsibility of health care professionals for patient safety culture|
a4 Health care professionals should not tolerate uncertainty in patient care.
a6 Learning how to improve patient safety is an appropriate use of time in health programs in school.
a7 Health care professionals should routinely share information about medical errors and what caused them.
a8 Patient safety is a high priority to health care professionals.
| Factor A3. Error reporting and disclosing|
a9 Health care professionals should routinely report when certain errors occur.
a11 Health care professionals should disclose errors to an affected patient and his or her family.
a13 If there is no harm to the patient, there is no need to report an error.
a14 If I saw an error, I would keep it to myself.
| Factor A4. The components of patient safety culture|
a2 Making errors in health care is preventable.
a3 Health care professionals should make an effort to improve patient safety.
Characteristics of Study Participants (N = 354)
|Characteristic||Total||School 1||School 2||School 3||School 4||School 5||School 6|
|Gendera (n [%])|
| Female||336 (94.9)||54 (94.7)||20 (95.2)||83 (96.5)||53 (98.1)||63 (92.6)||63 (92.6)|
| Male||17 (4.8)||3 (5.3)||1 (4.8)||2 (2.3)||1 (1.9)||5 (7.4)||5 (7.4)|
|Agea (M [SD])||22.38 (1.59)||23.16 (1.87)||22.24 (1.51)||21.81 (1.16)||22.52 (1.60)||22.38 (1.22)||22.37 (1.87)|
|Semester (n [%])|
| Junior second semester||142 (40.1)||15 (71.4)||59 (68.6)||68 (100)|
| Senior first semester||212 (59.9)||57 (100)||6 (28.6)||27 (31.4)||54 (100)||68 (100)|
Item Loading From the Exploratory Factor Analysis (N = 354)
Confirmatory Factor Analysis (CFA) of Patient Safety Competency Self-Evaluation
|Factor/Item||Initial CFA||Final CFA|