Quality improvement (QI) in nursing education became a national goal in the early 2000s, when the Institute of Medicine (IOM) published reports on the current status of quality and safety in health care (IOM, 2003, 2010). The Quality and Safety Education for Nurses (QSEN) identified specific knowledge, skills, and attitudes (KSAs) related to QI competence that every BSN graduate should achieve (Cronenwett et al., 2007).
At the same time, multiple studies demonstrated that institutions employing nurses with a BSN or higher degree had better patient outcomes (Aiken, Clarke, Cheung, Sloane, & Silber, 2003; Aiken, Friese, Lake, Silber, & Sochalski, 2008; Estabrooks, Mididzi, Cummings, Ricker, & Giovannetti, 2005; Tourangeau et al., 2007; Van den Heede et al., 2009). Because of this, The Future of Nursing report (IOM, 2010), and institutions preferring to hire BSNs to meet Magnet® certification, the number of RN-to-BSN programs has increased dramatically. There are now more RN-to-BSN programs than there are traditional and accelerated BSN programs. In 2014, RN-to-BSN programs graduated 47,777 nurses, or 43% of the 111,634 graduates awarded a baccalaureate degree in nursing (AACN, 2015). This number increased in 2015, with 56,059 RN-to-BSN graduates comprising 46% of the baccalaureate degrees awarded (Fang, Li, Stauffer, & Trautman, 2016).
The second Essential emphasizes the need for nurses to understand “organizational and systems leadership,” and the QSEN QI competency provides the specific KSAs needed to demonstrate this competency. As RN-to-BSN students already have foundational nursing knowledge and skills, the RN-to-BSN curriculum provides an opportune time to integrate organizational leadership to improve quality and patient safety and QI competence. The purpose of this pilot study was to develop and test an assessment strategy to measure RN-to-BSN students' QI competence and their perceptions of QI knowledge and skill.
The literature was reviewed for ways to assess QI competence. Both the BSN Essential for Organizational and Systems Leadership and the QSEN QI competency suggest that nurses be able to use systems thinking by recognizing an event that prevents optimal patient outcomes and analyzing the causative factors to determine how to improve the quality of patient care (Cronenwett et al., 2007). Systems thinking is essential in QI because it broadens and expands the nurses' perspective to the systems level so that system failures can be recognized and improvements can be recommended (Dolansky & Moore, 2013). It is the ability to look not only at the individual level but at all levels within the system and to understand how nursing actions interact or counteract each other.
Kirkpatrick's evaluation framework was used to guide this pilot study (Kirkpatrick, 2009). The Kirkpatrick evaluation framework includes evaluation of four levels: reaction, knowledge, skill or behavior, and clinical outcomes. In this pilot, reaction was measured by student satisfaction of QI curricular content. Knowledge was measured using a QI knowledge test. Skill was measured by student rating of QI confidence and use of systems thinking. These assessment strategies were compiled and administered to RN-to-BSN students as a way to measure QI competence.
Institutional review board approval was granted from participating schools. An e-mail describing the pilot and the link to the survey was sent to a convenience sample of six AACN accredited programs, with 583 students completing their final semester of the program. The programs represented the south, midwest, southwest, and northern midwest regions. A total of 59 students completed the survey, reflecting a response rate of 10%.
Knowledge was measured by eight multiple choice questions from the Case Western Reserve University's “Take the Lead on Healthcare Quality Improvement,” offered as one of its massive open online courses (MOOCs). Questions selected were based on content that could be broad and basic enough for students at multiple sites with varying program content in QI. The questions were scored as correct or incorrect, and the total score ranged from 0 to 8.
QI skills were assessed using the Quality Improvement Knowledge Assessment Tool–Revised (QIKAT-R). The QIKAT-R assesses the ability to apply QI knowledge to systems-level problems using responses to clinical case scenarios. Participants read a case scenario and then answered three questions reflecting the aim (“What are we trying to improve?”), the measure (“How will we know that a change is an improvement?”), and the change (“What changes can be made that will lead to an improvement?”) (Langley et al., 2009). The answers were free text and were graded against a rubric. For this pilot study, the QIKAT-R case study responses were individually scored by three experts using a validated rubric. The final score was the average of the three scores and ranged from 0 to 9 points. The discriminative validity of the QIKAT-R is strong between excellent and poor responses, and the intraclass correlation coefficients is 0.66 overall for the 9-point scale (Singh et al., 2014).
The confidence to apply QI knowledge and skills was measured using the Quality Improvement Confidence Instrument (QICI) self-assessment scale. The QICI is a 31-item tool developed to measure confidence in applying specific types of QI skills (Hess, Johnston, Lynn, Conforti, & Holmboe, 2013). Participants completed a 5-point Likert-type scale (1 = not at all to 5 = very confident). A total score was calculated by adding up all the items and could range from 31 to 155. Construct validity was confirmed using confirmatory factor analysis, and factor correlations ranged from 0.63 to 0.85. Internal consistency and reliability estimates were high, ranging from 0.86 to 0.96 (Hess et al., 2013).
Another dimension of QI skill was measured using the Systems Thinking Scale (STS), a 20-item self-assessment tool that uses a 5-point Likert-type scale (0 = never to 4 = most of the time) that ranks the frequency with which the respondent agrees with the statement (Moore, Dolansky, Singh, Palmieri, & Alemi, 2010). Participants self-rated how often they believed the statements related to their practice regarding improvements. Items were added together to provide a total STS score, which could range from 0 to 80. Higher scores indicated a higher level of systems thinking. Reliability was acceptable with a Cronbach's alpha of .89 and test–retest reliability of 0.74. Discriminant validity and concurrent criterion-related validity were confirmed (Moore et al., 2010).
Participants' satisfaction with the QI content covered in their curriculum was measured using five questions that related to time spent on QI education, time spent on systems-thinking education, competence of faculty teaching QI, competence of faculty teaching systems thinking, and satisfaction with the evaluation methods used to assess QI knowledge and skills. Responses were based on a 5-point Likert-type scale (1 = extremely dissatisfied to 5 = extremely satisfied). A total score was calculated by summing the five scores and could range from 5 to 25.
To obtain a baseline measure of QI experience, students were first asked if they participated in a QI practicum or project in a setting where health care was delivered during the RN-to-BSN program. If so, they were asked to indicate which of the following components of QI they had worked on during the practicum: collected data, used a control or run chart, used a fishbone diagram or process map, and implemented an intervention. Participants were also asked if they had prior QI work-related experience or had taken QI courses as a part of their work experience.
Fifty-nine students responded to the survey, and 34 completed the majority of the survey. Respondents were Caucasian (78.1%), female (93.5%), and between the ages of 30 to 39 years old (37.5%). Forty-two percent of the respondents had been an RN for 1 to 3 years, and 71% worked in an acute care setting.
The majority (60%) of students replied they did not do any QI activities, such as collecting data or using a fishbone diagram or process map, during their QI practicum. The students answered an average of 66.4% of the eight QI questions correctly, and only 77% answered six or more questions correctly. The students scored low on QI skills, with an average score of 48.8% correct. The mean score was 40.5 on the STS, indicating a low level of systems thinking. The students' perception of their self-confidence in QI was high; 72.4% reported that they were somewhat confident or very confident in their QI skills. RN-to-BSN students in the final semester of their program were satisfied or very satisfied with the competence of faculty teaching QI (70%) and the competence of faculty teaching systems thinking (66.7%). The students were the least satisfied with the amount of time spent on systems thinking (66.7%).
This pilot study presents a method to assess QI competence in RN-to-BSN students. Although a small number of students stated that they had participated in a QI practicum, none of them stated that they had performed all the essential components of a QI project (i.e., collect data, use a control or run chart, use a fishbone diagram or process map, and implement an intervention). In addition, although it is possible that students did indeed use these methods, it is questionable as to whether they associated these methods with QI because 13% to 20% of the students stated that they did not know whether they had used some of the components of QI.
Less than half (45.8%) of the students achieved the minimum standard on the QI knowledge questions. The majority (68.6%) of the students missed the questions related to the use of a Plan-Do-Study-Act (PDSA) cycle even though this is a common strategy in all of the improvement models (i.e., Lean, Six Sigma, Model for Improvement). Based on their replies, students were confused about the difference between improvement and research, and only 50% answered the question about the purpose of the institutional review board correctly.
The majority of RN-to-BSN students were unable to reach competency on the QIKAT-R that measures QI skills. A score of 7 indicates competence, and only 45.8% reached this competency level. This means that less than half of the students were able to identify what the aim of the QI project would be, what needed to be measured, and one change that would be worth testing related to the scenario provided. These scores further support the scores found in the QI knowledge section of this assessment indicating that students struggled with methods and measures of QI.
Scores for overall confidence in QI indicated that students had confidence in their abilities related to QI projects. However, it is not surprising that the two individual items with the lowest percentage of self-reported confidence were related to collecting data (48.5%) and using control charts (57.6%), which also had the highest percentage rate of not at all confident, at 12.1% for each. The students scored the lowest on the QI knowledge question about using the PDSA cycle. This indicates that students are not only unsure how to use a methodical approach such as PDSA but also that they have a lower level of confidence in this area. It is interesting to note that although scores on QI knowledge and skills were low, students' perceptions of self-confidence in QI was high. This demonstrates a disconnect between the students' perceptions of their QI confidence and their actual QI knowledge and skills.
Although systems thinking is an essential component of the BSN Essentials on organizational and systems leadership related to improving quality and patient safety and the QSEN competency of QI, the students scored poorly on this measure. Systems thinking is a newer concept (Phillips & Statler, 2016), so it is possible that students are not receiving instruction to increase their systems thinking competence.
A limitation to this pilot study was the low response rate, which was likely due to the length of the survey and the questions on QI skills that required written responses. It must be emphasized that this is beginning work to develop and test assessment strategies to measure the second BSN Essential and therefore the results cannot be generalized. However, these assessment strategies could be used in RN-to-BSN programs for use in both formative and summative evaluation of students' QI knowledge, skills, and confidence.
Overall, the study provides an assessment strategy for further testing and refinement. The results of this pilot indicate that RN-to-BSN students believe they are prepared for QI projects upon graduation. However, the scores indicate the students lack QI knowledge and skills. According to Kirkpatrick's (2009) evaluation model, students must master knowledge before adopting the skill to confidently apply the knowledge and skills at the systems level.
The use of QI assessment strategies would provide baseline data on students' QI knowledge, skills, confidence, and systems thinking. The BSN Essentials suggest that students have the opportunity to implement a QI project and that the QI assessment strategies can be used at the end of the project to assess competency. In addition to a QI project, programs that have a face-to-face component and include simulation as an option may consider implementing the QI components with a simulation exercise.
Many resources to teach QI are available. The Institute for Healthcare Improvement (2016) offers free online education. Students work through QI modules on specific topics such as safety, leadership, and receive a certificate of completion for each module. The Case Western Reserve University's MOOC, “Take the Lead on Healthcare Quality Improvement,” is another course available free of charge for students to learn about and engage in QI (Coursera, 2016). Faculty would benefit from taking these courses. These free resources are valuable for schools that do not have faculty with QI expertise, given that students learn virtually using modules and apply what they learn to an experiential QI project.
With the dramatic increase in RN-to-BSN students, nursing education has an opportunity to integrate the second BSN Essential, organizational and systems leadership related to quality and safety and the QSEN QI competency. Emphasis on QI KSAs in RN-to-BSN programs will contribute to the nursing profession's efforts to ensure that nurses have the knowledge and skills to continually improve the care they deliver.
- Aiken, L.H., Clarke, S.P., Cheung, R.B., Sloane, D.M. & Silber, J.H. (2003). Educational levels of hospital nurses and surgical patient mortality. Journal of the American Medical Association, 12, 1617–1623. doi:10.1001/jama.290.12.1617 [CrossRef]
- Aiken, L.H., Friese, C.R., Lake, E.T., Silber, J.H. & Sochalski, J. (2008). Hospital nurse practice environment and outcomes for surgical oncology patients. Health Services Research, 43, 1145–1161. doi:10.1111/j.1475-6773.2007.00825.x [CrossRef]
- American Association of Colleges of Nursing. (2008). Essentials of baccalaureate education for professional nursing practice. Retrieved from http://www.aacn.nche.edu/education-resources/BaccEssentials08.pdf
- American Association of Colleges of Nursing. (2015). Amid calls for a more highly educated RN workforce, new AACN data confirm enrollment surge in schools of nursing. Retrieved from http://www.aacn.nche.edu/news/articles/2015/enrollment
- Coursera. (2016). Take the lead on healthcare quality improvement, Case Western Reserve University. Retrieved from https://www.coursera.org/course/hcqualityimprovement
- Cronenwett, L., Sherwood, G., Barnsteiner, J., Disch, J., Johnson, J., Mitchell, P. & Warren, J. (2007). Quality and safety education for nurses. Nursing Outlook, 55, 122–131. doi:10.1016/j.outlook.2007.02.006 [CrossRef]
- Dolansky, M. & Moore, S. (2013). Quality and safety education for nurses (QSEN): The key is systems thinking. The Online Journal of Issues in Nursing, 18(3). doi:10.3912/OJIN.Vol18No03Man01 [CrossRef]
- Estabrooks, C.A., Mididzi, W.K., Cummings, G.C., Ricker, K.L. & Giovannetti, P. (2005). The impact of hospital nursing characteristics on 30-day mortality. Nursing Research, 45, 74–84.
- Fang, D., Li, Y., Stauffer, D.C. & Trautman, D.E. (2016). 2015–2016 enrollment and graduations in baccalaureate and graduate programs in Nursing. Washington, DC: American Association of Colleges of Nursing.
- Hess, B.J., Johnston, M.M., Lynn, L.A., Conforti, L.N. & Holmboe, E.S. (2013). Development of an instrument to evaluate residents' confidence in quality improvement. The Joint Commission Journal on Quality and Patient Safety, 39, 502–510. doi:10.1016/S1553-7250(13)39066-7 [CrossRef]
- Institute for Healthcare Improvement (2016). IHI open school. Retrieved from http://www.ihi.org/education/ihiopenschool/Pages/default.aspx
- Institute of Medicine. (2003). Health professions education: A bridge to quality. Washington, DC: The National Academies Press.
- Institute of Medicine. (2010). The future of nursing: Leading change, advancing health. Washington, DC: The National Academies Press.
- Kirkpatrick, D. (2009). The Kirkpatrick Model. Retrieved from http://www.kirkpatrickpartners.com/OurPhilosophy/TheKirkpatrickModel/tabid/302/Default.aspx
- Langley, G.J., Moen, R.D., Nolan, K.M., Nolan, T.W., Norman, C.L. & Provost, L.P. (2009). The improvement guide: A practical approach to enhancing organizational performance (2nd ed.). San Francisco, CA: Jossey-Bass.
- Moore, S.M., Dolansky, M.A., Singh, M., Palmieri, P. & Alemi, F. (2010). The Systems Thinking Scale. Unpublished manuscript. Retrieved from https://nursing.case.edu/media/nursing/pdf-dox/STS_Manual.pdf
- Phillips, J.M. & Stalter, A.M. (2016). Integrating systems thinking into nursing education. The Journal of Continuing Education in Nursing, 47, 395–397. doi:10.3928/00220124-20160817-05 [CrossRef]
- Singh, M.K., Orginc, G., Cox, K., Dolansky, M., Brandt, J., Morrison, L. & Headrick, L. (2014). The quality improvement knowledge application tool revised (QIKAT-R). Academic Medicine, 89, 1386–1391. doi:10.1097/ACM.0000000000000456 [CrossRef]
- Tourangeau, A.E., Doran, D.M., Hall, L.M., Pallas, L.O., Pringle, D., Tu, J.V. & Cranley, L.A. (2007). Impact of hospital nursing care on 30-day mortality for acute medical patients. Journal of Advanced Nursing, 57, 32–44. doi:10.1111/j.1365-2648.2006.04084.x [CrossRef]
- Van den Heede, K., Lesaffre, E., Diya, L., Vleugels, A., Clarke, S.P., Aiken, L.H. & Sermeus, W. (2009). The relationship between inpatient cardiac surgery mortality and nurse numbers and educational level: Analysis of administrative data. International Journal of Nursing Studies, 46, 796–803. doi:10.1016/j.ijnurstu.2008.12.018 [CrossRef]