Journal of Nursing Education

Research Briefs 

Early Indicators of NCLEX-RN Performance

Angela Schooley, MSN, RN; Jonathan Richard Dixon Kuhn, PhD

Abstract

Health Education Systems Incorporated (HESI) test results, course grades, and National Council Licensure Examination-Registered Nurse (NCLEX-RN®) outcomes of students in an associate degree nursing program at a midwestern public university were investigated. Statistical analysis revealed that introductory Fundamentals HESI test scores, more than either comprehensive HESI Exit Exam scores or other specialty HESI test scores, significantly predicted NCLEX-RN outcomes in this study (p < 0.05), while controlling for grade point average and high school percentile rank. In addition, of the general education courses and the nursing courses in the associate nursing program examined, Pediatric Nursing, Medical–Surgical Nursing, and Maternity Nursing course grades were found most statistically significantly influential of all the HESI test scores (p < 0.01). [J Nurs Educ. 2013;52(9):539–542.]

Ms. Schooley is Assistant Professor, Department of Nursing, and Dr. Kuhn is Associate Professor of Statistics, Department of Mathematics, Statistics and Physics, Purdue University North Central, Westville, Indiana.

This study was supported in part by a grant from the Mu Omega chapter of Sigma Theta Tau for novice nurse researchers. Sigma Theta Tau Mu Omega supported the writing of the report by providing resources for printing and revisions. The funding also facilitated the decision to submit the study for publication.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

The authors thank Dr. Yeou Lan Chen, Professor of Nursing, and Dr. Charlotte Strahm, Assistant Professor of Nursing, Purdue University North Central, for their mentorship and support. They also thank Amy Moberg for her assistance with data collection.

Address correspondence to Angela Schooley, MSN, RN, Assistant Professor, Department of Nursing, Purdue University North Central, 1401 South US Highway 421, Westville, IN 46391-9542; e-mail: aschool@pnc.edu.

Received: November 21, 2012
Accepted: February 27, 2013
Posted Online: August 19, 2013

Abstract

Health Education Systems Incorporated (HESI) test results, course grades, and National Council Licensure Examination-Registered Nurse (NCLEX-RN®) outcomes of students in an associate degree nursing program at a midwestern public university were investigated. Statistical analysis revealed that introductory Fundamentals HESI test scores, more than either comprehensive HESI Exit Exam scores or other specialty HESI test scores, significantly predicted NCLEX-RN outcomes in this study (p < 0.05), while controlling for grade point average and high school percentile rank. In addition, of the general education courses and the nursing courses in the associate nursing program examined, Pediatric Nursing, Medical–Surgical Nursing, and Maternity Nursing course grades were found most statistically significantly influential of all the HESI test scores (p < 0.01). [J Nurs Educ. 2013;52(9):539–542.]

Ms. Schooley is Assistant Professor, Department of Nursing, and Dr. Kuhn is Associate Professor of Statistics, Department of Mathematics, Statistics and Physics, Purdue University North Central, Westville, Indiana.

This study was supported in part by a grant from the Mu Omega chapter of Sigma Theta Tau for novice nurse researchers. Sigma Theta Tau Mu Omega supported the writing of the report by providing resources for printing and revisions. The funding also facilitated the decision to submit the study for publication.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

The authors thank Dr. Yeou Lan Chen, Professor of Nursing, and Dr. Charlotte Strahm, Assistant Professor of Nursing, Purdue University North Central, for their mentorship and support. They also thank Amy Moberg for her assistance with data collection.

Address correspondence to Angela Schooley, MSN, RN, Assistant Professor, Department of Nursing, Purdue University North Central, 1401 South US Highway 421, Westville, IN 46391-9542; e-mail: aschool@pnc.edu.

Received: November 21, 2012
Accepted: February 27, 2013
Posted Online: August 19, 2013

Effective program design encourages higher level thinking and prepares nursing students to manage patient care, prioritize nursing interventions, and delegate tasks (Herrman & Johnson, 2009; Rogers, 2010). Program evaluation efforts include examining course grades, National Council Licensure Examination-Registered Nurse (NCLEX-RN®) pass rates, preparatory tests such as the Health Education Systems Incorporated (HESI) specialty examinations, and the comprehensive HESI Exit Exam (Alameida et al., 2011; Morrison, Adamson, Nibert, & Hsia, 2008). The first-time NCLEX-RN pass rate in particular is one measure used to evaluate the ability of nursing programs to develop competent graduates. The Board of Directors of the National Council of State Boards of Nursing (2013) evaluates the NCLEX-RN pass rate standard every 3 years and implemented tighter passing standards in December 2012. Pressure is intense to meet the average national standard pass rate, which ranged from 86.2% to 87.6% during 2008 to 2011 (National League for Nursing [NLN], 2012; Pennington & Spurlock, 2010). Identifying factors influencing NCLEX-RN performance assists in establishing progression criteria, targets problem areas within the curriculum, and supports program graduation outcomes (Sewell, Culpa-Bondal, & Colvin, 2008).

Preparatory tests, which mimic the NCLEX-RN examination, are used in many nursing programs as a method of evaluating the readiness of senior-level nursing students for taking the NCLEX-RN examination. The HESI tests provide an external independent assessment of a student’s competency at the higher cognition level of application, analysis, and synthesis (Morrison, Nibert, & Flick, 2006). Many colleges adapt their progression policies according to data not only from course grades but also from the various specialty HESI tests and the HESI Exit Exam (Alameida et al., 2011; Morrison et al., 2008). A specialty HESI test is used to determine whether students meet competencies in a specific course and is given as a final test to evaluate mastery of course concepts. The HESI Exit Exam is an all-inclusive test, examining student competency across the nursing program, and measures students’ readiness to take the NCLEX-RN. Spurlock and Hunt (2008) found that first-time HESI Exit Exams predicted NCLEX-RN outcomes most accurately (87%) and matched observed (actual) NCLEX-RN outcomes when the first-time HESI Exit Exam cutoff score to pass the NCLEX-RN was assumed to be either 625 or 650 of a maximum of typically 1200 to 1250 points.

Morrison et al. (2006) suggested measuring the validity of a variety of specialty HESI tests by demonstrating test questions on any individual HESI test that measures an intended latent characteristic, such as the final grades of the course the individual HESI was designed to evaluate. This makes sense because if a specialty HESI test is used to determine whether a student meets competencies in a specific course, there should be a firm correlation between course grades and the specialty HESI test scores for the respective course. Nursing programs can identify problem areas by examining the relationship between course grades and HESI test scores.

Purpose

The purpose of this study was to examine the relationship between HESI test results, course grades, and NCLEX-RN outcomes of students in an associate degree nursing program. Two related research questions were addressed:

  • Which of the HESI test scores best predicted NCLEX-RN results?
  • Which course grades best predicted the HESI test scores?

Method

Sample

The sample included 306 students in a six-semester associate degree nursing program at a small public midwestern university who graduated from spring 2007 to fall 2010. The university is located in a rural community and is considered a commuter campus. The nursing program is accredited by the National League for Nursing Accrediting Commission. Students must maintain a 2.0 grade point average (GPA) and must receive at least a “C” in all nursing courses to remain in the nursing program.

Data Collection

After obtaining permission from the Institute Research Board, data were collected from three sources. Student NCLEX-RN outcomes were obtained from the state board of nursing NCLEX-RN quarterly reports from 2007 to 2010. Students’ scores from the HESI examinations (Fundamentals, Maternity, Pathophysiology, Medical–Surgical, Pediatrics, Psychiatric–Mental Health, and Exit) were obtained from the Registered Nurse Specialty Examination Summary Reports found on the Elsevier Web site, faculty section (Elsevier, 2012). Students’ academic records, including grades for the general education and nursing courses and high school percentile rank were obtained from the registrar’s office. The general education courses included English, Math, Sociology, Human Anatomy and Physiology I and II, Psychology, Nutrition, Chemistry, and Microbiology. The nursing courses included Introduction to Professional Nursing, Pharmacology, Human Development, Fundamentals Nursing, Health Assessment, Pediatric Nursing, Nursing Issues, Medical–Surgical Nursing, Maternity Nursing, Psychiatric–Mental Health Nursing, Pathophysiology, and Nursing Synthesis.

Ninety percent of students in the study completed the program in 3 years. Many students (84.7% [249 of 294]) passed the NCLEX-RN on first attempt; most students (93.5% [275 of 294]) ultimately passed the NCLEX-RN after a second attempt, although seven students had between three and five attempts. No more than seven students took any specialty HESI test a second time. Specialty HESI tests were retaken only if a student failed the course. The HESI Exit Exam was taken twice by 63 students, although 28 students took between three and five attempts. Students were allowed to retake the HESI Exit Exam a second time if a score of 850 was not achieved. Remediation was required for further retakes after the second failure.

Information was collected on all but 12 of 306 possible student NCLEX-RN outcomes. A median of 43 HESI scores, a median of 60 general education course grades, and a median of six grades from the nursing courses were missing from the data. In addition, 84 high school percentile ranks were missing.

Results

Two different statistical models were used to analyze the relationships between NCLEX-RN examination outcomes, the specialty HESI test scores and the HESI Exit Exam scores, and the grades of the general education courses and the nursing courses taken by students in the study. Both of these statistical models—the binary logistic model and the analysis of variance (ANOVA) model—are considered robust to deviations from nonnormality assumptions when fitting data (Agresti, 1996; Kutner, Nachtsheim, Neter, & Li, 2005). All analyses were performed using the SPSS 17.0 software package.

Which of the HESI Test Scores Best Predicted NCLEX-RN Results?

The first of the two research questions—which of the HESI test scores best predicted NCLEX-RN results—was analyzed with 16 different binary logistic models. All 16 models used the forward statistical likelihood selection technique. Of the 16 models, eight involved NCLEX-RN (first attempt) outcomes and eight involved NCLEX-RN (last attempt) outcomes. Different HESI predictor variables (specialty and first and last attempts), as well as the controlling covariates of high school rank, nursing GPA, and general education courses GPA, were used in the 16 binary logistic models of NCLEX-RN outcomes regressed on HESI variables. The HESI variables most frequently selected as significant in all 16 analyses were considered the most important predictors of NCLEX-RN outcomes.

The statistically significant (p < 0.05) predictors of either the first or last attempt of the NCLEX, from most important to least important, were found to be the Fundamentals HESI examination and the HESI Exit Exam (both last attempt and first attempt together), then Maternity HESI. The clearly dominant Fundamentals HESI was selected as significant in three quarters (12) of the 16 analyses, whereas the HESI Exit Exam and the Maternity HESI were selected as significant eight times and two times, respectively. The Fundamentals HESI was an especially important predictor of the NCLEX-RN (first attempt) outcomes because it was a significant predictor in all eight models that included NCLEX-RN (first attempt) outcomes. However, the Fundamentals HESI was significant predictor in only four of the eight models that included NCLEX-RN (last attempt) outcomes. The HESI Exit Exam (first or last attempt) appeared as a significant predictor in eight of 16 models that included NCLEX-RN (both first and last attempts) outcomes. The Maternity HESI appeared as a significant predictor in only two of eight models that included NCLEX-RN (first attempt) outcomes. All models were statistically significant (p < 0.05) and were rated as a fair to good fit to the data, as the value of the concordance statistic, c, was between 0.746 and 0.794 for all 16 models. Of all the HESI tests, the Fundamentals HESI, followed by the HESI Exit Exam and the Maternity HESI, were most predictive of NCLEX-RN outcomes.

Which Course Grades Best Predicted the HESI Test Scores?

One hundred eighty-four one-way ANOVA models, each with two controlling variables (overall program GPA and high school percentile rank), were created to answer the second research question—which course grades best predicted the HESI test scores? Final course grades (A, B, and C only) of four courses (Maternity Nursing, Pediatric Nursing, Psychiatric–Mental Health Nursing, and Medical–Surgical Nursing) were most significantly predictive of their corresponding specialty HESI test scores (Maternity, Pediatrics, Psychiatric–Mental Health, and Medical–Surgical Nursing). Of note, three nursing courses were in the top five significant predictors of a majority (at least six of eight) of the HESI tests (p < 0.01): Pediatrics Nursing, Medical–Surgical Nursing, and Maternity Nursing courses. The course grades of Fundamentals Nursing was only the fifth most significantly predictive of Fundamentals HESI test scores, F(2, 139) = 8.07; p = 0.00. The general education courses of Human Anatomy and Physiology I and II were the fourth and fifth most significant predictors of Maternity HESI, Pathophysiology HESI, and the first- and last-attempt HESI Exit Exam.

Discussion

The Fundamentals HESI test scores, which is the first specialty HESI test taken in the nursing program studied, were more predictive of NCLEX-RN outcomes than scores from the last HESI test taken (the HESI Exit Exam). The Fundamentals HESI appears able to identify the skills necessary for NCLEX-RN success at a very early juncture in the nursing student’s education program. Uyehara, Magnussen, Itano, and Zhang (2007) reported a fundamentals course as being predictive of NCLEX-RN results in a baccalaureate nursing program, but they did not investigate the link between an external independent test, such as the Fundamentals HESI test scores, and NCLEX-RN outcomes. After evaluation of a number of Assessment Technologies Institute (ATI) and NLN test scores (similar to HESI test scores), Ukpabi (2008) found that the ATI Critical Thinking, the NLN Percentile Adult II and, singularly, the Fundamental Percentile ATI test scores were all highly significantly (p < 0.01) predictive of NCLEX-RN outcomes. This early identification of struggling students can be used to guide remediation and progression efforts in the program.

It is intriguing that, despite the importance of Fundamentals HESI scores in predicting NCLEX-RN outcomes, the Fundamentals Nursing course grades were only the fifth most significant predictor, after the Pediatric Nursing, Medical–Surgical Nursing, Health Assessment, and Maternity Nursing course grades, of Fundamentals HESI test scores. This observation possibly indicated that a broad range of content, as specified by these other courses, was required of a Fundamentals Nursing course to meet the standards set by the external independent Fundamentals HESI test and, ultimately, of the national NCLEX-RN examination. Recently, the Fundamentals Nursing course in this study was divided into two semesters to spread course content over a longer period of time to allow students more time to digest the large amount of material and to develop stronger foundational skills.

The study findings indicate that the three nursing courses of Pediatrics Nursing, Medical–Surgical Nursing, and Maternity Nursing significantly influenced the HESI tests, particularly the key Fundamentals HESI, HESI Exit, and Maternity HESI tests. It appears these three courses provided students with the core nursing knowledge needed to not only succeed in the HESI tests but also the NCLEX-RN. To strengthen the curriculum, these courses could be used as benchmarks to determine whether learning outcomes are being met. Students performing poorly in these three courses could be directed to tutoring assistance to improve their scholastic performance or to develop a plan for improved performance. Reintroduction of pediatric, maternity, and medical–surgical nursing concepts in the final semester would support students’ successful performance on the NCLEX. The results of these data were presented to the nursing program’s curriculum committee to inform program changes.

Although significant (p = 0.01), Nursing Synthesis course grades were not one of the top five significant predictors of either first-attempt HESI Exit Exam test scores or last-attempt HESI test scores, despite Nursing Synthesis being the capstone nursing course in the program. This course focused on advanced medical–surgical concepts, such as high-risk cardiac care, organ transplants, brain injuries, and trauma. The current narrow scope of this course may be the reason why it did not significantly influence the broad-based, comprehensive HESI Exit Exam. Broadening the focus of this course to include acute care nursing across the lifespan and revisiting the important concepts in both the Maternity Nursing and Pediatric Nursing courses may better match the Nursing Synthesis course content with the standards of both the HESI Exit Exam and the NCLEX-RN.

Two general education courses had a significant impact on the HESI tests (p < 0.01). The appearance of Human Anatomy and Physiology I and II among the five most significant predictors in three of the specialty HESI tests demonstrated the importance of a firm understanding of the function of the human body and the need for strong performance in these two pivotal courses.

Conclusions

The Fundamentals HESI test (and to a lesser extent, the HESI Exit Exam and the Maternity HESI), as well as the benchmark courses of Pediatric Nursing, Medical–Surgical Nursing, and Maternity Nursing, specifically, could be used to inform progression policies. Poor performance on the first Fundamentals HESI test indicates a need for early intervention. This early identification provides faculty with ample time to provide remediation and build the needed skills to progress in the program. Mediocre course grades in Pediatric Nursing, Medical–Surgical Nursing, and Maternity Nursing would indicate a need for remediation and tutorial support to meet required competencies.

References

  • Agresti, A. (1996). An introduction to categorical data analysis. New York, NY: John Wiley & Sons.
  • Alameida, M.D., Prive, A., Davis, H.C., Landry, L., Renwanz-Boyle, A. & Dunham, M. (2011). Predicting NCLEX-RN success in a diverse student population. Journal of Nursing Education, 50, 261–267. doi:10.3928/01484834-20110228-01 [CrossRef]
  • Elsevier, Inc. (2012). Registered nurse speciality examination summary report. Retrieved from: https://evolve.elsevier.com/cs/Satellite/FacultyHome?Audience=Faculty
  • Herrman, J.W. & Johnson, A.N. (2009). From beta-blockers to boot camp: Preparing students for the NCLEX-RN. Nursing Education Perspectives, 30, 384–388.
  • Kutner, M.H., Nachtsheim, C.J., Neter, J. & Li, W. (2005). Applied linear statistical models (5th ed.). Boston, MA: McGraw-Hill.
  • Morrison, S., Adamson, C., Nibert, A. & Hsia, S. (2008). HESI exams: An overview of reliability and validity. CIN: Computers, Informatics, Nursing, 26, S39–S45. doi:10.1097/01.NCN.0000336442.01671.c6 [CrossRef]
  • Morrison, S., Nibert, A. & Flick, J. (2006). Critical thinking and test item writing (2nd ed.). Houston, TX: Health Education Systems.
  • National Council of State Boards of Nursing. (2013). Passing standard: Setting the NCLEX-RN passing standard. Retrieved from https://www.ncsbn.org/2630.htm
  • National League for Nursing, Board of Governors. (2012). The fair testing imperative in nursing education: A living document from the National League for Nursing. Retrieved from http://www.nln.org/aboutnln/livingdocuments/pdf/nlnvision_4.pdf
  • Pennington, T.D. & Spurlock, D. Jr.. (2010). A systematic review of the effectiveness of remediation interventions to improve NCLEX-RN pass rates. Journal of Nursing Education, 49, 485–492. doi:10.3928/01484834-20100630-05 [CrossRef]
  • Rogers, T.L. (2010). Prescription for success in an associate degree nursing program. Journal of Nursing Education, 49, 96–100. doi:10.3928/01484834-20091022-03 [CrossRef]
  • Sewell, J, Culpa-Bondal, F. & Colvin, M. (2008). Nursing program assessment and evaluation: Evidence based decision making improves outcomes. Nurse Educator, 33, 109–112. doi:10.1097/01.NNE.0000312176.67529.c6 [CrossRef]
  • Spurlock, D.R. & Hunt, L.A. (2008). A study of the usefulness of the HESI Exit Exam in predicting NCLEX-RN failure. Journal of Nursing Education, 47, 157–166. doi:10.3928/01484834-20080401-07 [CrossRef]
  • Ukpabi, C.V. (2008). Predictors of successful nursing education outcomes: A study of the North Carolina Central University’s nursing program. Educational Research Quarterly, 32(2), 30–39.
  • Uyehara, J., Magnussen, L., Itano, J. & Zhang, S. (2007). Facilitating program and NCLEX-RN success in a generic BSN program. Nursing Forum, 42, 31–38. doi:10.1111/j.1744-6198.2007.00063.x [CrossRef]

10.3928/01484834-20130819-08

Sign up to receive

Journal E-contents