Identifying the indicators that predict students’ potential for success on the National Council Licensure Examination-Registered Nurse (NCLEX-RN®) is critical because passing it is required to practice nursing. First-time NCLEX-RN pass rates often serve as an outcome measure in the evaluation of program quality (Beeson & Kissling, 2001; Fraher, Belsky, Carpenter, & Gaul, 2008). Predictors have included pre-entry admission criteria, academic performance in various courses, performance on standardized achievement tests, and demographic characteristics. Meta-analyses conducted by Campbell and Dickson (1996) and Grossbach and Kuncel (2010) have shown that there is considerable variation in the reported predictive value of numerous variables associated with passing the NCLEX-RN.
The most productive application of variables related to NCLEX-RN performance may be in a model that considers a multiple set of predictors. The applicability of any single variable may be questionable, but generally the value is enhanced when aggregated. The variables are typically combined in terms of a multivariate regression model, such as discriminant analysis or logistic regression. A discriminant analysis by Haas, Nugent, and Rule (2004) using verbal and quantitative SAT scores, age, race, campus location, and gender correctly identified 71% of those who passed and 70% of those who failed. A better and equally proportional hit rate was achieved by Beeman and Waterhouse (2001) on the basis of a discriminant analysis with 7 predictors; they correctly identified 94% of those who passed and 92% of those who failed. However, not all multivariate studies are equally successful. On the basis of logistic regression, McGahee, Gramling, and Reid (2010) correctly predicted 99% of those who passed but only 29% of those who failed. A logistic regression by Seldomridge and DiBartolo (2004) likewise found it more difficult to predict who will fail, compared with predicting who will pass.
An alternative to the use of regression models is to simply combine the variables without any weights. Barkley, Rhodes, and Dufour (1998) pointed to practitioners’ need for instruments that do not require complicated regression-based formulas for scoring. Frequently, just adding up the predictors into a simple sum score (termed unit weighting) can produce results that are as predictive as regression-based models when applied to another sample (Cohen, 1990; Dawes, 1979; Kerby, 2003; Schmidt, 1971; Wilks, 1938).
The Risk Assessment Profile, Strategies for Success (RAPSS) (Breckenridge, 1992, 1995, 2004, 2006a, 2006b, 2008, 2009) assesses students’ risk of failure on the basis of a simple sum of risk factors. The RAPSS evolved over two decades and consists of 13 items. Risk threatens the ability of students to progress in a nursing program, graduate, and pass the NCLEX-RN. The risk factors associated with the completion of a Bachelor of Science in Nursing (BSN) degree are not necessarily the same as those associated with NCLEX-RN performance (Fraher et al., 2008).
The RAPSS instrument has been used at a baccalaureate nursing program to identify at-risk BSN candidates and target interventions. The investigators address the predictive potential for a 13-item protocol in which the items are unit weighted to predict whether a graduate of a BSN degree at a private university will pass or fail the NCLEX-RN on the first attempt. To better understand the variables, they were also studied using logistic regression.
A prediction design (Burns & Grove, 2009) was used with the RAPSS, a criterion-referenced instrument (Waltz, Strickland, & Lenz, 1984), with BSN graduates. The RAPSS aims to identify students at risk for failing the NCLEX-RN.
A university institutional review board approved the study. A convenience sample of BSN graduates (N = 255) from a private, mid-Atlantic university was included; 133 passed the NCLEX-RN on the first attempt and 122 failed on the first attempt. Students were enrolled full time (62%) or part time (38%). The age of the sample students ranged from 21 to 57 years; the average age was 28 years (SD = 9 years). The sample was 85% women and 61% White. The non-Hispanic ethnic distribution was 151 White, 71 Black, 14 Asian, and 15 unknown; there were 4 Hispanics (White). Approximately 35% were minorities. Passing or failing the NCLEX-RN for the first time was transcribed from state board reports. Research assistants and one researcher transcribed data from student files to the RAPSS; 100% inter-rater agreement was achieved.
The RAPSS items are scaled ordinally:
- Speaks English as a second language (yes = 1; no = 0).
- Works while attending school full time (works ⩾ 40 hours/week = 2; works ⩽ 20 hours/week = 1; does not work = 0).
- Marital status (married/partner/divorced/separated/widowed = 0; single = 1);
- Support to care for children while at school (no or little support for childcare = 1; has support or does not have children = 0).
- Completed college preparatory algebra (no high school algebra = 2; grade lower than B or ⩽ 79 = 1; grade B or higher or ⩾ 80 = 0).
- Completed college preparatory biology (no high school biology = 2; grade lower than B or ⩽ 79 = 1; grade B or higher or ⩾ 80 = 0).
- Completed college preparatory chemistry (no high school chemistry = 2; grade lower than B or ⩽ 79 = 1; grade B or higher or ⩾ 80 = 0).
- Took prerequisite college science courses again to obtain a C or C+ grade (repeated 4 science courses = 4; repeated 3 science courses = 3; repeated 2 science courses = 2; repeated 1 science course = 1; no repeat of science course = 0).
- Scored below acceptable level in pre-entrance standardized test (SAT < 1000; ACT < 21; National League for Nursing Pre-Admission Examination < 40; Nursing Entrance Test composite < 63; Test of Essential Academic Skills < 74%: yes = 1; no = 0).
- Undergraduate science grade point average (GPA) (< 3.0 = 1; ⩾ 3.0 = 0).
- Undergraduate GPA prior to the nursing major (< 3.0 = 1; ⩾ 3.0 = 0).
- First in family to attend college or university (yes = 1; no = 0).
- Financial status based on family income and size (poverty level = 1; higher than poverty level = 0); ⩽ $20,800 in a 1-person family to $71,240 in an 8-person family = 1; above poverty level = 0).
Table 1 presents the distribution of the sample of the 13 predictors.
Table 1: Distribution of 13 Items Used to Predict NCLEX-RN Pass or Fail (N = 255)
Using SPSS version 18 software, statistical models for ordinal level predictors are reported but include parametric procedures for comparison. The 13 items were summed into a total score (Table 2). The group that failed the NCLEX-RN had greater risk scores in terms of the mean rank (81.97 versus 51.19), median (6.93 versus 4.23), and mean (6.93 versus 4.23). The difference in mean ranks was statistically significant (Mann–Whitney U = 825.50, p = 0.000, effect size = 0.41). The t test for the equality of means revealed a statistically significant difference and produced a large effect size (t  = 5.20, p > 0.001, Cohen’s d = 0.90).
Table 2: Associations between the NCLEX-RN Pass/Fail Predictors and Criterion in the Overall Sample (N = 255) and the Reduced Sample (n = 122)
The predictive value of each of the 13 items was also tested; the univariate relationship of each predictor with the criterion was examined using asymmetric Sommer’s d, Spearman rho statistic, and point biserial coefficients. The predictors are scaled so that a higher value reflects greater risk of failure; therefore, the criterion was coded in the same direction: 1 = fail and 0 = pass. Coded in this manner, positive relationships indicate greater association with failure. Missing values existed on all but one predictor, and only 122 cases had legitimate values on all 13 variables. To determine whether the subsample of 122 was systematically different from the overall sample of 255, the correlations were computed on all available cases and on the subsample of 122, with no missing values. As seen in Table 2, the values of d, rho, and the point biserial r were similar and did not differ markedly in the overall and reduced samples.
In both samples, the best single predictor was the science GPA. The other strong predictors were family income, undergraduate GPA prior to the nursing major, and having to retake college science courses. All four predictors achieved statistical significance in overall and reduced (n = 122) samples. English as a second language also achieved statistical significance in both samples; however, the magnitude of the relationship was smaller. Being the first in the family to attend college was a statistically significant predictor in the larger sample, but it did not reach significance in the reduced sample of 122. The magnitude of the association was fairly comparable (0.16 and 0.12, respectively). The variables that failed to achieve statistical significance were associated with the criterion in the correct direction.
Several logistic regression models were explored. The goal of logistic regression was to develop the best fitting model to describe the relationship between the passing and failing of the NCLEX-RN and the 13 predictors. Initially, all 13 predictors were submitted in a single step into a binary logistic regression, coding passed as 0 and failed as 1. Using all 13 variables, approximately 94% of students who passed and 68% of students who failed were classified correctly using this model when equal probability of failure and passing are assumed (Cox & Snell R2 = 0.43; Nagelkerke R2 = 0.60). With all 13 predictors in the model, the Wald chi-square tests indicate that only three of the predictors have beta weights that are statistically different from zero (p < 0.05): family income (odds ratio = 13.33), English as second language (odds ratio = 6.25), and science GPA (odds ratio = 4.62). Other predictors (support to care for children, retaking science courses, marital status) had low probabilities, but not at the p < 0.05 cutoff.
Next, all six predictor reduction options available in SPSS were tried; specifying p = 0.05 for entry and p = 0.05 for removal produced the same results. Under all stepwise options, only the three variables shown in Table 3 were retained. The variable providing the best discrimination was family income (poverty or nonpoverty) followed by science GPA and retaking science courses. The classification of actual and predicted pass or fail group membership declined slightly with only three variables instead of 13, falling from 85% to 83%. The R2 approximations with three predictors were Cox and Snell = 0.37 and Nagelkerke = 0.52. The prediction was more accurate about students who passed (93%) than those who failed (63%). The reduction in accuracy with three instead of 13 predictors was more pronounced for students who failed than for those who passed.
Table 3: Predictors of NCLEX-RN Pass/Fail Retained in Logistic Regression (n = 122)
These three variables were totaled into a simple sum score, and comparisons were conducted between students who passed and those who failed. With three instead of 13 predictors, the number of cases without any missing values increased from 122 to 164. To allow for an exact comparison with the 13 predictor total score model, data were examined in the cohort of 122. In the sample of 122 (81 passed, 41 failed), the three-item-based risk scores differed in the correct direction on mean rank (89.50 versus 47.33), mean (2.05 versus 0.72), and median (2.00 versus 1.00). The difference was statistically significant using the Mann–Whitney U test (U = 512.5, p < 0.001; effect size = 0.61) and the t test, (t  = 7.34, p < 0.001, Cohen’s d = 1.30). In the cohort of 164 (100 passed, 64 failed), the results were similar. Differences existed on the mean rank (115.75 versus 61.22), mean (1.98 versus 0.70), and median (2.00 versus 1.00), and again statistical significance was reached on the basis of non-parametric and parametric tests (U = 1072.00, p < 0.001, effect size = 0.59; t  = 8.55, p < 0.001, Cohen’s d = 1.21). In the samples of 122 and 164, the difference in risk scores between students who failed and students who passed was statistically significant with both the nonparametric and parametric tests. The effect sizes were similar in the smaller and larger samples.
The RAPSS instrument differentiated between the completers of a bachelor’s undergraduate degree program who pass and those who fail the NCLEX-RN on the first attempt. Although all of the 13 predictors included in the instrument had a theoretical basis for the prediction of failure or success of graduating, not all were significant predictors of NCLEX-RN performance. An efficient model consists of the sum of (a) family income (poverty or nonpoverty), (b) science GPA prior to the nursing major, and (c) whether the student had to repeat college science courses. Using these three predictors instead of all 13 in a sum improved discrimination. In a logistic regression model, these three variables correctly identify 93% of the candidates who passed the NCLEX-RN and 63% of those who failed it. This compares favorably with other prediction studies (McGahee et al., 2010; Seldomridge & DiBartolo, 2004), although some did better (Haas et al., 2004). It was more difficult to predict failure than success, consistent with previous studies.
The investigators were struck by the predictive power of poverty, a variable that has not received much attention in previous studies attempting to predict NCLEX-RN pass rates. Early identification of at-risk nursing students assists with identifying interventions, and poverty is a variable known at admission. In the current study, poverty income is not a proxy for minority status, given that the poverty rate was 60% among White participants and 63% overall. Disparities by income level have been documented on many other standardized tests (McCallum & Demie, 2001; Nyhan & Alkadry, 1999; Sackett, Borneman, & Connelly, 2008). The role of income on NCLEX-RN performance is not surprising, but it is curious that it remains in the logistic regression with the largest odds ratio after controlling for subject mastery variables. It may serve as a proxy variable for unmeasured factors associated with performing well on the NCLEX-RN. For instance, low-income students do not have the resources for examination preparation courses or may experience generalized test or stereotype anxiety on high stakes tests.
The availability of the necessary predictors and ease of scoring are a consideration. Investigators using the RAPSS could use a sum score weighted in terms of weights based on their own logistic regression, or they could sum the items without any weights. The results of the logistic regression should be considered as merely a rough guide, given the small sample size (Peduzzi, Concato, Kemper, Holford, & Feinstein, 1996), and the investigators recommend that those interested in using the RAPSS simply add the items without weighting. Because the findings are based on experiences at a single university, users are urged to validate the system at their own institutions. The investigators coded the predictor marital status, single = 1 and all others as 0; the literature varies on this risk factor (Yess, 1981).
The RAPSS is a simple-to-score 13-item scale that can be used to evaluate students’ potential for success on the NCLEX-RN following graduation from a baccalaureate nursing program. Not all 13 predictors were significant; the number of necessary predictors may be reducible to three, which provided similar results found in other samples. The most powerful predictor was income at poverty level, followed by science GPA.
Identifying indicators that predict students’ potential for success on the NCLEX-RN is extremely critical because passing it is required to practice nursing. Risk factors associated with the completion of a nursing degree are not necessarily the same as those associated with performance on the NCLEX-RN (Fraher et al., 2008). The RAPSS was originally developed on application to a nursing program to predict graduation, but these results demonstrate that it can be used to predict NCLEX-RN performance.
- Barkley, T.J., Rhodes, R.S. & Dufour, C.A. (1998). Predictors of success on the NCLEX-RN among baccalaureate nursing students. Nursing and Health Care Perspectives, 19, 132–137.
- Beeman, P.M. & Waterhouse, J.K. (2001). NCLEX-RN performance: Predicting success on the computerized examination. Journal of Professional Nursing, 17, 158–165. doi:10.1053/jpnu.2001.24860 [CrossRef]
- Beeson, A. & Kissling, G. (2001). Predicting success for baccalaureate graduates on the NCLEX-RN. Journal of Professional Nursing, 17, 121–127. doi:10.1053/jpnu.2001.23382 [CrossRef]
- Breckenridge, D.M. ( 1992, June. ). The new access student in response to the decrease in student enrollment in schools of nursing. Paper presented at the Forum on Doctoral Education. , University of Maryland, Baltimore, MD. .
- Breckenridge, D.M. ( 1995, March. ). Risk factor assessment parameters for the new access student. Paper presented at the Forum on Doctoral Education. , University of Maryland, Baltimore, MD. .
- Breckenridge, D.M. ( 2004, May. ). Meeting the challenges of the new access student to overcome risk factors and succeed in nursing education. Poster session presented at the Twelfth Annual Conference for Nurse Educators in Practice Settings and Schools of Nursing, Saint Anselm College and Dartmouth-Hitchcock Medical Center. , Manchester, NH. .
- Breckenridge, D.M. (2006a). Hospital diversity initiative for new access students: NETS, a career ladder program. Pennsylvania Nurse, 61(4), 22–23.
- Breckenridge, D.M. ( 2006b, September. ). The new access student. Paper presented at the National League for Nursing Educational Summit. , New York, NY. .
- Breckenridge, D.M. ( September, 2008. ). Risk Assessment Profile and Strategies for Success (RAPSS). Poster presented at the National League for Nursing Education Summit. , San Antonio, TX. .
- Breckenridge, D.M. ( 2009, November. ). Language assessment strategies to increase nursing workforce for culturally competent care. Paper presented at the 40th Biennial Convention of Sigma Theta Tau International, Honor Society of Nursing. , Indianapolis, IN. .
- Burns, N. & Grove, S.K. (2009). The practice of nursing research: Appraisal, synthesis, and generation of evidence (6th ed.). St. Louis, MO: Saunders Elsevier.
- Campbell, R.C. & Dickson, C.J. (1996). Predicting student success: A 10-year review using integrative review and meta-analysis. Journal of Professional Nursing, 12, 47–59. doi:10.1016/S8755-7223(96)80074-3 [CrossRef]
- Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304–1312. doi:10.1037/0003-066X.45.12.1304 [CrossRef]
- Dawes, R.M. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34, 571–582. doi:10.1037/0003-066X.34.7.571 [CrossRef]
- Fraher, E.P., Belsky, D.W., Carpenter, J.M. & Gaul, K. (2008, October). A study of associate degree nursing program success: Evidence from the 2002 Cohort. Chapel Hill, NC: The Cecil G. Sheps Center for Health Services Research at the University of North Carolina. Retrieved from http://www.shepscenter.unc.edu/hp/publications/NCCCS_ADN_Report.pdf
- Grossbach, A. & Kuncel, N.R. (2010). The predictive validity of nursing admission measures for performance on the National Council Licensure Examination: A meta-analysis. Journal of Professional Nursing, 27, 124–128. doi:10.1016/j.profnurs.2010.09.010 [CrossRef]
- Haas, R.E., Nugent, K.E. & Rule, R.A. (2004). The use of discriminant function analysis to predict student success on the NCLEX-RN. Journal of Nursing Education, 43, 440–446.
- Kerby, D.S. (2003). CART analysis with unit-weighted regression to predict suicidal ideation from Big Five traits. Personality and Individual Differences, 35, 249–261. doi:10.1016/S0191-8869(02)00174-5 [CrossRef]
- McCallum, I. & Demie, F. (2001). Social class, ethnicity and educational performance. Educational Record, 43, 147–159. doi:10.1080/00131880110051146 [CrossRef]
- McGahee, T.W., Gramling, L. & Reid, T.F. (2010). NCLEX-RN success: Are there predictors?Southern Online Journal of Nursing Research, 10(4), Article 13. Retrieved from http://www.resourcenter.net/images/SNRS/Files/SOJNR_articles2/Vol10Num04Art13.html
- Nyhan, R.C. & Alkadry, M.G. (1999). The impact of school resources on student achievement test scores. Journal of Education Finance, 25, 211–228.
- Peduzzi, P., Concato, J., Kemper, E., Holford, T.R. & Feinstein, A.R. (1996). A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology, 49, 1373–1379. doi:10.1016/S0895-4356(96)00236-3 [CrossRef]
- Sackett, P.R., Borneman, M.J. & Connelly, B.S. (2008). High-stakes testing in higher education and employment: Appraising the evidence for validity and fairness. American Psychologist, 63, 215–227. doi:10.1037/0003-066X.63.4.215 [CrossRef]
- Schmidt, F.L. (1971). The relative efficiency of regression and simple unit predictor weights in applied differential psychology. Educational and Psychological Measurement, 31, 699–714. doi:10.1177/001316447103100310 [CrossRef]
- Seldomridge, L. & DiBartolo, M. (2004). Can success and failure be predicted for baccalaureate graduates in the computerized NCLEX-RN?Journal of Professional Nursing, 20, 361–368. doi:10.1016/j.profnurs.2004.08.005 [CrossRef]
- Waltz, C.F., Strickland, O.L. & Lenz, E.R. (1984). Measurement in nursing research. Philadelphia, PA: F.A. Davis.
- Wilks, S.S. (1938). Weighting systems for linear functions of correlated variables when there is no dependent variable. Psychometrika, 3, 23–40. doi:10.1007/BF02287917 [CrossRef]
- Yess, J.P. (1981). The influence of marriage on community college student achievement in specific programs of study. Research in Higher Education, 14, 103–111. doi:10.1007/BF00976288 [CrossRef]
Distribution of 13 Items Used to Predict NCLEX-RN Pass or Fail (N = 255)
|Speaking English as second language|
| Yes||46 (18)|
| No||209 (82)|
| ⩾ 20 hours/week while attending school full time||69 (27.1)|
| ⩾ 40 hours/week while attending school part time and full time||77 (30.2)|
| Not working||105 (41.2)|
| Missing||4 (1.6)|
| Married, partner, divorced, separated, or widowed||72 (28.2)|
| Single||177 (69.4)|
| Missing||6 (2.4)|
|Support to care for children while at school|
| Little or no support for childcare||14 (5.5)|
| Have no children||226 (88.6)|
| Missing||15 (5.9)|
|Retaking prerequisite science courses to obtain C or C+ grade|
| Repeat 3 science courses||1 (0.4)|
| Repeat 2 science courses||13 (5.1)|
| Repeat 1 science course||49 (19.2)|
| No repeat science courses||187 (73.3)|
| Missing||5 (2)|
|Score on pre-entrance standardized test (SAT, ACT, National League for Nursing Pre-Admission Examination, Nursing Entrance Test, Test of Essential Academic Skills)|
| Average or above average||73 (28.6)|
| Not at acceptable level||80 (31.4)|
| Missing||102 (40)|
|Science grade point average|
| < 3.0||83 (32.5)|
| ⩾ 3.0||172 (67.5)|
|Overall grade point average prior to nursing major|
| < 3.0||110 (43.1)|
| ⩾ 3.0||143 (56.1)|
| Missing||2 (0.8)|
|First in family to attend college or university|
| Yes||52 (20.4)|
| No||127 (49.8)|
| Missing||76 (29.8)|
| Above poverty level||58 (22.7)|
| At or below poverty level||107 (42)|
| Missing||90 (35.3)|
|Completed college preparatory courses|
| B or ⩾ 80||113 (44.3)|
| Below B or ⩽ 79||79 (31)|
| No high school||7 (2.7)|
| Missing||56 (22)|
| B or ⩾ 80||118 (46.3)|
| Below B or ⩽ 79||73 (28.6)|
| No high school||6 (2.4)|
| Missing||58 (22.7)|
| B or ⩾ 80||115 (45)|
| Below B or ⩽ 79||74 (29)|
| No high school||10 (3.9)|
| Missing||56 (22)|
Associations between the NCLEX-RN Pass/Fail Predictors and Criterion in the Overall Sample (N = 255) and the Reduced Sample (n = 122)
|Predictor||n||Asymmetric Sommer’sd||Spearmanrho||Point Biserialr|
|English as a second language||255||0.14**||0.18**||0.18**|
|Working while attending school||251||0.09||0.08||0.08|
|Support to care for children||240||0.06||0.13*||0.13*|
|Completed college preparatory algebra||199||0.06||0.07||0.08|
|Completed college preparatory college biology||197||0.05||0.05||0.07|
|Completed college preparatory chemistry||199||0.12||0.12||0.11|
|Retaking science courses||250||0.26***||0.29***||0.30***|
|Pre-entry standardized test||153||0.05||0.05||0.05|
|Science grade point average||255||0.38***||0.41***||0.41***|
|Overall grade point average prior to nursing major||253||0.38***||0.38***||0.38***|
|First in family to attend college||179||0.16*||0.17*||0.12|
Predictors of NCLEX-RN Pass/Fail Retained in Logistic Regression (n = 122)
|B||Wald (df = 1)||p||Odds Ratio|
|Retaking science courses||1.36||3.90||0.048||3.91|
|Science grade point average||2.04||10.98||0.001||7.69|