As part of their ongoing exploration of factors contributing to the success or failure of graduate nurses taking the National Council Licensure Examination for RNs (NCLEX-RN), faculty in a school of nursing at a private college in Oregon interviewed 19 recent graduates of their baccalaureate program. The purpose of this study was to uncover themes that may help faculty understand the testing experience from the graduates' point of view and help future students increase their likelihood of passing the examination.
It is important for nursing faculty to pay close attention to individual as well as institutional pass-fail rates on the NCLEX-RN for many reasons. Students who fail the examination may experience difficulty with self-esteem, as well as economic hardship if professional practice opportunities are lost (Billings et al., 1996). In addition, the authors' state requires ongoing minimum pass rates to maintain program accreditation. Inability to meet this minimum requirement threatens program viability from both accreditation and admission standpoints.
There is a wealth of literature about quantitative predictors of success on the NCLEX-RN (Barkley, Rhodes, & Dufour, 1998; Campbell & Dickson, 1996; Endres, 1997; Krupa, Quick, & Whitley, 1988; Payne & Duffey, 1986; Wall, Miller, & Widerquist, 1993; Waterhouse, Carroll, & Beeman, 1993). Measurable indicators of success have included standardized test scores, grade point averages in nursing and science courses, clinical nursing grades, and National League for Nursing (NLN) comprehensive examination scores. Faculty at the authors' institution have seen a similar pattern of predictive variables among their graduates.
What has been studied less frequently are nonacademic, qualitative predictors of success or failure. The few studies that have examined these variables, such as test anxiety, self-esteem, and self-perceived competence, generally have shown these to be weaker predictors when compared to more robust, quantitative measures (Arathuzik & Aber, 1998, Dell & Valine, 1990; Poorman & Martin, 1991). However, feeling there might be an unidentified "something" contributing to declining first-time pass rates for their graduates, the authors decided to try to understand the experience from the graduate's point of view.
The evaluation committee of the School of Nursing chose to limit the study to 20 participants, approximately 25% of the 1998 graduating class. A random numbers table was used to select 10 prospective participants who had passed the NCLEX-RN at the first sitting. AU 10 graduates who had failed the first time were included in the sample. The final sample consisted of 19 participants, 10 who passed and 9 who failed on the first testing (one participant dropped out of the study due to relocation). The sample included 15 women and 4 men, with a mean age of 29.7 and a range of 21 to 53. Also included in the study were 3 students who identified English as their second language, all of whom were in the no-pass group.
Data Collection. A research assistant contacted the participants approximately 2 weeks prior to the planned phone interviews, inviting their voluntary participation and scheduling a time at their convenience. All 19 readily agreed to participate.
The authors and a third faculty member used a semistructured interview guide during the interviews to increase standardization, but allowed ample time and space for unstructured comments. The interview guide (available on request) was developed by members of the evaluation committee, and included student, faculty, and administrative input. The authors took extensive notes during the interviews and aUowed time for informants to supplement their responses to the 15 interview questions with observations and suggestions not directly related to the questions. The interviewers obtained informed consent prior to beginning the interviews and reassured the participants of the voluntary and confidential nature of their participation in the study. The faculty development, research, and review committee of the school of nursing approved the study prior to its inception.
Data Analysis. After transcription of the text-based data, the interviewers, both independently and as a group, identified common data clusters or themes (Streubert & Carpenter, 1999). As verification of credibility, the authors offered the data to two administrators (the dean of the school of nursing and the campus director) for their independent assessment of relevant themes and also solicited feedback from two study participants in each group (Denzin & Lincoln, 1994). The results of the study represent a mix of direct quotes and derived themes arising from each of the 15 questions. Limited quantitative analysis (descriptive statistics and comparison of means) was accomplished on questions best explicated by that type of data.
Preparing for and Experiencing the NCLEX-RN
The first set of questions focused on the participants' experiences around the time of final preparation for and taking the examination.
Number of Weeks Between Graduation and Taking the Examination. Because anecdotal data had suggested that some graduates felt pressured by family, peers, and employers to take the examination earlier than they would have liked, the authors asked about the number of weeks between graduation and taking the NCLEX-RN. The mean number of weeks between graduation and taking the examination was 3.77 (SD = 1.79) for the participants who failed on the first attempt, and 5.55 (SD = 3.21) for the group who passed. Although this showed some association between taking the examination early and failing, the difference was not statistically significant (t(18) = -1.46, p = .16).
Student Preparation for the NCLEX-RN. When asked the question "How did you prepare for the NCLEX-RN?," 3 of the 9 participants who failed on the first attempt readily admitted they had not prepared as weU as they should have. One participant who did not pass "glanced'' through the review book and another "crammed'' with the book. Although 18 of the 19 participants had taken the review course offered by the school the previous spring, only 1 participant felt it helped, but only in confidence, not content. Participants reported that preparation materials offered by the school did not even begin to simulate the perceived difficulty of the NCLEX-RN examination questions.
Most participante had not studied for the NCLEX-RN while in school. Their preparation occurred mostly during the 2 to 3 weeks prior to the examination, but there was a wide range of preparation time. One participant stated she had prepared throughout the previous year, while another stated he studied only 10 hours the week before taking the examination. Almost all participants stated the spring semester of their senior year was too busy with paper and project requirements for them to think about the examination.
One participant, who passed the first time, used an eclectic and interactive preparation process. She "took a 1-week break after graduation, then met twice weekly with a study partner reviewing the book from the National Student Nurses Association review course.'' Each student selected an answer, and discussed their choices, then looked up the correct answer and discussed it. She also studied on her own with software self-tests. She and her study partner drove to the testing site 1 week prior to their scheduled examination appointment, walked around to get a feel of the place, and discussed their stress and possible relaxation techniques. This participant felt her approach was successful.
Feelings During the Examination. Responses to the question, "What were your feelings during the examination?" included:
However, a pattern did emerge. Participants who passed the first time were concerned prior to the examination, but were able to manage the stress. These participants stated:
* "I was nervous at first, but the more I sat there the more relaxed I got."
* "I tried to stay focused."
* "I felt stressed, but gave myself messages that I could do it."
In contrast, the group who failed the first time felt, overall, more confident prior to the examination, but had little ability to cope with its reality. They made comments such as:
* "I felt confident prior to the exam.. .but my tension grew as I continued to take the test."
* "I was not really anxious before the NCLEX-RN, but on seeing the test, I thought, holy cow, this is hard!" There were feelings of anxiety and lack of confidence in both groups.
Environmental Influences During the Testing Process. In response to the question, "Did anything happen during the examination that affected your performance?," 7 of the 10 participants who passed said no, but 8 of the 9 who failed at the first attempt said yes. For the passing group, two had concerns about noise, and one cited the temperature of the room. This group was able to manage distractions, stating:
* "I tuned him out, it didn't affect my test-taking."
* "I was very focused, not aware of the environment."
Comments made by the unsuccessful group included:
* "It was too warm."
* It's noisy, you can hear keyboards clicking."
* "I was hungry. I didn't bring food for the breaks."
* "I had a terrible headache."
Perceived Reasons for Success or Failure at First Examination Attempt. To ascertain salient factors influencing the NCLEX-RN, the authors asked the participants, "Why do you think you passed/didn't pass the examination on the first try?" Most participants who passed stated they had "prepared well," and "I went through thousands of questions on disks." These participants again demonstrated coping skills with comments such as:
* "It's important to take care of your body, like exercise, eating right, and resting."
* "I walked in comfortable."
* "I paid attention."
All participants who passed stated they were prepared, but 5 of the 9 who failed cited lack of preparation as the reason. In addition, 3 of the 9 who failed felt they had not known what to expect and how to answer NCLEX- RNtype questions. One participant stated:
I feel that I didn't pass because I didn't know how to take the test. Even though you told us it would be critical thinking, I was used to taking fact-based tests. These questions were about what to do first in a situation, then next, etc. I wasn't used to thinking that way.
Conversely, one participant stated she didn't pass because of lack of facts in some content areas. Four participants felt they failed because they took the examination too soon, and one said she wished she had listened to herself (about how and when to study), not to others." Another participant stated:
I don't think I could have passed the first time. It just didn't feel right. There was lots of pressure from peers and family to hurry up and take it. I needed to fail. It was the only way I knew how to study!"
Most Difficult Aspect of the NCLEX-RN. Both cognitive and affective themes rapidly emerged from the question, "What aspect of the NCLEX-RN did you find the most difficult and why?" Participants from both groups felt nothing had prepared them for answering NCLEX-RN-type questions, particularly questions about critical thinking and prioritizing care. One participant felt she didn't have the factual background to be successful. Another participant who identified English as her second language stated she had difficulty reading the questions and understanding them.
Several participants stated the most difficult aspect was "not knowing how you were doing." This affective component was demonstrated in comments such as:
* "The fear factor- controlling your reactions, keeping your head about you."
* "Stress associated with taking the test... knowing I couldn't get a job unless I passed."
* "Difficult being timed."
Examination Duration. In the interest of understanding any possible association between time spent taking the examination and success at the first attempt, interviewers asked the participants, "How long did you spend taking the examination?" In addition, if students used the entire 5 hours allotted, they were asked if they had as much time as they felt they needed. The participants in the pass group took approximately 2 hours (mean = 1.88, SD = 1.32) answering the examination questions, whereas those who failed on the first attempt took approximately 3 hours (mean = 3.11, SD = 1.02). This resulted in a statistically significant difference, with participants who passed taking less time than those who failed (t(18) = 2.25, p = .04). Two participants used the entire time allotted, and one passed and one failed. The participant who failed stated, 4T didn't finish the exam. I needed more time because I wasn't prepared enough."
Surprising Aspects of the NCLEX-RN. In another attempt to learn about perceived salient aspects, the authors asked the question, "Was there anything that particularly surprised you about the examination?" Several participants were surprised at the level of difficulty and obscurity of the questions, "...I got asked about diseases I never heard of," but another participant stated, "it was easier than I expected it to be." Two participants who failed on the first attempt were surprised the test was so "application-oriented." One stated, "I expected it to be more scientific based," and the other said:
It's got a lot to do with safety. Not so much about things like how the liver or lungs function- not physiology. It's that you are safe in the practice of nursing.
Several participants were unprepared for questions on prioritizing care among groups of patients and delegating care to other personnel. One felt she didn't have the background in patient education about risk factors for particular diseases. Another participant said:
No, nothing surprising. I knew I hadn't passed. If I had passed I wouldn't be proud to have an RN after my name after how poorly I knew I did.
Number of Examination Questions Answered. Participants were asked, "How many questions did you answer before the examination concluded?" In this sample, approximately half (n = 9) of the participants answered only the minimum of 75 questions, with a range of 75 to 265 questions. Seven (70%) of the 10 participants in the pass group answered only the minimum 75 questions, compared to only 2 (22%) of the 9 participants in the no-pass group.
Several participants commented on how surprised they were when the computer turned off, and how different it felt to "not know" how many questions they would be required to answer. One participant stated, "When the computer shut off, I freaked. Seventy-five questions, I either really passed or really failed."
Relationship Between Nursing Education and the NCLEX-RN
Whereas the first set of questions was directed toward participants' experiences of the NCLEX-RN, the second set was designed to explore the relationship between participants' nursing education and success or failure on the examination. The authors asked a combination of externally oriented questions about the program as a teacher and more internally oriented questions about the student as a learner.
Self-Perceived Test-Taking Abilities. In answering the question, "How would you describe your test-taking abilities during the nursing program?," approximately half of each group described themselves as above average or good test takers, and the other participants felt test taking was an area of weakness. The difference was in the type of questions they felt were their strengths. More of the pass group felt they did better on multiple choice questions than on essay or short answer questions, whereas the nopass group felt just the opposite.
Although a number of learning styles were represented in the sample, most participants identified themselves as interactive, hands-on learners:
* "I learn more from quizzing others."
* "I figured out my best learning style is talking it through.... I need interactive, participatory activities. I have to go through a thinking process."
* "I learn best by doing. Clinical [experiences]... helped me understand better."
Only one participant identified lecture as her primary style of learning, stating, "I like sound instruction, lecture. In reality, I didn't have time to do all the reading. They [faculty] gave us the information."
Helpfulness of Classroom Activities. The participants were asked to describe ways in which their classroom experiences were "helpful or not helpful in preparing for the examination." Aspects they found helpful included using case studies and spending time analyzing real-world clinical situations. One participant stated, "The teachers encouraged us to think and give answers, like why we would or wouldn't do something." Several participants said small classes were helpful because they felt lees inhibited questioning each other and faculty.
Aspects of classroom experiences the participants identified as unhelpful included:
* Incongruity between the way examination questions were written in class and on the NCLEX-RN.
* Lack of time for learning to think through and prioritize care when case studies were used.
* Lack of focus on preparation for the NCLEX-RN.
The perceived value of in-class group work as preparation for the NCLEX-RN varied, but one participant felt the most helpful classroom experiences were those in which students focused on scrutinizing information they had read, using "critical thinking [and] group participation, [and] helping each other understand differing viewpoints."
Salience of Clinical Activities. Participants also were asked to answer, "How were your clinical experiences helpful or not helpful in preparing for the NCLEX-RN?" In general, most participants felt clinical experiences were helpful in answering NCLEX-RN-type questions. However, several participants noted choices they had made in clinical situations often were not options for similar situations on the examination. One participant commented, "Sometimes my actions in clinical were not given as [choices] on NCLEX," and another stated:
What's not helpful is that in clinical we were not expected to do things by the book, but the test is by the book. You have to answer the questions differently than how you would really practice"
Participants in both the pass and no-pass groups echoed this sentiment.
Several participants were able to see they could learn about specific clinical experiences from others without having to actually be there. One participant noted, "I don't feel I learned acute psych as I was in an outpatient unit, but I learned from students who were in inpatient settings during weekly group meetings." However, others found lack of experience in particular clinical areas problematic when taking the NCLEX-RN.
Several participants noted the examination set up a "best practices" environment, as if they had only one patient:
On the NCLEDC-RN, they assume you have all the time you want to take care of each patient, and you are supposed to take care of all their needs. We didn't get to see or practice this kind of care, as we had to take care of four patients.
Three participants who passed on the first attempt noted clinical experiences would have been more helpful if instructors would have pushed them more, "If s important to let students know that this is all they will get, and to push them more." However, two participants were able to identify ways in which they had control over their learning. One participant who was not successful the first time noted, "Clinical was pretty helpful. It depends on you. If you want to learn, you'll learn. Sometimes I was too tired to want to learn." Another participant praised a clinical instructor who said, "If you don't know, look it up and get back to me. I didn't like that at first, but it ended up helping the most."
Needed Changes in the Program. Participants were asked to "identify at least one thing the nursing program could have done differently that would have helped them on the examination." Six participants restated their inability to prepare for the NCLEX-RN during the spring semester of the senior year due to the amount of work in courses "unrelated to boards." Some participants wanted NCLEX-RN preparation to be more of a priority for the school, "Teach to NCLEXI-RN] at the end of the senior year." Others recommended rearranging senior courses, "The last semester is a real burn-out."
Four participants suggested emphasizing prioritization of clinical actions throughout the entire curriculum, and this was followed closely by a suggestion to focus more on how to use critical thinking in clinical situations. Three participants wanted less group work and "more solid information, less touchy-feely stuff." One participant stated that by graduation she "felt like she knew how to care about people but not how to take care of them." Finally, two participants asked that the administration and faculty not place so much pressure on them to do well on boards, "The message I got was you guys don't know what you're doing... kind of like setting us up to fail."
Needed Individual Change. Hoping to get a sense of what graduates may suggest for current students, the authors asked, "Can you identify one thing in your own behavior while a student that would have helped you on the examination?" A follow-up probe of, "If you had it to do over again, what would you change about your study habits, outside, activities, etc.?," was used, if necessary. Responses to this question tended to differ by group. More suggestions from the pass group were related to things participants could change about themselves, whereas suggestions from the no-pass group were related to things others could change for them. Three participants in the pass group said they would work less while in school and "dedicate more time to school." Several said they would take school more seriously.
One participant who failed on the first attempt stated, "The school should have helped us more the last semester," and another said, "We needed a more intensive review course." A third mentioned the school should have provided more clinical time.
Feelings About Results. Participants were asked the question, "Were you surprised about your NCLEX-RN results, and why or why not?" Of the 10 participants who passed on the first attempt, 6 stated they were not surprised and felt they had passed, "I was pretty sure I passed." One who left feeling she had failed said, "as rational thought returned, I realized I probably had passed." These participants felt prepared for "not knowing." One participant said, "I had prepared myself not to know. I left feeling good but unsure."
Only 2 of 9 participants from the no-pass group were surprised they had not been successful. Both of those participants admitted they felt they had failed but "so did everyone else." One participant described the difficulty telling others about failing, "...I was in shock, depressed, couldn't tell anybody. I was ashamed. It was hard to call my manager. My parents still don't know I failed the first time."
Final Comments and Suggestions. Participants were provided a final opportunity to comment on or suggest something that previously had not been discussed. Five participants urged the school to be more actively involved in helping students prepare for the NCLEX-RN, and one specifically mentioned the experience of computer-adapted testing, "I'm fairly computer literate, but you can't go back and change an answer. Include as much as possible on computer testing."
The need for more emphasis on critical thinking and prioritization in clinical situations reemerged, as did the need to wait until graduates are ready to take the examination, despite pressure to test early. Finally, several participants mentioned there was not really anything else the school could do, "Students just have to study."
These data provided a unique view of the NCLEX-RN experience from the participants' point of view. Three distinct but overlapping areas of interest were identified in this study:
* Internal issues particular to the learner-examinee.
* Examination issues.
* Programmatic issues.
Internal Learner-Specific Issues
Although there were individual variations, the participants differed by group in their perception of responsibility for learning. The participants who passed the NCLEXRN on the first attempt took responsibility for their own learning, and when they made suggestions for programmatic change, they tended to be positive and collaborative. The graduates who were not successful tended to attribute responsibility for not passing onto others and were less able to see how they, the examinees, may have affected the outcomes. Although there was general agreement that participants felt unprepared for the NCLEX-RN and needed more direction in this area, there was less agreement about whether this was the responsibility of the school or themselves.
In addition, and certainly related, were the issues of examinee readiness for testing and ability to manage stress. The no-pass group often cited employers, family, and others pressured them to take the examination before they were ready, which refers to their external locus of control. These unsuccessful testers also demonstrated less ability to manage stress and were more vulnerable to environmental annoyances. In contrast, the pass group seemed more proactive in test preparation, visualizing the examination experience and mapping a plan to cope with their stress.
Almost all participants felt unprepared to answer NCLEX-RN-type questions and believed nothing had prepared them to do so. The most extreme response came from one graduate who told the authors that taking the test and failing was his way of preparing for the examination. The participants felt confused between the "real world" of their clinical experiences and the "best practices" world asked about on the examination, and although they found their clinical experiences valuable preparation, they were surprised when their actual clinical interventions were not options on related test questions.
Participants may have been surprised by the difference between the depth required on course examinations and the breadth experienced on the NCLEX-RN. In nursing course examinations, all questions are related to the same subject, whereas on the NCLEX-RN, questions are about all areas of nursing and in no particular order.
Although some participants' concerns were easily rectified (i.e., lack of practice with prioritizing and delegating, inadequate NCLEX-RN preparation resources, intensity of the last semester course work), the more difficult concerns reflected ongoing tension between the authors' goals for educating professional nurses and goals involved in passing the NCLEX-RN. As a baccalaureate program, the authors' school tends to focus heavily on professional nursing and liberal arts concepts that may put their students at a disadvantage on a standardized examination such as the NCLEX-RN. The core issue is the emphasis the authors' school places on critical thinking, often assessed by evaluating formal papers rather than multiple choice tests, versus the emphasis placed on teaching for the examination. This is an issue that needs to be explored by baccalaureate nursing education in general.
Based on the findings of this study, the authors recommend that:
* Nursing curricula be based on active learning strategies that encourage student responsibility for learning.
* Graduates be encouraged to reflect on and determine their readiness to test.
* Faculty help students identify appropriate test-taking strategies.
Further, in response to nursing programs' desire to not "teach to the test," the authors recommend nurse educators communicate clearly to students that it is expected they take an NCLEX-RN review course following graduation. The goal of baccalaureate nursing education is to prepare students to practice as professional nurses, a skill quite different than that needed to respond to singleanswer, context-free test questions.
Finally, the fact that the interviewers knew the NCLEX-RN success/fail status of the participants prior to the interviews served as a methodological limitation to the validity of this study, although the semi-structured questionnaire partly ameliorates this concern. A related concern is that the participants knew whether or not they had succeeded when answering the questions, which may have affected their sense of control. However, both philosophical tensions around programmatic emphasis and locus of control differences hold promise for furthering understanding of the way nursing students perceive and respond to the NCLEX-RN and thus deserve further study.
- Arathuzik, D., & Aber, C. (1998). Factors associated with National Council Licensure Examination-Registered nurse success. Journal of Professional Nursing, 14, 119-126.
- Barkley, T.W., Rhodes, R.S., & Dufour, CA (1998). Predictors of success on the NCLEX-RN among baccalaureate nursing students. Nursing and Health Care Perspectives, 19, 132-136.
- Billings, D., Hodson-Carolton, K, Kirkpatrick, J., Aaltonen, P., Dillard, N., Richarson, V, et al. (1996). Computerized NCLEXRN preparation programs: A comparative review. Computers in Nursing, 14, 272-286.
- Campbell, AR., & Dickson, C.J. (1996). Predicting student success: A 10-year review using integrative and meta-analysis. Journal of Professional Nursing, 12(1), 47-59.
- Dell, M.S., & Valine, W. J. (1990). Explaining differences in NCLEX-RN scores with certain cognitive and noncognitive factors for new baccalaureate nurse graduates. Journal of Nursing Education, 29, 158-162.
- Denzin, N.K., & Lincoln, Y.S. (1994). Handbook of qualitative research. Thousand Oaks, CA Sage.
- Endres, DA (1997). A comparison of predictors of success on NCLEX-RN for African American, foreign-born, and White baccalaureate graduates. Journal of Nursing Education, 36, 365-371.
- Krupa, K.C., Quick, M.M., & Whitley, R.W. (1988). The effectiveness of nursing grades in predicting performance on the NCLEX-RN. Journal of Professional Nursing, 4, 294-298.
- Payne, M.A., & Duffey, MA (1986). An investigation of the predictability of NCLEX scores of BSN graduates using academic predictors. Journal of Professional Nursing, 2, 326-332.
- Poorman, S.G., & Martin, E.J. (1991). The role of nonacademic variables in passing the National Council Licensure Examination. Journal of Professional Nursing, 7, 25-32.
- Streubert, H.J., & Carpenter, D.R. (1999). Qualitative research in nursing: Advancing the humanistic imperative. Philadelphia: Lippincott.
- Wall, B.M., Miller, D.E., & Wìderquist, J.G. (1993). Predictors of success on the newest NCLEX-RN. Western Journal of Nursing Research, 15, 628-643.
- Waterhouse, J.K., Carroll, M.C., & Beeman, RB. (1993). National Council Licensure Examination success: Accurate prediction of student performance on the post-1988 examination. Journal of Professional Nursing, 9, 278-283.