The landscape of nursing education has changed dramatically in the past two decades, resulting in a transformation of how nurse educators “educate.” This change can be attributed to a shortage of nursing faculty paired with a growing student population, characteristics of the students themselves, and the growth of technology. Despite a decrease in the number of nursing faculty, the number of nursing students enrolled in entry-level baccalaureate programs has continued to increase in recent years (American Association of Colleges of Nursing, 2008). As a result of high faculty-student ratios, nursing classes often are taught in large lecture halls, which are perceived by many students as a distant and intimidating learning environment (DeBourgh, 2008).
Technology has been integrated into the classroom as a way to bridge the distance between faculty and students through active engagement and collaborative learning (Black & Watties-Daniels, 2006). The use of technology also capitalizes on the characteristics and learning styles of current college students, who were born in 1982 or later and are referred to as millennial learners (Pardue & Morgan, 2008; Raines, 2002). These students have a preference for active learning and group activities, as well as the use of technology (Mangold, 2007). Educators have demonstrated an understanding of this generation by beginning to adopt teaching strategies that incorporate technology and focus on active learning (DeBourgh, 2008; Stein, Challman, & Brueckner, 2006), critical thinking (DeBourgh, 2008), and peer collaboration (Skiba, 2006).
Incorporating personal response system (PRS) technology into teaching is one pedagogical approach that can be used to stimulate active learning in large classrooms and possibly even in small classrooms. Literature supports the use of PRS technology with millennial learners in various disciplines in large classrooms (Billings & Kowalski, 2006; Copas & Del Valle, 2004; Guthrie & Carlin, 2005; Skiba, 2006; Vernaza, 2007). Researchers found an improvement in comprehension, class participation, course satisfaction (Vernaza, 2007), and faculty-student interaction (Skiba, 2006).
Personal response system technology uses a combination of infrared and radio waves and can be easily adapted to Smart Classrooms with overhead LCD projectors. Individual handheld devices allow students in the room to respond to a question simultaneously through a receiver linked to the faculty’s computer, which collects and records responses. Collective student responses are displayed to the class as a bar graph-histogram via the LCD projector (eInstruction, 2008). The PRS allows students to receive immediate feedback about their responses, and students also can see how the class responded as a whole (Guthrie & Carlin, 2005; Skiba, 2006). The PRS allows faculty to provide students with feedback during the lecture and thereby engage them in active learning, assess comprehension of content, and clarify challenging concepts (DeBourgh, 2008; Moredich & Moore, 2007; Vernaza, 2007).
Research related to PRS efficacy in nursing courses is limited. Although studies support the use of PRS technology in large classes (>75 students) (DeBourgh, 2008; Moredich & Moore, 2007; Stein et al., 2006), no studies have evaluated its effect in small classes (<35 students). The usefulness of integrating multiple-choice and true-false questions into lectures has been explored (DeBourgh, 2008; Stein et al., 2006), but no PRS research has integrated the new NCLEX-RN® alternate format-style questions (NCLEX-RN, 2007) or evaluated the effect of using PRS on student learning. Several nursing studies examined the value of using PRS technology for quiz and examination reviews (Abdallah, 2008; DeBourgh, 2008; Stein et al., 2006), but none have incorporated graded reading quiz questions into didactic lectures.
This study evaluated and compared the perceived effectiveness of PRS technology in enhancing student learning in large and small classrooms. Specifically, researchers evaluated the use of PRS technology to actively engage students, foster critical thinking, and improve learning outcomes.
This study was supported by an internal academic technology grant and was approved with exempt status by the university’s institutional review board. This mixed-method study began in the summer of 2008 with the review of several operating systems (eInstruction, iclicker, Turning Point). Cost of the individual keypads ranged from $15 to $45. The researchers determined it was essential for the system to have the capability of constructing NCLEX-RN alternate format-type questions (NCLEX-RN, 2007). After careful consideration, eInstruction was selected because it provided the most flexibility in designing different types of questions, such as polling, multiple choice, true-false, short answer, multiple response, and multiple response with prioritization of items (eInstruction, 2008).
Sample and Setting
In the fall 2008 semester, PRS technology was integrated into two undergraduate nursing courses, nursing research (n = 33) and junior medical-surgical nursing (n = 116). The research course was held in a traditional, one-level classroom with movable tables and chairs, and the medical-surgical course was held in a large amphitheater-style classroom with fixed seating.
Researchers incorporated multiple-choice, true-false, fill-in-the-blank, multiple-response, and chart-exhibit response questions within didactic PowerPoint® presentations. The questions were focused on content review, case studies, and reading quizzes. At the beginning of each class, students obtained a numbered, personal responder keypad assigned to them for the semester. Students then joined the class session, which allowed individual responses to be saved and evaluated at a later time.
During the didactic portion of the class, content review questions were displayed on the screen and students responded by selecting their answer on the individual keypads. After student responses were received, a visual count registered on the screen. The software also incorporated a count-down clock that alerted students to the time remaining to enter responses.
Tabulated results of the question were displayed in a bar graph format, indicating percentages of students who selected each answer. The Figure shows a bar graph for a multiple-response question posed to students during a lecture on diabetic ketoacidosis and hyperglycemic hyperosmolar nonketotic syndrome. The results indicated the majority of students (86.9%) answered the question correctly (B and C); however, several students selected an additional response (D) that was incorrect. Receiving instant student response feedback allowed faculty to use an alternative method of explanation to discuss why D was incorrect.
Figure. Sample Multiple-Response Question and Bar Graph.
In the junior medical-surgical course, a video clip from YouTube was incorporated as a case study on diabetes. Students viewed the video and responded to questions using PRS technology. In the nursing research course, chart and exhibit questions were presented and reading quiz questions were interwoven with lecture content. Students responded to content-based questions throughout the class session.
Evaluation of study outcomes used mixed methods. Students completed a 14-item evaluation tool that collected demographic data and students’ perceptions of the effectiveness of PRS technology. The demographic data included gender, age, years of experience with technology, the course (medical-surgical or nursing research), and previous use with PRS. Students’ perceptions were collected by asking level of agreement statements about PRS using a 6-point Likert scale. Students in the nursing research course answered an additional item evaluating effectiveness of reading quizzes. Data were analyzed using Statistical Package for Social Sciences software version 17 (SPSS Inc, Chicago, IL).
Qualitative data was obtained from students by answering the following open-ended questions on the evaluation tool:
- Describe how the PRS technology enhanced your learning in this course.
- Identify other uses for PRS technology that you feel would improve your undergraduate nursing classes or laboratories.
Faculty provided evaluation data at the end of the course. Researchers analyzed the effectiveness of PRS technology in increasing class participation, identifying concepts that required reinforcement, and integrating testing for student comprehension.
A total of 139 undergraduate nursing students in either the nursing research (22.3%) or junior medical-surgical (77.7%) course completed the evaluation form. The majority of students were women (92.8%) aged 18 to 22 (95.7%). Most of the students indicated they had >5 years of experience with technology (92.8%) but no previous experience using PRS (88.5%).
Using a 6-point Likert-type scale (1 = strongly disagree, 6 = strongly agree), students in both classes were asked to evaluate the effectiveness of PRS technology in enhancing their learning. Specifically, questions addressed the ability to clarify key concepts (Mean = 4.71, SD = 1.36), provide instantaneous feedback (Mean = 4.89, SD = 1.33), and increase classroom participation (Mean = 5.14, SD = 1.29). Students also were asked to evaluate their comfort (Mean = 5.34, SD = 1.15) and satisfaction using the PRS (Mean = 5.20, SD = 1.22), as well as whether they believed the technology could be useful in other nursing courses (Mean = 5.16, SD = 1.20). Summative results supported PRS technology as an easy-to-use, effective way to engage students and enhance learning.
To determine the efficacy of PRS technology with smaller classes, an independent samples t test was performed to compare the perceived effectiveness of PRS technology between students in the large (n = 116) and small (n = 33) classes. There were no statistically significant differences between classes for any of the Likert-type questions. Thus, students in the small and large classes rated PRS technology equally effective.
Responses to the first qualitative question mirrored the quantitative results supporting enhancement of student learning through PRS technology. One student commented that the technology made it “easier to connect course material to critical thinking questions.” Another student said:
The PRS technology helped evaluate my level of knowledge for that particular subject. It displayed what areas I needed to focus on and those areas I already knew.
When asked to identify other uses for PRS technology, students suggested multiple uses including test preparation sessions, examination reviews, graded quizzes, attendance, polling questions, discussion topic questions during lectures, and NCLEX-RN practice questions.
Students also commented on the effectiveness of the NCLEX-RN alternate-format questions and reading quiz questions to enhance their learning. Specifically, students appreciated the exposure to various types of questions and the opportunity to apply content learned in lecture. One student noted that the PRS technology “gave good examples of the types of questions we will see on exams.” Another student commented:
The PRS allows us to answer similar questions that would be asked on our nursing exams. It allows us to get used to these types of questions.
In addition, students responded favorably to the use of graded reading quiz questions during lecture. Comments included:
When the clicker was used to take a quiz that was spread throughout material, it was useful. I felt like I paid more attention and learned better.
PRS technology for more quizzes would be helpful because if you get something wrong, you know right away and you have a better understanding of what the right answer is.
Faculty evaluation data reinforced student results, with faculty indicating a noted increase in class participation and an enhanced ability to identify concepts for reinforcement during lecture. One faculty member also supported the benefit of inserting reading quiz questions in the lecture to enhance student engagement and learning.
Students and faculty perceived PRS technology to be effective in engaging students, fostering critical thinking, and improving learning outcomes in both the large and small classes. Similar to students in DeBourgh’s (2008) study, respondents in the current study enjoyed using PRS and were comfortable using PRS during class.
Faculty and students reported increased classroom participation in both classes. Studies support increased student engagement in large classrooms by providing a safe (Moredich & Moore, 2007) and anonymous environment (DeBourgh, 2008). The results of this study suggest PRS technology can be equally effective in small classes where peer intimidation and fear of discomfort also exist and may go unrecognized by faculty.
The PRS technology enhanced students’ ability to understand lecture material, think critically, and link key concepts in the large and small classes. The evaluation comments made by students supported the use of all types of NCLEX-RN format questions to enhance their ability to understand content and prepare for examinations. This finding may be attributed to the instant individual feedback students received, discussion of wrong answer choices (distracters), and the ability of faculty to provide clarification and reinforcement of content.
Similar to previous research (DeBourgh, 2008; Moredich & Moore, 2007; Stein et al., 2006), faculty respondents agreed that incorporating PRS into the large and small classrooms encouraged active learning and enhanced critical thinking skills. Although examination reviews were not part of the current study, several students identified this as a potential use of the PRS technology in the future.
Students in the nursing research class indicated that reading quiz questions during lecture were an effective means of using PRS technology. The nursing research faculty concurred, stating that integration of quizzes increased students’ preparedness for class and attentiveness during class. Similarly, Moredich and Moore (2007) found students who used PRS in class often were better prepared when they expected quizzes.
The current study did not evaluate the effect of PRS technology on examination grades. Future research to evaluate the effect of PRS technology on examination grades, overall course grades, and NCLEX-RN scores may be beneficial.
The integration of PRS technology is an effective strategy for educating millennial learners in both large and small classrooms. The PRS technology promotes active learning, increases participation, and provides students and faculty with immediate feedback that reflects comprehension of content and increases faculty-student interaction.
Nursing faculty are encouraged to consider PRS technology as a pedagogical tool that can be readily incorporated into existing class lectures. This study supports the integration of PRS technology across undergraduate nursing curriculums. Students suggested the inclusion of PRS in a variety of nursing courses from freshman to senior year. The adoption of PRS technology in the freshman year would allow students to adapt early and absorb the cost across several courses (Moredich & Moore, 2007). By using PRS technology in the classroom, nurse educators can provide the essential thread of active learning to engage millennial learners.
- Abdallah, L. (2008). Reflective teaching with technology: Use of a personal response system and publisher’s web site to enhance students’ performance in a nursing assessment and skills course. Online Journal of Nursing Informatics, 12(1). Retrieved from http://ojni.org/12_1/abdallah.html
- American Association of Colleges of Nursing. (2008). Advancing higher education in nursing: Annual state of the schools. Retrieved from http://www.aacn.nche.edu/media/pdf/AnnualReport08.pdf
- Billings, D.M. & Kowalski, K. (2006). Keeping participants enrolled: Participation. The Journal of Continuing Education in Nursing, 37, 152–153.
- Black, C.D. & Watties-Daniels, A.D. (2006). Cutting edge technology to enhance nursing classroom instruction at Coppin State University. The ABNF Journal, 17(3), 103–106.
- Copas, G.M. & Del Valle, S. (2004). Where’s my clicker? Bringing the remote into the classroom—Part II. Usability News, 6.1. Retrieved from http://psychology.wichita.edu/surl/usabilitynews/61/remote_testing.htm
- DeBourgh, G.A. (2008). Use of classroom “clickers” to promote acquisition of advanced reasoning skills. Nurse Education in Practice, 8, 76–87. doi:10.1016/j.nepr.2007.02.002 [CrossRef]
- eInstruction (2008). Interwrite PRS®. Retrieved from http://www.einstruction.com/products/assessment/prs/
- Guthrie, R.W. & Carlin, A. (2005). Waking the dead: Using interactive technology to engage passive listeners in the classroom. Retrieved from http://www.einstruction.com/News/index.cfm?fuseaction=news.display&menu=news&content=showArticle&id=131
- Mangold, K. (2007). Educating a new generation: Teaching baby boomer faculty about millennial students. Nurse Educator, 32, 21–23. doi:10.1097/00006223-200701000-00007 [CrossRef]
- Moredich, C. & Moore, E. (2007). Engaging students through the use of classroom response systems. Nurse Educator, 32, 113–116. doi:10.1097/01.NNE.0000270225.76661.74 [CrossRef]
- NCLEX-RN: 250 new-format questions (2nd ed.). (2007). Philadelphia, PA: Lippincott Williams & Wilkins.
- Pardue, K.T. & Morgan, P. (2008). Millennials considered: A new generation, new approaches, and implications for nursing education. Nursing Education Perspectives, 29, 74–79.
- Raines, C. (2002). Managing millennials. Retrieved from http://www.generationsatwork.com/articles/millenials.htm
- Skiba, D.J. (2006). Got large lecture hall classes? Use clickers. Nursing Education Perspectives, 27, 278–280.
- Stein, P.S., Challman, S.D. & Brueckner, J.K. (2006). Using audience response technology for pretest reviews in an undergraduate nursing course. Journal of Nursing Education, 45, 469–473.
- Vernaza, K.M. (2007). Using personal response system technology and concept check modules to improve students’ learning experience: A case study. Proceedings of the 37th ASEE/IEEE Frontiers in Education Conference, F1D, 16–17.