Teaching methods in higher education are evolving from a didactic approach to one that engages with diverse learners and leverages technology in innovative ways. Teachers and learners must work to engage and evolve with this progression (Verkuyl et al., 2018). A promising method for meeting the needs of an increasingly technological world are open educational resources (OERs). These tools are learning and teaching materials, available online for all to access (Toven-Lindsey et al., 2015). OERs are positioned to change the manner in which teachers and learners access and interact with educational materials.
Open Educational Resources
OERs were first defined by the United Nations Educational, Scientific and Cultural Organization (UNESCO) in 2002, as learning, teaching, and research materials in any medium. These materials exist publicly with open access and require no payment for use, adaptation, or redistribution (Ontario Council of University Libraries, 2017; UNESCO, 2012). Compared with textbooks, OERs can be frequently updated, customized, or reworked by the authors or by individual teachers to serve specific purposes (Cooney, 2017). OERs include, but are not limited to, full courses, electronic textbooks, articles, and video presentations (Swigart & Liang, 2016).
OERs often engage learners in self-selecting educational materials, which adjusts the students' position in the learning process. Through self-selection, learners contribute to their personal educational progression and can mutually create or customize educational materials to meet the learning objectives (DeRosa & Robison, 2017). The use of OERs has been infrequently studied, and limited literature exists describing teachers' or learners' experiences with these open access sources, particularly in nursing education.
OERs and Nursing Education
The existing literature examining nursing students' experiences with OERs uses student performance metrics to assess students' perceptions of the materials. For example, one study evaluated student performance measures through grades and withdrawal rates (Elf et al., 2015). Additionally, one study showed that students using OERs as a course guide received equal or better grades compared with students enrolled in comparable courses using textbooks and standard teaching methods (Fischer et al., 2015). Additional measurement tools used for OER assessment include student surveys and focus group interviews (Cooney, 2017). Limited literature reflects students' positive attitudes toward OERs. Students disclosed that the use of OERs made course sessions more interactive due to the accessibility of materials. Two additional concepts noted were the students' role as self-educators and challenges using technology for material access (Cooney, 2017; Elf et al., 2015).
The purpose of this evaluation study was to describe students' perspectives on the quality, integration, and experience of a course-specific OER. The OER was implemented over a 15-week semester through a course entitled “Introduction to Advanced Practice Nursing: Knowledge for Practice in the Global Community.” The course focused on nursing theories and models as part of an advanced practice RN graduate curriculum taken by both master's (MS) and Doctor of Nurisng Practice (DNP) students. Prior to the course, the professor and the university librarian collaboratively created a library resource guide that was used in place of course textbooks. The OERs included in the library resource guide included free, open access materials relevant to the course subject matter. The materials included eTextbooks, websites, social media, scholarly articles, and grey literature relevant to the course objectives. The library resource guide was available to students through the class website, as well as throughout the World Wide Web.
We conducted a nonexperimental retrospective study to evaluate the use of OERs in a graduate nursing course.
Participants and Setting
The participants were MS and DNP students in a graduate nursing course in a single large urban public university on the east coast of the United States. The study was approved through the University's institutional review board and was introduced to students during the semester. Survey participation was voluntary, and consent was obtained through a survey question.
To assess the student experience with OER, the OER Satisfaction Scale adapted from Jaggars et al. (2018) was used. The scale assessed three domains of the student experience using the library resource guide: quality, integration, and experience. The Likert scale items described students' experience with OERs in relation to textbooks or traditional printed textbooks or reading materials. The OER Satisfaction Scale reports “strong inter-item reliability and predictive validity in terms of student interest” (Jaggars et al., 2018, p. 72) with OER utilization. The use of this scale allows for an assessment of course materials based on student experience and not student outcomes, such as course grades. Additionally, three open-ended questions addressing facilitators and barriers to using the OER were included. Library resource guide traffic data were also collected throughout the semester.
At the completion of the semester, the survey was administered through a secure web application, facilitated by one of the research assistants. Participation was voluntary and the research team did not see the survey data until after course grades were turned in.
All 57 students enrolled in the course during the semester were eligible to participate, and 21 students completed the survey at the end of the semester. Responses from the OER Satisfaction Scale were analyzed with descriptive statistics using SPSS® version 25. Each item in the quality, integration, and experience subscales is scored from 1 (most negative) to 5 (most positive). We calculated the mean and standard deviation.
Quality, Integration, and Experience. The overall score on the OER Satisfaction Scale was 4.01 (SD = 0.7) on a 5-point Likert scale, suggesting that students felt the course-specific OER was better than materials they had used in other courses. We calculated the mean, standard deviation, and top box scores (sum of responses for the two most favorable answer categories) for each item (Table 1). The mean score for quality was 4.03 (SD = 0.75). Scores were lowest for engaging and interesting writing, understandable and clear writing, and comfort while using the OER (e.g., lack of eye strain, can relax comfortably while using the OER). Scores were highest for relevant content. The mean score was highest in the integration category at 4.5 (SD = 0.64), indicating that students appreciate the relevance of the OERs to the course itself. Additionally, 90% to 95% of students responded that the OER was well integrated into coursework. The mean score for experience was 3.83 (SD = 0.86). Scores were lowest for shutting out other distractions and taking useful notes. Scores were highest for collaborating with other students. The survey responses indicate that using OERs enhance student engagement with the course by requiring learners to assume a more active role as course participants and providing relevant, integrated content.
Open Educational Resource Satisfaction Scale Questions, With Corresponding Scores
Open-Ended Responses. Respondents (n = 20) provided answers to the open-ended questions. Students appreciated that the resources were free and found that the information was manageable and easy to navigate. Students commented that the organization of the OER and navigation page made the resource easy to use. Overall, students appreciated that content relevant to the course topic was readily available to them. Although accessibility was the most commonly cited helpful aspect, four respondents mentioned that access was limited because of technical issues, such as “My computer is slow,” and “If I had a weak Wi-Fi signal, [the resources] took a bit [of time] to load.” Additionally, several students commented on having difficulty not having print material, stating, “It's harder for me, personally, to organize my thoughts and take helpful notes with online materials. I still prefer textbooks.” Finally, two students commented that they believed it cost more to print material than to purchase a book.
The survey responses indicate that using OERs enhanced student engagement with the course by requiring learners to assume a more active role as course participants. The high mean score for integration indicates that students appreciate the relevance of the OER to the course itself. Survey responses also stressed that students needed to be oriented to using OERs and thus appreciated support from the university's librarian. This orientation is not limited to training regarding the resource's technological aspects; students must also be positioned to take on a role as active participants in the course and in their own education. OERs such as the ones used in this graduate nursing course allow students to self-select materials, requiring them to assume responsibility for directing the course of their own learning. Survey responses indicate alignment with previous studies' findings that state OERs helped facilitate increased interaction and longer discussions, and they prompted students to assume the role of self-educators (Cooney, 2017; Elf et al., 2015). These characteristics are evident through the high scores related to being prepared for class activities or discussions and collaborating with fellow students.
One of the most widely advertised advantages of OER is that they cost students nothing. Therefore, they provide access to course content from the first day of the semester. Although responses to the open-ended questions indicated that students appreciated not needing to pay for course readings, it appears that one challenge for more widespread OER adoption is that many students still prefer print course materials. Although OERs are free resources, one must remember there are costs associated with printing materials if students prefer not reading electronic documents. Future work should include assessing graduate nursing students' preferred medium for course materials.
This study has evaluated graduate (MS and DNP) nursing students' perspectives on the quality, integration, and experience of a course-specific OER. The graduate nursing students' satisfaction regarding the integration and the quality of the OERs supports further exploring a future transition from traditional course materials to OERs. This work lends support to the use of OERs in nursing schools, emphasizes the importance of focusing course-specific OERs, and highlights students' satisfaction with these free electronic resources.
Choosing to use a course-specific OER represents a real pedagogical opportunity for a course to be redesigned. Allowing students to self-select materials from a resource guide allows them to identify and seek specific materials to supplement personal knowledge gaps. The main aim of OERs is to provide open access to different educational resources through the internet. Although OERs are being used in graduate education, further work is needed to understand their effectiveness, particularly their value in graduate nursing education.
- Cooney, C. (2017). What impacts do OER have on students? Students share their experiences with a health psychology OER at New York City College of Technology. The International Review of Research in Open and Distributed Learning, 18, 155–178 doi:10.19173/irrodl.v18i4.3111 [CrossRef]
- DeRosa, R. & Robison, S. (2017). OER to open pedagogy: Harnessing the power of open. In Jhangiani, R. S. & Biswas-Diener, R. (Eds.), Open: The philosophy and practices that are revolutionizing education and science (pp. 115–124). Ubiquity Press. doi:10.5334/bbc.i [CrossRef]
- Elf, M., Ossiannilsson, E., Neljesjö, M. & Jansson, M. (2015). Implementation of open educational resources in a nursing programme: Experiences and reflections. Open Learning, 30, 252–266 doi:10.1080/02680513.2015.1127140 [CrossRef]
- Fischer, L., Hilton, J. III. , Robinson, T.J. & Wiley, D. A. (2015). A multi-institutional study of the impact of open textbook adoption on the learning outcomes of post-secondary students. Journal of Computing in Higher Education, 27, 159–172 doi:10.1007/s12528-015-9101-x [CrossRef] PMID:32269452
- Jaggars, S. S., Folk, A. L. & Mullins, D. (2018). Understanding students' satisfaction with OERs as course materials. Performance Measurement & Metrics, 19, 66–74 doi:10.1108/PMM-12-2017-0059 [CrossRef]
- Ontario Council of University Libraries. (2017). Open educational resources white paper. https://ocul.on.ca/sites/default/files/2017-11-17%20no.%2002.12.01.%20OER%20white%20paper%20for%20Directors%20Nov%202017.pdf
- Swigart, V. & Liang, Z. (2016). Digital resources for nursing education: Open courseware and massive open online courses. International Journal of Nursing Sciences, 3, 307–313 doi:10.1016/j.ijnss.2016.07.003 [CrossRef]
- Toven-Lindsey, B., Rhoads, R. A. & Lozano, J. B. (2015). Virtually unlimited classrooms: Pedagogical practices in massive open online courses. The Internet and Higher Education, 24, 1–12 doi:10.1016/j.iheduc.2014.07.001 [CrossRef]
- UNESCO. (2012). 2012 Paris OER declaration. http://iite.unesco.org/files/news/639202/Paris%20OER%20Declaration_01.pdf
- Verkuyl, M., Lapum, J., St-Amant, O., Tan, A. & Garcia, W. (2018). Engaging nursing students in the production of open educational resources. Nurse Education Today, 71, 75–77 doi:10.1016/j.nedt.2018.09.012 [CrossRef] PMID:30261366
Open Educational Resource Satisfaction Scale Questions, With Corresponding Scores
|Question||Mean Score (1–5)||± SD||Responses Scoring Positively (Above 4)|
|Quality: Compared with traditional printed textbooks or readings used in your other courses, how high-quality was this course's digital library guide in terms of the following characteristics? (1 = much worse, 5 = much better)|
| 1. Good search capabilities||4||0.85||80%|
| 2. High-quality visuals||3.5||0.89||62%|
| 3. Engaging and interesting writing||3.5||0.85||57%|
| 4. Understandable and clear writing||3.5||0.94||62%|
| 5. Relevant content (e.g., to the particular course's focus and instructor||5||0.86||76%|
| 6. Current content (e.g., up-to-date with recent advances in the field)||4||0.85||71%|
| 7. Comfort while using (e.g., lack of eye strain, can relax comfortably while using)||3.5||0.96||70%|
|Integration: To what extent were this course's materials integrated into assignments and assessments? (1 = never, 5=every time)|
| 1. Did this instructor explicitly make you review the course materials (e.g., by requiring quizzes on them, or requiring you to discuss them in class)?||4||0.74||95%|
| 2. Was reviewing the course materials necessary to understand in-class material,||4||0.80||90%|
|lectures, or discussions?|
| 3. Was reviewing the course materials necessary to doing well on this course's assessments (e.g., quizzes, papers, examinations)?||4||0.65||90%|
|Experience: Compared with traditional printed textbooks or readings used in your other courses, did this course's digital library guide make it easier or harder to do the following activities? (1 = much harder, 5 = much easier)|
| 1. Access course materials whenever I needed them||3||1.02||71%|
| 2. Find and get started on the correct assignments in time||3||1.01||62%|
| 3. Read and understand the material||3.5||0.93||57%|
| 4. Shut out other distractions while studying||2||1.08||43%|
| 5. Take useful notes on the material||2||1.16||48%|
| 6. Complete assignments on time||3.5||0.89||70%|
| 7. Review and remember the material||3||0.96||48%|
| 8. Be prepared for class activities or discussions||3.5||0.97||62%|
| 9. Collaborate with fellow students||4||0.94||62%|