In graduate nursing education, statistical knowledge is identified by the American Association of Colleges of Nursing (AACN, 2011) as being essential to the curriculum. Knowledge of statistics is essential for nurses so they can competently interpret evidence that informs clinical decisions. In addition, knowledge of statistical concepts and methods is necessary for understanding study designs, measurement and instrumentation, data analysis techniques, data output, published results of studies, and relevant clinical guidelines. Meeting the essential requirement for statistical content can be challenging as schools of nursing seek faculty with statistical expertise. The issue becomes more complex for faculty of graduate programs who know that undergraduate nursing students report anxiety and fear of mandatory statistical courses (Hagen, Awosoga, Kellett, & Damgaard, 2013) and that baccalaureate-prepared nurses also report ongoing discomfort with statistics (Gaudet, Singh, Epstein, Santa Mina, & Gula, 2014).
Teaching Statistics in Health Sciences and Nursing
At the 2012 Joint Statistical Meeting, an expert panel of experienced biostatisticians convened to discuss teaching statistics in Health Sciences, and this panel determined that Master of Science in Nursing (MSN) and Doctor of Nursing Practice (DNP) students need a strong foundation in statistical literacy and reasoning (Baghi & Kornides, 2012). In 2016, the American Statistical Association endorsed a revision of the Guidelines for Assessment and Instruction in Statistics Education (GAISE) College Report that included six recommendations for teaching statistics; two recommendations were focused on what to teach and four recommendations were focused on how to teach (Carver et al., 2016). These recommendations included:
- Teaching statistical thinking.
- Emphasizing conceptual understanding.
- Using real data in context and for a purpose.
- Fostering active learning.
- Using technology to explore concepts and analyze data.
- Using assessments to improve and evaluate learning.
It is important that professors of statistics in graduate nursing programs use strategies that diminish angst about statistics. As many as 80% of graduate students have reported that statistics anxiety diminished performance and contributed to procrastination on completing assignments (Onwuegbuzie, 2004). Qualitative studies of nursing students identified themes about statistics courses that reflect fear of equations and a need for help (Hagen et al., 2013). Lack of establishing statistical knowledge in baccalaureate programs may impact nursing careers. Alumni nurses reported the juxtaposition of valuing statistical knowledge and possessing math skill, but still experienced ongoing discomfort in using statistics (Gaudet et al., 2014).
Although statistical content is recognized as essential, there are no set guidelines for schools or faculty on best practices to integrate statistical content into graduate programs. It generally is accepted that traditional statistics education, with the focus on mathematics and calculations, is outdated and ineffective, and that the focus should be on statistics literacy (Hayat, Eckardt, Higgins, Kim, & Schmiege, 2013). However, when seeking best pedagogical approaches for teaching statistics in health science, more studies are needed. Recent literature suggests that for nursing students, attitudes toward statistics can improve with appropriate course design (Hagen, Awosoga, Kellett, & Dei, 2013).
Courses that include learning exercises that reflect actual clinical scenarios (Wonder & Otte, 2015) with real data or that give students additional opportunities to interact with course materials (Schwartz, 2014), such as a flipped classroom experience, may enhance learning. Finally, the literature suggests web-based approaches are an effective tool for nurses in learning statistics (Justham & Timmons, 2005). The purpose of this study was to discover the impact of an open and free online statistics course on student grades and student evaluations of instruction (SEI) for students enrolled in an online MSN nursing research and statistics course.
This study was exempted by the West Virginia University Institutional Review Board. A retrospective design was used to collect and analyze changes in course grades and course evaluation reports from 92 graduate nursing students who completed a graduate research and statistics course during the first year of an MSN program. The course was a required course in the graduate nursing curriculum routinely offered during the spring semester. The course was evaluated annually and adapted after course reviews for each spring semester for 3 years, aiming to address student concerns and embrace faculty suggestions. In Spring 1, the course included weekly synchronous online lectures and a face-to-face 2-day statistics seminar. In Spring 2, the course included the same weekly synchronous online lectures and discussions, and was adapted to include the completion of a free and open online statistics course. In Spring 3, the course included the same weekly synchronous online lecture, and free and open online statistics course, but the course was adapted to include 4 additional hours of synchronous online discussion dedicated to statistical concepts. The research question for this study was: What were the differences in class grade point average (GPA), student grades, and SEI scores for three groups of students enrolled in three spring semesters?
The MSN Nursing Research Course
The graduate research nursing course was a 5-credit course with three credits focused on research methods and two credits focused on statistical knowledge. The course included objectives to focus on the use of research, statistics, and epidemiological principles to guide the advanced practice nurse in the translation of research to practice. The course includes lessons on both quantitative and qualitative methods, designs for feasibility and acceptability pilot studies, and critiquing and understanding research literature.
The Embedded Open Online Statistics Course
The Carnegie Mellon University Open Learning Initiative (CMU OLI, 2019) was used to access the free and open online statistics course. The course, Probability & Statistics—Open & Free, was accessed weekly by students taking the nursing research course. Students established an account at the CMU OLI and completed self-guided materials and interactive activities. The course includes 5 units that are divided into 17 modules. Course materials include online content, opportunities for practice, simulations, computer-based tutorials, virtual experiences, and opportunities for self-assessment that include hints for students and immediate feedback on answers.
Expected Learning Outcomes for the Course
The learning outcomes documented in the syllabus of this 5-credit research course were detailed and comprehensive. Students were expected to:
- Demonstrate understanding of research methodologies.
- Critically evaluate conceptual, ethical, methodological, analytical, and interpretive dimensions of the research process.
- Develop skill for searching for research and clinical literature.
- Use a framework to critique scientific literature.
- Demonstrate ability to synthesize findings from scientific literature.
In the course, learning outcomes were assessed through examinations, statistical exercises, and completion of a literature synthesis paper. Therefore, course grades were a reflection of students' ability to apply what they had learned; students were required to identify a clinical question, develop a search strategy for the literature, identify papers for inclusion in a systematic review, critique each paper for inclusion in a primary report of a research study with a literature critique matrix, and then synthesize the findings from the included study reports in a course paper. Course papers were graded based on a rubric to ensure equity in grading.
Data Collection and Analysis
Final course grades and numerical data from SEI evaluations were deidentified and entered into a data file for analysis using SPSS® version 26. Data were evaluated for outliers, impossible values, and missing data prior to analysis. Analysis included descriptive data (frequencies, means, medians, modes, and percentiles), as well as ANOVA and Pearson correlation to determine differences across groups and relationships between teaching method and outcome measures. Qualitative comments from students on SEIs were not identifiable, and examples of student comments are presented as supportive of meeting learning outcomes.
Course grade data were collected for 108 students who completed the course. When looking at individual grades earned by students based on teaching method, there was a statistically significant difference (χ2 = 9.598, p = .048) with the second and third group of students earning an equal number of A grades (n = 24). The course change resulted in fewer students earning B and C grades. In the Spring 1 semester, 4 students earned a C grade compared with only 1 student in the Spring 3 semester earning a C grade (Figure 1).
Changes in course grades by teaching method. Note. CMU OLI = Carnegie Mellon University Online Learning Initiative.
Overall class GPA differed by teaching method, with the Spring 3 group (Wimba Live Classroom plus CMU OLI plus statistics discussions) earning the highest class GPA of 3.65. In mean comparison testing with post hoc tests, the difference between the Spring 1 group (Wimba Live Classroom and Statistics Seminar) and the Spring 3 group was significant (Tukey's honestly significant difference, p = .045). There was no significant difference in overall class GPA between the Spring 2 and 3 groups. Figure 2 shows the change in overall class GPA based on teaching method.
Change in overall class mean grade point average by teaching method. Note. CMU OLI = Carnegie Mellon University Online Learning Initiative.
To assess changes in course evaluations, SEI data were compared for 92 students who completed the evaluations (n = 52 for Spring 1, n = 27 for Spring 2, and n = 13 for Spring 3). Student evaluations of instruction changed favorably with the course adaptations. Comparison of mean SEI scores indicated improvement for critical thinking (F = 6.924, p = .002), teacher effectiveness, (F = 6.406, p =.003), overall course rating (F = 10.119, p < .001), and overall learning (F = 9.820, p < .001). Students in the Spring 2 group (Wimba Live classroom plus CMU) had the highest evaluation scores on all four questions. In post hoc testing, the group differences remained significant.
Additional SEI comments indicated students met learning outcomes related to application of course content. Across all three semesters, students commented that the course design was student-centered, organized, and informative. Multiple students reported they had a better appreciation of research and a greater understanding of the process of writing a review paper, were encouraged about taking research to practice, had developed competence for reading and understanding research, and were considering future research or obtaining a PhD degree. Students commented that the statistical content was easy to understand and that the self-pace course with immediate feedback was helpful in understanding statistics.
This study demonstrates that statistical content from an open and free online statistics course from a reputable university easily can be integrated into a research course for clinical graduate nursing students. Delivering statistics content to clinical graduate nursing students can be challenging for schools of nursing that do not have statistician faculty. Clinical graduate students are RNs who generally are engaged in work and school, and incorporating the self-paced CMU OLI course into the required graduate nursing course meets these students' needs for flexibility while providing essential content for MSN and DNP students.
A randomized trial that compared face-to-face courses with the CMU OLI statistics course (n = 3,046 students at seven universities) demonstrated the CMU OLI course resulted in increased final examination scores and statistical knowledge (Bowen, Chingos, Lack, & Nygren, 2012). Although these differences were not significant, the study is a landmark study as it highlights that the online, self-paced completion of learning modules on statistics can be effective or even better in producing student outcomes and enhancing student knowledge of statistics. The findings of this study lend support for using the CMU OLI course for basic statistics knowledge required by the AACN to meet MSN and DNP essentials. The improvements in course grades in this study are congruent with the report on the CMU OLI statistics courses (Bowen et al., 2012).
Overall, the students in this study had favorable comments for the live classroom plus the CMU OLI self-paced course, and they were successful using this format. This is congruous with reports that research methods courses could be delivered completely online (Campbell, Gibson, Hall, Richards, & Callery, 2008). Additional benefits of teaching completely online include the ability to accurately determine the frequency of students' interaction with material, making it evident when students are engaged and participating (Campbell et al., 2008). The CMU OLI course also affords this opportunity with the ability for faculty to determine whether students are engaged and completing modules.
The inclusion of an existing online course that was developed by statistical experts at CMU can provide rigor in statistical instruction for schools of nursing that are seeking to meet the AACN essentials but lack a dedicated statistics professor on faculty. As previously emphasized in the nursing education literature, statistics is a separate knowledge discipline (Hayat, 2014) and requires expertise to deliver content in a meaningful way so that students can apply what they learn in clinical settings. The CMU OLI course reflects significant expertise and meets the six recommendations in the GAISE report as it emphasizes statistical literacy, emphasizes conceptual thinking, gives students the opportunity to work with data, has interactive exercises for active learning, gives the students opportunities to work with technology to analyze data, and provides assessments (Hayat, 2014).
Study limitations include the use of a convenience sample of graduate student data. It is possible that study results would vary if every student completed the course evaluations; however, this is not mandated. In addition, the study compared student grades and SEIs sequentially with different students. Although students are admitted to the program based on similar accomplishments, it is possible that the three different cohorts of students had different baseline abilities and preferences for learning.
Future studies aimed at understanding how existing free courses can be incorporated into nursing education courses to enhance student retention of content, student preconceptions about statistics, and student motivation toward research would enhance the nursing knowledge base about the pedagogy of statistics in nursing. In addition, studies that evaluate how the CMU OLI course may impact student anxiety or fear of statistics are needed. Finally, studies on the utility of the CMU OLI course as a basic preparatory course prior to entry into doctoral education could provide insight into baseline statistical knowledge of students newly admitted to doctoral programs.
- American Association of Colleges of Nursing. (2011). AACN essentials. Retrieved from https://www.aacnnursing.org/Education-Resources/AACN-Essentials
- Baghi, H. & Kornides, M. ( 2012, July. ). Current and future health care professionals' attitudes toward and knowledge of statistics: How confidence influences learning. Paper presented for the Section on Teaching of Statistics in the Health Sciences at Statistics: Growing to Serve a Data-Dependent Society, 2012 Joint Statistical Meeting. , San Diego, CA. .
- Bowen, W., Chingos, M., Lack, K. & Nygren, T. (2012). Interactive learning online at public universities: Evidence from randomized trials. Retrieved from https://sr.ithaka.org/wp-content/uploads/2015/08/srithaka-interactive-learning-online-at-public-universities.pdf
- Campbell, M., Gibson, W., Hall, A., Richards, D. & Callery, P. (2008). Online vs. face-to-face discussion in a web-based research methods course for postgraduate nursing students: A quasi-experimental study. International Journal of Nursing Studies, 45(5), 750–759.
- Carnegie Mellon University Open Learning Initiative. (2019). Probability & statistics—Open & free. Retrieved from https://oli.cmu.edu/courses/probability-statistics-open-free/
- Carver, R., Everson, M., Gabrosek, J., Horton, N., Lock, R., Mocko, M. & Wood, B. (2016). Guidelines for assessment and instruction in statistics education (GAISE) College Report 2016. Retrieved from https://www.amstat.org/asa/files/pdfs/GAISE/GaiseCollege_Full.pdf
- Gaudet, J., Singh, M.D., Epstein, I., Santa Mina, E. & Gula, T. (2014). Learn the game but don't play it: Nurses' perspectives on learning and applying statistics in practice. Nurse Education Today, 34(7), 1080–1086.
- Hagen, B., Awosoga, O., Kellett, P. & Dei, S.O. (2013). Evaluation of undergraduate nursing students' attitudes towards statistics courses, before and after a course in applied statistics. Nurse Education Today, 33(9), 949–955.
- Hagen, B., Awosoga, O.A., Kellett, P. & Damgaard, M. (2013). Fear and loathing: undergraduate nursing students' experiences of a mandatory course in applied statistics. International Journal of Nursing Education Scholarship, 10.
- Hayat, M.J. (2014). Guidelines for Assessment and Instruction in Statistics Education (GAISE): Extending GAISE into nursing education. Journal of Nursing Education, 53(4), 192–198.
- Hayat, M.J., Eckardt, P., Higgins, M., Kim, M. & Schmiege, S.J. (2013). Teaching statistics to nursing students: An expert panel consensus. Journal of Nursing Education, 52(6), 330–334 doi:10.3928/01484834-20130430-01 [CrossRef]
- Justham, D. & Timmons, S. (2005). An evaluation of using a web-based statistics test to teach statistics to post-registration nursing students. Nurse Education Today, 25(2), 156–163.
- Onwuegbuzie, A. (2004). Academic procrastination and statistics anxiety. Assessment & Evaluation in Higher Education, 29(1), 1–19.
- Schwartz, T.A. (2014). Flipping the statistics classroom in nursing education. Journal of Nursing Education, 53(4), 199–206.
- Wonder, A.H. & Otte, J.L. (2015). Active learning strategies to teach undergraduate nursing statistics: Connecting class and clinical to prepare students for evidence-based practice. Worldviews on Evidence-Based Nursing, 12(2), 126–127.