Dr. Sears and Ms. Goldsworthy are Nursing Professors, Durham College/University of Ontario Institute of Technology Collaborative Bachelor of Science in Nursing Program, and Dr. Goodman is Associate Professor, Faculty of Business and Information Technology, University of Ontario Institute of Technology, Oshawa, Ontario, Canada.
This study was supported in part by Durham College/University of Ontario Institute of Technology Innovation Research Funding.
Address correspondence to Kimberley Sears, PhD, RN, Nursing Professor, Durham College, 2000 Simcoe Road North, Oshawa, Ontario, L1H 7K4 Canada; e-mail: firstname.lastname@example.org.
In 2005, Ontario became the first and largest jurisdiction in North America to mandate a baccalaureate degree requirement for nursing registration (Clarke & Connolly, 2004). Research has shown that the higher the proportions of baccalaureate nurses in a unit, the lower the rates of mortality and failure-to-rescue (Aiken, Clarke, Cheung, Sloane, & Silber, 2003), with improved outcomes for both patients and nurses (O’Brien-Pallas et al., 2004). Nonetheless, concerns have been raised about the link between decreased clinical placement time in baccalaureate education and the students’ ability to safely perform skills such as medication administration. Robinson Wolf, Hicks, and Farley Serembus (2005) investigated medication errors linked to student practice and discovered that the primary causes of these errors were inexperience and distractions.
Recognizing the difficulty for Canadian nursing schools to find enough appropriate clinical placements to meet nursing students’ needs, Ontario’s Ministry of Health and Long Term Care provided $20 million ($CDN) in funding to schools of nursing to create simulation education laboratories (The Nursing Secretariat News, 2006). These laboratories provide a safe environment where realistic cases can be practiced, repeated, and debriefed to help students develop specific competencies.
To date, studies have been found that evaluated the students’ satisfaction with the simulation experience (Bearnson & Wiker, 2005). However, there is a gap in the research to demonstrate whether the knowledge and skills acquired in the simulated environment are transferable to actual clinical experiences.
To assess the effectiveness of these laboratories, a randomized control study was conducted to test whether a simulation-based educational intervention can in fact contribute to the success of new nurses in overcoming the risks of error and increase their safety in medication administration. Participants in this study were second-year bachelor of science in nursing (BScN) students, scheduled for placement in medical surgical or maternal child field environments.
Two types of errors were reported: actual medication administration errors and potential medication administration errors. As expected, the medication errors reported were generally potential errors because instructors intervened to prevent actual misadministration of medication.
The primary research question was “Does the use of a simulation by second year BScN students in medical/surgical and maternal child placement environments influence the students’ ability to safely administer medications?” For errors that occurred, additional descriptive data were also collected on possible contributing factors; however, at this stage, the factors were not formally analyzed. The test hypothesis was that students’ experience in the simulation laboratory would increase the students’ abilities to safely administer medications in the clinical area.
Ethics approval was granted for this research by the research ethics board (comparable to an institutional review board in the United States). In a preliminary session, second-year BScN students who were entering either a medical-surgical or an obstetrical placement were informed about the study and their opportunity to participate voluntarily. The volunteer list became the sampling frame.
The study used a randomized control group, posttest-only design. Fifty-four of the students who volunteered were randomly selected and initially assigned to one of three treatment groups (24 students in total) or to one of three control groups (30 students). The experiment also controlled for type of placement: There were 28 students assigned to maternal units, and 26 assigned to medical-surgical units (Table 1).
Table 1: Original Distribution of Student Placements
Sample sizes were constrained by available placement opportunities. For a simple test of whether the treatment group exhibited fewer errors than expected based on the relative numbers of treatment and control participants, the planned sample size could detect a minimum difference of approximately 14% with 70% power. For reasons of scheduling and student requests, some individuals shifted positions within the experimental structure prior to the commencement of the project; and an additional placement context was provided for several student controls for medical-surgical units.
The intervention consisted of replacing some early-term clinical hours with exposure to simulated case scenarios, related to the type of placement. The students were taking their pharmacology course in conjunction with this study, and none of the students could administer medications until the seventh week of the rotation, which occurred after the simulation intervention was completed. Therefore, all students were exposed to real-time medication administration in the clinical setting at the same time and had an equal opportunity to demonstrate success.
The data collection instrument was adapted from a survey developed by one of the authors (K.S.) in 2006. Clinical instructors completed one form for each medication error (or near-miss) that was observed. To preserve confidentiality for the students, no personal or demographic data were recorded for the incidents. Reports were stored in a locked cabinet and results were entered into SPSS version 13 software for analysis. These data were collected:
- The unit in which the incident occurred and the clinical instructor.
- Which one of the five rights was being violated by the error (i.e., right patient, right route, right drug, right time, or right dose) and what was the actual or potential severity of the outcome.
- Which factors (e.g., distraction or lack of experience) contributed to the incident, as measured on a Likert scale.
Once a report was generated from the instructor and the student, the report was forwarded to one of the principal investigators.
Face validity for the instrument was confirmed by the validation of several experts, who critiqued the survey for accuracy, wording, and appropriateness. The survey was developed from the literature and from an assessment of existing pediatric incident reporting systems. Content validity was assured through an in-depth examination of research in the field, as well as by the assessment of a group of experts asked to critique the level of appropriateness of each question’s content. Because judgment was required to answer many of the questions, interrater reliability was established through information sessions that were conducted with the clinical instructors prior to the commencement of the study.
The simulation experience took this form: When the students arrived in the simulation laboratory, they were dressed in their uniforms and prepared to practice as would be expected in a clinical session. They were assigned in groups of 5 students to work with a client on whom they received a report, and they were given 1 hour to prepare. They were then taken through a simulated experience in which they were expected to assess, plan, and perform appropriate nursing interventions, including medication administration.
A debriefing period at the bedside then followed, in which the students discussed what went well and what they noted as gaps in their knowledge. The students had 2 hours to fill these knowledge gaps and perform additional preparation. Thereafter, the students returned to the laboratory to reengage with the same simulation scenario as before to strengthen their knowledge and confidence.
The scenarios were built by experts in the obstetric and the medical surgical areas to incorporate two aspects: typical assessment and interventions (including types of medication) that the students would normally encounter on the units, and the most common emergency scenarios that students may encounter, to encourage critical thinking and prioritization in a rapidly changing situation. Although scenarios differed between the two placement groups, the students received a consistent format and list of expectations from the researchers. The outcomes from the scenarios were aligned with the assessments and the interventions that a second-year nursing student with a similar clinical scenario would be expected to perform. This expectation was based on the perceptions of two seasoned clinical professors and on the program-established competencies for the second-year clinical experience.
Following data collection, the number of incidents reported for the different experimental and control groups were compared. Conventional chi-square tests were used for an initial comparison of the actual to the expected frequencies. Differences in proportions (for error numbers relative to numbers of participants) were also tested. However, it was determined that the numbers of documented errors observed per group could be modeled more appropriately by the Poisson distribution as opposed to the approximately normal distribution expected by the chi-square test. This is because the variable “numbers of observed errors” per group is analogous to a variable like “numbers of recorded defects” per batch or shift, in a quality control test for production. Variation about the mean for such variables tends not to be symmetrical, but rather right skewed. Questionnaire results about incident details were also analyzed to assess for common threads and intercorrelations.
There was compelling evidence that collectively, students in clinical placement generate fewer medication errors if they have had prior exposure to a related, simulation-based experience. The controls had a disproportionately larger share of the errors (24 errors among a sample of 30 control group participants, compared with 7 errors among a sample of 24 treatment group participants) (Table 2). Assuming a Poisson distribution of the error numbers for testing significance, the p value becomes the probability, under the null hypothesis (given an overall error rate of 31 errors per 54 person-semesters), that the treatment group (of 24 person-semesters) would output no more than 7 of the errors (Table 3). The p value is significant, at less than 0.05.
Table 2: Recorded Errors for Treatment and Control Participants
Table 3: Expected and Actual Distribution of the Errors
If a conventional chi-square test is performed to compare the error rates of the treatment and control groups, respectively, for the two placement settings, the difference appears highly significant, with p < 0.001. Alternatively, using a simulation method that assumes intracell variation is Poisson distributed, the difference between actual and expected distributions is reconfirmed, with p < 0.01.
Other data collected in the form enabled a preliminary look at what contributing factors may be associated with the errors that occur. “Lack of knowledge” was a common element. Possible groupings and interactions of these factors will be examined in a future study.
The results of this study demonstrate that simulation did have an effect on the reduction of medication administration errors; however, some study limitations should be acknowledged. Although both groups were randomly assigned students, the two groups came from one collaborative nursing program; thus the results may not be generalizable to all nursing programs. Two community hospitals were used in this study to provide the clinical placements; therefore, one of the hospital medication systems may have been more user friendly for the students than the other because it used unit dose.
The necessity that different student groups had different clinical instructors could also potentially bias the reporting of the errors. Given that all clinical instructors are required (independently of this study) to record the occurrence and details of medication errors during placements, it seemed reasonable to expect a certain common level of training and experience among those observing the incidents.
To further validate this study, it should be replicated on a larger scale. It would be useful to explore for clusters among contributing factors for errors, as well as to explore whether there are interactions between the clusters and the types of errors.
A concern that was identified by the students prior to the commencement of this study was that their inability to perform well in the simulation laboratory might result in a failing mark in the clinical experience. This concern may have deterred some students from participating in the study. Students within this cohort were not familiar with the simulation equipment; after they had the opportunity to run through a night of simulation and see what the exercise consisted of, they noted that they no longer had this fear. Rather, the students noted that they were thrilled for this experience as it helped them identify their knowledge gaps and provided them with a safe opportunity to learn without potentially harming their client. In an attempt to reduce this concern for students, it may be advisable for the researchers who conduct the scenarios to not have a clinical group at the same time. However, given the funding received for this study, the professors were still expected to have one clinical group.
One caution would be the time and intensity required on the part of the professors to run the students through an 8-hour experience such as this. It would be worthwhile to retain the approach of rerunning a scenario after the students are given time to research, as this increased the students’ confidence in their ability. This self-confidence appears to have aided in the students’ performance in actual medication delivery.
This study adds to the knowledge in the area of simulation education in nursing, and its findings suggest that simulation education may contribute to a reduction in medication errors among novice nurses. The study further iden-tifies areas for further investigation in the area of simulation and patient safety and recommends that the study be replicated on a larger scale.
- Aiken, L.H., Clarke, S.P., Cheung, R.B., Sloane, D.M. & Silber, J.H. (2003). Educational levels of hospital nurses and surgical patient mortality. Journal of the American Medical Association, 290, 1617–1623. doi:10.1001/jama.290.12.1617 [CrossRef]
- Bearnson, C.S. & Wiker, K.M. (2005). Human patient simulators: A new face in baccalaureate nursing education at Brigham Young University. Journal of Nursing Education, 44, 421–425.
- Clarke, S.P. & Connolly, C. (2004). Nurse education and patient outcomes: A commentary. Policy, Politics & Nursing Practice, 5, 12–20. doi:10.1177/1527154403261623 [CrossRef]
- The Nursing Secretariat News. (2006, Spring/Summer). Retrieved June 2, 2006, from http://www.health.gov.on.ca/english/providers/program/nursing_sec/newsletters/nsnl_0506.pdf
- O’Brien-Pallas, L., Murphy, G., White, S., Hayes, L., Baumann, A. & Higgins, A., Tomblin et al. (2004). Building the future: An integrated strategy for nursing human resources in Canada. Mississauga, Ontario, Canada: The Nursing Sector Study.
- Robinson Wolf, Z., Hicks, R. & Farley Serembus, J. (2005). Characteristics of medication errors made by students during the administration phase: A descriptive study. Journal of Professional Nursing, 22, 39–51. doi:10.1016/j.profnurs.2005.12.008 [CrossRef]
Original Distribution of Student Placements
Recorded Errors for Treatment and Control Participants
Expected and Actual Distribution of the Errors