Journal of Nursing Education

Research Briefs 

Quasi-Experimental Evaluation of a Substance Use Awareness Educational Intervention for Nursing Students

David M. Cadiz, PhD; Sue S. Butell, MS, RN; Beverly J. Epeneter, EdD, RN; Basilia Basin, BS, RN, BC; Chris O'Neill, DMin, RN

Abstract

 

Click here to read a Letter to the Editor about this article. 

This article reports on a study that evaluated the effectiveness of an educational intervention, Addressing Nurse Impairment, for addressing nursing students’ knowledge acquisition, changes in self-efficacy to intervene, and changes in substance abuse stigma. A gap exists in nursing students’ education regarding the risks of addiction within the profession and how to handle a colleague suspected of having a substance use disorder. The seminar was adapted from an existing evidence-based prevention program called Team Awareness, as well as information from focus groups and a pilot test. A quasi-experimental pretest–posttest design was used to evaluate the effect of the seminar. When the control and experimental groups were compared, the results indicated that the seminar significantly affected knowledge and self-efficacy to intervene but did not significantly affect stigma. This research contributes to the body of evidence related to educational interventions for nursing students regarding substance abuse in the nursing profession.

Note. A Letter to the Editor about this article was published in the February 2013 issue and can be accessed here.

Abstract

 

Click here to read a Letter to the Editor about this article. 

This article reports on a study that evaluated the effectiveness of an educational intervention, Addressing Nurse Impairment, for addressing nursing students’ knowledge acquisition, changes in self-efficacy to intervene, and changes in substance abuse stigma. A gap exists in nursing students’ education regarding the risks of addiction within the profession and how to handle a colleague suspected of having a substance use disorder. The seminar was adapted from an existing evidence-based prevention program called Team Awareness, as well as information from focus groups and a pilot test. A quasi-experimental pretest–posttest design was used to evaluate the effect of the seminar. When the control and experimental groups were compared, the results indicated that the seminar significantly affected knowledge and self-efficacy to intervene but did not significantly affect stigma. This research contributes to the body of evidence related to educational interventions for nursing students regarding substance abuse in the nursing profession.

Note. A Letter to the Editor about this article was published in the February 2013 issue and can be accessed here.

Dr. Cadiz is Research Associate, and Dr. O’Neill is Program Director, WorkHealthy Oregon, Oregon Nurses Foundation, Tualatin; Ms. Butell is Professor, and Dr. Epeneter is Professor, Linfield-Good Samaritan School of Nursing; and Ms. Basin is Doctoral Student, Oregon Health and Sciences University School of Nursing, Portland, Oregon.

Funding for the development and evaluation of the Addressing Nurse Impairment seminar was provided to Dr. Cadiz and Dr. O’Neill by the Oregon Health Authority. The views expressed in this article do not reflect the official policies of the funder.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

Address correspondence to David M. Cadiz, PhD, Research Associate, Oregon Nurses Foundation, 18765 SW Boones Ferry Road, Suite 200, Tualatin, OR 97062; e-mail: cadiz@oregonrn.org.

Received: October 06, 2011
Accepted: March 21, 2012
Posted Online: May 15, 2012

The prevalence of substance use disorders among nurses has been estimated to be 6% to 8%, which is approximately the same rate as in the general population (Snow & Hughes, 2003). Evidence suggests that younger health professionals may be at higher risk (Kenna & Lewis, 2008). Risk factors include routine access to drugs in their practices and the stressful nature of their jobs (Trinkoff, Storr, & Wall, 1999). In addition, nursing faculty report having frequent encounters with nursing students whom they suspect of having a substance use disorder (Kornegay, Bugle, Jackson, & Rives, 2004).

Unfortunately, health care professionals often lack the awareness and skill set to recognize or assist a colleague with a substance abuse problem (Quinlan, 2003). In fact, nurses report feeling unprepared to identify and address substance use disorders (Rassool, 2004). This lack of preparedness often results in individuals not intervening when a colleague has performance problems suspected to be related to a substance use disorder. Nurses may enable nurse colleagues to continue unsafe performance by making excuses, ignoring problems, covering up mistakes, and accepting incomplete work (Quinlan, 2003). Thus, patient safety may be compromised when a nurse fails to intervene with a colleague’s unsafe practice in a timely way. In addition, the nurse with the substance use disorder may delay getting help or avoid seeking help altogether.

The nursing profession has clearly indicated a need for education. The National Student Nurses Association (NSNA, 2002) and the American Nurses Association (ANA, 2002) have passed resolutions that call for greater education of nursing students about the risk of addiction. Evidence demonstrates that education can develop a more positive and nonjudgmental attitude toward people with substance use disorders, develop confidence to intervene when impairment is present, and develop the skills to identify and assist people with a substance use disorder (Hagemaster, Handley, Plumlee, Sullivan, & Stanley, 1993; Rassool, 2004). However, the evidence is limited for nursing students, although it generally supports the efficacy of educational interventions to affect knowledge acquisition, attitude, and confidence to intervene (Rassool & Rawaf, 2008).

The aim of this article is to report on an educational intervention called Addressing Nurse Impairment, which was developed, implemented, and evaluated at a baccalaureate school of nursing located in the Pacific Northwest.

Seminar Development

A systematic approach was used to develop the Addressing Nurse Impairment seminar. A thorough review of the educational literature related to drug and alcohol intervention identified an evidence-based training program called Team Awareness (Bennett, Lehman, & Reynolds, 2000) that fit the purpose of the seminar. Team Awareness focuses on increasing positive social interactions to promote a healthy work culture and reduce the stigma associated with seeking help for substance use disorders (Bennett et al., 2000). Because a majority of the interventions using the Team Awareness program is with younger workers in a team environment (Bennett, Aden, Broome, Mitchell, & Rigdon, 2010; Einspruch, O’Neill, Jarvis, Vander Ley, & Raya-Carlton, 2011), it seemed well-suited to nursing students who are generally younger adults about to enter a workforce where a team-based work model is prevalent. However, two adjustments of the Team Awareness program were needed to meet the seminar’s requirements.

First, the Addressing Nurse Impairment seminar had to be completed in 2 hours, and the Team Awareness program is 8 hours in length. Therefore, two modules were selected from the original training based on their relevance to the purpose of the seminar—destigmatizing help-seeking for substance use disorders and positive social interactions. Destigmatizing help-seeking was selected because it increases awareness by discussing the stigma of addiction and the effectiveness of treatment and it promotes professional ethics as guidance. The positive social interactions module was selected because students in triads practice a communication model called NUDGE (Notice, Understand, Decide, use Guidelines, and Encourage; Bennett et al., 2010). Practicing the NUDGE skill increases confidence to speak up when impaired performance is suspected and how to counteract resistance when communicating concerns. Group exercises are more effective in improving help-seeking behaviors than an informational approach alone (Bennett & Lehman, 2001).

Second, the Team Awareness program needed to be customized for nursing students. Focus groups with nurse faculty and nursing students were used to guide this adaptation process, capturing information about the culture, vocabulary, relevant examples, and concerns specific to the academic institution. Several themes were integrated into the seminar, including the school’s substance use disorder policy, obstacles to seeking help, and the importance of confidentiality to reduce the stigmatization of someone who seeks treatment for a substance use disorder.

The seminar was piloted with a sample of nursing students. A pretest–posttest survey evaluation assessed changes in knowledge, self-efficacy, and stigma and solicited feedback about the seminar content and presentation. A matched sample of 42 pretest–posttest responses was used in our analysis. For each analysis, cases were removed due to missing data, which is the reason for the discrepancy between the matched sample and degrees of freedom. Using paired t tests, significant mean increases in knowledge (t = 3.62, df = 39, p < 0.01), self-assessed knowledge (t = 10.31, df = 39, p < 0.01), and self-efficacy (t = 6.93, df = 39, p < 0.01) were observed. However, there was no significant change in stigma (t = 1.31, df = 35). On the basis of the pilot, additional stigma-related content and increased role-play time were added to the seminar.

Method

Evaluation Design

A nonequivalent control group pretest–posttest design with a sample of nursing students was used to examine the effectiveness of the 2-hour seminar. This design was selected because random assignment to groups was not feasible. However, by collecting preintervention data, differences between groups were statistically controlled (Cook, Campbell, & Paracchio, 1990). For instance, differences on the preassessments on all of the outcome variables can be controlled. Moreover, to address threats to validity, such as history, the control group participants were from the same institution and were only one semester behind the experimental group in the program, so they were exposed to the same curriculum and institutional events. Maturation was also not a concern because the entire evaluation process was contained within a 4-week span. Also, by collecting preintervention and postintervention data for all participants, testing and reactivity effects stemming from being exposed to the same measures twice can be statistically examined (Cook et al., 1990).

Based on the results from similar educational interventions (e.g., Rassool & Rawaf, 2008), three positive changes were expected: (a) increased knowledge and awareness of substance use disorders in the workplace, (b) increased self-efficacy to intervene if a colleague is practicing unsafely, and (c) reduced stigma about colleagues with a substance use disorder.

Procedure

Several steps were involved in implementing the evaluation design, all of which received institutional review board approval from the school of nursing. The first step was participant recruitment. The instructors of the senior seminar course coordinated recruitment of the experimental group. The control group recruitment consisted of the first author speaking to two classes of students who were two semesters from graduation.

Second, survey invitations were e-mailed to each student enrolled in the three participating classes. Informed consent was received from all participants. Reminder e-mails were sent 1 week after the original survey invitation and again 2 days prior to the seminar. The postseminar survey was distributed immediately following the completion of the seminar. E-mail reminders were sent 1 week after the initial survey distribution and 2 days prior to closing the survey. The two surveys were the same except that demographic data were collected only in the preseminar survey, and self-rated knowledge was collected only in the postseminar survey.

Incentives were offered to increase the rate of participation. Participants were offered the chance to win one of three $50 gift cards. Participants qualified for the random drawing by filling out the preseminar and postseminar surveys.

Participants

Surveys were distributed to 173 nursing students (107 experimental and 66 control) at Time 1 (preseminar) and Time 2 (postseminar). At Time 1, 112 survey responses were received (74 experimental and 38 control) for a response rate of 65%. At Time 2, 99 survey responses were received (64 experimental and 35 control) for a response rate of 57%. The matched data from Time 1 and Time 2 resulted in 86 responses (56 experimental and 30 control) for a response rate of 50%. However, nine participants (2 experimental and 7 control) were removed from the analyses because they responded to less than 25% of the survey items on one or both surveys. Therefore, there was a total matched sample of 77 (54 experimental and 23 control); for a response rate of 45%. The mean age of the sample participants was 28.11 years (SD = 7.24 years), and 92% of the sample was female. The sample was 71% Caucasian, 9% Hispanic, 9% Asian/Pacific Islander, 3% Native American, and 8% indicated mixed ethnicity. Analysis of variance was used to examine for differences on age and gender between the two samples. The analysis identified that the control group was significantly older (Mcontrol = 32.96 and Mexp = 26.00; F[1, 74] = 18.18, p < 0.01) and had more men than the experimental group (Control = 22% and Experimental = 2%; F[1, 75] = 9.78, p < 0.01). Given these results, age and gender were controlled for in our analyses.

Measures

Demographics. Survey participants were asked for their age, gender, and ethnicity.

Knowledge Test. Ten knowledge test items were developed to sample seminar content and were adapted from a previous evaluation of a similar training program (Cadiz, Truxillo, & O’Neill, 2012). The knowledge test items were developed using a multiple choice format with three or four response alternatives each. Participants were asked to take the knowledge test in the preseminar and postseminar surveys.

Self-Rated Knowledge. Eleven self-rated knowledge items were developed to measure participant assessment of their level of knowledge and understanding of the seminar skills and goals. At Time 2, participants were asked to retrospectively assess their preseminar and postseminar levels of knowledge. Participants were asked to rate their knowledge after the training because they would not be able to rate their understanding of the seminar goals and skills prior to the seminar. Participants responded using a 5-point scale ranging from very low to very high. Cronbach’s alpha reliabilities for the self-assessed preknowledge and postknowledge were 0.94 and 0.95, respectively.

Self-Efficacy. Self-efficacy was measured because it has been shown to be a consistent predictor of learning and retention (Colquitt, LePine, & Noe, 2000). Self-efficacy was assessed using eight items capturing participant confidence to identify impairment, intervene, and respond to resistance during an intervention. Participants responded using a 5-point agreement scale. Self-efficacy was collected at Time 1 and Time 2. Cronbach’s alpha reliabilities for Time 1 and Time 2 were 0.82 and 0.86, respectively.

Substance Abuse Stigma. Substance abuse stigma was assessed with seven items from the perceived substance abuse stigma scale developed by Luoma et al. (2007). These items measured participant perceptions of the amount of stigma recovering nurses would face in their practice. Participants responded using a 5-point agreement scale. Stigma data were collected at Time 1 and Time 2. The Cronbach’s alpha reliabilities for Time 1 and Time 2 were 0.79 and 0.81, respectively.

Results

Analysis of covariance was used to examine differences between the control and experimental groups on knowledge, self-efficacy, and stigma. This type of analysis allows statistical control for differences between the two nonrandomly assigned groups. Age, gender, and the presurvey mean scores of knowledge, self-efficacy, and stigma were chosen as covariates in each of the respective analyses. These sources of variation were accounted for to provide a baseline for the statistical validity of the study and limit the effects of previous alcohol and drug education and personal and professional experiences. Of note, instead of using a calculated replacement method for missing data, participants were removed when one or more items for a measure were missing participant responses. This was deemed appropriate because no discernible missing data pattern was observed. For each statistically significant outcome, eta-squared is reported to reflect the effect size of the examined relationship. Eta-squared is the ratio of sum of squares for an effect to the total sum of squares, which is often interpreted as the variance in outcome by the independent variable and covariates.

Knowledge Test

Participant knowledge of the seminar material was higher after the seminar than it was before the seminar. A significant difference between the control (M = 6.13) and experimental groups (M = 7.81) was observed on the postseminar knowledge test after controlling for mean score on the preseminar knowledge test, age, and gender (F[1, 69] = 19.14, p < 0.01). The eta-squared of this difference was 0.21, which indicates that seminar participation and covariates account for 21% of the variance in the postknowledge test.

Self-Assessed Knowledge

Participants’ assessment of their level of understanding of the seminar material increased when retrospectively compared with preseminar knowledge. A significant difference was observed between the control (M = 2.84) and experimental (M = 4.02) groups on the postseminar assessment of understanding the seminar material (F[1, 68] = 53.13, p < 0.01). The partial eta-squared associated with this difference was 0.28, which means seminar participation and the covariates accounted for 28% of the variance in the postseminar self-assessed knowledge.

Self-Efficacy

Participant confidence to identify and intervene in cases where safe performance is in question was higher after the seminar than before the seminar. A significant difference between the control (M = 3.45) and experimental (M = 3.93) groups was observed on the postseminar measure of self-efficacy (F[1, 70] = 11.20, p < 0.01). The eta-squared of this difference was 0.12, which means that seminar participation and the covariates accounted for 12% of the variance in postseminar self-efficacy.

Stigma

The seminar did not significantly reduce participant-perceived substance abuse stigma. A significant difference between the control (M = 3.15) and experimental (M = 3.38) groups was not observed on the postseminar substance abuse stigma (F[1, 69] = 2.33).

Discussion and Implications

The results support the effectiveness of the Addressing Nurse Impairment seminar to affect knowledge about substance use disorders and a nurse’s ethical role in intervening when impaired practice behaviors are observed. Moreover, the seminar increased nursing students’ confidence to address colleagues whose unsafe practice may be related to a substance use disorder. Therefore, these findings contribute to addressing the existing gap in nursing students’ education about the risks of addiction within the profession and how to handle a colleague suspected of having a substance use disorder. In addition, the seminar fulfills the call from the ANA (2002) and the NSNA (2002) for greater education of nursing students about the risks of addiction. Ultimately, the knowledge, skills, and confidence gained from the seminar could reduce the potential for compromising patient safety when a nurse fails to intervene with a colleague’s unsafe practice in a timely way and may also increase the chance that nurse colleagues with a substance use disorder may be directed to the help they need.

The nonequivalent control group pretest–posttest design strengthens the interpretation that the seminar is effective. Control group preintervention data allowed for statistical control of preexisting differences between the groups and allowed investigation of the threats to the study’s validity. Specifically, testing effects (i.e., test score improvement due to taking the same knowledge test twice) were investigated, and no significant pretest and posttest differences were found in the control group on the knowledge test (t[22] = 0.94). This suggests the experimental group’s knowledge test improvement is related to seminar participation, rather than from taking the pretest. The reactivity effect—participants altering responses to conform to what they believe is expected—is also a concern of pretesting. However, control group self-efficacy did not significantly change (t[21] = −0.37), suggesting that multiple exposures to the measures did not affect participant responses. Therefore, the evaluation method used strengthens support for the seminar as a valuable educational intervention for schools of nursing to increase knowledge and confidence to handle colleagues with substance use disorders and safe practice concerns.

Finally, the results support that the Team Awareness (Bennett et al., 2000) program could be effectively adapted and used with nursing students. Introducing an established evidence-based substance abuse prevention program into the nursing curriculum is an important contribution because it provides a “best practice” that is identified as generally lacking in the nursing profession (National Council of State Boards of Nursing, 2011). In addition, Team Awareness is especially valuable because it focuses on the social context as a way to prevent substance use disorders and impaired practice that may drive a cultural change with regard to help-seeking and proactively addressing practice concerns in the workplace.

Potential Limitations and Future Research

As with all research, there are potential limitations that could have affected the results. First, the generalizability of the results could be of concern because the research was conducted at a single academic institution. Future research should examine the effectiveness of the training seminar at additional institutions. Second, the seminar’s long-term effect on future behavior was not evaluated. Future research should use a longitudinal design to investigate knowledge erosion and long-term behavioral effects. Finally, the nonsignificant change in stigma could be attributed to the seminar being a single 2-hour event. Future research should investigate whether multiple interventions over time may have a greater effect on stigmatized attitudes.

References

  • American Nurses Association. ( 2002, June–July. ). Resolution: Reaffirming the profession’s response to the problem of addictions and psychological dysfunctions in nursing. Resolution presented at the American Nurses Association House of Delegates. , Philadelphia, PA. .
  • Bennett, J.B., Aden, C., Broome, K., Mitchell, K. & Rigdon, W. (2010). Team resilience for young restaurant workers: Research-to-practice adaptation and assessment. Journal of Occupational Health Psychology, 15, 223–236. doi:10.1037/a0019379 [CrossRef]
  • Bennett, J.B. & Lehman, W.E.K. (2001). Workplace substance abuse prevention and help-seeking: Comparing a team-oriented and informational training. Journal of Occupational Health Psychology, 6, 243–254. doi:10.1037/1076-8998.6.3.243 [CrossRef]
  • Bennett, J.B., Lehman, W.E. & Reynolds, G.S. (2000). Team awareness for workplace substance abuse prevention: The empirical and conceptual development of a training program. Prevention Science, 1, 157–172. doi:10.1023/A:1010025306547 [CrossRef]
  • Cadiz, D., Truxillo, D. & O’Neill, C. (2012). Evaluation of a training program for nurse supervisors who monitor nurses in an alternative-to-discipline program, Advances in Nursing Science, 35, 1–10.
  • Colquitt, J.A., LePine, J.A. & Noe, R.A. (2000). Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85, 678–707. doi:10.1037/0021-9010.85.5.678 [CrossRef]
  • Cook, T.D., Campbell, D.T. & Paracchio, L. (1990). Quasi-experimentation. In Dunnette, M.D. & Hough, L.M. (Eds.), The handbook of industrial/organizational psychology (2nd ed., pp. 491–576). Palo Alto, CA: Consulting Psychologists Press.
  • Einspruch, E., O’Neill, C., Jarvis, K., Vander Ley, K. & Raya-Carlton, P. (2011). Substance abuse prevention in the electrical industry: The NECA-IBEW team awareness and team vigilance programs. In Bray, J., Galvin, D. & Cluff, L. (Eds.), Young adults in the workplace: A multi-site initiative of substance use prevention programs (pp. 73–101). Research Triangle Park, NC: RTI International.
  • Hagemaster, J., Handley, S., Plumlee, A., Sullivan, E. & Stanley, S. (1993). Developing educational programmes for nursing that meet today’s addiction challenges. Nurse Education Today, 13, 421–425. doi:10.1016/0260-6917(93)90117-K [CrossRef]
  • Kenna, G.A. & Lewis, D.C. (2008). Risk factors for alcohol and other drug use by healthcare professionals. Substance Abuse Treatment, Prevention, & Policy, 3, 1–8.
  • Kornegay, K., Bugle, L., Jackson, E. & Rives, K. (2004). Facing a problem of great concern: Nursing faculty’s lived experience of encounters with chemically dependent nursing students. Journal of Addictions Nursing, 15, 125–132. doi:10.1080/10884600490491222 [CrossRef]
  • Luoma, J.B., Twohig, M.P., Waltz, T., Hayes, S.C., Roget, N., Padilla, M. & Fisher, G. (2007). An investigation of stigma in individuals receiving treatment for substance abuse. Addictive Behaviors, 32, 1331–1346. doi:10.1016/j.addbeh.2006.09.008 [CrossRef]
  • National Council of State Boards of Nursing. (2011). Substance use disorder in nursing: A resource manual and guidelines for alternative and disciplinary monitoring programs. Chicago, IL: Author.
  • National Student Nurses Association. ( 2002, April. ). Resolution: In support of nursing school policies to assist and advocate for nursing students experiencing impaired practice. Resolution presented at the National Student Nurses Association House of Delegates. , Philadelphia, PA. .
  • Quinlan, D. (2003). Impaired nursing practice: A national perspective on peer assistance in the U.S. Journal of Addictions Nursing, 14, 149–155. doi:10.1080/10884600390245749 [CrossRef]
  • Rassool, G.H. (2004). Curriculum model, course development, and evaluation of substance misuse education for health care professionals. Journal of Addictions Nursing, 15, 85–90. doi:10.1080/10884600490450245 [CrossRef]
  • Rassool, G.H. & Rawaf, S. (2008). Predictors of educational outcomes of undergraduate nursing students in alcohol and drug education. Nurse Education Today, 28, 691–701. doi:10.1016/j.nedt.2007.11.005 [CrossRef]
  • Snow, D. & Hughes, T. (2003). Prevalence of alcohol and other drug use and abuse among nurses. Journal of Addictions Nursing, 14, 165–167. doi:10.1080/jan.14.3.165.167 [CrossRef]
  • Trinkoff, A.M., Storr, C.L. & Wall, M.P. (1999). Prescription-type drug misuse and workplace access among nurses. Journal of Addictive Diseases, 18, 9–17. doi:10.1300/J069v18n01_02 [CrossRef]
Authors

Dr. Cadiz is Research Associate, and Dr. O’Neill is Program Director, WorkHealthy Oregon, Oregon Nurses Foundation, Tualatin; Ms. Butell is Professor, and Dr. Epeneter is Professor, Linfield-Good Samaritan School of Nursing; and Ms. Basin is Doctoral Student, Oregon Health and Sciences University School of Nursing, Portland, Oregon.

Funding for the development and evaluation of the Addressing Nurse Impairment seminar was provided to Dr. Cadiz and Dr. O’Neill by the Oregon Health Authority. The views expressed in this article do not reflect the official policies of the funder.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

Address correspondence to David M. Cadiz, PhD, Research Associate, Oregon Nurses Foundation, 18765 SW Boones Ferry Road, Suite 200, Tualatin, OR 97062; e-mail: .cadiz@oregonrn.org

10.3928/01484834-20120515-02

Sign up to receive

Journal E-contents