Journal of Nursing Education

Educational Innovations 

Teaching Nursing Students the Importance of Treatment Fidelity in Intervention Research: Students as Interventionists

Amanda S. Eymard, DNS, APRN, PMHNP-BC, CNE; Gerry Altmiller, EdD, APRN, ACNS-BC

Abstract

Background:

Treatment fidelity is the extent to which a study team adheres to the dictates of a study protocol. It ensures that the intervention has been implemented as intended. Including and monitoring treatment fidelity reduces threats to validity and is vital to providing reputable, sound research.

Method:

This article describes specific measures taken to teach nursing students the key concepts and methodologic issues associated with treatment fidelity in intervention research studies through active learning. Twelve senior-level nursing students participated as interventionists in a study that spanned 4 months.

Results:

Treatment fidelity strategies included development of a manual, proper training and supervision of interventionists, a follow-up meeting with the interventionists, checklists, and a controller.

Conclusion:

The treatment fidelity strategies implemented throughout the study provided an opportunity for students to learn firsthand how such strategies are implemented and how they strengthened the confidence that study findings are indeed related to the intervention. [J Nurs Educ. 2016;55(5):288–291.]

Abstract

Background:

Treatment fidelity is the extent to which a study team adheres to the dictates of a study protocol. It ensures that the intervention has been implemented as intended. Including and monitoring treatment fidelity reduces threats to validity and is vital to providing reputable, sound research.

Method:

This article describes specific measures taken to teach nursing students the key concepts and methodologic issues associated with treatment fidelity in intervention research studies through active learning. Twelve senior-level nursing students participated as interventionists in a study that spanned 4 months.

Results:

Treatment fidelity strategies included development of a manual, proper training and supervision of interventionists, a follow-up meeting with the interventionists, checklists, and a controller.

Conclusion:

The treatment fidelity strategies implemented throughout the study provided an opportunity for students to learn firsthand how such strategies are implemented and how they strengthened the confidence that study findings are indeed related to the intervention. [J Nurs Educ. 2016;55(5):288–291.]

Treatment fidelity in intervention research is the extent to which a study team adheres to the dictates of a study protocol (Persch & Page, 2013). Bellg et al. (2004) described this as the methodologic strategies used to monitor and enhance the reliability and validity of intervention studies. Its importance cannot be overstated as it ensures that the intervention has been implemented as intended (Resnick et al., 2009). Because treatment fidelity establishes the reliability of the independent variable in intervention research, it ultimately bears on the application of study findings to clinical practice (Hinckley & Douglas, 2013), making it a key methodologic requirement of any sound study (Robb, Burns, Docherty, & Haase, 2010). Although many prelicensure nursing education programs require coursework focused on conducting and applying research findings, the concepts that support sound methodology are difficult for students to grasp because of their complexity. Active learning strategies can increase learning (Boctor, 2013; Everly, 2013) and change attitudes (Samra, Griffiths, Cox, Conroy, & Knight, 2013). For that reason, students were provided with an opportunity to learn through direct involvement in an ongoing research study. This article describes the specific measures undertaken to teach prelicensure nursing students concepts of treatment fidelity through participation in an interventional research project.

Background

Treatment fidelity is fundamental to the inference of validity in nursing intervention research and should not be considered optional. It supports both internal and external validity of intervention research by increasing specificity and understanding of what was tested during the study and by decreasing unintended interventions that can influence results (Resnick et al., 2009). In the past decade, investigators have increasingly addressed treatment fidelity when reporting research from behavior change-related interventions (Faulkner, 2012; Song, Happ, & Sandelowski, 2010).

Treatment fidelity consists of two key components, both of which are essential. The first component is the extent of adherence to the protocol, meaning how strictly the protocol is followed, and the second component is the competence of the interventionist's delivery of an intervention as intended, meaning skillfulness in technique (Faulkner, 2012). Both adherence and competence are required to achieve a high level of treatment fidelity, as competent implementation is impossible without adherence to treatment guidelines, and adherence alone is insufficient to ensure competent delivery (Torrey, 2012).

Significance to Nursing Education

Most baccalaureate nursing programs require a course specific to research. As novices to the profession, this course may seem irrelevant or insignificant to nursing students, but involvement in an intervention study provides an opportunity for students to experience active learning with application of the knowledge acquired in the research course. Such learning methods promote student engagement and have been shown to have positive effects on problem solving, critical thinking, and persistence (Popkess & McDaniel, 2011).

As students transition into practice, many will be called on to participate in research in their clinical practice. Facilities designated as Magnet status, as well as those seeking Magnet status, want to employ nurses prepared to conduct research. The Institute of Medicine (2011) recommends expanding opportunities for nurses to lead and manage collaborative efforts with transdisciplinary health care teams to conduct research for improving practice environments and health systems. Because it has become an expected skill, nurse educators need to provide opportunities for students to learn not only theory but also hands-on skills for implementing research.

Project Overview

Students were provided with an opportunity to participate in the “Take a Walk in My Shoes” study, which was designed by faculty at a southeastern state university using a two-phase mixed-methods sequential explanatory design. The study's focus was the effect of an educational intervention on changing the attitudes of certified nursing assistants (CNAs) toward older adults in long-term care settings by using a low-fidelity simulation. Senior-level nursing students were invited to participate as the interventionists. The results of the study are reported elsewhere in two publications (Eymard, Crawford, & Keller, 2010; Eymard & Douglas, 2012); therefore, for the purpose of this article, discussion will focus on the student learning experience rather than on the study findings.

Method

Twelve senior-level baccalaureate nursing students were recruited to deliver the intervention. Institutional review board approval was obtained from the university. Ethical consideration provided that student participation in the study was voluntary and at no time were any of the participating students graded or evaluated for coursework by the researcher. Student participants received information regarding the purpose of the study before agreeing to participate, and they were informed that they could relinquish participation at any time during the study. All nursing home administrators gave written consent for the study to be conducted at their facilities, and written informed consent was obtained from each CNA participant.

A convenience sample of 165 CNAs was recruited from 12 nursing homes, ranging in size from 110 to 125 beds. The nursing homes were assigned to Group A or B using the technique of mixed randomization. This resulted in 126 CNAs at eight facilities being assigned to the treatment group, and 39 CNAs at the remaining four facilities were assigned to the comparison group, which did not receive the intervention; a hand washing in-service program was provided instead.

The “Take a Walk in My Shoes” study consisted of a simulation experience designed to be an interactive educational intervention for CNAs working in long-term care residences or nursing homes. The hypothesis was that experiencing the common health conditions associated with aging and challenges of activities of daily living would positively impact the attitudes of CNAs toward those for whom they care.

The simulation consisted of four different stations. The CNAs as learners were placed into one of four groups and rotated through each station in 15-minute intervals. Each station consisted of a table holding equipment and necessary props that would allow learners to experience some of the common conditions of aging, such as decreased visual acuity, difficulty with movement, delayed wound healing, and generalized de-conditioning (Table). Role-playing in the simulation activity was meant to sensitize learners to the experience of aging with disability and promote self-examination of feelings toward caring for older adults (Samra et al., 2013; Yu & Chen, 2012).


Simulation Stations and Equipment

Table:

Simulation Stations and Equipment

Treatment Fidelity Plan

Manual

A well-developed standardized training manual is the gold standard of intervention therapies (Resnick et al., 2009). Developing and providing a manual with a set protocol supports internal validity as it reduces deviations from the original research plan. It provides an objective means for comparing interventions, reducing variability resulting from interventionist effects, and fostering replication (Hinckley & Douglas, 2013). For this study, an extensive training manual was developed with specific guidelines for the simulation equipment and a detailed standardized intervention protocol to promote the application of the treatment in a consistent manner.

Training and Supervision

The purpose of training and supervising is to refine the skills of the interventionists in delivering the intervention and decrease variability that occurs with protocol violations (Persch & Page, 2013). Because the interventionists were senior-level nursing students and novices to the interventionist's role, great attention was paid to this aspect of establishing treatment fidelity. Students attended a mandatory 4-hour training session that allowed them to become familiar with the equipment and learn the detailed intervention protocol. The manual was reviewed and referred to frequently during the training session.

The training included a presentation developed by the primary investigator (A.S.E.) to review respiratory, musculoskeletal, vision, and hearing disease processes experienced by older adults. As suggested by Faulkner (2012), the interventionists were given the opportunity to role-play the intervention, practicing techniques and strategies during their training session as part of their preparation. The role-playing experiences were impromptu, with interventionists being presented with possible scenarios regarding a simulation station or particular simulation equipment and then asked to demonstrate the appropriate response. Checklists were used to ensure that treatment and interventionists' responses were consistent. Having a well-defined protocol with a high degree of structure in the intervention design worked to reduce variance while serving to increase the comfort level of these novice interventionists.

Follow-Up Meeting

One week after the initial training session, a follow-up meeting was held to ensure the protocol was well understood. Interventionists were challenged to restate what was going to occur at the educational sessions and to demonstrate the simulations they would be providing for the CNAs to ensure adherence to the protocol. Student interventionists also were given the opportunity to pose questions to the primary investigator.

Checklists

Intervention monitoring instruments, such as a criterion checklist, serve as a method of verifying treatment fidelity. A dichotomous checklist indicating present or absent was developed, and the checklist included all necessary equipment to be used for each simulation station. Interventionists used the checklists to assess for appropriate equipment use, as well as protocols regarding implementation of the intervention and the number of CNAs attending each simulation station. The checklists were used at each of the eight participating nursing homes and served as a safeguard to ensure the treatment was applied reliably throughout the study.

Controller

A controller was recruited and trained regarding the standardized protocol. Having a controller to conduct checks that verify treatments are being delivered according to the protocol supports treatment fidelity by ensuring consistency among the interventionists at the different locations (Persch & Page, 2013). The controller, who was not identified to the interventionists to reduce bias, randomly visited two of the intervention locations. Neither the primary investigator nor the interventionists were notified regarding which locations the controller would visit during the intervention phase of the study. The same checklists developed for the interventionists, as well as field notes, were used by the controller when visiting the intervention locations. No deviations from protocol were identified, but had they been the controller would have reported these findings to the primary investigator so that further training could be initiated.

Evaluation

At the end of the first phase of the study, students completed evaluations consisting of open-ended questions asking them to describe how they were affected by the study and what benefits they gained from serving as interventionists. Open-ended questions that allow for students to reflect on and share personal meaning gained from the experience have been shown to be an effective method of evaluating and validating student learning (Everly, 2013).

Student responses indicated that they obtained experience interacting with and educating health care employees, acquired leadership skills, and gained experience in a long-term care environment. Responses demonstrated students' recognition of the importance of treatment fidelity strategies in strengthening a study's findings. Some students noted that before this experience, they did not realize the importance of adherence to the protocol as a way to ensure that everyone received the same treatment so that the measurement of the treatment reflected consistency. Students also noted that until participating in this study as interventionists, they did not realize different investigators could perceive and interpret the instructions of a study in different ways that could erroneously impact a study's findings.

These types of statements aligned with understanding the concepts of adherence and competence that the nurse educators involved in this teaching strategy had hoped would be realized by the students. Several of the students expressed they had never visited a nursing home and identified it as a good learning experience that brought nursing research to life for them. Comments included that they enjoyed their work as interventionists and that they learned about older adults. In addition to providing an opportunity for students to reflect on their learning, the evaluation challenged them to think about research in practice by asking for suggestions for future studies.

Impact on Students

The students who participated in the “Take a Walk in My Shoes” study gained hands-on experience with the research process and participated in activities to enhance reliability and validity in an intervention study by incorporating treatment fidelity measures. The authors of this article, who are nurse educators, valued this learning experience for the students because although these students were exposed to intervention studies and the importance of treatment fidelity in their coursework, research indicates that simulation and active learning has a greater effect on knowledge building and changing attitudes than solely providing information (Everly, 2013; Popkess & McDaniel, 2011; Samra et al., 2013; Sullivan-Mann, Perron, & Fellner, 2009). Including students as interventionists for this study supported active learning methods.

The student interventionists received additional information at the beginning of the “Take a Walk in My Shoes” study focused on how treatment fidelity was supported within the study, including a manual, training and supervision, follow-up meeting, checklists, a controller, and evaluation. The students not only learned treatment fidelity was a methodologic requirement, but they also were exposed to the strategies to enhance treatment fidelity and learned about the significance of each strategy throughout the study. The students actively participated in incorporating the treatment fidelity strategies within the study, which enhanced their learning by making the content relevant to their future roles of practitioner and researcher. The “Take a Walk in My Shoes” study provided students with an engaging and active learning environment that allowed them to learn through participation in research.

Limitations

Although students were tested to determine knowledge gained about implementing and understanding research, it is difficult to determine whether that knowledge was gained from classroom teaching or gained specifically through this active learning experience. Future endeavors related to teaching strategies can include research using a control group to determine the gains made by utilizing active teaching and simulation.

Conclusion

Establishing treatment fidelity is a crucial component of intervention research. It must be addressed by the researcher when planning, implementing, and evaluating the results of the study. Involving students as the interventionists in this study provided an opportunity for students to learn firsthand the importance of addressing treatment fidelity in intervention research, as well as the importance of constructing a detailed plan to ensure that the protocol is followed. The students were involved in each component of establishing treatment fidelity, and the concepts of adherence and competence were stressed. By having the opportunity to actively participate in this study as interventionists, senior nursing students learned that including and monitoring treatment fidelity is vital to providing reputable, sound research results.

References

  • Bellg, A.J., Borrelli, B., Resnick, B., Hecht, J., Minicucci, D.S., Ory, M. & Czajkowski, S. (2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH behavior change consortium. Health Psychology, 23, 443–451. doi:10.1037/0278-6133.23.5.443 [CrossRef]
  • Boctor, L. (2013). Active-learning strategies: The use of a game to reinforce learning in nursing education: A case study. Nurse Education in Practice, 13, 96–100. doi:10.1016/j.nepr.2012.07.010 [CrossRef]
  • Eymard, A., Crawford, B. & Keller, T. (2010). “Take a walk in my shoes”: Nursing students take a walk in older adults' shoes to increase knowledge and empathy. Geriatric Nursing, 31, 137–141. doi:10.1016/j.gerinurse.2010.02.008 [CrossRef]
  • Eymard, A. & Douglas, D. (2012). Ageism among healthcare providers and interventions to improve their attitudes toward older adults: An integrative review. Journal of Gerontological Nursing, 38(5), 26–35. doi:10.3928/00989134-20120307-09 [CrossRef]
  • Everly, M.C. (2013). Are students' impressions of improved learning through active learning methods reflected by improved test scores?Nurse Education Today, 3, 148–151. doi:10.1016/j.nedt.2011.10.023 [CrossRef]
  • Faulkner, M.S. (2012). Intervention fidelity: Ensuring application to practice for youth and families. Journal for Specialists in Pediatric Nursing, 17, 33–40. doi:10.1111/j.1744-6155.2011.00305.x [CrossRef]
  • Hinckley, J.J. & Douglas, N.F. (2013). Treatment fidelity: Its importance and reported frequency in aphasia treatment studies. American Journal of Speech-Language Pathology, 22, S279–S284. doi:10.1044/1058-0360(2012/12-0092) [CrossRef]
  • Institute of Medicine. (2011). The future of nursing: Leading change, advancing health. Washington, DC: National Academies Press.
  • Persch, A.C. & Page, S.J. (2013). Protocol development, treatment fidelity, adherence to treatment, and quality control. The American Journal of Occupational Therapy, 67, 146–153. doi:10.5014/ajot.2013.006213 [CrossRef]
  • Popkess, A.M. & McDaniel, A. (2011). Are nursing students engaged in learning? A secondary analysis of data from the national survey of student engagement. Nursing Education Perspectives, 32, 89–94. doi:10.5480/1536-5026-32.2.89 [CrossRef]
  • Resnick, B., Galik, E., Pretzer-Aboff, I., Gruber-Baldini, A.L., Russ, K., Cayo, J. & Zimmerman, S. (2009). Treatment fidelity in nursing home research. Research in Gerontological Nursing, 2(1), 30–38. doi:10.3928/19404921-20090101-09 [CrossRef]
  • Robb, S., Burns, D., Docherty, S. & Haase, J. (2010). Ensuring treatment fidelity in a multi-site behavioral intervention study: Implementing NIH behavior change consortium recommendations in the SMART trial. Psycho-Oncology, 20, 1193–1201. doi:10.1002/pon.1845 [CrossRef]
  • Samra, R., Griffiths, A., Cox, T., Conroy, S. & Knight, A. (2013). Changes in medical student and doctor attitudes toward older adults after an intervention: A systematic review. Journal of the American Geriatrics Society, 61, 1188–1196. doi:10.1111/jgs.12312 [CrossRef]
  • Song, M., Happ, M. & Sandelowski, M. (2010). Development of a tool to assess fidelity to a psycho-educational intervention. Journal of Advanced Nursing, 66, 673–682. doi:10.1111/j.1365-2648.2009.05216.x [CrossRef]
  • Sullivan-Mann, J., Perron, C.A. & Fellner, A.N. (2009). The effects of simulation on nursing students' critical thinking scores: A quantitative study. Newborn and Infant Nursing Reviews, 9(2), 111–116. doi:10.1053/j.nainr.2009.03.006 [CrossRef]
  • Torrey, A.R. (2012). Treatment fidelity instrument to measure a brief opportunistic intervention for prenatal substance use. Journal of Obstetric, Gynecologic, and Neonatal Nursing, 41, 809–820. doi:10.1111/j.1552-6909.2012.01413.x [CrossRef]
  • Yu, C.Y. & Chen, K.M. (2012). Experiencing simulated aging improves knowledge of and attitudes toward aging. Journal of the American Geriatric Society, 60, 957–961. doi:10.1111/j.1532-5415.2012.03950.x [CrossRef]

Simulation Stations and Equipment

StationEquipmentDescription
Physical LimitationsThe Physical Limitations Simulation SuitAddresses limited movement, dull touch, and stiff joints
Vision and HearingChronic Challenges Simulation Glasses SetSix pairs of glasses that simulate cataracts, glaucoma, macular degeneration, retinopathy, hemianopia, and detached retina
Empathy LungThe Empathy Lungs® C.O.P.D. SimulatorWorn around the torso along with a customized airway
Skin IntegrityDecubitus Wound Care ModelDepicts four stages of pressure ulcers
Unhealthy Foot Care KitA foot replica that shows an invasive wound
Wound Healing DisplayShows three different stages of wounds
Authors

Dr. Eymard is Associate Professor of Nursing, Nicholls State University, Thibodaux, Louisiana; and Dr. Altmiller is Associate Professor of Nursing, The College of New Jersey, Ewing Township, New Jersey.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

Address correspondence to Amanda S. Eymard, DNS, APRN, PMHNP-BC, CNE, Associate Professor of Nursing, Nicholls State University, 151 East 26th Place, Larose, LA 70373; e-mail: Amanda.eymard@nicholls.edu.

Received: May 05, 2015
Accepted: February 18, 2016

10.3928/01484834-20160414-09

Sign up to receive

Journal E-contents