Journal of Nursing Education

Research Briefs 

Evaluation of an Asynchronous Online Interprofessional Clinical Experience for DNP Students

Mary Lou Manning, PhD, CRNP; Monika Pogorzelska-Maziarz, PhD, MPH

Abstract

Background:

Little evidence exists guiding interprofessional experiences in Doctor of Nursing Practice (DNP) programs, especially in online learning environments. The primary aim of this study was to evaluate a DNP program interprofessional clinical experience.

Method:

Students enrolled in an online asynchronous DNP 15-week course used direct observation and individual and group reflection to evaluate team interprofessional collaboration. Observations were measured using the Jefferson Teamwork Observation Guide (JTOG).

Results:

Eleven students conducted 84 interprofessional observations. The overall mean score across all JTOG observations was 3.44 (SD ± .48). No statistically significant differences exist in the overall or domain-specific scores by observer participation in the team or by type of setting. Corresponding reflections tended to support JTOG scores.

Conclusion:

Findings suggest that this experience increased students' own personal awareness of interprofessional practice and helped them connect new learning to existing knowledge. How this translates into actual clinical or leadership practice is unknown and provides an important area for future investigation. [J Nurs Educ. 2017;56(10):618–622.]

Abstract

Background:

Little evidence exists guiding interprofessional experiences in Doctor of Nursing Practice (DNP) programs, especially in online learning environments. The primary aim of this study was to evaluate a DNP program interprofessional clinical experience.

Method:

Students enrolled in an online asynchronous DNP 15-week course used direct observation and individual and group reflection to evaluate team interprofessional collaboration. Observations were measured using the Jefferson Teamwork Observation Guide (JTOG).

Results:

Eleven students conducted 84 interprofessional observations. The overall mean score across all JTOG observations was 3.44 (SD ± .48). No statistically significant differences exist in the overall or domain-specific scores by observer participation in the team or by type of setting. Corresponding reflections tended to support JTOG scores.

Conclusion:

Findings suggest that this experience increased students' own personal awareness of interprofessional practice and helped them connect new learning to existing knowledge. How this translates into actual clinical or leadership practice is unknown and provides an important area for future investigation. [J Nurs Educ. 2017;56(10):618–622.]

In 1999, the Institute of Medicine (IOM) released To Err Is Human: Building a Safer Health System, and in 2001, its companion, Crossing the Quality Chasm: A New Health System for the 21st Century. The two reports document the scope of quality and safety problems in the U.S. health care system, offer analysis of the problems, and recommend strategies for major reform. Recognizing the magnitude of the changes required to address the challenge, the IOM launched the Quality Chasm series. To date, 12 reports have been produced in this series addressing various aspects of the national agenda for health care reform. Threaded through this series are key attributes of the architecture needed to build a safe, consistently high-performing health care system (Wakefield, 2008). One such attribute is the need for teamwork and interprofessional collaboration among health care providers. In addition, the IOM (2010) report The Future of Nursing: Leading Change, Advancing Health also identifies interprofessional collaboration (IPC) as an essential component of improving the accessibility, quality, and value of health care in the United States.

The American Association of Colleges of Nursing's (2006) Essentials of Doctoral Education for Advanced Nursing Practice states that Doctor of Nursing Practice (DNP) graduates should have preparation in “methods of effective team leadership and are prepared to play a central role in establishing interprofessional teams, participating in the work of the team, and assuming leadership of the team when appropriate” (p. 14). More specifically, DNP Essential VI, Interprofessional Collaboration for Improving Patient and Population Health Outcomes, indicates that DNP programs integrate IPC content and clinical practice opportunities.

The Interprofessional Education Collaborative (IPEC) expert panel (2011) core competencies provide a useful framework for guiding the integration of IPC content in DNP curriculum. Developed in 2011, the nationally defined competencies include four core domains: values/ethics, roles/responsibilities, communication, and teams/teamwork. Each domain identifies specific interprofessional competencies. In 2016, the IPEC expert panel (2016) updated the competencies positioning IPC as the central domain under which the original four core domains are now organized. Although the IPEC expert panel (2016) competencies help guide program and course content, little guidance exists on how to apply the competencies in development of clinical practice learning opportunities incorporating IPC in online asynchronous DNP programs.

Realizing the importance of advancing DNP student IPC competencies and the potential positive effects that DNP students had on interprofessional collaborative practice, a new clinical assignment was created and tested in the authors' existing 15-week asynchronous online post-Master's DNP Leadership and Interprofessional Collaboration course. The assignment, linked to course objectives and supported by existing theoretical content, consisted of three interrelated, learner-centered activities:

  • Conduct interprofessional team observations and evaluate the interaction using the Jefferson Teamwork Observation Guide (JTOG), a survey tool designed to assess interprofessional behavior of teams (Lyons, Giordano, Speakman, Smith, & Horowitz, 2016).
  • Write a reflective narrative for each observation.
  • Participate in a reflective group discussion.

The goals of the assignment were to provide a vehicle for students to apply course content by evaluating their ability to recognize and assess competency-based characteristics of interprofessional teams and to reflect on their attitudes, beliefs, or assumptions about what drives successful IPC practice. The primary objective of this article is to share the authors' experience in integrating an IPC clinical experience into an existing asynchronous online DNP course. The secondary objective is to evaluate the usefulness of the JTOG in measuring IPC by doctoral nursing students. To the authors' knowledge, this is the first report of DNP students using the JTOG to evaluate team IPC.

Method

The university's institutional review board approved the use of the JTOG by doctoral students in course-related settings. IPC was defined as “When multiple health workers from different professional backgrounds work together with patients, families, and communities to deliver the highest quality of care” (World Health Organization, 2010, p. 13).

Interprofessional Team Observations

Students were instructed to (a) observe 10 interprofessional teams in action in their workplace, their organization, or other health care organization, (b) evaluate the team interaction using the JTOG, and (c) keep a structured journal with a description of each observation (including location, date, and time) with a reflective narrative.

The JTOG is an easy-to-use survey tool consisting of 14 items (Table 1), each item corresponding to an IPEC competency and indicative of highly collaborative teams (Lyons et al., 2016). The tool includes five interprofessional competency areas: values and ethics, roles and responsibilities, communication, teamwork, and leadership. Each item is measured on a 4-point Likert scale (1 = strongly disagree to 4 = strongly agree). Total possible score range is 14 to 56 points. Additional data regarding the type of team observed (e.g., clinical rounds, patient care meeting, simulation), the health professional represented on the observed team (e.g., nurse, physician, pharmacist), the setting where the observation occurred (e.g., inpatient, primary care office), and whether the student is a member of the team were also included.

Jefferson Teamwork Observation Guide Items and Corresponding IPECa

Table 1:

Jefferson Teamwork Observation Guide Items and Corresponding IPEC

The JTOG tool and related guidelines were discussed with students via a conference call conducted during the first week of class. The JTOG guidelines provide specific examples for behaviors, verbal and nonverbal, that might indicate the presence or absence of each of the team characteristics (Lyons et al., 2016).

Students entered JTOG results into a SurveyMonkey® database via their smartphone, tablet, or computer. Statistical evaluation included calculation of mean overall scores, as well as scores for each of the five domains, to examine the behaviors of the observed teams. Bivariate analyses were conducted to assess whether scores differed by setting (i.e., inpatient, out-patient, other), type of observation (i.e., patient care planning or process/system improvement), or observer participation in the team.

Reflection

Students were instructed to write a one- to two-paragraph reflective narrative for each IPC observation. They were asked to consider their own IPC strengths and weaknesses and identify additional learning needs (Mann, Gordon, & MacLeod, 2009). Guiding questions included: “During this observation, how did you feel, what went well and what concerned you, and what have you learned about your own IPC competencies?” The narratives were not entered into the SurveyMonkey database, but collated into a cumulative journal and submitted to the course board. To facilitate learning from and with peers, students also participated in a synchronous reflective group discussion during the final week of class. They were asked to share stories about their observational experiences; reflect on their attitudes, beliefs, or assumptions about what drives IPC practice; and suggest how they can apply what they learned to lead and facilitate IPC in their own clinical practice.

Results

Eleven students were enrolled in the course. All students were employed full time and had a working knowledge of SurveyMonkey. Between June 1 and August 5, 2016, students conducted 120 observations in 17 health care facilities or settings. Thirty-six observations were not interprofessional and were not included in the analysis. The majority of observations occurred in acute care hospitals (51%) and outpatient settings (43%), and most frequently included nurses (95%), physicians (81%), social workers (40%), and pharmacists (35%). Observation characteristics are included in Table 2. The overall mean score across all JTOG observations was 3.44 (SD ± .48). The mean scores across the five JTOG domains ranged from 3.54 (SD ± .49) for roles/responsibilities and 3.37 (SD ± .65) for values/ethics. No statistically significant differences existed in the overall or domain-specific scores by observer participation in the team or by type of setting. However, patient care observations were scored higher on the values and teamwork domains (p = .014 and .061, respectively), as compared with the observations of the process/system improvement meeting.

Characteristics of Students' Observations (N = 84)

Table 2:

Characteristics of Students' Observations (N = 84)

A written reflection accompanied each observation (100%). In light of the consistently high JTOG scores, the scores were carefully compared with the corresponding written reflections. In most cases, the assigned scores were supported by detailed, positive, narrative statements and thoughtful analysis. In addition, the reflections provided a much richer description of the teamwork encounter that was not captured in the JTOG scores, as demonstrated by students' comments related to communication:

  • The team demonstrated that they can troubleshoot through conflict in a diplomatic fashion. The team focused on the problem at hand and not any individual.
  • During this patient/family meeting, the physician did most of the talking. The nurse, social worker, patient, and family members were placed in the role of listeners. It made me uncomfortable and more aware of the effect of my own behavior.

The asynchronous reflective group discussion was intended as a complementary reflective learning approach. Within this interactive community of practice, students were able to build on one another's experiences, explore differing perspectives, support and challenge each other's IPC assumptions, and gain new insights, as highlighted by the following:

  • I find myself more interested in the negative [team] comments and wanting to gain a deeper understanding of the opposite view of others.
  • These experiences gave me a greater appreciation for the roles of different health professionals and how they can be engaged or silenced by the team facilitator [or] others.

Discussion

DNP programs must prepare graduates to make measurable, relevant contributions to a clinical practice environment that is highly collaborative and interprofessional; however, little evidence exists to guide faculty in incorporating interprofessional activities in asynchronous online courses. Post-Master's nurses enrolled in DNP programs represent a unique group of learners that bring diverse backgrounds of clinical and leadership experiences to the learning environment. This clinical assignment provided these students the opportunity to build on their experiences by challenging their assumptions about IPC practice, both as a participant and as a leader. Students also had the freedom to choose which teams to observe, providing flexibility in meeting their individual learning needs and preferences.

Several instruments are available to measure the features related to IPC practice. The JTOG was selected because it is a validated tool, is based on the IPEC competencies, and has been demonstrated to be easy to use in real-time observational situations. Students indicated that conducting multiple observations using the JTOG reinforced their knowledge of the “specific components of a successful interprofessional team experience,” as well as introduced them to a tool they could consider using in their own practice settings. Prelicensure nursing students and graduate students have successfully used the JTOG (Lyons et al., 2016), and this experience indicates the usefulness of the JTOG for advanced nursing students and professionals. To the authors' knowledge, this is the first report to describe doctoral students' use of the tool.

Reflection is a core skill needed for successful interprofessional practice (Doll et al., 2013) and may be most useful when viewed as a learning strategy (Mann et al., 2009). Self-reflection is also a key to effective leadership (Heckemann, Schols, & Halfens, 2015). The use of written narratives is a pedagogical approach that provides a reflective and interpretive framework for analyzing clinical experiences (Asselin, 2011; Asselin, Schwartz-Barcott, & Osterman, 2012). Group reflective discussions have been found to help students reach deeper levels of exploration (Miraglia & Asselin, 2015). Student reflections revealed thoughtful insights about the facilitators and barriers of interprofessional practice and consistently identified an increase in their own personal awareness of IPC practice.

Course evaluations were overwhelmingly positive. All students strongly agreed that the team observation/evaluation assignment facilitated meeting course objectives, 100% strongly agreed or agreed that the assignment enabled them to apply leadership theory to practice, and 100% strongly agreed or agreed that the assignment enhanced their critical thinking skills. Written evaluation comments included, “I now valued the importance of building new interprofessional leadership relationship,” “[I am] using my new interprofessional team observation skills to improve patient rounds in my workplace,” and “The team observation assignment required me to really dig deep within myself (it was difficult!), which helps me understand myself better as a person and as a future leader.”

Limitations

The limitations of this educational strategy must be noted. First, the assignment required students to observe 10 inter-professional teams within a 15-week period, and this specific number was chosen for reasons of feasibility and flexibility. In the authors' experience, the number of observations assigned was sufficient for most students to observe different types of clinical teams, in different settings, and become proficient with the JTOG tool. However, fewer observations or observations within a shorter time frame may also be sufficient to achieve the objective. Second, several students had limited access to interprofessional teams, most notably students employed as nurse educators and health care administrators. Many of their planned interprofessional encounters were unexpectedly uniprofessional. Another student employed as a nurse practitioner at a minute clinic was the only provider on staff, and all interprofessional interaction was conducted by telephone. The JTOG tool was not designed to assess such interaction. Thus, 30% of observations were not interprofessional. This finding provided an important insight for faculty. Faculty did not anticipate this student challenge when creating the assignment and will need to address this in the future. Finally, observations were self-reported and not validated by faculty. Faculty did compare each JTOG submission with the corresponding written description/reflection and found no evidence of fabricated observations.

Conclusion

Experiential learning (i.e., IPC observations using a standardized tool), coupled with individual and group reflection, helped DNP students enrolled in an asynchronous online course to reach new levels of understanding, meaning, and insights about IPC practice and increased their own personal awareness of IPC practice. How this translates into actual clinical or leadership practice is unknown and provides an important area for future investigation. Although this is a report describing one clinical practice assignment in one course, it provides an exemplar on how to successfully integrate IPC experience into an existing asynchronous online DNP course.

References

  • American Association of Colleges of Nursing. (2006). The essentials of doctoral education for advanced nursing practice. Retrieved from http://www.aacnnursing.org/Portals/42/Publications/DNPEssentials.pdf
  • Asselin, M.E. (2011). Reflective narrative: A tool for learning through practice. Journal for Nurses in Staff Development, 27, 2–6. doi:10.1097/NND.0b013e3181b1ba1a [CrossRef]
  • Asselin, M.E., Schwartz-Barcott, D. & Osterman, P.A. (2012). Exploring reflection as a process embedded in experienced nurses' practice: A qualitative study. Journal of Advanced Nursing, 69, 905–914. doi:10.1111/j.1365-2648.2012.06082.x [CrossRef]
  • Doll, J., Packard, K., Furze, J., Huggett, K., Jensen, G., Jorgensen, D. & Maio, A. (2013). Reflections from an inter professional education experience: Evidence for the core competencies for inter professional collaborative practice. Journal of Interprofessional Care, 27, 194–196. doi:10.3109/13561820.2012.729106 [CrossRef]
  • Heckemann, B., Schols, J.M. & Halfens, R.J. (2015). A reflective framework to foster emotionally intelligent leadership in nursing. Journal of Nursing Management, 23, 744–753. doi:10.1111/jonm.12204 [CrossRef]
  • Institute of Medicine. (1999). To err is human: Building a safer health system. Washington, DC: National Academy Press.
  • Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.
  • Institute of Medicine. (2010). The future of nursing: Leading change, advancing health. Retrieved from http://books.nap.edu/openbook.php?record_id=12956&page=R1
  • Interprofessional Education Collaborative Expert Panel. (2011). Core competencies for interprofessional collaborative practice: Report of an expert panel. Retrieved from https://www.aamc.org/download/186750/data/core_competencies.pdf
  • Interprofessional Education Collaborative Expert Panel. (2016). Core competencies for interprofessional collaborative practice: 2016 update. Washington, DC: Interprofessional Education Collaborative.
  • Lyons, K.J., Giordano, C., Speakman, E., Smith, K. & Horowitz, J.A. (2016). Jefferson Teamwork Observation Guide (JTOG): An instrument to observe teamwork behaviors. Journal of Allied Health, 45, 49–53.
  • Mann, K., Gordon, J. & MacLeod, A. (2009). Reflection and reflective practice in health professions education: A systematic review. Advances in Health Sciences Education, 14, 595–621. doi:10.1007/s10459-007-9090-2 [CrossRef]
  • Miraglia, R. & Asselin, M.E. (2015). Reflection as an educational strategy in nursing practice: An integrative review. Journal for Nurses in Professional Development, 31, 62–72. doi:10.1097/NND.0000000000000151 [CrossRef]
  • Wakefield, M.K. (2008). The Quality Chasm series: Implications for nursing. In Hughes, R.G. (Ed.), Patient safety and quality: An evidence-based handbook for nurses. Rockville, MD: Agency for Healthcare Research and Quality. Retrieved from https://www.ncbi.nlm.nih.gov/books/NBK2677/
  • World Health Organization. (2010). Framework for action on interprofessional education & collaborative practice. Geneva, Switzerland: Author. Retrieved from http://www.who.int/hrh/resources/framework_action/en/.

Jefferson Teamwork Observation Guide Items and Corresponding IPECa

Competency

There appeared to be a team leader that coordinated the discussion. (L)

The team leader facilitated the discussion rather than dominated it. (L)

Members of the team came prepared to discuss the case/situation from their profession-specific perspective. (R)

Members of the team who were involved in the case/situation contributed to the discussion. (C)

Discussion was distributed among all team members. (C)

Members of the team appeared to understand the roles and responsibilities of other members of the team. (R)

Team members appeared to have respect, confidence, and trust in one another. (V)

Team members listened and paid attention to each other. (C)

Team members listened to and considered the input of others before pressing their own ideas. (C)

Team members added other supporting pieces of information from their profession-specific perspective regarding the case/situation. (R)

The opinions of team members were valued by other members. (V)

Team members appeared to feel free to disagree openly with each other's ideas. (V)

Team members sought out opportunities to work with others on specific tasks. (T)

Team members engaged in friendly interaction with one another. (T)

Characteristics of Students' Observations (N = 84)

Characteristicn (%)
Location of observation
  Acute care hospital43 (51)
  Outpatient setting36 (43)
  Other/unknown5 (6)
Type of team observed
  Patient rounds52 (62)
  System improvement32 (38)
Participant by professional discipline
  Nurse80 (95)
  Physician68 (81)
  Social workers34 (40)
  Pharmacist29 (35)
  Nutritionist/dietician19 (23)
  Health profession student17 (20)
  Respiratory therapist16 (19)
  Physical therapist10 (12)
  Occupational therapist10 (12)
  Patients/family members and other nonprofessionals9 (11)
Student was member of team
  Yes55 (66)
  No28 (33)
  Unknown1 (1)
Authors

Dr. Manning is Professor, and Dr. Pogorzelska-Maziarz is Assistant Professor, Jefferson College of Nursing, Thomas Jefferson University, Philadelphia, Pennsylvania.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

Address correspondence to Mary Lou Manning, PhD, CRNP, Professor, Jefferson College of Nursing, Thomas Jefferson University, 901 Walnut Street, Philadelphia, PA 19107; e-mail: marylouman@gmail.com.

Received: January 05, 2017
Accepted: April 25, 2017

10.3928/01484834-20170918-07

Sign up to receive

Journal E-contents