Every year, >1 million seriously ill Medicare recipients in the United States elect to receive hospice care when disease-directed treatment is unlikely to be effective and life expectancy is limited (National Hospice and Palliative Care Organization [NHPCO], 2018). Most hospice care is provided to older adults in the place they call home (e.g., private residence, long-term care facility) rather than an acute care setting. An interdisciplinary team (IDT), including nurses, social workers, chaplains, physicians, and others, delivers hospice services according to individualized care plans that focus on management of patients' physical symptoms in addition to the provision of psychosocial support to patients and their families. From the perspective of hospice providers, family members are co-providers of care (family caregivers partner with hospice professionals to meet patients' care needs) and co-recipients of care (hospice providers explicitly acknowledge a responsibility to address family caregivers' well-being) (Raveis & Waldrop, 2016).
U.S. federal regulations require that hospice IDTs meet no less frequently than every 15 days to update care plans based on an ongoing assessment of patients' and families' needs (Centers for Medicare & Medicaid Services, 2014). Although health care providers tend to value the opportunity for collaboration and mutual support IDT meetings provide, they also voice frustrations over meeting inefficiencies, attendees' emphasis on biomedical information to the exclusion of psychosocial content, and providers' inability to access up-to-date patient and family data (Demiris et al., 2008; Washington, Demiris, et al., 2017, Washington, Guo, et al., 2017).
The Envision Tool
To address the pressing need for improved communication in hospice IDT meetings, our team of researchers and clinicians set out to design a tool that would provide IDT meeting attendees with access to up-to-date patient and family data to inform clinical decision making. Based on a review of the literature and our preliminary research (Washington, Guo, et al., 2017), we concluded that such a tool should include information that could be quickly interpreted, that encompassed the full range of biopsychosocial issues, and that provided information on patients and their families. Because cancer is the most common primary diagnosis for U.S. hospice patients (NHPCO, 2018), we focused our initial design on supporting hospice cancer care specifically. The result of our efforts was a prototype of the ENVISION (ENgagement and Visualization to Improve Symptoms In ONcology care) tool.
While still in prototype stage, an operational version of ENVISION was conceptualized as functioning as follows: Using data electronically submitted by family caregivers, ENVISION summarizes the current status of patients and their primary family caregivers based on responses to the Edmonton Symptom Assessment Scale-Revised (ESAS-r; Watanabe et al., 2012). The ESAS-r is a widely used and well-validated instrument that allows respondents to rate clinically relevant symptoms (e.g., pain, shortness of breath) and other pertinent outcomes (e.g., overall well-being) on a scale of 0 (symptom is absent/best possible outcome) to 10 (symptom is at the worst severity possible/worst possible outcome). Whereas patient data include responses to all ESAS-r items, caregiver data are limited to items that focus on psychosocial well-being. Patient and family caregiver data are summarized on the ENVISION dashboard (Figure 1) using visualizations to indicate whether each symptom/indicator was below (better than), equal to, or above (worse than) patients' and caregivers' self-selected, individualized goals (or “thresholds”). By clicking on a specific symptom/indicator, users are able to view a line graph showing longitudinal patient and caregiver data. Displaying ENVISION dashboards during hospice IDT meetings grants providers access to “at-a-glance” summaries of care recipients' well-being, highlighting issues that might necessitate care plan modifications, while maintaining meeting efficiency. Equipped with a paper prototype of the ENVISION dashboard and a description of ENVISION's proposed functionality, our team sought to elicit detailed feedback as the next step in our user-centered design process (Sedlmayr et al., 2019; U.S. Department of Health and Human Services [USDHHS], 2017).
© 2016 Hospice Caregiving Research Network. Reprinted with permission from K.T. Washington, G. Demiris, and D. Parker Oliver.
The Technology Acceptance Model
With more than 3 decades of empirical support, the Technology Acceptance Model (TAM; Davis, 1989) provides a useful lens through which to study potential users' perceptions of ENVISION. TAM posits that potential users are more likely to adopt new technologies if they perceive them to be useful and easy to use. Thus, studies informed by TAM typically examine a proposed technology's perceived usefulness (i.e., degree to which a potential user believes that using a particular technology would enhance their job performance) and perceived ease-of-use (i.e., degree to which a potential user believes that using a particular technology would be free from effort). Correspondingly, our team posed the following research questions: (1) To what extent do potential users perceive that ENVISION would be useful? and (2) To what extent do potential users perceive that ENVISION would be easy to use?
We conducted a qualitative descriptive study (Sandelowski, 2000) to explore potential users' perspectives. After our study was classified as exempt by the University of Missouri Institutional Review Board, we recruited study participants by contacting hospice agencies and providers with whom we had existing partnerships and asking them to share information about the study via email or during agency meetings. Hospice providers were deemed eligible to participate in the study if they had been employed in their current position for at least 3 months and regularly attended IDT meetings. We also asked providers to inform current and former hospice family caregivers of the opportunity to participate in the study via existing communication channels. Caregiver inclusion criteria required that family caregivers were age ≥18, could speak and read English, and were either currently caring for a hospice patient with cancer or had done so within the past 2 years. Because hospice patients are terminally ill, their family caregivers are likely to be the primary users of ENVISION in the home. Given this reality and the well-documented challenges of involving dying patients in research (Sivell et al., 2019), we did not include patients in our study.
Participants took part in an approximately 30-minute, audio-recorded, semi-structured interview. Interviews were conducted in person or via telephone, depending on each participant's preference. To enable telephone interviewees to provide feedback on specific elements of the ENVISION tool, we emailed a document that contained multiple versions of the ENVISION dashboard for interviewees' reference. All study participants received $20 as an acknowledgment of their time.
Two members of our team performed a directed content analysis of interview transcripts (Hsieh & Shannon, 2005). First, we developed a structured codebook based on the TAM (Davis, 1989). We jointly coded 20% of the transcripts, refining our codebook in the process. Next, using our finalized codebook (Table 1), we independently coded an additional 20% of the transcripts and met to calculate intercoder reliability. Upon determining that we had achieved satisfactory reliability (>80%; Miles & Huberman, 1984) for both parent codes (perceived usefulness and perceived ease of use), we divided and independently analyzed the remainder of the transcripts.
We took numerous steps to ensure the trustworthiness of our study findings (Guba & Lincoln, 1994). Development of a codebook and calculation of intercoder reliability enhanced the confirmability and credibility of our findings, whereas audio recording and verbatim transcription of interviews strengthened their credibility and authenticity. By maintaining careful documentation and establishing an audit trail of our analytical processes, we supported our findings' dependability and confirmability. Further, our search for confirming and discon-firming evidence promoted the overall credibility of our study results.
We recruited 31 study participants: 21 hospice providers and 10 family caregivers. Providers included five nurse care managers, four social workers, four bereavement counselors, two chaplains, four physicians, and two volunteer coordinators. Fifteen providers were female, and six were male. Of the 10 family caregiver participants, six were female, and four were male.
Efficiency. Most hospice providers indicated that ENVISION would improve the efficiency of hospice IDT meetings. When commenting on accessing data visualizations in IDT meetings, one nurse care manager stated, “I think it would cut down on sharing of unnecessary information.” Several participants reflected on the dashboard's utility in quickly calling attention to concerning symptoms, thereby reducing the amount of time taken to identify problems. One nurse care manager commented on the use of darker colors to indicate poorly controlled symptoms. When asked about the most useful elements of the tool, she replied, “The colors. Immediately your eyes go to the [darker] ones. So, [there is] immediate prioritization.” Other providers commented on the usefulness of having a dashboard with key information readily available, without having to look through paper charts or the electronic health record.
Some providers were wary about the time-related implications of adopting ENVISION. Several indicated that contacting family caregivers to request data if they did not provide it without prompting would be “time consuming” and potentially “bothersome.” One nurse care manager worried that ENVISION might decrease rather than increase meeting efficiency. She suggested that providers should access ENVISION data in advance: “IDT meetings are short and sweet. The physicians are always in a hurry, so…to me, [it] would be useful…for all the team members to already have this [information] available and ready…before we get there.”
Family caregivers' discussions about efficiency focused primarily on the electronic submission of data. Approximately all caregivers thought daily entries would be manageable and indicated that they would be accepting of daily reminders; however, two suggested different frequencies. One caregiver explained that daily submission of data would make sense if the patient was acutely ill and, thus, the caregiver required daily support of the hospice team. Otherwise, she thought daily entries would be too frequent. Conversely, another caregiver advocated for thrice daily entries to capture fluctuations in symptom intensity throughout the day.
Effectiveness. Most comments about ENVISION's impact on effectiveness pertained to patient and family care. Providers commented on the comprehensive symptom data included in the ENVISION dashboards, noting that the dashboards allowed them to easily discern which symptoms needed to be addressed. Participants also commented on the longitudinal graphs and the ability for providers to see trends over time and adjust treatment accordingly. Several family caregivers emphasized the potential value in providing family members with access to the dashboards, not just limiting their role to electronically submitting data. One caregiver explained, “With this [longitudinal] graph, I as a caregiver also can see how the pattern of his condition changed over this period.” Another family caregiver suggested that ENVISION might improve communication with providers because it essentially functioned as a list of information caregivers should share with the hospice team.
Participants were specifically asked about ENVISION's inclusion of data visualizations that depicted symptom intensity in relation to patients' and caregivers' self-selected, individualized goals. Most endorsed this approach, emphasizing the benefit of tailoring treatment to individual preferences. One family caregiver stated, “The patient ultimately will get better management of pain or whatever the symptom might be if it's personalized.” However, some participants were critical of this approach (these criticisms pertained to ease of use rather than usefulness and are detailed later).
Several participants commented on the inclusion of family caregiver data on the ENVISION dashboards. Of the providers who discussed it, all were positive about the general idea; however, some wanted more or different family caregiver information. One provider pointed out that patients often have multiple caregivers and ENVISION only presented data for one. Other providers suggested including information such as family history and current functioning. A few family caregivers expressed hesitation about sharing their data with the hospice team. One commented, “I feel uncomfortable about sharing my personal information or my personal symptoms when I'm not the patient.”
Difficulty. Participants who commented on the ways ENVISION would likely impact task difficulty tended to focus on the tool as a memory aid. One family caregiver explained how challenging it could be to recall important things amid numerous stressors: “As caregivers we go through a lot of depression, anxiety…and having the boxes here and [recording daily symptom and well-being scores] will allow me to remember…. It's kind of hard to share that information right on the spot… because I forget.” One social worker explained, “It's so hard…to remember, 'Was it a couple months ago we had problems with this?' To be able to click on something like [the longitudinal graphs] to track that history is awesome.” A few providers commented on the value of the patient and caregiver pictures included on the ENVISION dashboard, indicating that they helped meeting attendees more easily match faces with names. In addition, a social worker explained that not all team members have necessarily met every patient and caregiver prior to the IDT meeting. She stated, “I love the pictures. [If] you've never met the person before…that's helpful.”
Several providers believed that listing comorbidities on the ENVISION dashboard in addition to patients' primary diagnoses would be useful. One social worker explained, “It's hard… to remember everybody's diagnoses, especially when we're only focused on [the diagnosis that] qualifies them [for hospice]. But they also have a lot of other medical history that is pertinent.” However, this individual also acknowledged that many patients have an extensive diagnosis list, which might be problematic given limited space available on the dashboard.
Perceived Ease of Use
Ease of Learning. Few participants provided in-depth insights about the ease with which they expected they would be able to learn to use ENVISION; however, approximately all reported that ENVISION would likely be easy to learn to use. The ENVISION dashboard was described using words such as “quite clear” and “intuitive,” and participants made numerous comments suggesting that ENVISION as a whole “probably requires minimal training.”
Ease of Operating. As with ease of learning, participants did not elaborate a great deal when asked about the ease with which they thought they would be able to operate ENVISION, though most comments were positive. Words and phrases such as “user-friendly” and “fairly self-explanatory” were used to describe the technical use of the tool. However, some participants noted that family caregivers might struggle to rate patient symptoms on a scale of 0 to 10, particularly for patients who were no longer able to communicate verbally. In addition, some participants believed that it would be difficult to quantify more complex indicators, such as well-being. Some providers also wondered whether all families would have the capacity or resources (e.g., computer, smartphone) needed to electronically submit data.
Ease of Interpreting. Participant comments on the ease of interpreting ENVISION data focused on the visualizations included in the dashboard. The longitudinal symptom graphs were almost universally described as easy to interpret (one exception was a family caregiver who stated, “I think it will be confusing…. I'm not used to reading graphs”). In addition, participants noted the ease with which they were able to interpret individual symptom/indicator boxes that were shaded with darker colors to reflect more poorly controlled symptoms. Feedback on using care recipients' individualized goals as reference points for the shading of the boxes was mixed. Although most participants saw value in tailoring the plan of care to individualized goals, several indicated that it nonetheless made interpreting the ENVISION dashboard data more challenging. Participants also recommended providing definitions for symptoms/indicators that they perceived as vaguely worded, such as well-being.
Capturing user feedback and preferences in the early developmental stages of a tool is critical in user-centered designs (Sedlmayr et al., 2019; USDHHS, 2017). Our exploration of perceptions regarding ENVISION's usability and ease of use assisted our team in identifying the tool's strengths, while also informing potential modifications as we move from a paper protype to a fully functional tool. Commonly cited factors promoting user acceptance included ENVISION's comprehensive patient and caregiver data and easy-to-interpret data visualizations. Factors potentially limiting acceptability included challenges quantifying patient and caregiver experiences, reliance on family caregivers to provide timely data, and—for some—basing data visualizations on individualized patient and caregiver goals.
Numerous study limitations warrant consideration. First, as an early step in our user-centered design process, we interviewed providers and family caregivers using a paper prototype of the ENVISION tool. Although this is a standard approach in iterative system design (Snyder, 2003), future responses to a functional prototype may differ from those presented here. Second, we collaborated with existing partners to recruit participants for this study. Although we strongly emphasized the importance of candid feedback at this early stage in ENVISION's development, it is nonetheless possible that participants' responses were influenced by the nature of our pre-existing relationships. Finally, use of a directed content analysis may have limited our ability to identify important factors influencing ENVISION's acceptability that fell outside the parameters of the TAM. Future research would benefit from consideration of additional theoretical frameworks and methodological approaches.
Implications for Nursing Practice
It is perhaps not surprising that nurse care managers highlighted enhanced meeting efficiency and the ability to prioritize patient and family needs as key ways ENVISION might be useful. As hospice care managers, nurses oversee the direction and coordination of services provided to patients and families (Ellington et al., 2013). Access to up-to-date, comprehensive symptom assessment data, as is provided by the ENVISION dashboard, could help nurse care managers make decisions about the best use of limited staffing and other tangible resources. In addition, because hospice physicians do not typically perform routine home visits (Parker Oliver et al., 2010), nurses are often the only members of the hospice team who provide direct patient care with expertise in the biomedical realities of advanced disease processes and their palliation. ENVISION's longitudinal graphs showing changes in key areas of interest over time would assist nurses in collaborating with patients and families in better managing problematic symptoms. ENVISION may also reduce after-hours calls to hospice providers, as research indicates that poorly controlled symptoms are the most common reason these triage services are accessed (Jiang et al., 2012). Finally, in addition to decreasing hospice providers' job satisfaction (Washington, Guo, et al., 2017), inefficient IDT meetings consume a significant amount of staff time, making them remarkably expensive (Fine et al., 2014). If ENVISION were shown to increase meeting efficiency, nurses in senior leadership positions could realize cost savings, while potentially improving patient and family care and increasing employee job satisfaction.
Hospice providers and family caregivers found ENVISION to be a generally useful and easy to use tool that would likely enhance communication in hospice IDT meetings. Following additional refinement of the tool, testing is warranted to determine the impact of ENVISION on outcomes for hospice IDTs and the patients and families they serve.
- Centers for Medicare & Medicaid Services. (2014). State operations manual appendix M - Guidance to surveyors: Hospice conditions of participation. https://www.cms.gov/Regulations-and-Guidance/Guidance/Manuals/downloads/som107ap_m_hospice.pdf
- Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. Management Information Systems Quarterly, 13, 319–339 doi:10.2307/249008 [CrossRef]
- Demiris, G., Washington, K., Oliver, D. P. & Wittenberg-Lyles, E. (2008). A study of information flow in hospice interdisciplinary team meetings. Journal of Inter-professional Care, 22(6), 621–629 doi:10.1080/13561820802380027 [CrossRef] PMID:19012142
- Ellington, L., Cloyes, K., Berry, P., Thomas, N. T., Reblin, M. & Clayton, M. F. (2013). Complexities for hospice nurses in supporting family caregivers: Opinions from U.S. thought leaders. Journal of Palliative Medicine, 16(9), 1013–1019 doi:10.1089/jpm.2013.0116 [CrossRef] PMID:23837448
- Fine, P. G., Davis, M. & Muir, J. C. (2014). An evaluation of time and cost variability in hospice interdisciplinary group meetings and comparative clinical quality outcomes. Journal of Hospice and Palliative Nursing, 16(7), 414–419 doi:10.1097/NJH.0000000000000088 [CrossRef]
- Guba, E. G. & Lincoln, Y. S. (1994). Competing paradigms in qualitative research. In Denzin, N. K. & Lincoln, Y. S. (Eds.),Handbook of qualitative research (pp. 105–117). Sage.
- Hsieh, H. F. & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288 doi:10.1177/1049732305276687 [CrossRef] PMID:16204405
- Jiang, Y., Gentry, A. L., Pusateri, M. & Courtney, K. L. (2012). A descriptive, retrospective study of after-hours calls in hospice and palliative care. Journal of Hospice and Palliative Nursing, 14(5), 343–350 doi:10.1097/NJH.0b013e31824f1ffa [CrossRef] PMID:22773920
- Miles, M. B. & Huberman, A. M. (1984). Qualitative data analysis: A sourcebook of new methods. Sage.
- National Hospice and Palliative Care Organization. (2018). Facts and figures. https://39k5cm1a9u1968hg74aj3x51-wpengine.netdna-ssl.com/wp-content/uploads/2019/07/2018_NHPCO_Facts_Figures.pdf
- Parker Oliver, D., Tatum, P., Kapp, J. M. & Wallace, A. (2010). Interdisciplinary collaboration: The voices of hospice medical directors. The American Journal of Hospice & Palliative Care, 27(8), 537–544 doi:10.1177/1049909110366852 [CrossRef] PMID:20439483
- Raveis, V. H. & Waldrop, D. P. (2016). The history and development of end-of-life care. In Kaplan, D. B. & Berkman, B. (Eds.), The Oxford handbook of social work in health and aging (2nd ed., pp. 597–606). Oxford.
- Sandelowski, M. (2000). Whatever happened to qualitative description?Research in Nursing & Health, 23(4), 334–340 doi:10.1002/1098-240X(200008)23:4<334::AIDNUR9>3.0.CO;2-G [CrossRef] PMID:10940958
- Sedlmayr, B., Schöffler, J., Prokosch, H. U. & Sedlmayr, M. (2019). User-centered design of a mobile medication management. Informatics for Health & Social Care, 44(2), 152–163 doi:10.1080/17538157.2018.1437042 [CrossRef] PMID:29504838
- Sivell, S., Prout, H., Hopewell-Kelly, N., Baillie, J., Byrne, A., Edwards, M., Harrop, E., Noble, S., Sampson, C. & Nelson, A. (2019). Considerations and recommendations for conducting qualitative research interviews with palliative and end-of-life care patients in the home setting: A consensus paper. BMJ Supportive & Palliative Care, 9(1), e14 doi:10.1136/bmjspcare-2015-000892 [CrossRef] PMID:26647042
- Snyder, C. (2003). Paper prototyping; The fast and easy way to design and refine user interfaces. Morgan Kaufmann.
- U.S. Department of Health and Human Services. (2017). User-centered design process map. https://www.usability.gov/how-toand-tools/resources/ucd-map.html
- Washington, K. T., Demiris, G., Parker Oliver, D., Swarz, J. A., Lewis, A. M. & Backonja, U. (2017). A qualitative analysis of information sharing in hospice interdisciplinary group meetings. The American Journal of Hospice & Palliative Care, 34(10), 901–906 doi:10.1177/1049909117693577 [CrossRef] PMID:28193105
- Washington, K. T., Guo, Y., Albright, D. L., Lewis, A., Parker Oliver, D. & Demiris, G. (2017). Team functioning in hospice interprofessional meetings: An exploratory study of providers' perspectives. Journal of Interprofessional Care, 31(4), 455–462 doi:10.1080/13561820.2017.1305950 [CrossRef] PMID:28406340
- Watanabe, S. M., Nekolaichuk, C. L. & Beaumont, C. (2012). The Edmonton Symptom Assessment System, a proposed tool for distress screening in cancer patients: Development and refinement. Psycho-Oncology, 21(9), 977–985 doi:10.1002/pon.1996 [CrossRef] PMID:21671304
|Parent Code||Child Code||Definition|
|Perceived usefulness||Effect on efficiency||Perception that ENVISION would affect the speed with which tasks are completed|
|Effect on effectiveness||Perception that ENVISION would affect the quality of care provided|
|Effect on difficulty||Perception that ENVISION would make one's job/caregiving easier or more difficult|
|Perceived ease of use||Ease of learning||Perception that it would be easy to learn to use ENVISION|
|Ease of operating||Perception that it would be easy to operate ENVISION|
|Ease of interpreting||Perception that it would be easy to interpret data displayed by ENVISION|