Dr. Alexander is Assistant Professor, Sinclair School of Nursing, University of Missouri, Columbia, Missouri.
Funding was supported by the Centers for Medicare & Medicaid Services in response to RFP-CMS-03-001/DB (Principal Investigator: Marilyn Rantz, PhD, RN, FAAN). Funding was also provided by the National Library of Medicine in response to T15-LM07089-12. The views expressed in this article are those of the author and may not reflect those who supported the research.
Address correspondence to Gregory L. Alexander, PhD, RN, Assistant Professor, Sinclair School of Nursing, S415, University of Missouri, Columbia, MO 65211; e-mail: AlexanderG@missouri.edu.
The U.S. population is aging, with the number of individuals older than age 85 increasing the fastest (Institute of Medicine [IOM], 2001b). The IOM indicated this rapid growth will affect demand for and supply of long-term care services; projected ranges for the number of Americans needing long-term care by 2030 are 10.8 million to 14 million. Continuing concerns about quality, cost, accessibility, adequacy of oversight, and enforcement issues are driving the need to implement information systems in nursing homes, where significant portions of the population reside. Information systems that provide valid, reliable, and timely data about the care provided to residents, facilities, and caregivers is fundamental to monitoring and improving the quality of nursing home care (IOM, 2001b).
The purpose of this study was to use the infrastructure of a nursing home clinical information system, OneTouch (now Optimus EMR, Inc., Irvine, CA), to evaluate a component of the information system called a clinical decision support system (CDSS). This study sought to determine:
- The frequency of active alerts in the CDSS in each nursing home and in residents with specified diagnoses.
- Whether positive correlations exist between the frequency of active alerts and number of secondary diagnoses.
- The frequency of active alerts and their triggers in residents with specified diagnoses.
No funds were provided by One-Touch to conduct this study. Approval for the research process in this study was obtained from the university’s institutional review board.
Functional Models in Clinical Information Systems
During the past decade, health care leaders have stressed the importance of integrating clinical information systems into health care to enhance clinical practices, improve quality, and reduce medical errors (IOM, 2000IOM, 2001a; National Center for Nursing Research, 1993). Recognizing the core functions of clinical information systems can assist developers in designing models that are more practical and usable. These core functions are to support care delivery, care management, care support processes, administrative processes, and patient self-management (IOM, 2003). Clinical information systems should be designed to enhance core functions by improving patient safety, facilitating the management of chronic conditions, and increasing efficiency (IOM, 2003).
This study included an evaluation of a CDSS a component of a clinical information system implemented in three nursing homes located in the midwestern United States. A model of CDSS in a clinical information system, developed by the author and based on theoretical ideas from human factors experts, was used as a framework to guide this investigation (Figure) (Nemeth, 2004; Salvendy, 2006; Sanders & McCormick, 1993; Sawyer, 1996; Staggers, 1991).
The model illustrates how alerting mechanisms aid in problem recognition and lead to clinical actions that improve resident outcomes. Resident assessment data used to document resident conditions are entered into the clinical information system. Predetermined criteria or triggers within the resident assessment data are then used to activate clinical alerts in the CDSS. For example, a skin integrity alert might be activated when staff document that a resident is comatose, has increased edema, or is incontinent, or when a turning or repositioning program is ordered but has not been documented as being completed within a certain amount of time. The selection of these triggers or predetermined criteria can be set by vendors and can often be manipulated by the CDSS administrator. When an alert becomes active, the computer automatically sends the user a message that a potential problem has been identified, which should then guide the staff to investigate. Staff can choose whether or not to take clinical actions. When the condition is resolved through appropriate documentation, the alert becomes inactive. In the absence of alerts, staff are responsible for assimilating resident assessment data and making decisions about care using their own recall and synthesis of vital information. In this study, active alerts associated with the CDSS were evaluated to determine the frequency of activation and to describe patterns of alerts across different resident diagnoses.
Participants and Setting
The three nursing homes in this study were participating in a larger research project designed to evaluate the use of bedside technology (provided by OneTouch Corporation) to improve quality of care (Rantz et al., 2006). The Table contains characteristics of the participating facilities and their residents.
Resident data were collected during the 6-month to 12-month postimplementation period. Six months of data were collected from each facility after fictitious, unique nursing home and resident identifiers were assigned in the data sets for resident confidentiality. Data set elements included resident diagnosis, alert status (including times when alerts were active and not active), triggers initiating the alerts, care plan problems identified when alerts were active and not active, and certified nursing assistant (CNA) task list items when alerts were active and not active. Each of the three clinical information systems was queried every day at 7:00 a.m. (the start of the day shift) from the 6-month postimplementation date through the 12-month postimplementation date.
Nursing Home Technology
The OneTouch system has technologies not previously available to the nursing home industry and represents a shift from the manual paper-and-pen method to a digital environment. This new level of data collection should have positive effects on the quality of individual resident care by improving detection of potential resident problems through automated alerts. Evidence of the positive effects of automation is found in critical practices using alerts to improve evaluation of patient-specific clinical variables and clinical decision making (Gandhi & Bates, 2001; Murff, Gosbee, & Bates, 2001). OneTouch integrates specialized technology used for electronically tracing resident care, personal digital assistants for bedside point-of-care data entry, and wireless technology through the corporation’s proprietary software to support the CDSS.
The bedside modules in this study were designed to provide a template for complete, verifiable documentation and the interactivity of specific items in the clinical record. Users of nursing home information systems have previously indicated that this kind of bedside documentation system benefited health care providers. They could view many things about resident care simultaneously from multiple areas in the facility and then trace the resident care information back to care providers (Alexander, Rantz, Flesner, Diekemper, & Siem, 2007). Information currently provided at the point of care includes vital sign clinical alerts, nurse-to-nurse messaging, CNA task lists, care plan items, active orders from physicians, and treatments.
In phase 1 of the analysis, the One-Touch system was queried daily for 6 months to identify types of alerts and active times for each alert. The overall frequency of active alerts was identified for each participating nursing home. The frequency of active alerts was further analyzed for alert type for residents who have specified diagnoses:
- Decline in condition.
- Improvement in condition.
- Loss of skin integrity.
- Weight loss.
- Weight gain.
Descriptive statistics were used to track trends between types of active alerts and the average time the alerts were active. Each alert type and resident diagnosis was given a unique, dummy-coded variable so they could be manipulated in the database. The type of primary diagnosis and the number of secondary diagnoses per resident were determined. The total numbers of active alerts for the most common primary diagnosis types were tabulated. The goal was to describe trends in the specific types of alerts identified for the most frequent primary diagnoses.
Each month, the average length of time alerts were active both overall and for residents with the same primary diagnoses were assessed. Average length of time was calculated using the length of time alerts were active on consecutive days each month; categories were established where natural breaks occurred in the data. These categories included:
- No active alerts (indicating the number of days no alerts occurred).
- Active for 1 to 3 days.
- Active for 4 to 9 days.
- Active for 10 to 19 days.
- Active for 20 or more days.
In phase 2, the admission, discharge, and transfer reports were used to assess whether a significant positive correlation would be found between the number of secondary diagnoses assigned to nursing home residents and the number of active alerts.
In phase 3, the frequency and types of triggers of active alerts in residents with specified diagnoses were determined. Data were organized by alert type and the resident’s primary diagnosis; descriptive statistics were used to track trends between trigger frequencies and alert type. All statistical analyses were performed using SPSS version 14.0 and Excel® software.
Initial queries were examined to determine whether patterns in the active alert data could be found at the facility level. The researchers assumed that significant variations in the frequencies of alerts on a daily basis would exist within each facility and between facilities. This assumption was based on the diversity of care delivered to residents, changes in resident conditions, and complexity of resident care in nursing homes (Alexander et al., 2007; Rantz et al., 2004).
Contrary to this expectation, little variation in alert frequencies was noted in facility A during the first 10 days. No active alerts were documented for dehydration, decline in condition, weight loss, or weight gain; other alert frequencies were 136 for constipation, 3 for skin integrity, and 40 for improvement in condition. Similarly, alert frequencies in facility A had virtually no variation from day 21 of the fourth month to the last day of the sixth month. During the period from day 11 of the first month through day 20 of the fourth month, frequent variation was found in all alert categories.
Alert frequencies in facility B were variable from day 1 until day 25 of the second month, when abruptly, the alert frequencies in each category became the same consistently from day to day. Alert frequencies in facility C remained the same consistently throughout the entire 6 months of data collection. Here, no active alerts were documented for dehydration, skin integrity, or decline in condition. Active alert frequencies for constipation and improvement in condition were 83 and 18, respectively, for each day of the analysis period; alerts for weight loss and weight gain also showed little to no variation.
As a result of these initial findings, only alerts in facility A from day 11 of the first month to day 20 of the fourth month were included, making a total of 101 days of data. For facility B, data from day 1 of the first month to day 24 of the second month were included, making a total of 54 days of data. Because of the complete lack of variation in data from facility C, none of its data was included in this analysis. Therefore, the final analysis includes only the frequencies of alerts and their triggers collected from two facilities during a period of 155 days.
The two most frequent alerts occurring in both facility A and facility B were dehydration (32.5%, 29.8%) and improvement in condition (23.2%, 24.8%), respectively. Constipation (21.2%) was the third most frequently occurring alert in facility A; skin integrity (16.1%) was the third most frequently occurring alert in facility B.
Ventilation pneumonitis was the most frequent primary diagnosis in facility A; the most frequent alerts associated with this diagnosis were dehydration (32%) and improvement in condition (23.3%). In facility B, dehydration (31.9%) was the most frequent alert and occurred in residents with osteoarthritis. Caregivers contributed to this alert rate by frequently selecting data triggers for residents with osteoarthritis, indicating when residents left 25% or more of their food uneaten (21.9% of all triggers selected) and when output exceeded input (3.7% of all triggers selected). The least frequent alert type (weight loss) also occurred in residents with osteoarthritis, experienced in only 1 of 7 residents.
In facility A, a wide range of alert frequencies were found in residents with the same primary diagnosis. Of 89 (65.4%) residents with a primary diagnosis of ventilation pneumonitis, 35 (39.3%) experienced at least one alert type for 101 days, the maximum number of days an alert could be active. Of these alerts, 25 (71.4%) were for improvement in condition. In facility B, 20 (8.8%) residents had a primary diagnosis of dementia, 7 (35%) of whom experienced at least one alert type for 54 days, the maximum number of days an alert could be active. Of these alerts, 5 (71.4%) were for improvement in condition.
Facility A had the most periods of active alert times. Here, 5,339 active alert periods were found during the 4-month study period; facility B had 5,276 active alert periods during the 2-month study period. The most frequent alerts that were never activated in facility A or facility B were for weight gain (82.1%, 64%) and weight loss (84.4%, 66.4%), respectively.
The most frequent alert for both facilities, according to consecutive days of active status, was skin integrity. Skin integrity alerts also had the shortest time interval of active status in both facilities. Of facility A’s 5,339 active alert periods, 1,148 (21.5%) were for skin integrity, 76.1% of which occurred for 1 to 3 consecutive days. Of facility B’s 5,276 active alert periods, 1,726 (32.7%) were for skin integrity, 77.9% of which occurred for 1 to 3 consecutive days. Alert types with the longest periods of activity (20 or more consecutive days) occurred for improvement in condition (facility A = 28.2%, facility B = 30.9%). In addition, the dehydration alert in facility B was active for 20 or more consecutive days ( 30.9%).
Facility A had a total of 136 residents. The most frequent (66.9%) primary diagnosis in this group was ventilation pneumonitis. The next most frequent diagnoses were hypertension and heart disease, with 2 residents in each category. Of facility B’s 228 residents, the most frequent (36.4%) primary diagnoses were dementia, cerebral vascular accident, Alzheimer’s disease, hypertension, pneumonia, osteoarthritis, and depressive disorder.
Nonparametric Spearman’s rank correlations were completed on all alert types. Only one moderately significantly positive correlation was found between the weight gain alert and residents with a primary diagnosis of cerebral vascular accident (N = 18, r = 0.531, p = 0.023). These residents experienced an average of 8.22 weight gain alerts . The mean number of secondary diagnoses was 6.06 (SD = 2.817, range = 3 to 13).
The most frequent trigger (facility A = 39.7%, facility B = 30.8%) was the dehydration alert and it was activated when staff documented residents who had left 25% or more of their food uneaten twice within 48 hours.
In facility A, the most frequent (6.6%) trigger activating the constipation alert indicated a resident had not had a regular bowel movement; in facility B, this same trigger was activated only 0.5% of the time for residents with dementia. In facility B, the most frequent (29%) trigger activating the skin integrity alert in residents with dementia indicated bladder incontinence; in facility A, this same trigger was activated only 6% of the time.
This study includes an evaluation of a CDSS in a clinical information system that has alerts for seven clinical conditions: decline in condition, improvement in condition, constipation, dehydration, loss of skin integrity, weight loss, and weight gain. Critical findings discussed elsewhere regarding the implementation of the clinical information system showed that data integrity used for this study may have been dramatically affected by completeness of documentation, system changes in clinical practice, choices made during implementation, and the design of the clinical information system (Alexander, 2006; Alexander et al., 2007). Poor data integrity in the initial steps of the evaluation may have contributed to the lack of variability of alert frequencies in resident data found across facilities. This lack of variability may have resulted from poor documentation practices, staff’s inability to find appropriate fields to document, not enough training, or unfamiliar terminology in the information system.
Alerts that involved dehydration, improvement in condition, constipation, and skin integrity were active the most frequently. Skin integrity alerts changed active status most frequently, with most alerts lasting approximately 1 to 3 days. This rapid frequency of change could be due to the documentation of changes in skin condition and effective clinical responses to the procedures performed by care providers. However, this descriptive study does not provide compelling evidence that staff were responding appropriately to the alerts. In a concurrent study involving the same information system and the same facilities, staff indicated that a workable component of an information system (i.e., CDSS) needed to provide consistent data to users and be accurate, and that information about the alerting condition needed to be available and easy to find (Alexander et al., 2007). In addition, during the early stages of the CDSS implementation, staff recognized workability issues that created situations the staff did not normally chart, which affected the CDSS’ reliability and validity. Such workability issues were associated with employee problem resolution: When the documentation systems did not work, employees did not chart. Other workability issues included equipment availability, system speed, and terminology used in the system. These issues could have contributed to the long time periods alerts were active (facility A = 101 days, facility B = 54 days) in nursing homes implementing the CDSS.
Weight gain and weight loss alerts had the greatest frequency of not active status. The CDSS’ internal decision tree could have affected this level of inactivity. Specifically, the weight gain alert will become active if a resident gains more than 3.5% of his or her total body weight over 30 days or 7% over 180 days. These parameters may be too broad and may not capture subtle changes in weight. The alert with the longest consecutive active time was improvement in condition. This was a positive finding, because it indicated that staff focused on documenting positive aspects of resident care, such as improved decision making and improved participation in activities of daily living.
Another surprising finding was that frequencies of active alerts do not seem to be affected by the number of residents’ secondary diagnoses. The researchers assumed that as the complexity of care and the number of potential problems associated with secondary diagnoses increased, so would the number of active alerts. Perhaps this finding would be different if primary diagnoses were combined into larger, broader groups to increase the sample size. One recognized limitation about this part of the study is that nursing home records, including the diagnoses and Minimum Data Set, are notoriously inaccurate (Lum, Lin, & Kane, 2005). Perhaps this is one explanation for the high rate of ventilation pneumonitis found in facility A’s clinical records.
One benefit of these kinds of computerized systems yet to be determined is how the automated, traceable records enhance system administrators’ ability to evaluate care being delivered and recorded in real time. With the introduction of these information systems into long-term care, further research is needed to determine whether they can improve accuracy of documentation, increase responsiveness of caregivers, and enhance accountability for care delivery.
Evaluations of the CDSS in settings in which they are actually being used provide knowledge about how these tools might improve resident safety, facilitate management of chronic conditions, and improve efficiency. Other areas in which the CDSS might be useful for resident care but which remain largely unexplored because they have not been fully developed in long-term care settings are pain management, fall detection, and physiological changes in resident conditions (e.g., increased restlessness). What this study did not consider were the human factors, or human-computer interaction principles, including how staff interact with the computer, the physical nature of the information system and its effect on staff, and the environment in which the information system is implemented. Applications of these principles in future research would provide better information about the effectiveness of a CDSS in nursing homes.
Leaders in the nursing home industry are encouraging the integration of clinical information systems into these settings to improve practice, quality, and safety. Because these systems have not been used very often, little research exists on their effectiveness in improving these components of clinical information systems. It is important for nursing home leaders and designers of clinical information systems to work together to improve the usability of these systems. This study provides initial evidence of how clinical information systems can be used to describe and monitor resident care in nursing homes.
Important challenges that nursing home leaders, as early adopters of these systems, will face are constructing an implementation strategy for a CDSS that informs users how to use the system, ensuring that the CDSS accurately and reliably guides users to specific resident conditions, and providing information about what to do for those conditions (i.e., procedures, protocols) to improve outcomes for nursing home residents. The main purpose of this article is to acknowledge that implementing an information system in a nursing home is only the first step. Staff must understand how to use it and use it so maximum benefits can be obtained. Administrators need to facilitate optimum staff use of the system to improve care. The data presented in this article exemplify that use of the system in these three nursing homes could be improved greatly, and if it were, the system has great potential to monitor and improve resident care.
- Alexander, GL2006. Issues of trust and ethics in computerized clinical decision support systems. Nursing Administration Quarterly, 301, 21–29.
- Alexander, GL, Rantz, MJ, Flesner, M, Diekemper, M & Siem, C2007. Clinical information systems in nursing homes: An evaluation of initial implementation strategies. Computers Informatics Nursing, 25, 189–197. doi:10.1097/01.NCN.0000280589.28067.18 [CrossRef]
- Centers for Medicare & Medicaid Services. 2004. Nursing home compare. Retrieved February 10, 2004, from http://www.medicare.gov/Nursing/Overview.asp
- Centers for Medicare & Medicaid Services. 2005. Nursing home compare. Retrieved February 4, 2005, from http://www.medicare.gov/Nursing/Overview.asp
- Gandhi, TK & Bates, DW2001. Computer adverse drug event (ADE) detection and alerts. In Agency for Healthcare Research and Quality, Making health care safer: A critical analysis of patient safety practices (AHRQ Publication No. 01–E058, pp. 79–85). Rockville, MD: Agency for Healthcare Research and Quality.
- Institute of Medicine. 2000. To err is human: Building a safer health system. Washington, DC: National Academies Press.
- Institute of Medicine. 2001a. Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academies Press.
- Institute of Medicine. 2001b. Improving the quality of long-term care. Washington, DC: National Academies Press.
- Institute of Medicine. 2003. Key capabilities of an electronic health record system. Washington, DC: National Academies Press.
- Lum, TY, Lin, W & Kane, RL2005. Use of proxy respondents and accuracy of minimum data set assessments of activities of daily living. Journals of Gerontology. Series A, Biological Sciences and Medical Sciences, 60A, 654–659.
- Murff, HJ, Gosbee, JW & Bates, DW2001. Human factors and medical devices. In Agency for Healthcare Research and Quality, Making health care safer: A critical analysis of patient safety practices (AHRQ Publication No. 01-E058, pp. 459–470). Rockville, MD: Agency for Healthcare Research and Quality.
- National Center for Nursing Research. 1993. Enhancing patient care: A report to the NCNR priority expert panel on nursing informatics (NIH Publication No 93-2419) Bethesda, MD: U.S. Department of Health and Human Services.
- Nemeth, CP2004. Human factors methods for design: Making systems human-centered. Boca Raton, FL: CRC Press.
- Rantz, MJ, Hicks, LL, Petroski, GF, Madsen, RW, Mehr, DR & Conn, V et al. . 2004. Stability and sensitivity of nursing home quality indicators. The Journals of Gerontology. Series A, Biological Sciences and Medical Sciences, 59, 79–82.
- Rantz, MJ, Scott-Cawiezell, J, Alexander, GL, Conn, V, Madsen, RW & Petroski, GF et al. . 2006. Evaluation of the use of bedside technology to improve quality of care in nursing facilities (Report No. RFP-CMS-03-001/DB). Retrieved December 4, 2007, from the Centers for Medicare & Medicaid Services Web site: http://www.cms.hhs.gov/reports/downloads/Rantz.pdf
- Salvendy, G2006. Handbook of human factors and ergonomics (3rd ed) Hoboken, NJ: Wiley & Sons.
- Sanders, MS & McCormick, EJ1993. Human factors in engineering and design (7th ed) New York: McGraw Hill.
- Sawyer, D1996. Do it by design: An introduction to human factors in medical devices. Retrieved December 4, 2007, from the U.S. Food and Drug Administration Web site: http://www.fda.gov/cdrh/humfac/doitpdf.pdf
- Staggers, N1991. Human factors. The missing element in computer technology. Computers in Nursing, 9, 47–49.
Figure. Model for Clinical Decision Support Systems in Clinical Information Systems.
Characteristics of Nursing Home Facilities and Residents
|Number of beds
|Nursing staff hours per resident daya
|Selected Facility Quality Measures
||Facility A (%)
||Facility B (%)
||Facility C (%)
|Residents who are depressed or anxious
|High-risk residents with pressure sores
|Low-risk residents with pressure sores
||Number of residents too small to report.
|Low-risk residents who lose control of their bowels or bladder
|Residents who spend most of their time in bed or in a chair
|Residents who lose too much weight
Table: Characteristics of Nursing Home Facilities and Residents