Patients who fall while in the hospital increase health care costs for both the patient and facility, experience prolonged lengths of stay, use more hospital resources, and may experience physical as well as emotional injuries. In 2008, the Centers for Medicare and Medicaid Services (CMS) established regulations and reimbursement policies that limit hospital compensation for care regarding fall-related injuries (Inouye, Brown, & Tinetti, 2009). Findings from the current research study using Microsoft® Kinect™ motion sensors inside real hospital rooms offer a viable, cost-efficient solution in an unobtrusive manner to prevent and detect falls. This technological approach enables conducting quality improvement root-cause analysis of patient falls to “see” what happened leading up to the actual fall. I t also offers future research opportunities for real-time staff notification of potential falls or increasing fall risk.
The loss of Medicare coverage for in-hospital related falls has been the impetus for many organizations to rethink how to prevent such events (Titler, Shever, Kanak, Picone, & Qin, 2011). Falls are the most common safety incident among hospitalized patients, with fall rates from 2.9 to 13 per 1,000 patient days (Oliver, 2008a, 2008b; Rubenstein, 2006; Rubenstein & Pugh, 2006). Major fall risk factors include (a) a previous fall; (b) mobility limitations (weakness of the lower limbs, gait, and instability); (c) polypharmacy (more than four medications) and use of specific types of medications (e.g., psycho-active drugs); (d) urinary incontinence, frequency, or need for toileting assistance; (e) dizziness or orthostasis; and (f) confusion or cognitive impairment (Oliver, Daly, Martin, & McMurdo, 2004; Tinetti & Kumar, 2010). The prevalence of patient falls, coupled with multiple comorbidities of today’s hospitalized patient, is one of the American Nurses Association’s 10 nurse-sensitive quality indicators (Dunton, Gajewski, Klaus, & Pierson, 2007).
Although fall risk factors have been identified, little research has been reported on automatic sensing systems or using depth images (“ghost-like” shadows) inside hospital rooms to detect falls, assess fall risks, and understand events leading up to the fall. One particularly promising method uses the Microsoft® Kinect™ as a sensor. The Kinect, which became commercially available in 2010, was designed to allow controller-free game play on the Microsoft Xbox™. The device contains both an RGB (red, green, blue) camera, a microphone array, and an infrared-sensitive camera, from which a depth image can be produced based on a pattern of projected infrared light. Depth data are continuously and unobtrusively captured, the foreground is extracted, and a three-dimensional (3D) image is created using the depth information obtained from the sensor. The 3D image can be thought of as camera pixels that combine into a two-dimensional picture from a typical camera, but in 3D. Although other images are available from the Kinect to address privacy concerns, only information from the depth images were used in this study, which effectively provided a 3D anonymous ghost-like silhouette (Banerjee et al., 2012). Figure 1 shows the image of a patient and visitor in a hospital room using the Kinect.
Example of a depth image showing the patient and a visitor in the hospital room.
The Microsoft Kinect sensor uses a pattern of actively emitted infrared light to produce the depth image and a 3D image using a single Kinect device. Thus, the Kinect is robust, even in limited and variable lighting conditions. It is also relatively inexpensive (approximately $150) and one sensor can be used effectively to provide coverage for monitoring the safety of patients inside their hospital rooms, depending on the size and configuration of the room. Importantly, as can be seen in Figure 1, the depth images address privacy concerns of the patients while still allowing for collection of important activity information (Stone & Skubic, 2011).
A previous study used a wireless five-sensor, motion detection system (5S-MDS) with hospitalized older adults (Ferrari et al., 2011). Multiple wireless accelerometer-based sensors were placed on the patient’s body, which in turn were used to represent the body orientation through a privacy-protecting virtual 3D avatar. Although the sensor data accurately detected patient movement, the 5S-MDS was only feasible for 4 hours’ use with hospitalized older adults to maintain skin integrity (Ferrari et al., 2011). This provides limited usefulness for continuously monitoring patients for falls and risk of falls throughout hospitalization.
The descriptive pilot study described herein began in July 2012; the first 8 months of continuous data collection and automated analysis are reported. University of Missouri (MU) Health Care is an academic health care system that provides comprehensive inpatient and outpatient services. MU Health Care is a fully integrated health care delivery system, which provides its health care services through a wide range of avenues including home health, primary care, inpatient acute care, long-term acute care, outpatient clinics, and supporting ancillary services. MU Health Care consists of six hospitals with 536 licensed beds and 52 outpatient clinics. The study protocol was approved by the university institutional review board and the hospital’s human investigation committee.
Staff and leadership of Progressive Care, a 20-bed unit located within University Hospital, agreed to participate. Progressive Care was chosen due to its high patient acuity levels and risk for falls. Microsoft Kinect sensors were installed on a weekend when the census was lower. Sensors were installed in six private, unoccupied rooms. The average time for the initial four installations (performed by four members of the research team) was approximately 40 minutes; by the installation of the sixth sensor, installation time was only 11 minutes. After installing the Kinect sensor in each room, the team posted a patient/visitor sign with an explanation and a picture of the ghost-like depth image being collected for fall prevention to address any issues with patients’ rights, questions, or potentially patients not wanting to be included in the project (Banerjee et al., 2012). Figure 2 is a picture of a typical installation.
Front and side view of a Microsoft® Kinect™ sensor (circled in red) mounted above the television in a hospital room. The EEEBox computer along with the external hard drive is circled in blue.
Prior to installations in the six patient rooms in Progressive Care, a feasibility pilot study was conducted in an empty hospital room to test the Kinect sensor. Using protocols from prior fall research (Cuddihy et al., 2012), three researchers performed 18 falls (e.g., walking then falling down or falling down from the bed) and 17 non-fall events (e.g., crouching down, stooping to tie shoe laces, lying on the floor) for a total duration of 44 minutes. Based on algorithms originally developed for identifying falls using privacy-protecting images constructed from web-cams (Anderson et al., 2009a, 2009b), algorithms were modified so that all falls and non-fall events were correctly identified in the Kinect depth image data (Banerjee et al., 2012).
The Kinect sensors installed in the six patient rooms in Progressive Care were then configured to log data continuously for 24 hours per day.
Over the first 8 months, a single fall in one of the six rooms was reported by the hospital staff; unfortunately, the fall was not recorded by the system because it occurred during a snow storm that created a power outage. However, 50 actual falls were performed by stunt actors in the rooms over this period. These falls were from standing positions, while sitting on the bed, and rolling from the bed; all were captured by the system. These provided additional data for further refining and improving the algorithm for processing the Kinect data.
To evaluate false positives (i.e., the system detects a fall when none occurred), 100 weeks’ worth of data collected from all six hospital rooms were processed. Using the refined algorithm, the false positive rate was less than one fall per day per room (92% sensitivity, 95% specificity; 11 false alarms per month per room). For initial feasibility of using the Kinect, this rate was considered acceptable by clinicians, particularly because hospital staff and visitors who freely move in and out of patient rooms, as well as various equipment used with patients (e.g., wheelchairs, stretchers, infusion pump), create unique challenges for automated fall detection systems (Banerjee et al., 2012). Images of an actual fall of a person in a hospital room and an actual patient in a hospital bed are shown in Figure 3.
Depth (a, d), extracted foreground (b, e), and three-dimensional point clouds (c, f) of a person falling (a, b, c) and a hospitalized patient on the bed (d, e, f).
During the first 8 months of data collection in the actual hospital setting, it has been demonstrated that falls can be detected with an acceptable false alarm rate that supports ongoing deployment of the Kinect for fall detection. As the feasibility study was conducted, engineering staff worked with clinical staff to develop a “rewind” method, allowing them to see what happened before and during a fall that occurred in a patient room. Using the rewind method, clinicians have access to those rarely seen events that lead to the fall and can use the information in a post-fall quality improvement analysis process. Actually viewing images from the Kinect will provide a way to determine what body parts may or may not have struck equipment or the floor during the fall. There is potential to reduce unnecessary procedures for the patient (ruling out harm) by examining the actual fall event.
Preliminary data from this feasibility study provide detection of falls, fall risks, and quality improvement post-falls in real hospital environments unobtrusively, while taking into account patient privacy. An explanation with a picture of the ghost-like images from the Kinect was posted in all patient rooms, and patients could decline to participate in the study. No concerns were raised by patients or families, although one patient did put tape over the front of the Kinect lens during this patient’s stay. As in our other sensor research, residents living in senior housing have been accepting of technology that could potentially help them and, importantly, does not require them to “do” something (Skubic, Alexander, Popescu, Rantz, & Keller, 2009). They verbalize actually feeling safer because the technology is there so people can help them if they need it.
Findings from this research study using the Kinect inside real hospital rooms offer a viable, cost-efficient solution in an unobtrusive manner to prevent and detect falls. This technological approach enables conducting quality improvement root-cause analysis of patient falls to “see” what happened leading up to the actual fall. It also offers future research opportunities for real-time staff notification of potential falls or increasing fall risk. Probably the most telling result of this feasibility study is that the staff recommended the deployment be expanded to an entire unit of 24 beds to test the rewind feature indefinitely. Clinical staff recommended the sensors be placed in a unit with primarily older adults who have a high risk for falls. Planning for deployment is in process at this time.
Research that is conducted in a dynamic hospital environment requires not only assiduous pre-planning on behalf of the research team, but also buy-in and cooperation from all clinical partners, engineering, risk management, infection control, regulatory compliance, and human subject’s protection staff. For successful implementation and installation of the equipment, multiple layers of approval and supervision were required to assure that all regulatory standards were being met. Access to patient rooms for sensor installations posed another limitation, as room turnover must be prompt, with little time to install the equipment. Participants were not randomly assigned, and the sample is relatively small; thus, our sample may not be representative, making generalizability to other hospitalized patients challenging. As previously mentioned, false positive events are detected by the system and constant movement of the patient bed, tray table, and frequent visits from hospital staff or family are challenges that will need to be further refined in the algorithm. Our goal is to improve sensitivity and specificity for clinical utility so that all falls are detected with as close to no false alarms as possible.
The use of sensors for identifying falls will continue to evolve and could transition to become the standard fall prevention intervention used in health care. Interventions to prevent hospital-based falls in older adults are critically important to reduce morbidity, mortality, and health care costs (Ferrari et al., 2011). Using the Microsoft Kinect sensors inside hospital rooms offers promising information on depth data in a privacy-protecting, unobtrusive way that can capture the events leading up to a fall (Anderson et al., 2009a, 2009b). Through this ongoing pilot study, researchers are optimistic that despite the dynamic environment, an average of far less than one false positive per day can be achieved. Ultimately, using images to analyze falls and events leading up to falls will lead to better fall detection and better analysis of falls that can aid in developing future fall prevention strategies.
- Anderson, D., Luke, R.H., Keller, J.M., Skubic, M., Rantz, M. & Aud, M. (2009a). Linguistic summarization of video for fall detection using voxel person and fuzzy logic. Computer Vision and Image Understanding, 113, 80–89. doi:10.1016/j.cviu.2008.07.006 [CrossRef]
- Anderson, D., Luke, R.H., Keller, J.M., Skubic, M., Rantz, M.J. & Aud, M.A. (2009b). Modeling human activity from voxel person using fuzzy logic. IEEE Transactions on Fuzzy Systems, 17, 39–49. doi:10.1109/TFUZZ.2008.2004498 [CrossRef]
- Banerjee, T., Rantz, M., Mengyuan, L., Popescu, M., Stone, E., Skubic, M. & Scott, S. (2012, November). Monitoring hospital rooms for safety using depth images. Paper presented at the AI for Gerontechnology AAAI Fall Symposium. , Arlington, VA. .
- Cuddihy, P.E., Yardibi, T., Legenzoff, Z.J., Liu, L., Phillips, C.E., Abbott, C. & Rantz, M.J. (2012). Radar walking speed measurements of seniors in their apartments: Technology for fall prevention. Conference Proceedings of the 34th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 260–263.
- Dunton, N., Gajewski, B., Klaus, S. & Pierson, B. (2007). The relationship of nursing workforce characteristics to patient outcomes. Online Journal of Issues in Nursing, 12. doi:10.3912/OJIN.Vol12No03Man03 [CrossRef]
- Ferrari, M., Harrison, B., Rawashdeh, O., Hammond, R., Avery, Y., Rawashdeh, M. & Maddens, M. (2011). Clinical feasibility trial of a motion detection system for fall prevention in hospitalized older adult patients. Geriatric Nursing, 33, 177–183. doi:10.1016/j.gerinurse.2011.11.011 [CrossRef]
- Inouye, S.K., Brown, C.J. & Tinetti, M.E. (2009). Medicare nonpayment, hospital falls, and unintended consequences. New England Journal of Medicine, 360, 2390–2393. doi:10.1056/NEJMp0900963 [CrossRef]
- Oliver, D. (2008a). Evidence for fall prevention in hospitals. Journal of the American Geriatrics Society, 56, 1774–1775. doi:10.1111/j.1532-5415.2008.01830.x [CrossRef]
- Oliver, D. (2008b). Falls risk-prediction tools for hospital inpatients. Time to put them to bed?Age and Ageing, 377, 248–250. doi:10.1093/ageing/afn088 [CrossRef]
- Oliver, D., Daly, F., Martin, F.C. & McMurdo, M.E. (2004). Risk factors and risk assessment tools for falls in hospital in-patients: A systematic review. Age and Ageing, 33, 122–130. doi:10.1093/ageing/afh017 [CrossRef]
- Rubenstein, L.V. & Pugh, J.A. (2006). Strategies for promoting organizational and practice change by advancing implementation research. Journal of General Internal Medicine, 21(Suppl. 2), S58–S64. doi:10.1007/s11606-006-0276-8 [CrossRef]
- Rubenstein, L.Z. (2006). Falls in older people: Epidemiology, risk factors and strategies for prevention. Age and Ageing, 35(Suppl. 2), ii37–ii41. doi:10.1093/ageing/afl084 [CrossRef]
- Skubic, M., Alexander, G., Popescu, M., Rantz, M. & Keller, J. (2009). A smart home application to eldercare: Current status and lessons learned. Technology and Health Care, 17, 183–201. doi:10.3233/THC-2009-0551 [CrossRef]
- Stone, E. & Skubic, M. (2011). Evaluation of an inexpensive depth camera for in-home gait assessment. Journal of Ambient Intelligence and Smart Environments, 3, 349–361.
- Tinetti, M.E. & Kumar, C. (2010). The patient who falls: “It’s always a trade-off.”Journal of the American Medical Association, 303, 258–266. doi:10.1001/jama.2009.2024 [CrossRef]
- Titler, M.G., Shever, L.L., Kanak, M.F., Picone, D.M. & Qin, R. (2011). Factors associated with falls during hospitalization in an older adult population. Research & Theory for Nursing Practice, 25, 127–148. doi:10.1891/1541-6518.104.22.168 [CrossRef]