The worldwide population of adults 65 or older is predicted to grow from 524 million to approximately 1.5 billion between 2010 and 2050 (World Health Organization & National Institute of Aging, 2011). In Portugal, older adults comprise 19% of the population and this number is expected to double (to 35%) in the next decade (National Statistical Institute, 2011). The aging population has and will continue to have an enormous impact on the number of visits to emergency departments (EDs) (Pines, Mullins, Cooper, Feng, & Roth, 2013). Data from the largest Portuguese hospital showed that older adults account for 40% of all ED visits (Coimbra Hospital and University Centre, 2014). Compared to younger individuals, older adults are at increased risk of adverse outcomes during and after ED visits (Carpenter et al., 2015). During their ED stay, older adults are more likely to experience missed or incorrect diagnoses (Salvi et al., 2007), inadequate pain management (Hwang, Richardson, Harris, & Morrison, 2010; Iyer, 2011), and longer stays, and consume more resources (Banerjee, Dehnadi, & Mbamalu, 2011). On discharge, higher rates of admission and readmission (Pines et al., 2013), and higher functional loss and mortality rates have been reported (Niska, Bhuiya, & Xu, 2010).
Tools that predict poor outcomes can be used to facilitate the implementation of evidence-based practices that will then reduce the likelihood of these adverse outcomes. These tools are useful when creating a plan to prevent avoidable complications during and after an ED visit, and when transitioning to home or another setting. Validated and rapid risk screening instruments are desirable because of the increasing number of older adults presenting to the ED, and their increased risk for adverse outcomes (Carpenter et al., 2015). Early detection of older adults at risk for adverse outcomes through systematic screening in the ED can serve to identify high-risk groups in need of targeted assessment and early intervention (Carpenter et al., 2015).
The Geriatric Emergency Department Guidelines (American College of Emergency Physicians, American Geriatrics Society, Emergency Nurses Association, & Society for Academic Emergency Medicine, 2013) recommend screening older adult patients using the Identification of Seniors at Risk (ISAR) (McCusker et al., 1999) or a similar risk screening tool (Meldon et al., 2003) on the initial index visit rather than follow-up visits.
The ISAR was developed to identify older adults at risk of future adverse outcomes, such as functional decline, unplanned hospitalization or ED visit, nursing home admission, or death subsequent to an index ED visit (McCusker et al., 1999; Salvi et al., 2007). The ISAR represents the first of a two-step intervention; a second geriatric assessment includes the Standardized Evaluation and Intervention for Seniors at Risk (McCusker et al., 1999), which is a short, comprehensive evaluation of unmet needs used for older adults who are screened with ISAR and found to be at risk.
The ISAR is one of the most studied screening tools in the ED setting (Asomaning & Loftus, 2014; Carpenter et al., 2015; Thiem, Heppner, & Singler, 2015). Although the ISAR was originally developed in Canada (McCusker et al., 1999), many European countries, specifically Germany (Singler et al., 2014), Italy (Salvi et al., 2007; Salvi et al., 2012), The Netherlands (Buurman et al., 2011), Belgium (Braes et al., 2010; Moons et al., 2007), Denmark (Rosted et al., 2014), and Switzerland (Graf et al., 2012a), have translated and validated this tool for their populations.
Carpenter et al.'s (2015) systematic review reported that the ISAR predicted 30-day ED return, with a sensitivity of 0.47 to 1 (pooled estimate = 0.69) and specificity of 0.1 to 0.5 (pooled estimate = 0.39). For 180-day ED return, the sensitivity was 0.61 to 0.74 (pooled estimate = 0.61) and specificity was 0.39 to 0.69 (pooled estimate = 0.51). The pooled estimate of the sensitivity and specificity for 30-day hospital readmission was 0.85 and 0.21, respectively. Similar results were reported for 180-day hospital readmission (pooled estimate sensitivity was 0.8 and specificity was 0.34).
In Portugal, the translation and adaptation of the ISAR was completed in 2012 and the details are published elsewhere (Tavares, 2012). The aim of the current study was to examine the sensitivity and specificity of various cutoff points of the Portuguese version of the ISAR for post-ED discharge to predict early (30 days) and late (180 days) adverse outcomes in high-risk older adult patients. The study also included an evaluation of the tool's interrater reliability.
A prospective, single-center observational study was conducted in the ED of an urban university hospital in the central region of Portugal. A convenience sample of 402 patients was screened immediately after ED admission. Inclusion criteria were: community-dwelling or institutionalized adults 65 or older who were either cognitively intact (oriented to time and place) or, if cognitively impaired, had an informant (family member or caregiver) familiar with his/her health status. Exclusion criteria were: coma, required emergency procedures (surveys were completed after patients were stabilized), and absence of an informant. The current study was approved by the hospital's ethics committee.
Data Collection and Survey
The researcher (J.P.A.T.) conducted data collection after triage. Patients were screened from January 2013 to August 2014, every day per week, on a morning (8:00 a.m. to 4:00 p.m.) or afternoon (4:00 p.m. to 12:00 a.m.) shift. Only the initial visit (i.e., index visit) was considered in the case of repeated ED visits.
The survey included sociodemographic variables (e.g., age, gender), number of medications, responsibility for taking own medications, individual responsible for therapy management, living situation (e.g., home, nursing home, transferred from another hospital), triage code (Manchester Triage System), and presence of an informant (i.e., proxy or caregiver). ED length of stay and patient disposition (i.e., ED discharge or hospital admission) were also recorded.
The survey also included the ISAR tool. This tool has been used by different health care workers (e.g., nurses, ambulance technicians, other trained workers) as a screening tool for older adults at risk in the ED. The Portuguese translation and adaptation of ISAR was performed by Tavares (2012).
The ISAR is a brief screening tool that includes six items representing frequently observed problems (risk factors) in older adults presenting to the ED: (a) previous regular help before the illness or injury that brought the older adult to the ED, (b) need for more help than usual since the illness or injury that brought the older adult to the ED, (c) hospitalization in the past 6 months, (d) impaired vision, (e) serious problems with memory, and (f) polypharmacy (i.e., taking more than three different medications) (McCusker et al., 1999). In the current study, the medication item was adapted (cutoff of six drugs) to be more appropriate in the Portuguese clinical reality (Tavares, 2012). To analyze the sensitivity and specificity, three cutoffs were used (≥1, ≥2, and ≥3). Responses were dichotomized as yes or no, and 1 point was allocated for each yes answer. Individuals with a score ≥2 of 6 were considered at risk of adverse outcomes.
Information about early (ED return or unscheduled hospitalization within 30 days) and late (ED return or unscheduled hospitalization within 180 days) outcomes was extracted from the medical records.
Two raters (A [J.P.A.T.] and B [RN with 11 years of professional experience in EDs and a specialist in mental and psychiatric health]) performed interrater reliability over a 4-month period. Rater B was trained to complete the ISAR by Rater A. The ISAR was administered to eligible older adults first by Rater A and then Rater B.
Sensitivity and specificity analysis and the corresponding 95% confidence intervals for different ISAR cutoffs were estimated. Interrater reliability for total ISAR score and the three cutoff points were calculated using percentage agreement and Cohen's kappa statistic. Statistical analyses were performed using SPSS version 20.0 and p values ≤0.05 were considered significant.
The sample included 402 patients, of whom 55% (n = 221) were women, with a mean age of 80.9 years (SD = 6.7 years). Most (82.6%) resided in their homes, took an average of 6.4 medications (SD = 3.1 medications), were with caregivers or informants at the time of screening, and remained in the ED for approximately 12 hours.
According to the ISAR's cutoff ≥2, 308 older adults demonstrated positive results. More positive items were related to increased need of help (76.4%) and polypharmacy (82.8%). Previous help, impaired memory, recent hospitalization, and vision problems were positive in 36.8%, 33.1%, 30.6%, and 29.4% of participants, respectively.
At 30 days, 128 patients had returned to the ED and 53 (41.4%) had been readmitted at least once. For late outcomes (180 days), an ED return was recorded for 170 patients (42.3%); 73 (42.9%) were readmitted at least once.
Table 1 shows the results of the sensitivity and specificity analysis for each of the three cutoffs (≥1, ≥2, ≥3) for the ISAR considering ED return and hospital readmission outcomes (for 30 and 180 days). In all considered outcomes, a similar response pattern was achieved. Using a cutoff of ≥1, the ISAR was highly sensitive (range = 95.3% to 98.1%), but the specificity was very low (range = 4.2% to 9.9%). The original ISAR threshold (a cutoff of ≥2) showed decreased sensitivity (range = 81.8% to 88.7%) and improved specificity (range = 14.7% to 28.1%). The cutoff of ≥3 demonstrated the lowest sensitivity (range = 62.1% to 77.4%) and highest specificity (range = 50.9% to 53.5%). Table 2 shows the results of the interrater reliability testing. Cohen's kappa ranged from 0.81 to 0.9 for the three different thresholds and was ≥0.85 for the six ISAR items.
Sensitivity and Specificity of Different ISAR Cutoffs for 30 (Early) and 180 (Late) Days Considering ED Return and Hospital Readmission Outcomes
Interrater Reliability for the ISAR,b
The current study is the first to analyze the sensitivity and specificity of different cutoff points of the Portuguese version of the ISAR as a screening tool to detect adverse outcomes in the ED. The ISAR threshold (cutoff ≥2) presented a better compromise between sensitivity and specificity values when compared to cutoffs ≥1 or ≥3, respectively. The cutoff of ≥1 presented an excessively high value of sensitivity with a too low specificity value, whereas the cutoff of ≥3 presented intermediate/moderate values for sensitivity and specificity. The current results showed that the sensitivity is higher and specificity is lower, which is similar to findings from recently reported systematic reviews (Carpenter et al., 2015; Yao, Fang, Lou, & Anderson, 2015). The rewording of the cutoff used to define polypharmacy did not increase the specificity of the Portuguese version, as reported by Graf et al. (2012a).
The consistently high sensitivity of the ISAR for all outcomes does not allow risk to be accurately stratified (Carpenter et al., 2015). For this reason, it is not suitable to use the ISAR alone (Yao et al., 2015), but instead to assist clinical decision making when determining if older adults need the second-step intervention (i.e., geriatric assessment).
The current study tested the ISAR in a health care system, ED setting, and population that is different from the Canadian (McCusker et al., 1999) and European studies (Graf et al., 2012a; Rosted et al., 2014; Salvi et al., 2009; Singler et al., 2014). Differences in health care systems, community services, organizations, availability of home-based care services, the patient's profile (e.g., older, frailer), and pattern of ED admission (severity of index visit) among the countries may influence estimates of sensitivity and specificity (Yao et al., 2015).
The ISAR prognostic accuracy (sensitivity and specificity) for a cutoff ≥3 has been evaluated in other European countries, including Switzerland (Graf et al., 2012b) and Germany (Singler et al., 2014). Sensitivity was slightly lower compared with the cutoff ≥2, but showed moderate specificity. These results suggest a cutoff ≥3 be used instead of ≥2 because it yields a better overall predictive prognostic (Singler et al., 2014). However, in addition to the previously reported differences, there are differences in age cutoffs. These previous studies included adults 75 and older, which could have influenced early and late outcomes. A study in a Portuguese ED comparing these two thresholds for predictive validity related to poor outcomes is currently ongoing.
Interrater reliability results were excellent for all considered cutoffs and individual ISAR items. The agreement results were all >0.9 and the corresponding 95% confidence interval amplitudes were small. Cohen's kappa coefficients were also excellent (>0.8).
The increased demand for emergency services for older adults reinforces the significance of screening older adults at risk of adverse outcomes. The ISAR is a user-friendly screening instrument to identify older adults at risk of adverse outcomes after an ED visit. ED nurses play a crucial role in the identification of older adults at risk of adverse outcomes in the ED. A comprehensive, efficient geriatric evaluation promotes targeted clinical decision making based on local organizational and economic resources. According to Graf et al. (2012a), the ISAR remains clinically useful for avoiding further geriatric intervention in negative-screened individuals because of its high negative predictive value.
The ISAR can also be used as a strategy to raise awareness among nurses of the poor outcomes of at-risk older adults, and improve the ED experience. The screening of older adults by nurses is one of the first steps to improving quality of care in the ED, even without a specific geriatric initiative or program.
A convenience sample from one urban academic setting does not represent all possible EDs in Portugal. In addition, data collection was not conducted during night shifts (12:00 a.m. to 8:00 a.m.), and this may have resulted in selection bias. Finally, the outcomes were obtained from medical records, so it is possible that ED returns and hospital readmissions were missed. However, the study hospital is the major health care facility in the central region of Portugal. Considering these limitations (i.e., population and specific timing of work shift), more research is needed to validate changes to existing ISAR cutoffs.
Nurses have reported that the assessment of older adults is time-consuming and stressful for the older adult and nurse (Boltz, Parke, Shuluk, Capezuti, & Galvin, 2013). A short, standardized, comprehensive geriatric evaluation has been shown to be effective in reducing adverse outcomes in older adults, moreso than in the younger population (Yao et al., 2015). The ISAR is easy to administer and score, which adds to its clinical utility in the demanding ED setting.
The current study provides a sensitivity and specificity analysis for different ISAR cutoffs for early and late adverse outcomes (ED returns and hospital readmission). The validation results for cutoff ≥2 showed a higher sensitivity and low specificity values, as well as an excellent interrater reliability. The current findings suggest that the ISAR threshold should be modified according to population characteristics. A study of the predictive validity of the ISAR related to poor outcomes, such as ED revisits and hospital readmission within 30 and 180 days, is ongoing.
- American College of Emergency Physicians, American Geriatrics Society, Emergency Nurses Association & Society for Academic Emergency Medicine. (2013). Geriatric emergency department guidelines. Annals of Emergency Medicine, 63, e7–e25.
- Asomaning, N. & Loftus, C. (2014). Identification of seniors at risk (ISAR) screening tool in the emergency department: Implementation using the plan-do-study-act model and validation results. Journal of Emergency Nursing, 40, 357–364. doi:10.1016/j.jen.2013.08.014 [CrossRef]
- Banerjee, A., Dehnadi, H. & Mbamalu, D. (2011). The impact of very old patients in the ED. British Journal of Healthcare Management, 17, 72–74. doi:10.12968/bjhc.2011.17.2.72 [CrossRef]
- Boltz, M., Parke, B., Shuluk, J., Capezuti, E. & Galvin, J.E. (2013). Care of the older adult in the emergency department: Nurses views of the pressing issues. The Gerontologist, 53, 441–453. doi:10.1093/geront/gnt004 [CrossRef]
- Braes, T., Moons, P., Lipkens, P., Sterckx, W., Sabbe, M., Flamaing, J. & Milisen, K. (2010). Screening for risk of unplanned readmission in older patients admitted to hospital: Predictive accuracy of three instruments. Aging Clinical and Experimental Research, 22, 345–351. doi:10.1007/BF03324938 [CrossRef]
- Buurman, B., van den Berg, W., Korevaar, J.C., Milisen, K., de Haan, R.J. & de Rooij, S.E. (2011). Risk for poor outcomes in older patients discharged from an emergency department: Feasibility of four screening instruments. European Journal of Emergency Medicine, 18, 215–220. doi:10.1097/MEJ.0b013e328344597e [CrossRef]
- Carpenter, C.R., Shelton, E., Fowler, S., Suffoletto, B., Platts-Mills, T.F., Rothman, R.E. & Hogan, T.M. (2015). Risk factors and screening instruments to predict adverse outcomes for undifferentiated older emergency department patients: A systematic review and meta-analysis. Academic Emergency Medicine, 22, 1–21. doi:10.1111/acem.12569 [CrossRef]
- Coimbra Hospital and University Centre. (2014). Fluxo de doentes. Coimbra, Portugal: Author.
- Graf, C.E., Giannelli, S.V., Herrmann, F.R., Sarasin, F.P., Michel, J.-P., Zekry, D. & Chevalley, T. (2012a). Can we improve the detection of old patients at higher risk for readmission after an emergency department visit?Journal of the American Geriatrics Society, 60, 1372–1373. doi:10.1111/j.1532-5415.2012.04026.x [CrossRef]
- Graf, C.E., Giannelli, S.V., Herrmann, F.R., Sarasin, F.P., Michel, J.P., Zekry, D. & Chevalley, T. (2012b). Identification of older patients at risk of unplanned readmission after discharge from the emergency department—Comparison of two screening tools. Swiss Medical Weekly, 141, 1–9. doi:10.4414/smw.2011.13327 [CrossRef]
- Hwang, U., Richardson, L.D., Harris, B. & Morrison, R.S. (2010). The quality of emergency department pain care for older adult patients. Journal of the American Geriatrics Society, 58, 2122–2128. doi:10.1111/j.1532-5415.2010.03152.x [CrossRef]
- Iyer, R.G. (2011). Pain documentation and predictors of analgesic prescribing for elderly patients during emergency department visits. Journal of Pain Symptom Management, 41, 367–373. doi:10.1016/j.jpainsymman.2010.04.023 [CrossRef]
- McCusker, J., Bellavance, F., Cardin, S., Trépanier, S., Verdon, J. & Ardman, O. (1999). Detection of older people at increased risk of adverse health outcomes after an emergency visit: The ISAR screening tool. Journal of the American Geriatrics Society, 47, 1229–1237. doi:10.1111/j.1532-5415.1999.tb05204.x [CrossRef]
- Meldon, S.W., Mion, L.C., Palmer, R.M., Drew, B.L., Connor, J.T., Lewicki, L.J. & Emerman, C.L. (2003). A brief risk-stratification tool to predict repeat emergency department visits and hospitalizations in older patients discharged from the emergency department. Academic Emergency Medicine, 10, 224–232. doi:10.1111/j.1553-2712.2003.tb01996.x [CrossRef]
- Moons, P., De Ridder, K., Geyskens, K., Sabbe, M., Braes, T., Flamaing, J. & Milisen, K. (2007). Screening for risk of readmission of patients aged 65 years and above after discharge from the emergency department: Predictive value of four instruments. European Journal of Emergency Medicine, 14, 315–323. doi:10.1097/MEJ.0b013e3282aa3e45 [CrossRef]
- National Statistical Institute. (2011). Censos 2011–Resultados provisórios [in Portuguese]. Lisbon, Portugal: Author.
- Niska, R., Bhuiya, F. & Xu, J. (2010). National hospital ambulatory medical care survey: 2007 emergency department summary. National Health Statistics Reports, 26, 1–31.
- Pines, J.M., Mullins, P.M., Cooper, J.K., Feng, L.B. & Roth, K.E. (2013). National trends in emergency department use, care patterns, and quality of care of older adults in the United States. Journal of the American Geriatrics Society, 61, 12–17. doi:10.1111/jgs.12072 [CrossRef]
- Rosted, E., Schultz, M., Dynesen, H., Dahl, M., Sørensen, M. & Sanders, S. (2014). The Identification of Seniors at Risk screening tool is useful for predicting acute readmissions. Danish Medical Journal, 61, A4828.
- Salvi, F., Morichi, V., Grilli, A., Giorgi, R., De Tommaso, G. & Dessi-Fulgheri, P. (2007). The elderly in the emergency department: A critical review of problems and solutions. Internal Emergency Medicine, 2, 1–32. doi:10.1007/s11739-007-0081-3 [CrossRef]
- Salvi, F., Morichi, V., Grilli, A., Lancioni, L., Spazzafumo, L., Polonara, S. & Lattanzio, F. (2012). Screening for frailty in elderly emergency department patients by using the Identification of Seniors at Risk (ISAR). Journal of Nutrition, Health and Aging, 16, 313–318. doi:10.1007/s12603-011-0155-9 [CrossRef]
- Salvi, F., Morichi, V., Grilli, A., Spazzafumo, L., Giorgi, R., Polonara, S. & Dessì-Fulgheri, P. (2009). Predictive validity of the Identification of Seniors At Risk (ISAR) screening tool in elderly patients presenting to two Italian emergency departments. Aging Clinical and Experimental Research, 21, 69–75. doi:10.1007/BF03324901 [CrossRef]
- Singler, K., Heppner, H.J., Skutetzky, A., Sieber, C., Christ, M. & Thiem, U. (2014). Predictive validity of the Identification of Seniors at Risk screening tool in a German emergency department setting. Gerontology, 60, 413–419. doi:10.1159/000358825 [CrossRef]
- Tavares, J.P. (2012). Identification of Seniors at Risk in the emergency room: What reality? [article in Portuguese]. Journal of Aging and Innovation, 4, 14–25.
- Thiem, U., Heppner, H.J. & Singler, K. (2015). Instruments to identify elderly patients in the emergency department in need of geriatric care. Zeitschrift für Gerontologie und Geriatrie, 48, 4–9. doi:10.1007/s00391-014-0852-1 [CrossRef]
- World Health Organization, & National Institute of Aging. (2011). Global health and ageing. Retrieved from http://www.who.int/ageing/publications/global_health.pdf?ua=1
- Yao, J.-L., Fang, J., Lou, Q.-Q. & Anderson, R.M. (2015). A systematic review of the Identification of Seniors at Risk (ISAR) tool for the prediction of adverse outcome in elderly patients seen in the emergency department. International Journal of Clinical and Experimental Medicine, 8, 4778–4786.
Sensitivity and Specificity of Different ISAR Cutoffs for 30 (Early) and 180 (Late) Days Considering ED Return and Hospital Readmission Outcomes
|Cutoff||Sensitivity (%)||95% CI||Specificity (%)||95% CI|
|30 days (early)|
| ED returns|
| ≥1||95.3||[0.92, 0.99]||9.2||[0, 0.21]|
| ≥2||86.7||[0.8, 0.93]||28.1||[0.28, 0.36]|
| ≥3||66.4||[0.56, 0.76]||53.5||[0.45, 0.62]|
| Hospital readmission|
| ≥1||98.1||[0.94, 1]||6.7||[0, 0.3]|
| ≥2||88.7||[0.8, 0.98]||14.7||[0.16, 0.38]|
| ≥3||77.4||[0.64, 0.9]||50.9||[0.33, 0.64]|
|180 days (late)|
| ED returns|
| ≥1||96.5||[0.94, 0.99]||9.9||[0, 0.22]|
| ≥2||81.8||[0.75, 0.88]||27.2||[0.16, 0.38]|
| ≥3||62.1||[0.53, 0.71]||53.4||[0.45, 0.62]|
| Hospital readmission|
| ≥1||97.3||[0.93, 1]||4.2||[0, 0.24]|
| ≥2||82.2||[0.73, 0.92]||24.6||[0.01, 0.37]|
| ≥3||71.2||[0.59, 0.83]||51.2||[0.36, 0.66]|
Interrater Reliability for the ISARa,b
|Item||Rater A||Rater B||Agreement (95% CI)||Cohen's Kappa (p Value)|
|Previous regular help|
| No||64||60||0.94 [0.89, 0.99]||0.87 (<0.001)|
|Need for more help|
| Yes||71||72||0.97 [0.94, 1]||0.92 (<0.001)|
|Hospitalization (past 6 months)|
| No||80||80||0.98 [0.95, 1]||0.94 (<0.001)|
| No||66||69||0.95 [0.97, 0.99]||0.85 (<0.001)|
|Serious memory problems|
| No||62||64||0.96 [0.92, 1]||0.91 (<0.001)|
| Yes||79||81||0.98 [0.95, 1]||0.94 (<0.001)|
|ISAR cutoff ≥1|
| ≥1||93||94||0.99 [0.97, 1]||0.9 (<0.001)|
|ISAR cutoff ≥2|
| ≥2||78||80||0.94 [0.89, 0.99]||0.81 (<0.001)|
| 0 to 1||21||19|
|ISAR cutoff ≥3|
| ≥3||58||59||0.97 [0.94, 1]||0.94 (<0.001)|
| 0 to 2||41||40|