Delirium is common in older adults and leads to poor outcomes in hospitalized individuals, including decreased function, heightened emotional distress, and increased staff burden (Morandi et al., 2015). In addition, delirium costs the United States $164 billion annually in health care expenses and can lead to physical and mental decline, nursing home placement, and death (Inouye, Westendorp, & Saczynski, 2014; Leslie & Inouye, 2011). Despite these poor outcomes, delirium is frequently not recognized, especially by busy health care professionals and hospital staff. This under-recognition is particularly true for individuals with hypoactive (quiet) delirium and those with delirium superimposed on dementia (Fick, Agostini, & Inouye, 2002; Inouye, Foreman, Mion, Katz, & Cooney, 2001).
The Confusion Assessment Method (CAM) is the most commonly used tool to assess for delirium worldwide, and has been important in advancing delirium research. Moreover, there is growing interest in integrating the CAM into clinical care; however, this process has presented some challenges. The CAM is a diagnostic algorithm, not a delirium assessment. Previous research has shown that asking nurses to complete the CAM without formal mental status testing and with minimal training results in poor sensitivity (Inouye et al., 2001; Lemiengre et al., 2006). A brief, structured assessment tool that includes mental status testing and standardizes operationalization of the CAM algorithm might prove valuable to facilitate widespread implementation of delirium screening.
The CAM for the Intensive Care Unit (CAM-ICU) is an example of an assessment tool that has facilitated widespread delirium screening in intensive care settings (Wei, Fearing, Sternberg, & Inouye, 2008). However, the CAM-ICU has limited sensitivity for assessing delirium in general medical and surgical patients (Kuczmarska et al., 2016). Filling this void is the 3D-CAM, a relatively new, brief diagnostic assessment for CAM-defined delirium that is optimized for general medicine and surgical settings, can be completed in 3 minutes, and has 95% sensitivity and 94% specificity compared to a clinical gold standard (Marcantonio et al., 2014). Within the 3D-CAM, two items were identified (i.e., months of the year backwards and the day of the week) that can serve as an ultra-brief screener to rule out delirium in less than 40 seconds (Fick et al., 2015).
Another key area in delirium screening is to identify the most effective and efficient clinician to perform the screening and determine how to integrate assessment and management of delirium into clinician workflow and the health system. In health care research, it is rare to examine how three different disciplines perform the same task. Usually, a task is assigned to a single discipline, either because of a unique skill set or tradition. Evidence is not available for which discipline is most appropriate for delirium identification in the hospital setting, and no studies have compared physicians, nurses, and certified nursing assistants (CNAs). Each of these disciplines has unique daily responsibilities. Thus, in addition to measuring effectiveness and cost-efficiency of delirium screening, it is important to understand the specific barriers to and facilitators of delirium assessment faced by each discipline. Performing comparative effectiveness studies using mixed methods allows one to understand the most effective and efficient manner to screen for delirium. This information can help guide decisions within health care systems on how to assure the best information is available to provide optimal care at the bedside. This information is critical to clinical systems implementation, priority setting, and policy making for delirium care (Pincus, 2011).
Thus, the purpose of the current pilot study was to test the feasibility of a two-step delirium detection protocol completed by clinicians using the ultra-brief screener followed by the 3D-CAM (Marcantonio et al., 2014) in those with a positive screen. This process will be part of a systematic delirium identification process that will be tested in a larger trial. First, the ultra-brief screener, comprising two items—“Please tell me the day of the week” and “Please tell me the months of the year backwards (MOYB) starting with December”—is administered. Patients must indicate the exact day of the week and recite all 12 months backwards with no errors. They are allowed to look at a newspaper if one is nearby or a calendar on the wall. They are also allowed to be prompted twice on the MOYB if they get stuck; for example, they may be asked, “Please tell me the month that comes before July.” If either or both of the items in the ultra-brief screener is/are incorrect (or if the patient says “I don't know” or gives no meaningful response), physicians and RNs are instructed to move on to the 3D-CAM and CNAs are instructed to notify nurses. The 3-D CAM instrument can be accessed at the Hospital Elder Life Program website (access https://www.hospitalelderlifeprogram.org/delirium-instruments).
In the current pilot study, feasibility was determined and preliminary estimates were obtained regarding the effectiveness of this protocol as implemented by physicians, RNs, and CNAs relative to a trained researcher (D.M.F., J.G., J.M.) who served as a reference standard. Qualitative data were also collected from these clinician groups.
Study Sample and Setting
An embedded mixed methods design was used (Creswell & Plano-Clark, 2011). Participants were recruited via screening consecutive hospital admissions to two general medicine wards of a large urban academic medical center and a rural community hospital. Eligibility criteria were: (a) general medicine (hospitalist) service admission, (b) age 70 or older, (c) expected hospital length of stay ≥2 days, (d) ability to communicate adequately in English, and (e) not admitted for terminal care. The study protocol was approved by the Institutional Review Boards of all participating institutions.
Quantitative Data Collection Protocol
A series of four delirium assessments were performed on enrolled participants, blinded to each other's results, within a 2-hour time window. Reference standard assessments for this pilot were performed by senior research assistants with at least a master's degree. These assessors were trained in conducting the assessment using in-person and video methods. Training was performed by research personnel with each clinician in a one-on-one manner. There were slight differences between sites, as clinicians were often on call and busy, but all were trained using a procedures manual with step-by-step instructions for the ultra-brief screener and 3D-CAM (available from the author upon request). RNs and physicians were trained for approximately 15 to 20 minutes and then administered a video post-test. RNs at Mount Nittany Medical Center received 0.5 CNE credits for the training. CNAs were trained in the two-item, ultra-brief screener for 5 minutes and performed a practice administration to research personnel.
After obtaining informed consent from patients or designated surrogate decision makers, trained researchers conducted the reference standard delirium assessment. This assessment included cognitive testing using the Mini-Mental State Examination (Folstein, Folstein, & McHugh, 1975 [purchased from Psychological Assessment Resources]), supplemented with the Digit Span for additional attention testing (Palmer & Meldon, 2003), Delirium Symptom Interview (DSI) to elicit patient symptoms of delirium (Albert et al., 1992), and Geriatric Depression Scale (Yesavage, 1988). Using data from the cognitive testing and DSI, the researcher completed the 10-item (long) CAM, which generates a diagnosis of delirium using the CAM diagnostic algorithm (Inouye et al., 1990) and a measure of delirium severity (Inouye, Kosar, et al., 2014).
After trained researchers completed the reference standard delirium assessment, physicians and RNs were asked to complete the two-step delirium identification protocol, which comprised the two-item, ultra-brief screener; for participants who got either or both items incorrect (or who gave no response), physicians and RNs were also instructed to administer the 3D-CAM. CNAs were asked to complete the ultra-brief screener only, and notify the nurse of an abnormal result. All assessments were timed for both components—the ultra-brief screener and, if necessary, the 3D-CAM.
Qualitative Data Collection
After collecting patient quantitative data, researchers (N = 4, two at each site: D.M.F., J.G., J.M., and another) focused on the actual delirium-screening event. Non-participant observation (i.e., direct observation when researchers are not actively participating in the event they are observing) using ethno-graphic methodology (Speziale, Streubert, & Carpenter, 2007) was used to observe the delivery of the delirium assessment protocol followed by brief interviews focused on providers' perceptions of the protocol (e.g., How did the delirium screening go for you? What was the most difficult part? What did you like about it most?). These questions and observations addressed barriers and facilitators to bedside delirium screening. Observational data regarding the contextual features of the unit at the time of assessment (e.g., unit activities) were also collected. All data were recorded as field notes, then transcribed for analysis.
Sensitivity and specificity for the two-item screener alone (all three disciplines), and the two-step protocol (physicians, RNs only) were calculated for the quantitative analysis. Sensitivity was defined as percent that tested positive on the two-step protocol among the gold standard positives, and specificity as percent that tested negative on the two-step protocol among the gold standard negatives. Because of the pilot nature of the current study, with limited sample size, confidence intervals were not calculated for sensitivity or specificity results. Analyses were conducted using SAS version 9.3.
Qualitatively, an iterative process of thematic analysis of the field notes was performed. Two researchers (who were also data collectors; D.M.F., J.M.) reviewed and independently coded the dataset. Preliminary coding was discussed to generate common themes, categorized as either barriers interfering or facilitators assisting with administration of the delirium screens. Multiple coders (data collectors from both sites; D.M.F., J.M., J.G., J.P.) helped validate interpretation of the codes (Creswell, 2016). Lastly, two additional research team members who were not involved in earlier stages of thematic analysis reviewed the dataset and confirmed the findings. A third independent rater who was not in the field performed inter-coder agreement.
Over a 2-week period, 36 eligible patients were identified at both study sites, and 34 were approached (one patient was on respiratory precautions, and one patient was discharged before approach). Of these patients, 24 were enrolled (nine refused, one lacked capacity and the proxy was unable to be reached). One patient was excluded because he deviated greatly from the protocol with an approximate 4-hour gap (double the protocol) between delirium assessments and was noted to be fluctuating during this time. Therefore, the analysis was conducted on 23 enrolled patients.
All three clinical assessments were successfully completed in 100% of enrolled participants, and within the target 2-hour time window in 91%. Mean age of study patients was 81 years; 52% of patients were male and 100% were White (Table 1). Based on the reference standard rating, 5 (22%) of 23 patients had delirium. Seven physicians, 13 RNs, and seven CNAs participated in the assessments. Results suggest that the two-item, ultra-brief screener is effective in identifying delirium, with high sensitivity and moderate specificity, as designed (Table 2). Moreover, all three clinicians could administer the screener accurately. The full two-step delirium protocol, performed by physicians and RNs, retained very good to excellent sensitivity and substantially improved specificity, as designed. Physicians and RNs were able to administer the two-step protocol effectively.
Sensitivity and Specificity Results by Clinician Type
Qualitative analyses to identify barriers to and facilitators of protocol implementation based on the field notes from the observations and interviews are summarized in Table 3. Seven themes emerged under barriers (n = 4) and facilitators (n = 3). The qualitative data revealed many positive comments about the ease of use of the screener. Some barriers were noted, including finding time to perform the assessment; excessive encouragement or prompting by clinical staff; time between training and first use; interdisciplinary breakdowns in communication (i.e., instances when the CNA did not feel he/she had the power to report on cognitive changes); and challenges in screening older adults with severe cognitive impairment, impaired responsiveness, or anxiety about the cognitive testing.
Facilitators and Barriers Observed During Bedside Delirium Screening
Discussion and Implications for Practice and Research
The identification of delirium in the hospital setting continues to be a challenge. In the current preliminary work, the authors found that their two-step delirium identification protocol (ultra-brief screener followed in positives by the 3D-CAM) was feasible as administered by RNs and physicians, whereas the screener was feasibly administered by CNAs. Moreover, the protocol worked as designed and achieved high sensitivity and specificity. Finally, the protocol was time efficient, taking <40 seconds for the ultra-brief screener and 3 minutes for the 3D-CAM. The qualitative research found that all three disciplines expressed enthusiasm and confidence in completing the two-step protocol. Potential barriers were identified to the larger study of this protocol, such as time between training. Qualitative barriers and facilitators were also found that will aid in eventual implementation of the protocol.
Nurses and physicians report that lack of time and deciding who is responsible for delirium screening are barriers to assessing and managing delirium (Steis & Fick, 2008). Delirium screening that is quick, simple, highly sensitive, validated in patients with hypoactive delirium and those with underlying dementia, and that requires little training is urgently needed.
This study extends the current authors' prior work in which the 3D-CAM (Marcantonio et al., 2014) was developed and validated and the ultra-brief delirium screener was identified from secondary data analysis. The best two-item screener was the combination of “Please tell me the months of the year backwards” and “Please tell me the day of the week,” with a sensitivity of 93% (95% confidence interval [CI], [0.81, 0.99]) and specificity of 64% (95% CI, [0.56, 0.70]) (Fick et al., 2015). Based on these two studies, the two-step delirium identification protocol was developed, which comprises administering the ultra-brief screener first, and in “positives,” going on to administer the 3D-CAM. This study is one of the first to pilot test a two-step protocol in a prospective sample and test the real-world administration of this approach by RNs and physicians, and the ultra-brief screener by CNAs.
Previous screeners have not been as sensitive when used in the actual practice setting with clinicians as opposed to trained researchers. Testing the protocol as administered by the actual personnel who would implement it clinically is an important contribution. If the current protocol proves to be efficient and effective in a larger sample, it has the potential to streamline and individualize delirium screening and should be attractive to leaders of health care systems who may be searching for practical tools to identify delirium.
Although other studies found time to be a barrier in delirium identification (Morandi et al., 2017), the dilemma of who is most efficient and effective at delirium identification and whose responsibility it should be to screen for delirium are unanswered questions. Two issues surround this uncertainty: (a) Which discipline can administer the protocol most effectively (i.e., achieving closest concordance with a reference standard)?; and (b) Which discipline can administer the protocol most efficiently (i.e., in the shortest amount of time and at lowest cost)? The authors' planned larger study addresses both of these critical questions.
This pilot study demonstrated the feasibility of the proposed study approach, which is currently being tested in a new National Institutes of Health–funded interdisciplinary study “Researching Efficient Approaches to Delirium Identification” (READI), which will enroll 450 adults 70 and older and approximately 75 clinicians in Pennsylvania and Massachusetts. The study will use a protocol similar to this pilot, except that delirium will be assessed on 2 consecutive days in enrolled patients (assuming they are not discharged). Comparative effectiveness work such as this is critical for providing better information on delirium identification to patients, providers, and policymakers.
Limitations and Strengths
The current pilot study had some inherent limitations, such as a small, non-diverse sample, which may limit generalizability of the results. However, the study also had several important strengths and provided key information for proving feasibility of this complex, interdisciplinary protocol. Key feasibility metrics included: (a) successful enrollment of hospitalized older patients, (b) assessments completed by clinicians from all three disciplines with minimal training, (c) favorable comparison of the ultra-brief screener and two-step protocol with the reference standard, and (d) identification of barriers and facilitators using direct observation qualitative methods. Notably, this study was performed at two distinct study sites (i.e., a large, urban academic medical center and a small, rural, non-teaching community hospital), which should help address protocol effectiveness, efficiency, and implementation challenges across diverse settings, facilitating future large-scale implementation efforts.
Nurses and other health care professionals play a key role in bedside delirium identification. The ultra-brief screener followed by the 3D-CAM is a promising and innovative approach to improve delirium identification by clinicians. The current pilot study provided valuable results to guide future work. The authors plan to replicate this pilot in a much larger sample, collecting the data necessary to inform future widespread implementation, and furthering the goal of translating research into the real world of clinical practice.
- Albert, M.S., Levkoff, S.E., Reilly, C., Liptzin, B., Pilgrim, D., Cleary, P.D. & Rowe, J.W. (1992). The delirium symptom interview: An interview for the detection of delirium symptoms in hospitalized patients. Journal of Geriatric Psychiatry and Neurology, 5(1), 14–21.
- Creswell, J.W. (2016). 30 essential skills for the qualitative researcher. Thousand Oaks, CA: Sage.
- Creswell, J.W. & Plano-Clark, V.L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.
- Fick, D.M., Agostini, J.V. & Inouye, S.K. (2002). Delirium superimposed on dementia: A systematic review. Journal of the American Geriatrics Society, 50, 1723–1732. doi:10.1046/j.1532-5415.2002.50468.x [CrossRef]
- Fick, D.M., Inouye, S.K., Guess, J., Ngo, L., Jones, R., Saczynski, J. & Marcantonio, E. (2015). Preliminary development of an ultrabrief two-item bedside test for delirium. Journal of Hospital Medicine, 10, 645–650. doi:10.1002/jhm.2418 [CrossRef]
- Folstein, M.F., Folstein, S.E. & McHugh, P.R. (1975). Mini-mental state: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189–198. doi:10.1016/0022-3956(75)90026-6 [CrossRef]
- Inouye, S.K., Foreman, M.D., Mion, L.C., Katz, K.H. & Cooney, L.M. Jr. . (2001). Nurses' recognition of delirium and its symptoms: Comparison of nurse and researcher ratings. Archives of Internal Medicine, 161, 2467–2473. doi:10.1001/archinte.161.20.2467 [CrossRef]
- Inouye, S.K., Kosar, C.M., Tommet, D., Schmitt, E.M., Puelle, M.R., Saczynski, J.S. & Jones, R.N. (2014). The CAM-S: Development and validation of a new scoring system for delirium severity in 2 cohorts. Annals of Internal Medicine, 160, 526–533. doi:10.7326/M13-1927 [CrossRef]
- Inouye, S.K., Van Dyck, C., Alessi, C., Balkin, S., Siegal, A. & Horwitz, R. (1990). Clarifying confusion: The confusion assessment method. A new method for detection of delirium. Annals of Internal Medicine, 113, 941–948. doi:10.7326/0003-4819-113-12-941 [CrossRef]
- Inouye, S.K., Westendorp, R.G. & Saczynski, J.S. (2014). Delirium in elderly people. The Lancet, 383, 911–922. doi:10.1016/S0140-6736(13)60688-1 [CrossRef]
- Kuczmarska, A., Ngo, L.H., Guess, J., O'Connor, M.A., Branford-White, L., Palihnich, K. & Marcantonio, E.R. (2016). Detection of delirium in hospitalized older general medicine patients: A comparison of the 3D-CAM and CAM-ICU. Journal of General Internal Medicine, 31, 297–303. doi:10.1007/s11606-015-3514-0 [CrossRef]
- Lemiengre, J., Nelis, T., Joosten, E., Braes, T., Foreman, M., Gastmans, C. & Milisen, K. (2006). Detection of delirium by bedside nurses using the confusion assessment method. Journal of the American Geriatrics Society, 54, 685–689. doi:10.1111/j.1532-5415.2006.00667.x [CrossRef]
- Leslie, D.L. & Inouye, S.K. (2011). The importance of delirium: Economic and societal costs. Journal of the American Geriatrics Society, 59(Suppl. 2), S241–S243. doi:10.1111/j.1532-5415.2011.03671.x [CrossRef]
- Marcantonio, E., Ngo, L., O'Connor, M., Jones, R., Crane, P.K., Metzger, E. & Inouye, S.K. (2014). 3D-CAM: Derivation and validation of a 3-minute diagnostic interview for CAM-defined delirium: A cross-sectional diagnostic test study. Annals of Internal Medicine, 161, 554–561. doi:10.7326/M14-0865 [CrossRef]
- Morandi, A., Davis, D., Bellelli, G., Arora, R.C., Caplan, G.A., Kamholz, B. & Rudolph, J.L. (2017). The diagnosis of delirium superimposed on dementia: An emerging challenge. Journal of the American Medical Directors Association, 18, 12–18. doi:10.1016/j.jamda.2016.07.014 [CrossRef]
- Morandi, A., Lucchi, E., Turco, R., Morghen, S., Guerini, F., Santi, R. & Bellelli, G. (2015). Delirium superimposed on dementia: A quantitative and qualitative evaluation of patient experience. Journal of Psychosomatic Research, 79, 281–287. doi:10.1016/j.jpsychores.2015.07.010 [CrossRef]
- Palmer, R.M. & Meldon, S.W. (2003). Digit span test in acute care. In Hazzard, W.R. (Ed.), Principles of geriatric medicine and gerontology (5th ed., pp. 157–168). New York, NY: McGraw-Hill Professional.
- Pincus, H.A. (2011). Academic health centers and comparative effectiveness research: Baggage, buckets, basics, and bottles. Academic Medicine, 86, 659–660. doi:10.1097/ACM.0b013e3182188653 [CrossRef]
- Speziale, H.S., Streubert, H.J. & Carpenter, D.R. (2007). Qualitative research in nursing: Advancing the humanistic imperative. Philadelphia, PA: Lippincott Williams & Wilkins.
- Steis, M.R. & Fick, D.M. (2008). Are nurses recognizing delirium? A systematic review. Journal of Gerontological Nursing, 34(9), 40–48. doi:10.3928/00989134-20080901-12 [CrossRef]
- Wei, L.A., Fearing, M.A., Sternberg, E.J. & Inouye, S.K. (2008). The Confusion Assessment Method: A systematic review of current usage. Journal of the American Geriatrics Society, 56, 823–830. doi:10.1111/j.1532-5415.2008.01674.x [CrossRef]
- Yesavage, J.A. (1988). Geriatric Depression Scale. Psychopharmacology Bulletin, 2 4, 709–711.
|Variable||All Sites (N = 23)||MA Site (n = 14)||PA Site (n = 9)|
|Age (years) (mean, SD)||80.7 (7.1)||79.4 (7.5)||82.7 (6.3)|
|Gender (n, %)|
| Male||12 (52)||7 (50)||5 (56)|
| Female||11 (48)||7 (50)||4 (44)|
|Race (n, %)|
| White||23 (100)||14 (100)||9 (100)|
Sensitivity and Specificity Results by Clinician Type
|Clinician Type||Two-Item Ultra-Brief Screener||Two-Step Delirium Identification Protocol|
|Physicians||80 (4/5)||56 (10/18)||33 (4/12)||91 (10/11)||80 (4/5)||83 (15/18)||57 (4/7)||94 (15/16)|
|RNs||100 (5/5)||67 (12/18)||45 (5/11)||100 (12/12)||100 (5/5)||89 (16/18)||71 (5/7)||100 (16/16)|
|CNAs||100 (5/5)||61 (11/18)||42 (5/12)||100 (11/11)||N/A||N/A||N/A||N/A|
Facilitators and Barriers Observed During Bedside Delirium Screening
|Providers expressed confidence using the protocol with patients||“I think the two-item form is just as sensitive as the long form, I really like it...I've started doing it on several of my patients not in your study.” (Hospitalist)|
|“What I liked best was the first two questions and how quick and reliable it was and I liked knowing there was a process with the next form…if she didn't pass it that I would be confident in the step of the 3D-CAM.” (RN)|
|Providers rapidly gained comfort administering the protocol||“You can do it the first thing in the morning when you get the vital sign, it is just very easy to stick the two questions in.” (RN)|
|“She [CNA] said it was very easy and she felt comfortable doing the interview, and she didn't have any questions.” (Researcher field note)|
|Majority of staff were enthusiastic and positive about the protocol||“I liked working with you guys. You made it easy for me by handing me a clipboard. I felt comfortable with the questions, even though it is just my first day.” (Hospitalist)|
|“I just loved it! I was really happy to be able to do it [smiling].” (RN)|
|Too much time between training and first use||“I [researcher] had to correct the CNA on the ‘what is the day of the week?’ question; this is the first time for the CNA to do the study and the training was a few weeks ago. The next time she did the assessment, she did it correctly.” (Researcher field note)|
|Communication: Excess encouragement or prompting and breakdowns in interprofessional communication, and communication between clinicians and CNAs||“I wish I could ask these questions [as part of routine clinical practice—not just for a research study]. Sometimes I ask someone else to do them, like the nurse or physical therapist. I notice the confusion because I'm often one of the first people to notice it as I am taking care of the patient.” (CNA)|
|“There is not good communication [among nurses and CNAs] about the patients.” (CNA)|
|“I noticed the nurse praised the patient several times saying, ‘good, that is right’ and ‘that is wrong’ sort of thing. She didn't notice that she was doing this [correcting the patient].” (Researcher field note)|
|“Sometimes, a family member in the room tried to give patient hints.” (Researcher field note)|
|Clinician difficulties in engaging patients who were more challenging or less responsive||“The patient has severe dementia and was almost non-responsive and the physician was not exactly sure what to do when the patient was not answering.” (Researcher field note)|
|“The CNA was not confident with how to communicate with a patient who was actively delirious and not answering all of the questions.” (Researcher field note)|
|“The patient, an older woman, began to cry during the assessments saying she already answered these questions twice and she just wanted to go home.” (Researcher field note)|
|Busy unit, busy patient room traffic, and competing priorities||“I can do this, but first I have to do all the blood sugars for this side of the hallway and I have not had lunch and it's 11:15 already.” (CNA to research assistant)|
|“I'll try and let you know when I have time to do this [sighed twice and put hand on head while holding two intravenous lines in other hand].” (RN to research assistant)|
|“Many people coming and going throughout assessments (e.g., to check patient's vital signs or blood glucose, visitors or medical staff attending to roommate, etc.).” (Researcher field note)|