Journal of Gerontological Nursing

Feature Article Supplemental Data

The Language of Delirium: Keywords for Identifying Delirium from Medical Records

Margaret R. Puelle, BS; Cyrus M. Kosar, MA; Guoquan Xu, MD; Eva Schmitt, PhD; Richard N. Jones, ScD; Edward R. Marcantonio, MD, MSc; Zara Cooper, MD; Sharon K. Inouye, MD, MPH; Jane S. Saczynski, PhD

Abstract

Electronic medical records (EMRs) offer the opportunity to streamline the search for patients with possible delirium. The purpose of the current study was to identify words and phrases commonly noted in charts of patients with delirium. The current study included 67 patients (nested within a cohort study of 300 patients) ages 70 and older undergoing major elective surgery with evidence of confusion in their medical charts. Eight keywords or phrases had positive predictive values of 60% to 100% for delirium. Keywords were charted more often in nursing notes than physician notes. A brief list of keywords may serve as a building block for a methodology to screen for possible delirium from charts, with particular attention to nursing notes, for research and real-time clinical decision making. [Journal of Gerontological Nursing, 41(8), 34–42.]

Ms. Puelle is Research Associate, Mr. Kosar is Programmer Analyst, Dr. Xu is Research Associate, Dr. Schmitt is Assistant Scientist and Associate Director, Dr. Jones is Adjunct Senior Scientist, Dr. Inouye is Director, and Dr. Saczynski is Adjunct Scientist, Aging Brain Center, Institute for Aging Research, Hebrew SeniorLife; and Dr. Cooper is Assistant Professor of Surgery, Department of Surgery, Brigham and Women’s Hospital, Boston, Massachusetts. Dr. Jones is also Associate Professor, Departments of Psychiatry and Human Behavior, and Neurology, Warren Alpert Medical School, Brown University Medical School, Providence, Rhode Island. Dr. Marcantonio is Professor of Medicine, and Dr. Inouye is also Professor of Medicine, Harvard Medical School; Dr. Marcantonio is also Section Chief for Research, Division of General Medicine and Primary Care, Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts. Dr. Saczynski is also Associate Professor, Department of Medicine and Meyers Primary Care Institute, University of Massachusetts Medical School, Worcester, Massachusetts.

The authors have disclosed no potential conflicts of interest, financial or otherwise. This study was funded by grants (P01AG031720 [S. Inouye] and K07AG041835 [S. Inouye]) from the National Institute on Aging. Dr. Inouye holds the Milton and Shirley F. Levy Family Chair. Dr. Saczynski was supported in part by funding from the National Institute on Aging (K01AG33643) and the National Heart Lung and Blood Institute (U01HL105268). Dr. Marcantonio was supported in part by a grant (R01AG030618) and a Mid-Career Investigator Award (K24 AG035075) from the National Institute on Aging. The authors thank Bonnie Wong, PhD, for assistance with chart adjudications. This work is dedicated to the memory of Joshua Bryan Inouye Helfand. Drs. Saczynski and Inouye contributed equally to this manuscript.

*: Indicates multiple different endings, such as -um and -ous.*: Indicates multiple different endings, such as -ful and -ing.

Address correspondence to Jane S. Saczynski, PhD, Division of Geriatric Medicine, Department of Medicine, 377 Plantation St., Suite 315, Worcester, MA 01605; e-mail: Jane.saczynski@umassmed.edu.

Received: January 29, 2015
Accepted: July 09, 2015

Abstract

Electronic medical records (EMRs) offer the opportunity to streamline the search for patients with possible delirium. The purpose of the current study was to identify words and phrases commonly noted in charts of patients with delirium. The current study included 67 patients (nested within a cohort study of 300 patients) ages 70 and older undergoing major elective surgery with evidence of confusion in their medical charts. Eight keywords or phrases had positive predictive values of 60% to 100% for delirium. Keywords were charted more often in nursing notes than physician notes. A brief list of keywords may serve as a building block for a methodology to screen for possible delirium from charts, with particular attention to nursing notes, for research and real-time clinical decision making. [Journal of Gerontological Nursing, 41(8), 34–42.]

Ms. Puelle is Research Associate, Mr. Kosar is Programmer Analyst, Dr. Xu is Research Associate, Dr. Schmitt is Assistant Scientist and Associate Director, Dr. Jones is Adjunct Senior Scientist, Dr. Inouye is Director, and Dr. Saczynski is Adjunct Scientist, Aging Brain Center, Institute for Aging Research, Hebrew SeniorLife; and Dr. Cooper is Assistant Professor of Surgery, Department of Surgery, Brigham and Women’s Hospital, Boston, Massachusetts. Dr. Jones is also Associate Professor, Departments of Psychiatry and Human Behavior, and Neurology, Warren Alpert Medical School, Brown University Medical School, Providence, Rhode Island. Dr. Marcantonio is Professor of Medicine, and Dr. Inouye is also Professor of Medicine, Harvard Medical School; Dr. Marcantonio is also Section Chief for Research, Division of General Medicine and Primary Care, Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts. Dr. Saczynski is also Associate Professor, Department of Medicine and Meyers Primary Care Institute, University of Massachusetts Medical School, Worcester, Massachusetts.

The authors have disclosed no potential conflicts of interest, financial or otherwise. This study was funded by grants (P01AG031720 [S. Inouye] and K07AG041835 [S. Inouye]) from the National Institute on Aging. Dr. Inouye holds the Milton and Shirley F. Levy Family Chair. Dr. Saczynski was supported in part by funding from the National Institute on Aging (K01AG33643) and the National Heart Lung and Blood Institute (U01HL105268). Dr. Marcantonio was supported in part by a grant (R01AG030618) and a Mid-Career Investigator Award (K24 AG035075) from the National Institute on Aging. The authors thank Bonnie Wong, PhD, for assistance with chart adjudications. This work is dedicated to the memory of Joshua Bryan Inouye Helfand. Drs. Saczynski and Inouye contributed equally to this manuscript.

*: Indicates multiple different endings, such as -um and -ous.*: Indicates multiple different endings, such as -ful and -ing.

Address correspondence to Jane S. Saczynski, PhD, Division of Geriatric Medicine, Department of Medicine, 377 Plantation St., Suite 315, Worcester, MA 01605; e-mail: Jane.saczynski@umassmed.edu.

Received: January 29, 2015
Accepted: July 09, 2015

Delirium, an acute change in attention and cognition, is common among hospitalized older adults, occurring in 29% to 64% of inpatients and up to 70% of long-term care residents (Inouye, Westendorp, & Saczynski, 2014; McCusker et al., 2011a; Voyer, Richard, Doucet, & Carmichael, 2009). Development of delirium is associated with poor outcomes during the course of hospitalization (e.g., higher complication risk), in post-acute care settings, and long-term follow up (e.g., prolonged cognitive and functional impairment) (Inouye et al., 2014; McCusker et al., 2011b; Rudolph et al., 2010; Saczynski et al., 2012). Despite the frequency of delirium and its negative consequences, a high rate of underrecognition and lack of documentation exists, with fewer than 3% of cases of delirium noted by International Classification of Diseases-9 (ICD-9) codes in patients’ charts (Inouye et al., 2005). One reason for the underrecognition of delirium is that the most widely used assessment tool for delirium, the Confusion Assessment Method (Inouye et al., 1990), requires an in-person, bedside interview with the patient, which can be time-consuming and expensive. In addition, fluctuation is a defining characteristic of delirium and interview-based methods may miss delirium that was present outside the limited duration of bedside interviews. A chart-review method that does not require direct patient interaction and covers 24 hours exists and has been validated, but requires hand searches of all records (e.g., nursing and physician notes, discharge summaries) and relies on clinical judgment.

With the passing of the Health Information Technology for Economic and Clinical Health (HITECH) Act in 2009, there has been increased emphasis on use of electronic medical records (EMRs) (Blumenthal, 2010). Under the HITECH Act, there is emphasis on using EMRs in ways that are meaningful to clinicians (e.g., during a hospital stay) and researchers (e.g., to facilitate use of large administrative datasets) in real time (Blumenthal, 2010). An advantage of EMRs is that they are searchable for keywords associated with a diagnosis or event. Through automated searches, such as natural language processing, it is possible to search the large amount of the EMR considered unstructured clinical documentation for keywords and identify patients with a high probability of having a condition. This automated search approach has been successfully applied in identification of postoperative complications and chronic conditions (Murff et al., 2011; Wu et al., 2013). However, the identification of keywords is a necessary first step in the effective and systematic use of EMRs for clinical and research purposes. Automated approaches to identification of delirium would enhance in-hospital care of older patients and may also help facilitate transitions to post-acute and long-term care settings.

Although algorithms for chart identification of delirium exist (Inouye et al., 2005; Saczynski et al., in press) and phrases from charts describing mental status change have been identified (Steis & Fick, 2012), there is no list of keywords with which to search EMRs for identification of delirium. Foundational work of keyword identification will allow for the development of algorithms that combine words associated with delirium for use in searching large databases. This approach would be useful for research and clinical applications, such as searching EMRs in real time to enhance early identification of potentially delirious patients.

The purpose of the current study was to identify words and phrases commonly recorded in the charts of patients with possible delirium and report their predictive value for the diagnosis of delirium.

Method

Setting and Patients

The current study was conducted with the first 300 patients enrolled in a prospective observational study of older patients scheduled for major surgery, the Successful Aging after Elective Surgery (SAGES) study, described in detail previously (Schmitt et al., 2012). Potential participants were age 70 or older; scheduled for major orthopedic, vascular, or general surgery without delirium or dementia; and had a Modified Mini-Mental State Examination score of >68 or its education-adjusted equivalent at the baseline, pre-surgery interview (Table 1) (Teng & Chui, 1987). Written informed consent was obtained from all patients.

Sample Characteristics

Table 1:

Sample Characteristics

The current sample (N = 67) is a nested cohort of patients (drawn from the larger sample of 300 patients) who had evidence of confusion in their medical records. Abstraction of the entire medical record was performed within 2 weeks of hospital discharge. Evidence of confusion was based on the presence of one or more of nine trigger words (Table 2). A consort diagram is presented in (available in the online version of this article).

Trigger Words for Delirium Abstraction

Table 2:

Trigger Words for Delirium Abstraction

Chart-Based Delirium Instrument

The chart-based delirium instrument was developed to maximize sensitivity for identification of delirium and requires a chart abstraction (Inouye et al., 2005; Saczynski, in press). This method is not based on ICD-9 codes or discharge diagnoses and instead requires abstraction of clinician notes indicating confusion or related behaviors. The chart-based delirium method includes information on acute changes in mental status, time and duration of episodes, whether there was evidence of agitation associated with the episode, and whether there was reversibility or improvement of the acute confusion during hospitalization. Trained clinical chart abstractors obtained a baseline mental status rating (from which an acute change could be judged) from preoperative visits, previous discharge summaries, and outpatient visit notes.

Indicators of confusion or change in mental status during hospitalization were obtained through review of the chart, including admission and daily nursing notes, admission and daily physician progress notes, notes from specialist consultations (e.g., neurology, psychiatric, geriatrics, pain service), and the discharge summary. Abstractors were provided with trigger words or phrases that may be used to indicate delirium and prompted the rater to look into that section of the chart for details of episodes that might indicate delirium. When a potential case was identified, all notes with any indication of confusion were fully abstracted verbatim. Chart abstractors were nurses or physicians who underwent extensive training and standardization (Schmitt et al., 2012).

Reference Standard for Delirium

Delirium was diagnosed based on a previously validated chart-review method and was adjudicated independently by a geriatrician (S.K.I.) and neuropsychologist, both with extensive training and experience in delirium assessment, using a process previously reported (Saczynski, in press). Any discrepancies were resolved during consensus conferences.

Selection of Words Associated with Delirium

Abstractors were provided with trigger words that may be associated with confusion and delirium. These trigger words have been used in a previously validated chart-based approach to identifying delirium (Inouye et al., 2005) and were similar to those used in previous studies that extracted keywords for confusion from EMRs (Morandi et al., 2009; Steis & Fick, 2012). Examples of trigger words include mental status change, disoriented/re-oriented, unresponsive, and agitated. Table 2 includes a full listing of trigger words. When trigger words were identified, the full notes were transcribed verbatim. Other supporting words that could be associated with confusion were also abstracted (, available in the online version of this article). The difference between trigger and supporting words is that appearance of trigger words prompted a full review of the record, whereas appearance of supporting words did not. Together, trigger and supporting words represent keywords for identification of confusion from charts. Transcripts were entered and analyzed, and the most frequent relevant words and word combinations were selected.

Definition of Study Variables

Descriptive characteristics of the study patients were examined. Race, ethnicity, education, language of origin, and marital status were self-reported by patients at the preoperative baseline interview. Type of surgery, age, and sex were abstracted from the medical record.

Statistical Analysis

The current authors described each trigger word, number of charts (delirious and not) in which the word appeared, and who wrote the note in which the word appeared (e.g., nurse, physician). The positive predictive value (PPV) of each trigger word was calculated as the number of charts of patients diagnosed with delirium in which the word appeared divided by the total number of charts (i.e., total sample) in which the word appeared.

Results

The total sample from which the current nested cohort study was drawn comprised 300 hospitalized patients, with a mean age of 77 years. The nested cohort consisted of 63 hospitalized patients without evidence of confusion in their charts. The sample had a mean age of 77 years; one quarter was older than 80 (Table 1). Approximately one half of participants were female (57%) and 8% were non-White or Hispanic. Participants were well-educated, with three quarters having higher than a high school education. The majority (83%) of participants were scheduled for orthopedic surgery. Of the 63 charts with evidence of confusion, 35 (56%) were adjudicated as delirious. Rate of delirium differed slightly by surgery type. Seventy-two percent of patients who underwent orthopedic surgery, 86% of those who underwent vascular surgery, and 100% of those who underwent general surgery were adjudicated as delirious.

Language Representative of Delirium in Charts

Among patients who developed delirium, there were an average of 6.4 nursing notes containing keywords for delirium compared with an average of 2.8 notes from physicians, and less than 1 note, on average, from other sources (e.g., consults, discharge summaries) (data not shown).

Table 3 presents exemplar quotations from selected charts that were positive for delirium. Selected quotes represent hyperactive and hypoactive forms of delirium. In general, quotes from cases more representative of hyperactive delirium are easier to detect as abnormal or cause for concern. For example, notes from patients 1, 8, and 9 present hallucinations and paranoid and inappropriate behaviors. These notes can be interpreted and symptoms of delirium identified without much contextualization. That is, it is clear from the brief notes that the patient is experiencing an acute change in mental status and is confused.

Selected Quotations from Delirium-Positive Charts

Table 3:

Selected Quotations from Delirium-Positive Charts

In contrast, patients with behaviors more consistent with hypoactive delirium may be more difficult to identify from a single note. Notes from patients 2, 4, and 5 are examples of symptoms and behaviors that would be associated with the hypoactive delirium, such as extreme drowsiness. However, from the individual notes taken alone, it is difficult to identify delirium. That is, the notes and behaviors need to be put into context and a series of notes are needed to define the clinical course (e.g., fluctuation, reversibility) and establish the presence of delirium. These notes also highlight the inherent difficulty in identifying hypoactive, as compared to hyperactive, delirium from medical records.

Keywords for Identification of Delirium

Trigger Words. Trigger words (i.e., those prompting a full record review) found to be most useful in the identification of delirium are presented in Table 2, according to the source of the note (e.g., nurse, physician). Trigger words presented never appeared in charts that were not abstracted. Thus, a PPV was able to be calculated for these words based on the full sample of 300 patients.

In general, trigger words appeared in nursing notes more often than in physician notes, likely reflecting the higher frequency of nurses charting and also their longer duration of contact with patients. Several trigger words, although rare in abstracted charts, had high PPVs and served as clear indicators of the presence of delirium. That is, when the word appeared in the chart, the patient was highly likely to be delirious. For example, mental status appeared in 8 (13%) nursing notes and 11 (18%) physician notes and had a PPV of 100%; Deliri appeared in only nine charts and had a PPV of 90% to 100%. These are examples of words that could be used to identify high-probability delirium cases from clinical records on an ongoing basis or in real time, with little need for clinical interpretation.

Other trigger words required contextualization to determine whether symptoms of delirium were present. The appearance of the word alone did not necessarily mean that the patient was delirious and the context of the note needed to be carefully considered. For example, confus appeared in 39 (62%) nursing notes and 18 (29%) physician notes, with PPVs of 69% and 94%, respectively. Therefore, in nursing notes in particular, the context of a mention of confusion would need to be carefully considered before establishing the presence of delirium because 31% may represent false positive results.

Supporting Words. Other supporting words appeared more frequently in delirious and non-delirious charts and may serve as additional words with which to search to identify potential cases of delirium, such as in an EMR (, available in the online version of this article). Due to their relatively high frequency in the charts and broad potential for use in nurse and physician notes, the context in which these words are recorded requires careful examination for them to be useful in identification of delirium. For example, forget was included in the notes of 23 (37%) of all abstracted charts and in 14 (40%) of the notes of charts adjudicated as delirious. These words were not included in the list of trigger words and may have appeared in non-abstracted charts; thus, PPVs for these supporting words were not able to be calculated. However, these additional keywords may still be useful in flagging patients, who may represent possible delirium cases, during a hospitalization.

Discussion

Keywords associated with symptoms of delirium were identified and their prevalence and predictive values in nursing and physician notes were reported. A list of eight words associated with confusion that can be used for searching EMRs to identify symptoms of delirium is provided. PPVs of single keywords for a diagnosis of delirium ranged from 60% to 100%. Nurses used keywords associated with confusion in their notes more often than physicians, irrespective of delirium status.

Several previous studies have focused on nurse documentation of delirium (Fick, Hodo, Lawrence, & Inouye, 2007; Inouye, Foreman, Mion, Katz, & Cooney, 2001; Laurila, Pitkala, Strandberg, & Tilvis, 2004; Voyer, Cole, McCusker, St-Jacques, & Laplante, 2008) and underrecognition of delirium in nursing charts. However, few studies have compared physician and nurse charting of specific words associated with symptoms of delirium (Laurila et al., 2004; Morandi et al., 2009). In addition, the predictive value associated with specific words has not been previously calculated. Thus, the current study extends the previous literature on chart-based identification of symptoms of delirium by providing keywords associated with delirium and the prevalence and predictive value of each term. Development of a list of keywords is a crucial first step in automating the identification of patients at risk for delirium who may require further screening and evaluation.

Results showed that nurses charted more often than physicians. Nurses also used keywords associated with symptoms of delirium more often than physicians. These findings are similar to a study comparing nurse and physician charting of keywords for delirium in patients in post-acute care settings, where nurses recorded words associated with delirium 30% more often than physicians (Morandi et al., 2009), suggesting that it is important to review physician and nursing notes for symptoms of delirium and that using only physician notes would result in missing many cases.

Using keywords from a validated method for chart identification for delirium (Inouye et al., 2005; Saczynski et al., in press), eight words or phrases with high predictive validity for delirium were identified. Although chart-based delirium identification methods are available, they can be time-consuming and require the entire chart; thus, they can only be performed retrospectively. Increased use of EMRs offers the unprecedented opportunity for identifying patients at high risk for delirium in real time to inform clinical decision making, such as to guide further testing and treatment. A pilot study in the post-acute care setting exemplifies how using a keyword search could inform real-time clinical decision making (Morandi et al., 2009). Charts were retrospectively searched for keywords associated with delirium, and the discipline (e.g., nurse versus physician) charting each keyword was noted. Similar to the current findings, nurses were more likely to chart keywords for delirium and, importantly, when presence of the symptom was communicated to physicians, further evaluation or treatment (pharmacological and nonpharmacological) occurred in 70% of patients (Morandi et al., 2009). In contrast, no action (i.e., evaluation or treatment) was noted in the charts of patients in whom presence of symptoms of delirium was not communicated to physicians, suggesting that making physicians aware of keywords in the charts may facilitate early identification and treatment of delirium.

Several high-yield, low-frequency words or phrases that could be the building blocks for algorithms that could be used to identify delirium in large or administrative datasets were also identified. For example, altered mental status and delirium were noted in less than one quarter of the charts, but had a PPV of 100%. More frequently cited words may require additional clinical judgment and thus may be more useful to flag high-risk patients in real time, but would be less useful for large and administrative dataset use. For example, confusion and reorient were mentioned in 77% and 57% of nursing notes in the study and had PPVs between 65% and 69%, respectively. Although these keywords would be red flags in a patient chart and may highlight the need for closer monitoring or further testing, they would require additional contextualization and clinical evaluation to confidently identify delirium.

The current findings are strengthened by the rigorous chart-review method applied and adjudication of delirium diagnosis by two independent experts using consensus conferences. Detailed data on confusion were collected verbatim from notes in the medical record, allowing for identification of specific words and phrases for analysis.

Limitations

Several limitations of the current study must be considered. Only charts with evidence of confusion were fully abstracted; therefore, the supporting words and phrases may have appeared in the charts without evidence of confusion (e.g., charts not fully abstracted) and so PPVs could not be calculated. In addition, natural language processing approaches were not used because the charts were not fully electronic. The current study also included patients who were cognitively intact prior to surgery, primarily Caucasian, and healthy enough to undergo elective surgery from two hospitals in a single geographical area, limiting the potential generalizability of the findings.

Conclusion

The brief listing of identified keywords might serve as a starting point for building a methodology or program for detecting possible delirium from EMRs and administrative databases for research and real-time clinical decision making. With this list of words, future work to expand and refine these listings and to develop algorithms that combine keywords and phrases associated with possible delirium, with particular attention to nursing charts, can help increase sensitivity and specificity of the search. Eventually, EMR systems may automatically search for delirium keywords and flag patients with possible delirium, particularly for whom a bedside evaluation may be important and informative. Such work would carry important implications to improve identification of delirium in clinical practice and facilitate the transition of older patients to long-term care settings.

References

  • Blumenthal, D. (2010). Launching HITECH. New England Journal of Medicine, 362, 382–385. doi:10.1056/NEJMp0912825 [CrossRef]
  • Fick, D.M., Hodo, D.M., Lawrence, F. & Inouye, S.K. (2007). Recognizing delirium superimposed on dementia: Assessing nurses’ knowledge using case vignettes. Journal of Gerontological Nursing, 33(2), 40–47.
  • Inouye, S.K., Foreman, M.D., Mion, L.C., Katz, K.H. & Cooney, L.M. Jr.. (2001). Nurses’ recognition of delirium and its symptoms: Comparison of nurse and researcher ratings. Archives of Internal Medicine, 161, 2467–2473. doi:10.1001/archinte.161.20.2467 [CrossRef]
  • Inouye, S.K., Leo-Summers, L., Zhang, Y., Bogardus, S.T. Jr.. , Leslie, D.L. & Agostini, J.V. (2005). A chart-based method for identification of delirium: Validation compared with interviewer ratings using the confusion assessment method. Journal of the American Geriatrics Society, 53, 312–318. doi:10.1111/j.1532-5415.2005.53120.x [CrossRef]
  • Inouye, S.K., van Dyck, C.H., Alessi, C.A., Balkin, S., Siegal, A.P. & Horwitz, R.I. (1990). Clarifying confusion: The confusion assessment method. A new method for detection of delirium. Annals of Internal Medicine, 113, 941–948. doi:10.7326/0003-4819-113-12-941 [CrossRef]
  • Inouye, S.K., Westendorp, R.G.J. & Saczynski, J.S. (2014). Delirium in elderly people. The Lancet, 383, 911–922. doi:10.1016/S0140-6736(13)60688-1 [CrossRef]
  • Laurila, J.V., Pitkala, K.H., Strandberg, T.E. & Tilvis, R.S. (2004). Detection and documentation of dementia and delirium in acute geriatric wards. General Hospital Psychiatry, 26, 31–35. doi:10.1016/j.genhosppsych.2003.08.003 [CrossRef]
  • McCusker, J., Cole, M.G., Voyer, P., Monette, J., Champoux, N., Ciampi, A. & Belzile, E. (2011a). Prevalence and incidence of delirium in long-term care. International Journal of Geriatric Psychiatry, 26, 1152–1161. doi:10.1002/gps.2654 [CrossRef]
  • McCusker, J., Cole, M.G., Voyer, P., Monette, J., Champoux, N., Ciampi, A. & Belzile, E. (2011b). Use of nurse-observed symptoms of delirium in long-term care: Effects on prevalence and outcomes of delirium. International Psychogeriatrics, 23, 602–608. doi:10.1017/S1041610210001900 [CrossRef]
  • Morandi, A., Solberg, L.M., Habermann, R., Cleeton, P., Peterson, E., Ely, E.W. & Schnelle, J. (2009). Documentation and management of words associated with delirium among elderly patients in post-acute care: A pilot investigation. Journal of the American Medical Directors Association, 10, 330–334. doi:10.1016/j.jamda.2009.02.002 [CrossRef]
  • Murff, H.J., Fitzhenry, F., Matheny, M.E., Gentry, N., Kotter, K.L. & Crimin, K. (2011). Automated identification of postoperative complications within an electronic medical record using natural language processing. Journal of the American Medical Association, 306, 848–855. doi:10.1001/jama.2011.1204 [CrossRef]
  • Rudolph, J.L., Inouye, S.K., Jones, R.N., Yang, F.M., Fong, T.G., Levkoff, S.E. & Marcantonio, E.R. (2010). Delirium: An independent predictor of functional decline after cardiac surgery. Journal of the American Geriatrics Society, 58, 643–649. doi:10.1111/j.1532-5415.2010.02762.x [CrossRef]
  • Saczynski, J., Kosar, C.M., Xu, G., Puelle, M.R., Schmitt, E., Jones, R.N. & Inouye, SK. (in press). A tale of two methods: Chart and interview methods for identifying delirium. Journal of the American Geriatrics Society.
  • Saczynski, J.S., Marcantonio, E.R., Quach, L., Fong, T.G., Gross, A., Inouye, S.K. & Jones, R.N. (2012). Cognitive trajectories after postoperative delirium. New England Journal of Medicine, 367, 30–39. doi:10.1056/NEJMoa1112923 [CrossRef]
  • Schmitt, E.M., Marcantonio, E.R., Alsop, D.C., Jones, R.N., Rogers, S.O. Jr.. , Fong, T.G. & Inouye, S.K. (2012). Novel risk markers and long-term outcomes of delirium: The Successful Aging after Elective Surgery (SAGES) Study design and methods. Journal of the American Medical Directors Association, 13, 818.e1–818.e10. doi:10.1016/j.jamda.2012.08.004 [CrossRef]
  • Steis, M.R. & Fick, D.M. (2012). Delirium superimposed on dementia: Accuracy of nurse documentation. Journal of Gerontological Nursing, 38(1), 32–42. doi:10.3928/00989134-20110706-01 [CrossRef]
  • Teng, E.L. & Chui, H.C. (1987). The Modified Mini-Mental State (3MS) examination. Journal of Clinical Psychiatry, 48, 314–318.
  • Voyer, P., Cole, M.G., McCusker, J., St-Jacques, S. & Laplante, J. (2008). Accuracy of nurse documentation of delirium symptoms in medical charts. International Journal of Nursing Practice, 14, 165–177. doi:10.1111/j.1440-172X.2008.00681.x [CrossRef]
  • Voyer, P., Richard, S., Doucet, L. & Carmichael, P.-H. (2009). Detecting delirium and subsyndromal delirium using different diagnostic criteria among demented long-term care residents. Journal of the American Medical Directors Association, 10, 181–188. doi:10.1016/j.jamda.2008.09.006 [CrossRef]
  • Wu, S.T., Sohn, S., Ravikumar, K.E., Wagholikar, K., Jonnalagadda, S.R., Liu, H. & Juhn, Y.J. (2013). Automated chart review for asthma cohort identification using natural language processing: An exploratory study. Annals of Allergy, Asthma & Immunology, 111, 364–369. doi:10.1016/j.anai.2013.07.022 [CrossRef]
Appendix A. Consort diagram


Sample Characteristics

Variable Total Sample (N = 300), n (%) Nested Cohort (n = 63), n (%)a
Age at index surgery, mean (SD) 76.9 (5) 77.4 (6)
Age older than 80 81 (27) 16 (25)
Female 166 (55) 36 (57)
Non-White or Hispanic 21 (7) 5 (8)
English as a second language 20 (7) 2 (3)
Currently married/living with partner 185 (62) 34 (54)
Education
  0 to 12 years 86 (29) 16 (25)
  13 to 16 years 129 (43) 28 (44)
  >17 years 85 (28) 19 (30)
Surgery type
  Orthopedic 253 (84) 52 (83)
  Gastrointestinal 31 (11) 4 (6)
  Vascular 16 (5) 7 (11)

Trigger Words for Delirium Abstraction

Worda Source Total Charts with Word (N = 63) Delirious Charts with Word (n = 35) Non-Delirious Charts with Word (n = 28) PPV
n (%)
AMS or mental status Nurse 8 (13) 8 (23) 0 (0) 100
Physician 11 (18) 11 (31) 0 (0) 100
Other 1 (2) 0 (0) 1 (4) 0
Deliri* Nurse 9 (14) 9 (26) 0 (0) 100
Physician 10 (16) 9 (26) 1 (4) 90
Other 2 (3) 2 (6) 0 (0) 100
Alert and Oriented (<3) Nurse 17 (27) 14 (40) 3 (11) 82
Physician 2 (3) 2 (6) 0 (0) 100
Other 0 (0) 0 (0) 0 (0) 0
Hallucin* Nurse 5 (8) 4 (11) 1 (4) 80
Physician 1 (2) 1 (3) 0 (0) 100
Other 0 (0) 0 (0) 0 (0) 0
Confus* Nurse 39 (62) 27 (77) 12 (43) 69
Physician 18 (29) 17 (49) 1 (4) 94
Other 3 (5) 0 (0) 3 (11) 0
Reorient* Nurse 31 (49) 20 (57) 11 (39) 65
Physician 1 (2) 1 (3) 0 (0) 100
Other 0 (0) 0 (0) 0 (0) 0
Disorient* Nurse 15 (24) 9 (26) 6 (21) 60
Physician 3 (5) 3 (9) 0 (0) 100
Other 1 (2) 1 (3) 0 (0) 100
Encephalopathyb Nurse 0 (0) 0 (0) 0 (0) 0
Physician 3 (5) 3 (9) 0 (0) 100
Other 1 (2) 1 (3) 0 (0) 100
Any word Nurse 51 (81) 32 (91) 19 (68) 63
Physician 24 (38) 22 (63) 2 (7) 92
Other 7 (11) 3 (9) 4 (14) 43
No word Nurse 12 (19) 3 (9) 9 (32) 25
Physician 39 (62) 13 (37) 26 (93) 35
Other 56 (89) 32 (91) 24 (86) 57

Selected Quotations from Delirium-Positive Charts

Patient Quotea Note Source
1 Lethargic, disoriented at times to place/time. Patient removed IV access while confused. Nurse
A+Ox3 @ start of shift, although with frequent period of confusion. Patient bed alarmed for sedative and fall precaution. Nurse
A+Ox3. When asked questions, answered appropriately. Confused at times. Thought there was a shopping mall in the hospital. Nurse
2 Around 3 p.m., patient was found to be unresponsive, disoriented. Unable to answer questions or follow commands. Nurse
Per reports, patient was at his baseline mental status this morning, but became more confused over the course of the day. At that time, patient was unable to keep his eyes open for more than 10 seconds at a time. Altered mental status. ICU physician admission note
3 Mental status has been waxing and waning today. She is confused again, speaking nonsense, states she does not remember anything. Neuro: A+Ox3, answering questions appropriately, follows some commands. Attending physician
4 No major changes. Patient continues to be somnolent, easily arousable, but then falls back asleep. Able to state name, date, hospital, and reason for her admission. Nurse
A+Ox3 in the morning, but by the evening, confused and disoriented. The team is aware. At 2 p.m., patient is in bed and resting, very somnolent, and tachy is in the 140s. Not A+Ox3, but garbled speech. Nurse
5 Very somnolent and confused this morning. Unable to assess due to mental status. Unable to maintain conversation, drifts to sleep. Acute pain service physician note
Narcotic medication held to see if this helps confusion. Confusion clearing up, but not 100%. Unable to tell the date/year. Nurse
6 Patient initially impulsive, but mostly clear mentation early. Rapidly became profoundly delirious despite Seroquel®, Haldol®, and Ativan®, with limited effect. Required vigilant observation. Bed alarm and frequent intervention. Nurse
This morning, patient oriented to self/time plus hallucinations. Speech garbled. Patient confused but cooperative. Increasing clarity and wakefulness, but appropriate. A+Ox3 this afternoon and after 4 p.m. Nurse
7 Patient medicated with Ambien® 2.5 mg by mouth for sleep, but awake at 5 a.m. and is confused. A+Ox1, urinated on floor, assisted to bed, alarm on for safety. Reoriented to surroundings. Nurse
8 Event: confusion. Increasing confusion. At 9:30 p.m., she became more confused, yelling out, paranoid. Attempted to dial 911, telling nurse she is “scared, this isn’t a safe place.” Unable to reorient. Nurse
9 Patient went to sleep 12:15 a.m. to approximately 3 a.m. Found sitting up on the bedside chair without wearing any clothes. Stated: “I don’t know how I got here.” Assisted back to bed with maximal assist of two. Nurse
10 Acute change in conscious state—HCT 26.6 (decreasing 32.9). Into patient’s room @ 6:30 to give Tylenol® by mouth. Confused, unaware of time/place. Acute change in mental status—overnight when woken A+Ox3. Does not remember surgeon, name of surgery, etc. Able to tell DOB and spell last name. No BUE weakness, speech clear, but kept saying “I don’t know, I’m confused.” Nurse

Keypoints

Puelle, M.R., Kosar, C.M., Xu, G., Schmitt, E., Jones, R.N., Marcantonio, E.R., Cooper, Z., Inouye, S.K. & Saczynski, J.S. (2015). The Language of Delirium: Keywords for Identifying Delirium from Medical Records. Journal of Gerontological Nursing, 41(8), 34–42.

  1. Keywords for delirium can be abstracted from medical records.

  2. Keywords have high (i.e., 60% to 100%) predictive value for delirium.

  3. Keywords may serve as building blocks for a methodology to screen for delirium from charts.

10.3928/00989134-20150723-01

Sign up to receive

Journal E-contents