Research in Gerontological Nursing

Empirical Research 

Rapid Cognitive Assessment of Nursing Home Residents: A Comparison of the Brief Interview for Mental Status (BIMS) and Brief Cognitive Assessment Tool–Short Form (BCAT-SF)

Ryan A. Mace, BS; William E. Mansbach, PhD; Kristen M. Clark, BS

Abstract

In nursing homes, the ability to identify residents with cognitive impairment is critical; however, many providers fail to address symptoms of dementia due to insufficient time to assess cognition. In the current study, the authors compared two rapidly administered instruments, the Brief Interview for Mental Status (BIMS) and Brief Cognitive Assessment Tool–Short Form (BCAT-SF), in predicting specific cognitive stages. Two hundred twenty-five nursing home residents who were referred for neurocognitive evaluation and met inclusion criteria participated in the study. Both cognitive instruments were found to predict cognitive diagnoses generally, but only the BCAT-SF demonstrated sensitivity in differentiating among all cognitive levels. Receiver operating characteristic analyses did not confirm cutoff scores reported in the original BIMS study. However, sensitivity (0.91) and specificity (0.96) values for the BCAT-SF were robust for differentiating between dementia and non-dementia. The BCAT-SF appears to be a more sensitive instrument for predicting cognitive stages than the BIMS for nursing home residents.

[Res Gerontol Nurs. 2016; 9(1):35–44.]

Abstract

In nursing homes, the ability to identify residents with cognitive impairment is critical; however, many providers fail to address symptoms of dementia due to insufficient time to assess cognition. In the current study, the authors compared two rapidly administered instruments, the Brief Interview for Mental Status (BIMS) and Brief Cognitive Assessment Tool–Short Form (BCAT-SF), in predicting specific cognitive stages. Two hundred twenty-five nursing home residents who were referred for neurocognitive evaluation and met inclusion criteria participated in the study. Both cognitive instruments were found to predict cognitive diagnoses generally, but only the BCAT-SF demonstrated sensitivity in differentiating among all cognitive levels. Receiver operating characteristic analyses did not confirm cutoff scores reported in the original BIMS study. However, sensitivity (0.91) and specificity (0.96) values for the BCAT-SF were robust for differentiating between dementia and non-dementia. The BCAT-SF appears to be a more sensitive instrument for predicting cognitive stages than the BIMS for nursing home residents.

[Res Gerontol Nurs. 2016; 9(1):35–44.]

Cognitive impairment is prevalent in American nursing homes, as approximately 70% of residents have some type of cognitive impairment, much of it severe (Centers for Medicare & Medicaid Services [CMS], 2013). Studies estimate that approximately 50% of all residents meet criteria for dementia (Magaziner et al., 2000). The ability to identify residents with cognitive impairment is critical for resident-centered care to be effective and for staff to meet residents' needs (Singer & Luxenberg, 2003). Previous research has documented that inaccurate understanding of cognitive functioning can adversely affect management of medical conditions and resident life (Cohen-Mansfield, 2005; Cohen-Mansfield & Creedon, 2002). Since the 1980s, when the U.S. Congress legislated sweeping changes in the nursing home industry through the Omnibus Budget Reconciliation Act of 1987, a key provision and guiding principle has been the mandatory use of a standardized, comprehensive system to assist in assessment and care planning (Hawes et al., 1997). Recognizing the high prevalence and complex care needs of residents with dementia, this Minimum Data Set (MDS) has included cognitive assessments and has since been an evolving process.

Despite the potential use of screening older adults who present with a subjective or objective memory complaint, many providers fail to address symptoms of dementia due to insufficient time to assess the cognition of their patients (Boise, Camicioli, Morgan, Rose, & Congleton, 1999; Østbye et al., 2005). For nursing home providers to use cognitive screening tools, they must be brief and easily administered in addition to being psychometrically robust (Mansbach & MacDougall, 2012). When MDS 3.0 (the most recent iteration) was introduced, the Brief Interview for Mental Status (BIMS; Chodosh et al., 2008) was selected as the mandated cognitive assessment tool. Some of the strengths of the BIMS are that it is based on an interview format, has acceptable reliability and convergent validity, can be rapidly administered (average time is approximately 3 minutes), and appears to be more accurate than observational methods of assessment (Chodosh et al., 2008).

However, the BIMS has some shortcomings that may limit its use as a nursing home cognitive measure. First, it is not designed to be sensitive to the full cognitive continuum. Second, the cutoff scores differentiating those residents with and without cognitive impairments are based, in part, on the Modified Mini-Mental State Examination (3MS; Teng & Chui, 1987), a measure that may not be sensitive to mild cognitive impairment (MCI) (McDowell, Kristjansson, Hill, & Hébert, 1997). Third, memory is assessed using a simple word list (i.e., three words), with no story recall component. There is empirical support for using story recall items in cognitive screening instruments, especially for identifying patients with MCI (Rabin et al., 2009). Story recall has syntactic and other contextual elements largely absent from word lists that may differentially affect recall (Foldi, 2011; Waters & Caplan, 2005) and learning (Kintsch, 1994). Fourth, despite the fact that deficits in executive functions are common features of dementia, the BIMS used in MDS 3.0 does not assess executive capacities. Fifth, the BIMS was normed on a Veterans Administration population in which approximately 95% of the sample were men. Most nursing homes have more women than men, and it is unclear if women would have different normative values.

For the current study, the authors selected the Brief Cognitive Assessment Tool–Short Form (BCAT-SF; Mansbach & MacDougall, 2012), as it appears to address many of these limitations. The BCAT-SF is an abridged instrument of the original Brief Cognitive Assessment Tool (BCAT; Mansbach, MacDougall, & Rosenzweig, 2012) and was designed to be used in situations in which time is limited (Mansbach & MacDougall, 2012). The BCAT-SF can be completed in 5 minutes or less. It contains six items representing the cognitive domains of contextual memory, executive functions, and attentional capacity. The BCATSF preserves the psychometric rigor of the full BCAT in predicting cognitive diagnosis (i.e., dementia versus non-dementia) and functional status (i.e., basic activities of daily living [ADLs] and instrumental ADLs [IADLs]) (Mansbach & MacDougall, 2012).

The primary purpose of the current study was to compare the utility of the BCAT-SF and BIMS in predicting cognitive diagnoses in nursing home patients. An additional goal was to cross-validate the BCAT-SF and BIMS in a nursing home population. In so doing, the authors attempted to determine which brief instrument is more sensitive in identifying levels of cognitive functioning.

Method

Participants

Three hundred seven individuals residing in a Maryland skilled nursing facility were referred for neurocognitive evaluation (October 2012 to April 2014). Approximately all patients were residents receiving short-term subacute rehabilitation following hospitalization. Two hundred twenty-five individuals met inclusion criteria for retrospective data analysis, which required a completed BCAT-SF and BIMS, proficiency in English, and age older than 59. Individuals with medical or psychiatric symptoms that would prevent completion of the tests (e.g., global aphasia, disabling depression) were excluded.

Participants ranged in age from 61 to 102, with a mean age of 80.44 (SD = 8.98 years). As shown in Table 1, 61% of participants were female, 72% were Caucasian, 49% were widowed, and 78% had ≥12 years of education. Of the sample, 56% met diagnostic criteria for dementia (i.e., mild, moderate, or severe), 28% met diagnostic criteria for MCI, and 10% did not have a dementia diagnosis. Using a Geriatric Depression Scale Short Form (GDS-SF) cutoff score of >5 for self-reported depression (Sheikh & Yesavage, 1986), 28% of the sample showed evidence of depression, whereas 72% were categorized as not having depression.

Patient Demographics (N = 225)

Table 1:

Patient Demographics (N = 225)

Procedure

The nursing home's Medical Ethics Committee approved all procedures. All study participants were referred for neurocognitive evaluation by nursing home attending physicians. A licensed psychologist with specific expertise in the evaluation of cognitive dysfunction conducted all evaluations. Evaluations consisted of: (a) a clinical interview with the patient; (b) comprehensive review of the medical chart, inclusive of hospital discharge summaries (if appropriate); (c) administration of the AD8 Dementia Screening Interview (AD8; Galvin et al., 2005), Kitchen Picture Test (KPT; Mansbach, MacDougall, Clark, & Mace, 2014), and GDS-SF; (d) staff interviews; and (e) interviews with family members. The AD8 and KPT were used in the clinical evaluation to help determine cognitive diagnosis and confirm convergent validity for the BCAT-SF and BIMS. The GDS was used to evaluate mood functioning as part of the clinical evaluation and confirm discriminate validity with respect to the BCAT-SF and BIMS.

Diagnoses were made by the licensed psychologist using the Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2000) criteria to diagnose dementia, with Clinical Dementia Rating Scale (CDR; Morris, 1993) criteria for the staging of dementia as mild, moderate, or severe. The CDR 0.5 and 1.0 were combined, as they are both mild impairment categories, to better capture those participants who meet criteria for MCI. The diagnosis of MCI was further confirmed based on the Petersen et al. (1999) criteria. Cognitive diagnoses were made without knowledge of BIMS and BCAT-SF scores. Four social workers trained in administering the BIMS independently administered this screening tool and two trained mental health technicians administered the BCAT-SF. Test order was determined by assigning participants to one of two protocols. In Protocol A, the order of testing was: BCAT-SF, AD8, KPT, GDS-SF, and BIMS. In Protocol B, the order of testing was: BIMS, BCAT-SF, KPT, AD8, and GDS-SF. Those who administered the BIMS had no knowledge of the BCAT-SF scores. Those who administered the BCAT-SF had no knowledge of BIMS scores. Finally, the licensed psychologist making the clinical diagnoses had knowledge of the AD8, KPT, and GDS-SF scores to help make cognitive and mood diagnoses, but did not have knowledge of the BIMS and BCAT-SF test results.

Measures

Brief Interview for Mental Status. The 7-item BIMS (Chodosh at al., 2008) is an integral part of MDS 3.0 mandated for American nursing homes (Saliba et al., 2012). The BIMS focuses on two domains of cognitive function, memory and orientation, with scores ranging from 0 to 15. In the CMS RAI Version 3.0 Manual (CMS, 2010, p. 14), the following BIMS score ranges are recommended for assessing cognition: 13 to 15 = cognitively intact, 8 to 12 = moderately impaired, and 0 to 7 = severe impairment. Verbal memory on the BIMS is assessed using immediate and delayed free recall (i.e., 1- to 2-minute delay) of an orally presented word list (i.e., three words). The development study (N = 374) established interrater reliability between research assistant and facility nurse administrations of the BIMS within 72 hours of each other (r = 0.72) and found a correlation (r = 0.79, p < 0.01) with the 3MS (Teng & Chui, 1987). Chodosh et al. (2008) selected BIMS cutoff scores, based on benchmarks from the 100-point 3MS, for identifying severe impairment or probable dementia (3MS <48). A cutoff score of 10 (3MS <48) had a sensitivity of 0.94 and specificity of 0.79 for identifying severe impairment (Chodosh et al., 2008).

Brief Cognitive Assessment Tool–Short Form. The BCAT-SF (Mansbach & MacDougall, 2012) was designed as a shorter version of the full BCAT to be used when time for cognitive screening is constrained (e.g., primary care settings). The BCAT-SF can be easily administered in less than 5 minutes, with scores ranging from 0 to 21. To maintain content validity, Mansbach and MacDougall (2012) selected six items from the original BCAT that represent the cognitive domains of contextual memory, executive functioning, and attentional capacity. One hundred four older adults from assisted living facilities (87.5%, n = 91) and the independent section within continuing care retirement communities (12.5%, n = 13) participated in the development and validation of the BCAT-SF. Psychometric analyses yielded an internal consistency reliability of 0.86 (Cronbach's alpha based on standardized items), a test– retest reliability of r = 0.98, and preliminary evidence of the construct validity of BCAT-SF score inferences through convergent, discriminant, and predictive validity analyses. Using a cutoff score of 16, the BCAT-SF differentiated between MCI and dementia with a sensitivity of 0.90 and a specificity of 0.81 (area under the curve = 0.93) (Mansbach & MacDougall, 2012). Similar to the BCAT, the BCAT-SF has been shown to predict both ADL and IADL functioning, as measured by the Lawton and Brody (1969) Physical Self-Maintenance Scale and Lawton IADL scale.

Convergent Validity

AD8 Dementia Screening Interview. The AD8 (Galvin et al., 2005) was designed to be a brief informant-based measure to differentiate individuals with and without dementia and is sensitive in detecting early signs of dementia. The 8-item screening interview was designed to assess memory, orientation, judgment, and function. Scores range from 0 to 8, with scores ≥2 predicting a likelihood of dementia in a sample of older adults with a 38% base rate of very mild dementia (i.e., CDR = 0.5). Galvin et al. (2006) investigated the psychometric properties of the scale through 225 patient–informant dyads. Evidence for interrater reliability through correspondence between independent raters (intraclass correlation coefficient = 0.82; 95% confidence interval [CI] [0.5 to 0.92]), intrarater reliability through the agreement of repeated administrations by a single rater (weighted kappa = 0.67; 95% CI [0.59 to 0.75]), and construct validity through correlations of individual AD8 items and other neuropsychological tests were found (Galvin et al., 2006). In the absence of an informant, Galvin, Roe, Coats, and Morris (2007) demonstrated that a self-reported AD8 can differentiate between individuals with and without dementia, albeit with greater use in individuals with mild impairment.

Kitchen Picture Test (KPT). The KPT (Mansbach, MacDougall, et al., 2014) was designed as a visually presented test of practical judgment. The KPT is an illustration of a kitchen scene in which three potentially dangerous situations are unfolding. Participants are asked to identify, rank the severity of, and provide solutions that would resolve these problems. Scores range from 0 to 8. In two independent studies (Study 1, N = 99 nursing home patients; Study 2, N = 163 nursing home and assisted living patients), psychometric analyses indicated strong evidence for reliability, construct validity, and predictive validity (Mansbach, MacDougall, et al., 2014). In addition to detecting problems in basic judgment, the KPT can significantly differentiate between individuals with and without dementia, with a suggested cutoff score of 7 (Mansbach, MacDougall, et al., 2014).

Divergent Validity

Geriatric Depression Scale Short Form. The 15-item GDS-SF (Sheikh & Yesavage, 1986) is the most commonly used scale to screen for depression in older adults. Scores range from 0 to 15, with higher scores indicating a greater likelihood of depression. Researchers have reported validity data for the GDS-SF from a variety of settings, including a nursing home setting (Gerety et al., 1994). de Craen, Heeren, and Gussekloo (2003) reported adequate sensitivity and specificity in adults older than 85. However, de Craen et al. (2003) also stated that the GDS-SF is most useful for older adults with minimal to no cognitive impairment. A reliability generalization study reported average GDS reliability (including all short forms) to be 0.85 (Kieffer & Reese, 2002). Chiang, Green, and Cox's (2009) Rasch analysis from 177 older adult participants in a variety of residential locations (from retirement communities to skilled nursing facilities) found the GDS-SF to assess a unidimensional construct.

Statistical Analysis

All statistical analyses were performed using SPSS version 21.0. Descriptive statistics were used to report participant demographics, assessment tool scores, and clinical diagnoses. Internal consistency was estimated by Cronbach's alpha reliability coefficient. Convergent and divergent validity were addressed with Spearman correlation coefficients. A one-way multivariate analysis of variance (MANOVA) and the Games–Howell post hoc test were selected to investigate mean differences of BCAT-SF and BIMS scores between diagnostic/CDR categories (i.e., normal, MCI, mild dementia, moderate dementia, and severe dementia). A hierarchical multiple regression analysis was used to estimate the amount of variance in cognitive diagnoses explained by the BCATSF and BIMS scores. The predictive utility of the BIMS and BCAT-SF was also addressed with analyses of sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Cutoff scores for identifying participants with dementia were selected that yielded the optimal balance between sensitivity and specificity, while emphasizing a higher sensitivity. Receiver operator characteristic (ROC) curves and area under the ROC were calculated from sensitivity and 1-specificity values for differentiating diagnoses of MCI and dementia.

Results

In the current sample, the distribution of BIMS (range = 2 to 15) and BCAT-SF scores (range = 0 to 21) were negatively skewed. Mean BIMS and BCAT-SF scores were not significantly different for marital status, race, or sex. However, participants with less than 12 years of education scored lower on the BCAT-SF (median = 13, n = 41) than those with 12 or more years of education (median = 15, n = 176), U = 2799.0, z = −2.59, p = 0.01. The BIMS (rs = −0.27, p < 0.001) and BCAT-SF (rs = −0.38, p < 0.001) were significantly correlated with age.

Reliability

In the current study, the internal consistency reliability, as estimated by Cronbach's alpha based on standardized items, was 0.84 for the BCAT-SF and 0.77 for the BIMS. Both coefficients are within the acceptable range for tests that are used for clinical decision making (Nunnally & Bernstein, 1994).

Validity

Table 2 presents descriptive statistics used for validity analyses measured in the current study. As reported in Table 3, construct validity was demonstrated by significant correlations between the BIMS and BCAT-SF (rs = 0.59, p < 0.001). Further convergence validity was confirmed by comparing the BCAT-SF and BIMS to two measures of cognitive impairment. The BIMS (rs = −0.20, p < 0.01) and BCAT-SF (rs = −0.28, p < 0.001) were significantly negatively correlated with scores on the AD8, which measures self-reported changes in cognition. Negative correlations are to be expected because items on the AD8 are reverse-scored relative to the BIMS and BCAT-SF. The BIMS (rs = 0.52, p < 0.001) and BCAT-SF (rs = 0.68, p < 0.001) were significantly correlated with the KPT, a measure of basic judgment that can differentiate between dementia and non-dementia. Table 3 also shows discriminant validity, which was supported by nonsignificant correlations between the BIMS (rs = −0.05, p = 0.50), BCAT-SF (rs = −0.03, p = 0.66), and GDS-SF, a measure of depression.

Descriptive Statistics for Validity Measures

Table 2:

Descriptive Statistics for Validity Measures

Matrix of Spearman's Rho Correlation Coefficients

Table 3:

Matrix of Spearman's Rho Correlation Coefficients

A one-way MANOVA was conducted to explore the mean differences of the BIMS and BCAT-SF scores between levels of cognitive functioning based on five categories: (a) normal, (b) MCI, (c) mild dementia, (d) moderate dementia, and (e) severe dementia. The cognitive assessment tool scores by levels of cognitive functioning are shown in Table 4. A statistically significant difference was found in BCAT-SF scores between the five cognitive categories (F(4, 207) = 241.13, p < 0.001). Likewise, a statistically significant difference was found in BIMS scores between the five cognitive categories (F(4, 207) = 41.12, p < 0.001). The effect size, calculated using partial eta squared, for the BCAT-SF and BIMS scores, was 0.82 and 0.44, respectively. Having met one-way MANOVA statistical significance, multiple comparisons were conducted for the BIMS and BCAT-SF scores among all five cognitive levels. Games–Howell post hoc tests revealed significant mean differences for BCAT-SF scores between all five CDR groups (p < 0.001 for all tests). However, mean BIMS scores did not significantly differ between normal and MCI (p = 0.74) or between mild dementia and moderate dementia (p = 0.07).

Cognitive Assessment Tool Scores By Diagnostic Category

Table 4:

Cognitive Assessment Tool Scores By Diagnostic Category

A hierarchical multiple regression analysis was also used to test the ability of the BIMS and BCAT-SF in significantly predicting diagnoses of normal, MCI, mild dementia, moderate dementia, and severe dementia. As shown in Table 5, the BIMS was entered at Step 1, explaining 42% of the variance in cognitive diagnosis (R2 = 0.42, F(1, 210) = 151.63, p < 0.001). After entry of the BCAT-SF at Step 2, the total variance explained by the model as a whole was 82% (R2 = 0.82, F(2, 210) = 461.14, p < 0.001). The BCAT-SF accounted for an additional 40% of the variance in diagnoses over and above BIMS scores.

Summary of Multiple Regression Analysis for Predicting Dementia Diagnosis

Table 5:

Summary of Multiple Regression Analysis for Predicting Dementia Diagnosis

Finally, the predictive utility of the BIMS and BCAT-SF for identifying dementia was addressed with analyses of sensitivity, specificity, PPV, and NPV. As shown in Table 6, a BIMS cutoff score of <13 (scores ≥13 indicate no dementia), yielded a sensitivity of 0.66 and a specificity of 0.87 (PPV = 0.88, NPV = 0.63). A BCAT-SF cutoff score of <16 (scores ≥16 indicate no dementia), yielded a sensitivity of 0.91 and a specificity of 0.96 (PPV = 0.97, NPV = 0.87). An ROC was calculated to differentiate diagnoses of MCI and dementia (i.e., mild, moderate, and severe). The resulting ROC curves are compared in the Figure. For identifying dementia, the area under the ROC curve for the BIMS was 0.84 (p < 0.001, 95% CI [0.78, 0.89]) and 0.97 for the BCAT-SF (p < 0.001, 95% CI [0.95, 0.99]).

Selected Cutoff Scores Based on Predictive Utility for Identifying Dementia

Table 6:

Selected Cutoff Scores Based on Predictive Utility for Identifying Dementia

Receiver operating characteristic (ROC) curves. Comparison of the Brief Interview of Mental Status and Brief Cognitive Assessment Tool–Short Form for identifying severe cognitive impairment or probable dementia (i.e., normal and mild cognitive impairment versus mild, moderate, and severe dementia).

Figure.

Receiver operating characteristic (ROC) curves. Comparison of the Brief Interview of Mental Status and Brief Cognitive Assessment Tool–Short Form for identifying severe cognitive impairment or probable dementia (i.e., normal and mild cognitive impairment versus mild, moderate, and severe dementia).

Discussion

The current study compared the abilities of the BIMS and BCAT-SF, two rapidly administered assessment instruments, in predicting cognitive diagnoses and accurately differentiating among cognitive functioning levels (i.e., normal, MCI, mild dementia, moderate dementia, and severe dementia) in nursing home residents. Both instruments demonstrated robust reliability and construct validity. However, meaningful differences were found with respect to their abilities to differentiate among levels of cognitive functioning. The BIMS significantly predicted cognitive diagnosis in general, but did not significantly differentiate between residents with normal cognition and those who had MCI. Furthermore, it did not significantly differentiate between individuals with mild dementia and moderate dementia. The BCAT-SF was found to significantly predict cognitive diagnosis and demonstrated sensitivity in differentiating among all five cognitive levels.

Perhaps this superior sensitivity can be attributed to scale construction. The BCAT-SF was designed to briefly assess three cognitive domains critical to cognitive functioning: (a) attentional capacity, (b) contextual memory, and (c) executive functions. The memory component is especially broad for a rapid cognitive screening tool (four items) and emphasizes story recall to assess verbal memory. Previous research has demonstrated the predictive utility of story recall metrics in assessing cognitive functioning in patients with impaired executive skills (Mansbach, Mace, & Clark, 2014), a problem common to nursing home residents who often have moderate to severe cognitive impairment. Furthermore, in their analysis of the use of individual verbal memory metrics in predicting cognitive functioning, Mansbach, Mace, et al. (2014) found that the BCAT delayed story recognition and delayed story recall items were stronger predictors of cognitive diagnosis than the BIMS delayed word list.

Furthermore, the BCAT-SF contains a cognitive set-shifting item, which in the original BCAT study was found to have the strongest factor loading for the executive functions principle component analysis cluster, and individually strongly predicted cognitive diagnosis (Mansbach et al., 2012). There is added utility to cognitive set-shifting items, as they have been found to be sensitive predictors of MCI to Alzheimer's disease conversion (Gomar, Bobes-Bascaran, Conejero-Goldberg, Davies, & Goldberg, 2011). The BIMS used in nursing homes primarily relies on a simple word list to assess memory, and as Chodosh et al. (2008) and Saliba et al. (2012) point out, does not assess executive functions.

From a clinical perspective, cutoff scores are useful for estimating cognitive functioning. In the current study, the authors emphasized differentiating between patients with and without dementia. This decision was based on the recognition that MCI is a unique clinical syndrome (with subtypes) that may progress to dementia (Bruscoli & Lovestone, 2004; Petersen et al., 2001). Therefore, cognitive instruments that can differentiate between MCI and dementia may be of particular value from a treatment and symptom management perspective. For nursing home patients, the authors recommend a BCAT-SF cutoff score of 16 to differentiate between individuals with dementia and individuals with MCI/normal functioning (i.e., patients with scores ≥16 are likely to have MCI or normal cognition, and individuals with scores <16 are likely to have dementia). This cutoff score is the same value recommended in the original development and validation study (Mansbach & MacDougall, 2012); therefore, the current study appears to cross-validate the original findings.

The current study did not confirm cutoff scores reported in the original BIMS study; however, Chodosh et al. (2008) found that a cutoff score of 10 had a sensitivity of 0.94 and specificity of 0.79 for detecting severe impairment. The current authors found different values using this cutoff score, especially for sensitivity (0.42) and specificity (0.98). In the current study, a BIMS cutoff score of 13 had the best balance of sensitivity (0.66), specificity (0.87), PPV (0.88), and NPV (0.63) for severe impairment. The fact that the authors could not validate the original values might be explained by the fact that the current study was based largely on nursing home residents receiving rehabilitation services, whereas the BIMS sample may have had proportionately more long-term care residents. In this respect, these samples may tap into somewhat different populations. Another possible explanation is that Chodosh et al. (2008) BIMS ROC analyses were based, in part, on the 3MS, which appears to have poor sensitivity in detecting MCI (McDowell et al., 1997). Therefore, the BIMS may be more vulnerable to diagnostic errors.

Limitations

There are important caveats and study limitations that bear highlighting. First, the BIMS and BCAT-SF are screening or single instrument assessments. The actual diagnosis of MCI and dementia should only be made with confirmatory evidence. Second, the current study is based primarily on test scores from nursing home short-term rehabilitation patients. It is possible that long-term care nursing home residents may represent a somewhat different population. Because PPV and NPV values are dependent on the base rate of cognitive impairment, different populations may yield different PPV and NPV values. Third, although the original BCAT-SF validation study did not find an education bias, the current study revealed a modest education effect. Mild educational biases are not unusual for screening tools, and might be attributed to the higher proportion of post-secondary education in this sample compared to the original BCAT-SF study. Of course, the clinician should take the patient's education into account. Fourth, although independent cognitive diagnoses were made using multiple clinical sources, a “gold standard” cognitive measure was not used, which is in contrast to the original BCAT-SF study where comprehensive neuropsychological evaluations were used to determine cognitive diagnoses. Finally, interrater reliability data for those administering the BIMS and BCAT-SF were not available in the current study. Future comparisons of the two measures should determine whether a possible lack of agreement between raters of the BIMS and BCAT-SF accounts for differences in predictive utility.

Conclusion

For nursing home providers to use and optimize cognitive screening tools, a balance of instrument administration brevity and psychometric rigor is key. The issue of insufficient time to complete assessments and plan effective interventions based on test results is often reported as a barrier (Boise et al., 1999; Østbye et al., 2005). The BCATSF, which can be completed in less than 5 minutes and effectively identify patients at different levels of cognitive functioning, appears to be a promising choice. Although the current study does not refute the validity of the BIMS as a cognitive screening tool, it suggests that it may not be optimal in a nursing home population. Given the shortcomings of the BIMS, despite the fact that it is currently the mandated cognitive screening tool for MDS 3.0, providers may elect to also administer the BCAT-SF. In the current study, the BCAT-SF accounted for an additional 40% of the variance in cognitive diagnoses over and above BIMS scores.

References

  • American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author.
  • Boise, L., Camicioli, R., Morgan, D.L., Rose, J.H. & Congleton, L. (1999). Diagnosing dementia: Perspectives of primary care physicians. The Gerontologist, 39, 457–464. doi:10.1093/geront/39.4.457 [CrossRef]
  • Bruscoli, M. & Lovestone, S. (2004). Is MCI really just early dementia? A systematic review of conversion studies. International Psychogeriatrics, 16, 129–140. doi:10.1017/S1041610204000092 [CrossRef]
  • Centers for Medicare & Medicaid Services. (2010). CMS RAI Version 3.0 Section C: Cognitive patterns. Retrieved from http://www.ahcancal.org/facility_operations/Documents/UpdatedFilesOct2010/Chapter%203%20-%20Section%20C%20V1.04%20Sept%202010.pdf
  • Centers for Medicare & Medicaid Services. (2013). Nursing home data compendium 2013 edition. Retrieved from https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/CertificationandComplianc/downloads/nursinghomedatacompendium_508.pdf
  • Chiang, K.S., Green, K.E. & Cox, E.O. (2009). Rasch analysis of the Geriatric Depression Scale-Short Form. The Gerontologist, 49, 262–275. doi:10.1093/geront/gnp018 [CrossRef]
  • Chodosh, J., Edelen, M.O., Buchanan, J.L., Yosef, J.A., Ouslander, J.G., Berlowitz, D.R. & Saliba, D. (2008). Nursing home assessment of cognitive impairment: Development and testing of a brief instrument of mental status. Journal of the American Geriatrics Society, 56, 2069–2075. doi:10.1111/j.1532-5415.2008.01944.x [CrossRef]
  • Cohen-Mansfield, J. (2005). Nursing staff members' assessments of pain in cognitively impaired nursing home residents. Pain Management Nursing, 6, 68–75. doi:10.1016/j.pmn.2005.05.002 [CrossRef]
  • Cohen-Mansfield, J. & Creedon, M. (2002). Nursing staff members' perceptions of pain indicators with severe dementia. Clinical Journal of Pain, 18, 64–73. doi:10.1097/00002508-200201000-00010 [CrossRef]
  • de Craen, A.J., Heeren, T.J. & Gussekloo, J. (2003). Accuracy of the 15-item geriatric depression scale (GDS-15) in a community sample of the oldest old. International Journal of Geriatric Psychiatry, 18, 63–66. doi:10.1002/gps.773 [CrossRef]
  • Foldi, N.S. (2011). Getting the hang of it: Preferential gist over verbatim story recall and the roles of attentional capacity and the episodic buffer in Alzheimer disease. Journal of the International Neuropsychological Society, 17, 69–79. doi:10.1017/S1355617710001165 [CrossRef]
  • Galvin, J.E., Roe, C.M., Coats, M.A. & Morris, J.C. (2007). Patient's rating of cognitive ability using the AD8, a brief informant interview, as a self-rating tool to detect dementia. Archives of Neurology, 64, 725–730. doi:10.1001/archneur.64.5.725 [CrossRef]
  • Galvin, J.E., Roe, C.M., Powlishta, K.K., Coats, M.A., Muich, S.J., Grant, E. & Morris, J.C. (2005). The AD8: A brief informant interview to detect dementia. Neurology, 65, 559–564. doi:10.1212/01.wnl.0000172958.95282.2a [CrossRef]
  • Galvin, J.E., Roe, C.M., Xiong, C. & Morris, J.C. (2006). Validity and reliability of the AD8 informant interview in dementia. Neurology, 67, 1942–1948. doi:10.1212/01.wnl.0000247042.15547.eb [CrossRef]
  • Gerety, M.B., Williams, J.W. Jr.. , Mulrow, C.D., Cornell, J.E., Kadri, A.A., Rosenberg, J. & Long, M. (1994). Performance of case-finding tools for depression in the nursing home: Influence of clinical and functional characteristics and selection of optimal threshold scores. Journal of the American Geriatrics Society, 42, 1103–1109. doi:10.1111/j.1532-5415.1994.tb06217.x [CrossRef]
  • Gomar, J.M., Bobes-Bascaran, M.T., Conejero-Goldberg, C., Davies, P. & Goldberg, T.E. (2011). Utility of combinations of biomarkers, cognitive markers, and risk factors to predict conversion from mild cognitive impairment to Alzheimer's disease in patients in the Alzheimer's disease neuroimaging initiative. Archives of General Psychiatry, 68, 961–969. doi:10.1001/archgenpsychiatry.2011.96 [CrossRef]
  • Hawes, C., Mor, V., Phillips, C.D., Fries, B.E., Morris, J.N., Steele-Friedlob, E. & Nennstiel, M. (1997). The OBRA-87 nursing home regulations and implementation of the Resident Assessment Instrument: Effects on process quality. Journal of the American Geriatrics Society, 45, 977–985. doi:10.1111/j.1532-5415.1997.tb02970.x [CrossRef]
  • Kieffer, K.M. & Reese, R.J. (2002). A reliability generalization study of the Geriatric Depression Scale. Educational and Psychological Measurement, 62, 969–994. doi:10.1177/0013164402238085 [CrossRef]
  • Kintsch, W. (1994). Text comprehension, memory and learning. American Psychologist, 49, 294–303. doi:10.1037/0003-066X.49.4.294 [CrossRef]
  • Lawton, M.P. & Brody, E.M. (1969). Assessment of older people: Self–maintaining and instrumental activities of daily living. The Gerontologist, 9, 179–186. doi:10.1093/geront/9.3_Part_1.179 [CrossRef]
  • Magaziner, J., German, P., Zimmerman, S.I., Hebel, J.R., Burton, L., Gruber-Baldini, A.L. & Kittner, S. (2000). The prevalence of dementia in a statewide sample of new nursing home admissions aged 65 and older: Diagnosis by expert panel. The Gerontologist, 40, 663–672. doi:10.1093/geront/40.6.663 [CrossRef]
  • Mansbach, W.E. & MacDougall, E.E. (2012). Development and validation of the short form of the Brief Cognitive Assessment Tool (BCAT-SF). Aging and Mental Health, 16, 1065–1071. doi:10.1080/13607863.2012.702729 [CrossRef]
  • Mansbach, W.E., MacDougall, E.E., Clark, K.M. & Mace, R.A. (2014). Preliminary investigation of the Kitchen Picture Test (KPT): A new screening test of practical judgment for older adults. Neuropsychology, Development, and Cognition. Section B, Aging, Neuropsychology, and Cognition, 21, 674–692. doi:10.1080/13825585.2013.865698 [CrossRef]
  • Mansbach, W.E., MacDougall, E.E. & Rosenzweig, A.S. (2012). The Brief Cognitive Assessment Tool (BCAT): A new test emphasizing contextual memory, executive functions, attentional capacity, and the prediction of instrumental activities of daily living. Journal of Clinical and Experimental Neuropsychology, 34, 183–194. doi:10.1080/13803395.2011.630649 [CrossRef]
  • Mansbach, W.E., Mace, R.A. & Clark, K.M. (2014). Story recall and word lists: Differential and combined utilities in predicting cognitive diagnosis. Journal of Clinical and Experimental Neuropsychology, 36, 569–576. doi:10.1080/13803395.2014.916656 [CrossRef]
  • McDowell, I., Kristjansson, B., Hill, G.B. & Hébert, R. (1997). Community screening for dementia: The Mini-Mental State Exam (MMSE) and modified Mini-Mental State Exam (3MS) compared. Journal of Clinical Epidemiology, 50, 377–383. doi:10.1016/S0895-4356(97)00060-7 [CrossRef]
  • Morris, J.C. (1993). The Clinical Dementia Rating (CDR): Current version and scoring rules. Neurology, 43, 2412–2414. doi:10.1212/WNL.43.11.2412-a [CrossRef]
  • Nunnally, J. & Bernstein, I.H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.
  • Østbye, T., Yarnall, K.S.H., Krause, K.M., Pollak, K.I., Gradison, M. & Michener, J.L. (2005). Is there time for management of patients with chronic diseases in primary care?Annals of Family Medicine, 3, 209–214. doi:10.1370/afm.310 [CrossRef]
  • Petersen, R.C., Doody, R., Kurz, A., Mohs, R.C., Morris, J.C., Rabins, P.V. & Winblad, B. (2001). Current concepts in mild cognitive impairment. Archives of Neurology, 58, 1985–1992. doi:10.1001/archneur.58.12.1985 [CrossRef]
  • Petersen, R.C., Smith, G.E., Waring, S.C., Ivnik, R.J., Tangalos, E.G. & Kokmen, E. (1999). Mild cognitive impairment: Clinical characterization and outcome. Archives of Neurology, 56, 303–308. doi:10.1001/archneur.56.3.303 [CrossRef]
  • Rabin, L.A., Paré, N., Saykin, A.J., Brown, M.J., Wishart, H.A., Flashman, L.A. & Santulli, R.B. (2009). Differential memory test sensitivity for diagnosing amnestic cognitive impairment and predicting conversion to Alzheimer's disease. Neuropsychology, Development, and Cognition. Section B, Aging, Neuropsychology, and Cognition, 16, 357–376. doi:10.1080/13825580902825220 [CrossRef]
  • Saliba, D., Buchanan, J., Edelen, M.O., Streim, J., Ouslander, J., Berlowitz, D. & Chodosh, J. (2012). MDS 3.0: Brief interview for mental status. Journal of the American Medical Directors Association, 13, 611–617. doi:10.1016/j.jamda.2012.06.004 [CrossRef]
  • Sheikh, J.I. & Yesavage, J.A. (1986). Geriatric Depression Scale (GDS): Recent evidence and development of a shorter version. Clinical Gerontology, 5, 165–173. doi:10.1300/J018v05n01_09 [CrossRef]
  • Singer, C. & Luxenberg, J. (2003). Diagnosing dementia in long-term care facilities. Journal of American Medical Directors Association, 4(Suppl. 6), S134–S140. doi:10.1016/S1525-8610(04)70404-1 [CrossRef]
  • Teng, E.L. & Chui, H.C. (1987). The Modified Mini-Mental State (3MS) examination. Journal of Clinical Psychiatry, 48, 314–318.
  • Waters, G. & Caplan, D. (2005). The relationship between age, processing speed, working memory capacity, and language comprehension. Memory, 13, 403–413. doi:10.1080/09658210344000459 [CrossRef]

Patient Demographics (N = 225)

Characteristicn (%)
Gender
  Female137 (60.9)
  Male88 (39.1)
Race
  Caucasian163 (72.4)
  African American49 (21.8)
  Other8 (3.6)
  Missing5 (2.2)
Marital status
  Widowed111 (49.3)
  Married66 (29.3)
  Divorced23 (10.2)
  Single, never married16 (7.1)
  Missing9 (4)
Education (years completed)
  0 to 813 (5.8)
  9 to 1128 (12.4)
  1254 (24)
  13 to 1549 (21.8)
  1631 (13.8)
  17+42 (18.7)
  Missing8 (3.6)
Cognitive diagnosis
  Mild cognitive impairment63 (28)
  Mild dementia57 (25.3)
  Moderate dementia43 (19.1)
  Severe dementia27 (12)
  No dementia22 (9.8)
  No diagnosis13 (5.8)
Depression
  No162 (72)
  Yes63 (28)

Descriptive Statistics for Validity Measures

MeasurenMedianMeanSDRangeCronbach's Alpha
BCAT-SF2251513.525.470 to 210.84
BIMS2251311.513.622 to 150.77
AD819532.782.000 to 80.66
KPT20465.122.770 to 80.88
GDS-SF18444.072.590 to 100.46

Matrix of Spearman's Rho Correlation Coefficients

VariableAgeBCAT-SFBIMSAD8KPTGDS-SF
Age
BCAT–SF−0.38***
BIMS−0.27***0.59***
AD80.08−0.28***−0.20**
KPT−0.350.68***0.52***−0.08
GDS–SF−0.00−0.03−0.050.42***−0.03

Cognitive Assessment Tool Scores By Diagnostic Category

BCAT-SFBIMS
Diagnostic CategoryMedianMeanSDRangeMedianMeanSDRange
No dementia2120.181.2216 to 211514.181.4710 to 15
Mild cognitive impairment1818.021.9610 to 211413.731.598 to 15
Mild dementia1413.672.833 to 201211.582.933 to 15
Moderate dementia109.722.424 to 15119.843.413 to 15
Severe dementia43.932.670 to 1066.633.812 to 14

Summary of Multiple Regression Analysis for Predicting Dementia Diagnosis

Step 1Step 2
VariableBSE BßBSE Bß
BIMS−0.210.02−0.65**−0.050.01−0.15**
BCAT-SF−0.170.01−0.81**
R20.420.82
ΔR20.40
F for ΔR151.63461.14

Selected Cutoff Scores Based on Predictive Utility for Identifying Dementia

MeasureCutoff ScoreaAUCSensitivitybSpecificitybPPVbNPVb
BCAT-SF<160.970.91 (0.84, 0.95)0.96 (0.89, 0.99)0.97 (0.93, 0.99)0.87 (0.78, 0.93)
BIMS<130.840.66 (0.57, 0.74)0.87 (0.78, 0.93)0.88 (0.80, 0.94)0.63 (0.54, 0.72)
Authors

Mr. Mace is Research and Clinical Technician, Dr. Mansbach is Founder and CEO, and Ms. Clark is Senior Vice President, Mansbach Health Tools, LLC, Simpsonville, Maryland.

All authors are employed by the company that designed the Brief Cognitive Assessment Tool.

Address correspondence to Ryan A. Mace, BS, Research and Clinical Technician, Mansbach Health Tools, LLC, P.O. Box 307, Simpsonville, MD 21150; e-mail: rmace@thebcat.com.

Received: December 17, 2014
Accepted: March 27, 2015
Posted Online: May 28, 2015

10.3928/19404921-20150522-05

Sign up to receive

Journal E-contents