Journal of Gerontological Nursing

Research Brief 

Reliability and Validity of Two Measures of Toileting Skills in Frail Older Women Without Dementia

Kristine M.C. Talley, PhD, RN, GNP-BC; Jean F. Wyman, PhD, RN, GNP-BC, FGSA, FAAN; Becky G. Olson-Kellogg, PT, DPT, GCS, CEEAA; Ulf G. Bronas, PhD; Teresa C. McCarthy, MD

Abstract

Urinary incontinence (UI) affects 43% of frail, community-dwelling older women and threatens their independence. For these women, remaining continent depends on their ability to toilet. Treatments should include improving toileting skills. However, reliable and valid measures of toileting skills are not available to evaluate treatment efforts in this population. The current cross-sectional study of 24 frail older women (average age = 87 years) examines the reliability and concurrent validity of the Performance Oriented Timed Toileting Instrument (POTTI) and self-reported Minnesota Toileting Skills Questionnaire (MTSQ). The POTTI demonstrated strong interrater reliability (0.97) and the MTSQ demonstrated good internal consistency (Cronbach's alpha = 0.82). Both measures had statistically significant correlations with frailty, physical performance, UI-related quality of life, and UI self-efficacy. Both instruments demonstrated good psychometric properties and show promise as outcome measures for UI clinical trials. Further work is needed to establish their responsiveness to change and minimum clinically important difference values. [Journal of Gerontological Nursing, 42(9), 16–20.]

Abstract

Urinary incontinence (UI) affects 43% of frail, community-dwelling older women and threatens their independence. For these women, remaining continent depends on their ability to toilet. Treatments should include improving toileting skills. However, reliable and valid measures of toileting skills are not available to evaluate treatment efforts in this population. The current cross-sectional study of 24 frail older women (average age = 87 years) examines the reliability and concurrent validity of the Performance Oriented Timed Toileting Instrument (POTTI) and self-reported Minnesota Toileting Skills Questionnaire (MTSQ). The POTTI demonstrated strong interrater reliability (0.97) and the MTSQ demonstrated good internal consistency (Cronbach's alpha = 0.82). Both measures had statistically significant correlations with frailty, physical performance, UI-related quality of life, and UI self-efficacy. Both instruments demonstrated good psychometric properties and show promise as outcome measures for UI clinical trials. Further work is needed to establish their responsiveness to change and minimum clinically important difference values. [Journal of Gerontological Nursing, 42(9), 16–20.]

The ability to self-manage toileting is critical for frail older adults to maintain independent living. Once an older adult requires assistance with toileting, they are at high risk for nursing home placement and death (Carey et al., 2008). Toileting skills depend on the ability to walk, transfer positions, disrobe, and maintain continence. When lower urinary tract symptoms (e.g., urgency, leakage) are present, the time it takes to toilet can be the difference between being dry or wet. Older women are at increased risk for deteriorating toileting skills because they have higher rates of ambulation and transferring impairments, as well as urinary urgency and incontinence, than men (Talley et al., 2014). Thus, improving or preventing decline in toileting skills in frail older women may help them attain the highest possible level of independent living.

Before interventions can be tested, reliable and valid measures of toileting skills for this population must be established. Ouslander et al. (1987) developed a performance-based measure of toileting skills, the Performance Oriented Timed Toileting Instrument (POTTI), and reported its interrater reliability (90% agreement) with nursing home residents, but no published evidence exists on its reliability in frail, community-dwelling older adults. The POTTI demonstrated responsiveness to change in a pilot study of toileting skills in nursing home residents, but without report of its other psychometric properties (van Houten, Achterberg, & Ribbe, 2007). Existing self-report measures of toileting difficulties are often one categorical item in a survey of multiple activities of daily living (ADLs), making it impossible to assess difficulty performing specific toileting tasks.

Therefore, the purpose of the current study was to determine the reliability and validity of the POTTI and the newly created Minnesota Toileting Skills Questionnaire (MTSQ) in frail, community-dwelling older women without dementia. The specific aims were to establish the (a) interrater reliability (intraclass correlation coefficients [ICCs]) of the POTTI; (b) internal consistency reliability (Cronbach's alpha) of the MTSQ; and (c) concurrent validity of both instruments by determining their correlations with measures of frailty, urinary incontinence (UI) severity, UI-related quality of life, UI self-efficacy, and physical performance.

Method

Design

The current cross-sectional study used a convenience sample of 24 older women living in three senior apartment buildings. Two research assistants collected data in a private room in participants' buildings. To minimize instrumentation bias, one research assistant was randomly assigned to interview and administer the performance-based tests. Participants completed two trials of the POTTI while both research assistants simultaneously scored their performance. The study received approval from a university institutional review board.

Inclusion/Exclusion Criteria

Inclusion criteria were: must speak English, report UI, walk without human assistance, have difficulty performing ADLs (score ≥3 on the Vulnerable Elders Survey [Min et al., 2009]), and be cognitively intact according to the Mini-Cog test (Borson, Scanlan, Brush, Vitaliano, & Dokmak, 2000). Participants were excluded if they had acute onset UI, UI due to a central nervous system disorder, recent urogynecological surgery, or used a pessary, catheter, or ostomy.

Measures

Toileting skills were measured with a performance-based test and an interviewer-administered questionnaire. The POTTI is a performance-based test where participants are timed as they complete tasks that simulate toileting: walking a distance of 15 feet, pulling down an elastic waist skirt worn over usual clothing, transferring onto a chair, grabbing toilet paper and placing it into a wastebasket, rising, and pulling the skirt back up. The POTTI test was modified to use an elastic waist skirt instead of a vest with fasteners to reflect clothing worn by women and use supplies more accessible to users.

The MTSQ was designed by the first author (K.M.T.) in consultation with a gerontological occupational therapist. Participants used a 5-point Likert scale to rate their degree of difficulty completing five tasks (Table 1). Scores range from 0 to 20, with higher scores indicating more difficulty.


Items in the Minnesota Toileting Skills Questionnairea

Table 1:

Items in the Minnesota Toileting Skills Questionnaire

The Vulnerable Elders Survey (Min et al., 2009) measured frailty. Scores range from 0 to 10, with higher scores indicating greater activity limitations. Scores ≥3 indicate frailty.

The International Consultation on Incontinence Questionnaire (ICIQ; Avery et al., 2004) measured UI severity. Scores range from 0 to 21, with higher scores indicating greater severity.

The Incontinence Impact Questionnaire Short Form and the Urogenital Distress Inventory Short Form measured UI-related quality of life (Uebersax, Wyman, Shumaker, & McClish, 1995). Scores for both measures range from 0 to 100, with higher scores indicating worse qualities of life.

The Geriatric Self-Efficacy Index for UI (Tannenbaum et al., 2009) assesses older adults' level of confidence to prevent urine loss during 12 activities. Participants rate their confidence for each activity on a scale of 0 (not confident) to 10 (very confident). Total scores range from 0 to 120.

The Short Physical Performance Battery (SPPB; Guralnik et al., 1994) measures balance, time to walk 4 meters, and time to rise from a chair and return to a seated position five times. Each activity is scored categorically from 0 to 4, with total scores ranging from 0 to 12. Lower scores represent poorer performance.

Data Analysis

An a priori power calculation indicated a sample size of 24 would detect a difference of 0.2 correlations between raters' repeated measures on the POTTI, with 80% power and a two-sided alpha of 0.5. Interrater reliability was assessed using ICCs, with values of 0.75 to 1.0 denoting excellent agreement (Hallgren, 2012). ICCs were calculated using a two-way mixed effects model specifying consistency of agreement. Concurrent validity was assessed with Spearman correlation coefficients to accommodate the small sample and skewed distribution of POTTI and MTSQ scores. Internal consistency reliability of the MTSQ was assessed using Cronbach's alpha, with values >0.8 denoting excellent reliability (Tavakol & Dennick, 2011). All statistical tests were two-tailed. A p value of <0.05 was considered statistically significant. Analyses were performed using Stata/IC version 13.1.

Results

Participants were non-Hispanic White women with a mean age of 87 years (SD = 4.7 years). Most lived alone (83.3%) in independent living apartments (79.2%) for an average of 27 months. Mean POTTI scores for each trial and rater ranged from 34.2 (SD = 22.1) to 35.6 (SD = 18.7) seconds. MTSQ scores were skewed toward less difficulty (mean = 2.5, SD = 2.9). More than 20% of participants (n = 5) reported no difficulty with any of the toileting skills. Seventy-four percent of participants reported difficulty with putting on and taking off long pants, 30% reported difficulty walking through the house to the toilet, 30% reported difficulty getting on and off the toilet, 26% reported difficulty reaching their bottom to wipe, and 9% reported difficulty reaching for toilet paper. The POTTI and MTSQ had a correlation of 0.45 (p = 0.03).

Interrater reliability of the POTTI was high, with an ICC of 0.97 (95% confidence interval [0.94, 0.99]). Internal consistency reliability of the MTSQ was strong, with a Cronbach's alpha of 0.82.

In terms of concurrent validity, poorer performance on the POTTI was significantly associated with increased frailty, worse performance on the SPPB, worse UI-related quality of life, and lower UI self-efficacy. POTTI scores were not significantly correlated with incontinence severity (ICIQ scores). Poorer performance on the MTSQ was significantly associated with increased frailty, slower gait, worse UI-related quality of life, and lower UI self-efficacy scores. The MTSQ was not significantly correlated with UI severity (ICIQ scores), overall SPPB scores, or balance and chair stand subscores. Table 2 reports these correlations.


Mean Values and Correlations of the Performance Oriented Timed Toileting Instrument (POTTI) and Minnesota Toileting Skills Questionnaire (MTSQ) With Frailty, Physical Performance, and Urinary Incontinence Measures

Table 2:

Mean Values and Correlations of the Performance Oriented Timed Toileting Instrument (POTTI) and Minnesota Toileting Skills Questionnaire (MTSQ) With Frailty, Physical Performance, and Urinary Incontinence Measures

Discussion

Few measures of toileting skills are available to investigators interested in preventing dependency in this basic ADL. The current study provides preliminary evidence on the reliability and validity of two toileting skills instruments for use in frail, community-dwelling older women without dementia—a population at high risk for developing toileting dependencies. The POTTI demonstrated strong interrater reliability, similar to that established in nursing home residents (Ouslander et al., 1987), as well as validity with frailty, physical performance, UI-related quality of life, and UI self-efficacy.

The current article introduces a new self-reported measure of difficulty with toileting skills. The MTSQ demonstrated strong internal consistency. Like the POTTI, it demonstrated validity with measures of frailty, gait speed, UI-related quality of life, and UI self-efficacy. These findings add to the scant evidence supporting the relationship between gait speed and self-reported toileting difficulties (Verghese, Wang, & Holtzer, 2011). The MTSQ was not significantly correlated with total physical performance scores, balance measures, or chair stand scores. Others have found that performance-based measures detect early changes in physical abilities that older adults may not be able to detect with self-report measures (Brach, VanSwearingen, Newman, & Kriska, 2002). Thus, the MTSQ may reflect toileting skills participants have adapted in response to early physical changes (Rogers et al., 2003).

The performance-based POTTI and the self-reported MTSQ were moderately correlated, indicating overlap in concepts. The POTTI had stronger associations than the MTSQ with physical performance measures. Although evidence is not clear as to whether self-reported or performance-based measures of function are more reliable and valid (Brach et al., 2002), it is known that both predict poor outcomes, such as short-term mortality and nursing home admission (Guralnik et al., 1994).

Researchers and clinicians should select measures based on their purpose and resources. Benefits of the self-reported MTSQ include short administration time, not needing space to administer performance-based tests, and potential reporting of adaptive toileting behaviors. Limitations of the MTSQ include a potential ceiling effect with early physical changes. Benefits of using the performance-based POTTI include being a highly reliable objective measure and possible earlier detection of physical changes impairing toileting skills.

Limitations

Study limitations include convenience sampling of White, frail older women. However, the POTTI could be easily modified to use elastic waist shorts for frail older men. The sample most likely explains the non-significant correlations both instruments had with UI severity. A post-hoc analysis indicated a sample of 124 would have detected the reported correlation of 0.24, with a power of 0.80 and a two-tailed alpha of 0.05. It is worthwhile to explore these relationships in future larger trials because others have reported associations with UI and dependencies in ADLs (Greer, Xu, Propert, & Arya, 2015).

Conclusion

In the current sample of frail older women, the POTTI and MTSQ were reliable and valid measures of toileting skills. Both instruments show promise as outcome measures to evaluate the effectiveness of interventions to prevent or treat toileting difficulties. Further work is needed to confirm these findings in larger, more representative samples, and to establish their responsiveness to change and minimum clinically important difference values.

References

  • Avery, K., Donovan, J., Peters, T.J., Shaw, C., Gotoh, M. & Abrams, P. (2004). ICIQ: A brief and robust measure for evaluating the symptoms and impact of urinary incontinence. Neurourology and Urodynamics, 23, 322–330 doi:10.1002/nau.20041 [CrossRef]
  • Borson, S., Scanlan, J., Brush, M., Vitaliano, P. & Dokmak, A. (2000). The Mini-Cog: A cognitive ‘vital signs’ measure for dementia screening in multi-lingual elderly. International Journal of Geriatric Psychiatry, 15, 1021–1027 doi:10.1002/1099-1166(200011)15:11<1021::AID-GPS234>3.0.CO;2-6 [CrossRef]
  • Brach, J.S., VanSwearingen, J.M., Newman, A.B. & Kriska, A.M. (2002). Identifying early decline of physical function in community-dwelling older women: Performance-based and self-report measures. Physical Therapy, 82, 320–328 doi:10.1519/00139143-200124010-00022 [CrossRef]
  • Carey, E.C., Covinsky, K.E., Lui, L.Y., Eng, C., Sands, L.P. & Walter, L.C. (2008). Prediction of mortality in community-living frail elderly people with long-term care needs. Journal of the American Geriatrics Society, 56, 68–75 doi:10.1111/j.1532-5415.2007.01496.x [CrossRef]
  • Greer, J.A., Xu, R., Propert, K.J. & Arya, L.A. (2015). Urinary incontinence and disability in community-dwelling women: A cross-sectional study. Neurourology and Urodynamics, 34, 539–543 doi:10.1002/nau.22615 [CrossRef]
  • Guralnik, J.M., Simonsick, E.M., Ferrucci, L., Glynn, R.J., Berkman, L.F., Blazer, D.G. & Wallace, R.B. (1994). A short physical performance battery assessing lower extremity function: Association with self-reported disability and prediction of mortality and nursing home admission. Journal of Gerontology, 49, M85–M94 doi:10.1093/geronj/49.2.M85 [CrossRef]
  • Hallgren, K.A. (2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitaive Methods for Psychology, 8, 23–34.
  • Min, L., Yoon, W., Mariano, J., Wenger, N.S., Elliott, M.N., Kamberg, C. & Saliba, D. (2009). The vulnerable elders-13 survey predicts 5-year functional decline and mortality outcomes in older ambulatory care patients. Journal of the American Geriatrics Society, 57, 2070–2076 doi:10.1111/j.1532-5415.2009.02497.x [CrossRef]
  • Ouslander, J.G., Morishita, L., Blaustein, J., Orzeck, S., Dunn, S. & Sayre, J. (1987). Clinical, functional, and psychosocial characteristics of an incontinent nursing home population. Journal of Gerontology, 42, 631–637 doi:10.1093/geronj/42.6.631 [CrossRef]
  • Rogers, J.C., Holm, M.B., Beach, S., Schulz, R., Cipriani, J., Fox, A. & Starz, T.W. (2003). Concordance of four methods of disability assessment using performance in the home as the criterion method. Arthritis & Rheumatism, 49, 640–647 doi:10.1002/art.11379 [CrossRef]
  • Talley, K.M.C., Wyman, J.F., Bronas, U.G., Olson-Kellogg, B.J., McCarthy, T.C. & Zhao, H. (2014). Factors associated with toileting disability in older adults without dementia living in residential care facilities. Nursing Research, 63, 94–104 doi:10.1097/NNR.0000000000000017 [CrossRef]
  • Tannenbaum, C., Brouillette, J., Michaud, J., Korner-Bitensky, N., Dumoulin, C., Corcos, J. & Valiquette, L. (2009). Responsiveness and clinical utility of the Geriatric Self-Efficacy Index for Urinary Incontinence. Journal of the American Geriatrics Society, 57, 470–475 doi:10.1111/j.1532-5415.2008.02146.x [CrossRef]
  • Tavakol, M. & Dennick, R. (2011). Making sense of Cronbach's alpha. International Journal of Medical Education, 2, 53–55 doi:10.5116/ijme.4dfb.8dfd [CrossRef]
  • Uebersax, J.S., Wyman, J.F., Shumaker, S.A. & McClish, D.K. (1995). Short forms to assess life quality and symptom distress for urinary incontinence in women: The Incontinence Impact Questionnaire and the Urogenital Distress Inventory. Neurourology and Urodynamics, 14, 131–139 doi:10.1002/nau.1930140206 [CrossRef]
  • van Houten, P., Achterberg, W. & Ribbe, M. (2007). Urinary incontinence in disabled elderly women: A randomized clinical trial on the effect of training mobility and toileting skills to achieve independent toileting. Gerontology, 53, 205–210 doi:10.1159/000100544 [CrossRef]
  • Verghese, J., Wang, C. & Holtzer, R. (2011). Relationship of clinic-based gait speed measurement to limitations in community-based activities in older adults. Archives of Physical Medicine and Rehabilitation, 92, 844–846 doi:10.1016/j.apmr.2010.12.030 [CrossRef]

Items in the Minnesota Toileting Skills Questionnairea

How much difficulty do you have…
putting on and taking off long pants (including managing fasteners)?
walking through your home to get to the toilet in time?
sitting down on and standing up from the toilet?
reaching to the side to grab toilet paper?
reaching your bottom to wipe with toilet paper?

Mean Values and Correlations of the Performance Oriented Timed Toileting Instrument (POTTI) and Minnesota Toileting Skills Questionnaire (MTSQ) With Frailty, Physical Performance, and Urinary Incontinence Measures

VariableMean (SD)POTTI ρMTSQ ρ
POTTI (in seconds)34.2 (22.1)10.45*
MTSQ2.5 (2.9)0.45*1
Vulnerable Elders Survey5.3 (2.6)0.47*0.51*
Short Physical Performance Battery6.9 (3)−0.81*−0.28
  Balance subscore2.3 (1)−0.46*−0.3
  Gait subscore2.9 (1.1)−0.82*−0.48*
  Chair stand subscore1.7 (1.4)−0.79*−0.11
International Consultation on Incontinence Questionnaire9.5 (4)0.240.24
Incontinence Impact Questionnaire Short Form18.2 (18.7)0.53*0.67*
Urogenital Distress Inventory Short Form22.9 (21.1)0.20.51*
Geriatric Self-Efficacy Index for Urinary Incontinence68.9 (30.3)−0.49*−0.49*
Authors

Dr. Talley is Assistant Professor, Dr. Wyman is Professor and Cora Meidl Siehl Endowed Chair in Nursing Research, School of Nursing, Dr. Olson-Kellogg is Assistant Professor, Physical Medicine and Rehabilitation, Program in Physical Therapy, and Dr. McCarthy is Assistant Professor, Family Practice and Community Health, University of Minnesota, Minneapolis, Minnesota; and Dr. Bronas is Associate Professor, College of Nursing, University of Illinois at Chicago, Chicago, Illinois.

The authors have disclosed no potential conflicts of interest, financial or otherwise. This work was funded by grants (for K. Talley) from the Building Interdisciplinary Research Careers in Women's Health Program of the National Institutes of Child Health and Human Development to the Deborah E. Powell Center for Women's Health at the University of Minnesota (K12HD055887), and the National Center for Advancing Translational Sciences of the National Institutes of Health, University of Minnesota CTSI K to R01 Transition Award (UL1TR000114).

The authors thank Patricia L. Schaber, PhD, OTR/L, Associate Professor, University of Minnesota Program for Occupational Therapy, for her consultation on developing the Minnesota Toileting Skills Questionnaire.

Address correspondence to Kristine M.C. Talley, PhD, RN, GNP-BC, Assistant Professor, School of Nursing, University of Minnesota, 5-140 Weaver Densford Hall, 308 Harvard Street SE,Minneapolis, MN 55455; e-mail: carl0106@umn.edu.

Received: January 20, 2016
Accepted: April 27, 2016
Posted Online: June 03, 2016

10.3928/00989134-20160531-02

Sign up to receive

Journal E-contents