Journal of Gerontological Nursing

Global Aging 

Reliability and Validity of a Community Needs Assessment Instrument for Nursing Homes in Taiwan

Shu-li Chen, PhD, RN

Abstract

The Perception Evaluaù'on Tool is used to demonstrate a need for nursing home services, contribute to the development of services, ensure services meet demands, and improve existing services.

Abstract

The Perception Evaluaù'on Tool is used to demonstrate a need for nursing home services, contribute to the development of services, ensure services meet demands, and improve existing services.

In addition to the growing population and lengthening life span, changes in illness patterns and family life make long-term health care problems serious in the Taiwan area. Traditional Chinese individuals still cling to the value that it is the responsibility of family members to take care of their ill family members. However, because of urbanization and changing lifestyles, family members are faced with a dilemma when making the decision to place a loved one to a nursing home or long-term care facility. Several studies have been conducted which conclude there is an insufficient number of nursing homes in the Taiwan area (Chiou & Wang, 1987; Hon, 1993; Hurng, 1987; Jung, Tsai, & Lin, 1990; Wu, 1993). However, no research instruments were found which addressed specifically the perception of need for nursing home services.

Because of the lack of sound instruments, the author developed the Community Individual Informant Tool (CIIT) and the Health Care Provider Informant Tool (HCPIT) to assess community needs for nursing homes in Taiwanese societies (Chen, 1995). The original CIIT and HCPIT were revised and combined based on the principles of instrument development indicated by DeVellis (1991), Dillman (1978), Fink and Kosecoff (1985), Fowler, Jr. (1984), Mishel (1990), Nunnally and Bernstein (1994), and Witkin and Altschuld (1995). The purpose of this study was to evaluate the psychometric properties of the modified CIIT and HCPIT, which currently is tided the Perception Evaluation Tool (PET).

The PET is designed to identify the perception of:

* Need for nursing home services in a community.

* If needed, what services should be provided within nursing homes.

* The potential willingness of individuals to use services provided within nursing homes.

It is important that needs assessment studies be conducted prior to construction of any nursing homes to ensure the services that customers expect to have are available and also to reduce the risk of marketing failure (Neuber, 1980; Rakich, Longest, & Darr, 1992; Soriano, 1995; Warheit, Bell, & Schwab, 1977; Zaura, Brachrach, & Hess, 1983). The PET can be used by individuals who plan to build new nursing homes, and its results can be used to market nursing home services in the Taiwan area. Furthermore, the tool can be used by policy makers who plan or develop long-term health care programs. However, the people who will benefit most from this study are current nursing home customers (i.e., nursing home residents, their family members, service providers for nursing home living) and potential nursing home customers (i.e., the general public).

METHODOLOGY

Setting

For geographical and sampling convenience, 10 towns in Hsinchu County, Taiwan Province, Republic of China, were included in the study. Each was considered as a community for this survey. These 10 towns were selected based on the following inclusion criteria:

* They were located in Hsinchu County.

* They had a population of at least 10,000 people.

The selected towns are located in the northwest area of Taiwan Province. Based on Taiwanese designation, three of the selected communities are suburban, and the remainder are rural. The total population for these towns was 388,293. The population of the survey towns ranged from 10,778 to 81,552. The average population density for the survey towns was 621 individuals per square kilometer, which ranged from 1 89 to 1,645. On average, there were 4.3 individuals per house, and the gender ratio was 111 men per 100 women.

Sample

To evaluate the PET, an expected sample size of 270 was needed, based on the results of power analysis (Cohen, 1988) and the recommendations from Comrey and Lee (1992), Gorsuch (1983), and Nunnally and Bernstein (1994). Inclusion criteria for individual participants included:

* Residing in the target community for at least 6 months.

* Age 20 or older.

* Able to read and write Chinese.

In addition, key community informants in each surveyed town who had special knowledge of community needs also were invited to participate in the study. A simple random sampling method was used to select potential participants.

Instrument

The initial PET consisted of 71 closed-ended items and was organized into two sections - Factors and Perception. This study evaluated the reliability and validity of both sections of the tool. The items in Section I: Factors, were developed based on previous studies conducted in the Taiwan area. The three hypothesized subscales within Section I: Factors of the PET were:

* Awareness (10 items).

* Personal experience (10 items).

* Risk factors (24 items).

These items were designed to elicit information about awareness of nursing home services, the experience of nursing home services, and risk factors for nursing home entry. There also were three hypothesized subscales within Section II: Perception, including:

* Service needs (11 items).

* Utilization patterns (11 items).

* Residential preference (5 items).

It was hypothesized that the needs for nursing homes were influenced by utilization patterns and residential preference. Additionally, if a great need for providing nursing home services within the survey community was identified, the results from the subscale of Service Needs would become recommendations for services within newly constructed or preexisting nursing homes. The results from the subscales of Utilization Patterns and Residential Preference would indicate the potential and willingness to use nursing home services.

Design and Procedure of Data Collection

Prior to the mailing, a brief telephone call was made to describe the purpose of the study, solicit assistance, obtain the screening information, and, most important, obtain current mailing addresses. A cover letter, a survey questionnaire (i.e., the PET), and a stamped, self-addressed return envelope were mailed to the subjects who were willing to participate as indicated by the telephone screening. A thank you letter was sent to the subjects who responded.

Two weeks after the initial mailings, nonrespon dents were contacted by telephone to inform them that the questionnaire had not been received and to ask for its return. A follow-up letter as a friendly and courteous reminder was mailed to subjects who had not returned their questionnaires approximately 2 weeks after the telephone follow up. A pilot study initially was performed in the environment in which the survey was to be conducted. No trends within subscales of the PET could be identified during the pilot study, nor were there any differences among the survey towns related to administering the PET. A psychometric evaluation study with cross-sectional longitudinal sampling techniques using a mailed questionnaire then was conducted.

Table

TABLE 1PRINCIPAL COMPONENTS FACTOR ANALYSIS WITH VARIMAX ROTATION OF SECTION II: PERCEPTION OF THE PERCEPTION EVALUATION TOOL (N = 319)

TABLE 1

PRINCIPAL COMPONENTS FACTOR ANALYSIS WITH VARIMAX ROTATION OF SECTION II: PERCEPTION OF THE PERCEPTION EVALUATION TOOL (N = 319)

RESULTS

Demographic Characteristics of Participants

Based on the results of the telephone screening, the willingness to participate in the study was 91.3% for the general community individuals and 98.7% for the key community informants. The overall response rate for the survey was 69%, which was adequate for assessing the psychometric properties of the PET. Most participants who completed the questionnaires were general community individuals (52.7%), followed by health care providers (19.4%), social leaders (15.3%), and political leaders (12.6%). Overall, the sample contained 188 men and 131 women, ranging in age from 20 to 83 (mean = 44.8, SD = 14.8). Most participants indicated their annual household incomes ranged from adequate (52.8%) to more than adequate (36.7%). Approximately half of the participants (42.8%) indicated they had some religious preference at the time of survey. Also, approximately half of the participants reported there were one (33.9%) to two (16.3%) elderly family members living with them, and 48% of the participants indicated there was no elderly family member in their households.

Table

TABLE 2MEANS, STANDARD DEVIATIONS, AND CORRELATIONS OF TIME 1 AND TIME 2 MEASURES FOR THE FACTORED SUBSCALES WITHIN SECTION II: PERCEPTION OF THE PERCEPTION EVALUATION TOOL (Af s 79)

TABLE 2

MEANS, STANDARD DEVIATIONS, AND CORRELATIONS OF TIME 1 AND TIME 2 MEASURES FOR THE FACTORED SUBSCALES WITHIN SECTION II: PERCEPTION OF THE PERCEPTION EVALUATION TOOL (Af s 79)

Content Validity of the PET

The PET was viewed by 14 purposively selected Taiwanese individuals to check the items for clarity and readability. Face validation was obtained, and this helped assess the clarity of items, instructions, and format. After face validation, content validation was evaluated. Five Taiwanese individuals who were experts in nursing homes or in community needs assessment studies were invited to review the PET. Experts were asked to assess all items within the original PET related to their appropriateness in evaluating the perception of nursing home services, using a 4-point scale (from 1 = not appropriate to 4 = appropriate). No items had more than 20% of the expert panel rate them as either "not appropriate" or "probably not appropriate." Based on the comments made by the expert panel, closely related items were combined, and ambiguously worded items were clarified.

Construct Validity of the PET

Exploratory factor analysis with principal components and varimax rotation were used in this study to evaluate construct validity of Section II: Perception of the PET. Names of the factors and items together with factor loading, percentage of explained variance contributed to each factor, eigenvalues, and alpha reliability coefficients are shown in Table 1 . The orthogonally rotated three-factor solution contained 21 of the initial 27 perceptions of need items and accounted for 49.3% of the explained variance. Six items were eliminated from further instrument evaluation tests as a result of either low loading on the extracted factors or ambiguous loading on multiple factors. The results of factor analysis indicated Section II: Perception of the PET was multidimensional and measured three distinct domains, including Service Needs, Family Availability, and Potential for Use.

Stability Reliability of the PET

Stability by definition is the consistency of repeated measures across time. In this study, this aspect of reliability is referred to as testretest. A subset of the obtained sample (n = 79) was measured twice during a period of 2 weeks, called Time 1 and Time 2. According to the correlation coefficients in Table 2, the strong positive relationship between Time 1 and Time 2 measures have established the stability reliability of two of the factored subscales of the PET, Service Needs and Family Availability. However, the weak relationship (r = .39) for the subscale of Potential for Use indicated a low stability reliability, and further reliability evaluation of this subscale was needed.

Internal Consistency Reliability of the PET

Internal consistency reliability of Section I: Factors of the PET was established at Cronbach's alpha of .87. The alpha reliability coefficients for the subscales of Awareness, Personal Experience, and Risk Factor were alpha = .94, .93, and .73, respectively. The number of items within the subscales of Awareness and Personal Experience remained the same. However, the items which identified gender and migration from mainland China as risk factors for nursing home entry were deleted because of the low inter-item correlation and the alpha increase when the items were deleted. These two items originally were entered into the Risk Factor subscale based on Lin, Chen, Lee, & Lin's study in 1984. However, their study findings were not supported by recent studies (Lee, Wang, & Jung, 1990; Shiu, Shiung, Dai, Chen, & Huang, 1993; Wu, 1991). Based on these factors, it was decided to eliminate these two items from the Risk Factor subscale.

Table

TABLE 3PEARSON r CORRELATIONS OF THE SUBSCALES WITHIN THE PERCEPTION EVALUATION TOOL (Af =319)

TABLE 3

PEARSON r CORRELATIONS OF THE SUBSCALES WITHIN THE PERCEPTION EVALUATION TOOL (Af =319)

Internal consistency reliability for the three factored subscales within Section II: Perception of the PET (i.e., Service Needs, Family Availability, Potential For Use) was demonstrated by Cronbach's alphas of .93, .83, and .47, respectively. The low alpha reliability coefficient of .47 did not support the internal consistency reliability for the subscale Potential for Use. For the same subscale, the test-retest reliability was low at r = .39. Therefore, the four items within the factored subscale Potential for Use were eliminated from further instrument evaluation tests. The overall internal consistency alpha coefficient of the two-subscale model of Section II: Perception of the PET (alpha = .90) and of each of the retained subscales indicated acceptable evidence for reliability. Nunnally and Bernstein (1994) state that a reliability coefficient of .70 or higher is an acceptable level for a new scale. Therefore, the internal consistency reliability of both sections of the PET was established.

Convergent Validity of the PET

A Pearson product-moment correlation coefficient of the subscales within Section I: Factors and Section II: Perception of the PET was generated to evaluate how these two sections of the PET relate to each other (Table 3). The strong relationships between the subscales of Service Needs and Family Availability further support the convergent validity of the second section of the PET because those two subscales converted to each other to a certain degree. The correlation coefficients in Table 3 also indicate a moderate relationship between participants' awareness of nursing home services and their personal experience with nursing home services (r = .27), needs for nursing home services (r = .32), and the availability of family care (r = .25). A weak relationship occurred between the number of nursing home services experienced and the rest of the subscale scores within the PET. A weak relationship also was found between the number of risk factors for nursing home entry and the rest of the subscales. These weak relationships between the subscales of the PET may be because the items within the subscales Personal Experience and Risk Factors were measuring the "actual" factors for nursing home living when the rest of items were predicting the "ideal" way to provide nursing home services. The results of the Pearson correlation coefficient tests supported the convergent validity of Section II: Perception of the PET. The weak relationships among both the Personal Experience and Risk Factors subscales and the rest of the subscales within the PET may indicate the appropriateness of using these two subscales separately. However, further evaluation of the items within the subscales of Risk Factor and Personal Experience is needed, using different samples in different settings.

Discriminant Validity of the PET

Due to the large percentage of participants (49%) who never had heard of nursing homes, independent t tests were used to examine the difference between participants who had heard about nursing homes prior to the survey and those who had not. If the PET was able to discriminate these two group of participants, the discriminate validity of the PET then would be demonstrated. According to t test results, participants who had heard about nursing homes prior to the current survey had significantly higher subscale scores (i.e., Awareness, Personal Experience, Service Needs, Family Availability) than those who never had heard of nursing homes before. The results of the t tests have established the Discriminant validity of the PET because the PET was able to discriminate these two groups of participants. However, there was no statistically significant difference between these two groups of participants regarding the number of risk factors for nursing home entry reported by the participants (df= 315, t = .3, ? = .78). This indicates that whether or not the participants had heard of nursing homes before, they perceived a similar risk for nursing home entry. The results of the independent t tests are summarized in Table 4.

NURSING IMPLICATIONS

The availability of reliable and valid instruments to determine needed and desired services will contribute to the development of a more comprehensive health service system. For individuals involved with community program development, the results of a community needs assessment using the PET will demonstrate numerical evidence of a community's needs for nursing home services. A nationwide needs assessment study using the PET would contribute significantly to the development, implementation, and integration of nursing home services. It is especially important to conduct a wide-range needs assessment study using the PET in the Taiwan area because currently there are no specialized long-term care programs in the Taiwan area.

Table

TABLE 4MEANS, STANDARD DEVIATIONS, AND RESULTS OF INDEPENDENT TTEST OF THE AWARENESS SCORES, PERSONAL EXPERIENCE SCORES, SERVICES NEEDS SCORES, AND FAMILY AVAILABILITY SCORES ON PARTICIPANTS GROUPS

TABLE 4

MEANS, STANDARD DEVIATIONS, AND RESULTS OF INDEPENDENT TTEST OF THE AWARENESS SCORES, PERSONAL EXPERIENCE SCORES, SERVICES NEEDS SCORES, AND FAMILY AVAILABILITY SCORES ON PARTICIPANTS GROUPS

For health care professionals who already provide nursing home services in the Taiwan area, the PET will serve as a follow-up guide to ensure the services they provide have met their current or potential customers' demands. Nursing home service providers may use the PET to collect information regarding what kinds of nursing home services are needed by the communities close to their institutes. Results from the community needs assessment using the PET also may help service providers develop plans and strategies to improve their existing nursing homes. For example, if the results of a community needs assessment using the PET indicates a low level of awareness of nursing home services, education programs for nursing home services may be implemented. Certain types of services should be provided if they are needed and indicated by the surveyed communities. If the availability of family care is lacking in the surveyed communities, nursing home services providers may consider increasing the number of nursing home beds.

DISCUSSION AND CONCLUSIONS

Based on the results generated from a heterogeneous sample of 319 community individuals and key informants, the finalized PET is an instrument with good stability and internal consistency reliability, which has demonstrated adequate content, construct, convergent, and discriminant validity. The finalized PET consists of 59 closed-ended items and was restructured into two sections: Section I: Factors (42 items) and Section II: Perception (17 items). The first section of the PET consists of three subscales, Awareness (10 items), Personal Experience (10 items), and Risk Factors (22 items). The second section of the PET consists of two subscales, Service Needs (8 items) and Family Availability (9 items). The strength of the PET is that the underlying dimensions of Section II: Perception were established by means of the principal components factor analysis with varimax rotation, which allows for an interpretation of the construct of the tool. Another strength of the PET is that it was able to discriminate between people in Taiwanese societies who had heard of nursing homes and those who had not.

According to Norbeck (1985), the results of psychometric testing minimally should include two types of reliability testing (i.e., internal consistency, test-retest), at least one type of content validity, and at least one type of construct (or criterionrelated) validity. These also constitute minimum standards for the information required to evaluate an instrument for use in research. Thus, it is appropriate to use the PET for nursing home community needs assessment because the psychometric properties of the tool have been evaluated and established carefully.

This study also demonstrated a process of instrument evaluation which may draw some attention from Taiwanese researchers and enhance the recognition of the importance of instrument evaluation research. Although the results of this psychometric study are limited to a geographic area in Taiwan, the concepts, methods, and findings may be used as a blueprint for further research in other locations and in other populations. Use of a probability sampling method would increase the generalizability of the study findings. However, further research will be needed to provide better or refine existing services for older adults living in nursing homes.

REFERENCES

  • Chen, S.-L. (1995). Tool development of a community needs assessment for nursing homes in Taiwan. Unpublished master's thesis, University of Utah, Salt Lake City.
  • Chiou, C-J., & Wang, H.H. (1987). Self-care in the elderly population and its related factors. Kaohsiung Journal of Medical Science, 3(2), 69-78.
  • Cohen, J. (1988). Statistical power analysis for behavioral science (2nd ed.). Hillsdale, NJ: Erlbaum.
  • Comrey, A.L., & Lee, H.B, (1992). A first course in factor analysis. Hillsdale, NJ: Erlbaum.
  • DeVellis, R.F. (1991). Scale development: Theory and applications. Newbury Park, CA: Sage.
  • Dillman, D. A. (1978). MaU and telephone surveys: The total design method. New York: Wiley & Sons.
  • Fink, A., & Kosecoff, J. (1985). How to conduct surveys: A step-by-step guide. Newbury Park, CA: Sage.
  • Fowler, FJ., Jr. (1984). Survey research methods. Beverly Hills, CA: Sage.
  • Gorsuch, R.L. (1983). Factor analysa (2nd ed.). Hillsdale, NJ: Erlbaum.
  • Hon, B.-T. (1993). The needs and knowledge of health services among the community elderly. Public Health (ROC), 19(4), 353369.
  • Humg, L.-F. (1987). A study o/ hospital-oriented long-term care system. Taichung, Republic of China: Department of Medical Administration, Chinese Medical College.
  • Jung, J.-J., Tsai, C-H., & Lin, E.-J. (1990). The strain, attitude and related factors affecting the primary family caregivers who help bed-ridden patients. Family Medicine Bulletin, 8(3), 45-61.
  • Lee, K.-L, Wang, R.-G., & Jung, L.-G. (1990). Evaluation of twenty nursing homes in a suburban area. Journal of Public Health (ROC), 16(4), 416-423.
  • Lin, S.-N., Chen, J.-J., Lee, M.-B., & Lin, S. (1984). A two-year study of mental diseases of residents in long-term care facilities. Journal of Neuralgic and Psychiatric Medical Association of the Republic of China, 1(1), 47-64.
  • MisheL M.H. (1990). Methodological studies: Instrument development. In PJ. Bringk & M.J. Wood (Eds.), Advanced design in nursing research (pp. 224-284). New York: Sage.
  • Neuber, K. A. (1980). Needs assessment: A model for community planning. Beverly Hills, CA: Sage.
  • Norbeck, J.S. (1985). What constitutes a publishable report of instrument development? Nursing Research, 34, 380-381.
  • Nunnally, JC, & Bernstein, I.H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
  • Rakich, J.S., Longest, B.B., Jr., & Darr, K. (1992). Managing health services organizations (3rd ed.). Baltimore: Health Professions Press.
  • Shiu, I.-L, Shiung, B.-C, Dai, L-T, Chen, LJ., & Huang, G.-M. (1993). Characteristics of residents in long-term care facilities. Nursing Research (ROC), 1(1), 7-15.
  • Soriano, R.I. (1995). Conducting needs assessment: A multidisciplinary approach. Thousand Oaks, CA: Sage.
  • Warheit, G.J., Bell, R.A., & Schwab, J.J. (1977), Needs assessment approaches: Concepts and methods. Rockville, MD: National Institute of Mental Health.
  • Witkin, B.R., & Altschuld, J.W. (1995). Planning and conducting needs assessments: A practical guide. Thousand Oaks, CA: Sage.
  • Wu, N.-I. (1993). The needs of elder health care facilities in south Taiwan. Taipei, Taiwan, Republic of China: National Gerontology Conference Press.
  • Wu, S.-L. (1991). A survey of burdens and needs of primary famüy caregivers in the Taiwan area. Taipei, Taiwan, Republic of China: Taiwan Public Health Institution.
  • Zaura, A., Brachrach, K., & Hess, R. (1983). Strategies for needs assessment in prevention. New York: The Haworth Press.

TABLE 1

PRINCIPAL COMPONENTS FACTOR ANALYSIS WITH VARIMAX ROTATION OF SECTION II: PERCEPTION OF THE PERCEPTION EVALUATION TOOL (N = 319)

TABLE 2

MEANS, STANDARD DEVIATIONS, AND CORRELATIONS OF TIME 1 AND TIME 2 MEASURES FOR THE FACTORED SUBSCALES WITHIN SECTION II: PERCEPTION OF THE PERCEPTION EVALUATION TOOL (Af s 79)

TABLE 3

PEARSON r CORRELATIONS OF THE SUBSCALES WITHIN THE PERCEPTION EVALUATION TOOL (Af =319)

TABLE 4

MEANS, STANDARD DEVIATIONS, AND RESULTS OF INDEPENDENT TTEST OF THE AWARENESS SCORES, PERSONAL EXPERIENCE SCORES, SERVICES NEEDS SCORES, AND FAMILY AVAILABILITY SCORES ON PARTICIPANTS GROUPS

10.3928/0098-9134-20000401-08

Sign up to receive

Journal E-contents