Annals of International Occupational Therapy

Original Research 

Reliability and Responsiveness of the Pizzi Health and Wellness Assessment

Katrina M. Serwe, PhD, OTR/L; Angela L. E. Walmsley, PhD; Michael A. Pizzi, PhD, OTR/L, FAOTA

Abstract

Objective:

The Pizzi Health and Wellness Assessment (PHWA) is a unique tool that links occupational participation with client needs related to health and well-being. The assessment has established content validity; this study examined the reliability and responsiveness of the assessment to detect changes in response to an intervention.

Methods:

The PHWA was administered to 48 occupational therapy students from two universities at three time points. Assessment data from the first two time points (1 week apart) allowed assessment of the reliability of the instrument with intraclass correlation coefficients. Results from the first time point to the third time point (after an 8-week wellness intervention) were used to determine the responsiveness of the PHWA.

Results:

Significant strong to moderate intraclass correlations between weeks 1 and 2 showed good reliability. The results of paired t tests from week 1 to week 8 showed significant changes in three of the six categories (physical, family, and spiritual health), demonstrating responsiveness in detecting changes in response to the intervention.

Conclusion:

The PHWA is intended for use in both healthy and clinical populations to help adult clients to make a direct connection between occupational participation and health, well-being, and quality of life. This study was limited to a healthy population. Future research should examine the reliability and responsiveness of the PHWA in various clinical populations. [Annals of International Occupational Therapy. 2020; 3(1):7–13.]

Abstract

Objective:

The Pizzi Health and Wellness Assessment (PHWA) is a unique tool that links occupational participation with client needs related to health and well-being. The assessment has established content validity; this study examined the reliability and responsiveness of the assessment to detect changes in response to an intervention.

Methods:

The PHWA was administered to 48 occupational therapy students from two universities at three time points. Assessment data from the first two time points (1 week apart) allowed assessment of the reliability of the instrument with intraclass correlation coefficients. Results from the first time point to the third time point (after an 8-week wellness intervention) were used to determine the responsiveness of the PHWA.

Results:

Significant strong to moderate intraclass correlations between weeks 1 and 2 showed good reliability. The results of paired t tests from week 1 to week 8 showed significant changes in three of the six categories (physical, family, and spiritual health), demonstrating responsiveness in detecting changes in response to the intervention.

Conclusion:

The PHWA is intended for use in both healthy and clinical populations to help adult clients to make a direct connection between occupational participation and health, well-being, and quality of life. This study was limited to a healthy population. Future research should examine the reliability and responsiveness of the PHWA in various clinical populations. [Annals of International Occupational Therapy. 2020; 3(1):7–13.]

Quality of life is a client-based concept defined by the World Health Organization as “an individual's perception of their position in life in the context of the culture and value systems in which they live and in relation to their goals, expectations, standards and concerns” (World Health Organization, n.d., para. 2). An individual who is not actively engaged in meaningful productive occupation may become disorganized and may follow routines and habits that lead to ill health (Meyer, 1922).

With the recent commitment to health and wellness in occupational therapy, there is a strong need for both occupation- and health-based client-centered assessments that make a clear link between occupation and health, with an emphasis on well-being and quality of life as the outcomes of service delivery. Pizzi and Richards (2017) challenged the profession to commit to a paradigm shift, from occupational performance to occupational participation, and to “focus on quality of life and well-being as the primary outcomes of occupational therapy services for individuals, communities and populations” (p. 4). Further, Gillen (2013) strongly suggested a need to develop more occupation-based assessments instead of borrowing, adapting, and using those from other disciplines.

The Pizzi Health and Wellness Assessment (PHWA) is a client-centered, occupation-based instrument that links occupational participation with health, well-being, and quality of life (Pizzi, 2001; Pizzi & Richards, 2017). The assessment, which may be self-administered or facilitated by an occupational therapist, accounts for a client's perception of health. Its theoretical underpinnings are based on open systems theory (von Bertalanffy, 1968), the Health Promotion Model (Pender, 1996), and the Stages of Change Theory (Prochaska & DiClemente, 1982). It is also closely linked to a newly established clinical model that emphasizes quality of life and wellness as the outcomes of interactions among person, environment, and occupational participation (Pizzi & Richards, 2017). Most assessments used in occupational therapy focus on components, client factors, or singular skills (e.g., self-care). In addition, most have not been developed by occupational therapists and do not directly assess health, well-being, and quality of life (Scaffa, Reitz, & Pizzi, 2010). The assessment that most closely resembles the PHWA in terms of data captured is the Occupational Self-Assessment (OSA); however, this tool does not directly link health, well-being, and quality of life to occupational participation. Instead the focus of the OSA is on capturing the level of occupational competence (Baron, Kielhofner, Iyenger, Goldhammer, & Wolenski, 2006).

The PHWA has two sections. Section 1 consists of a Likert scale that provides a self-rating of health, focusing on clients' self-perception of their health in six categories. Section 2 asks three occupation-focused questions relative to clients' self-perceived level of health in each of the six categories, providing qualitative data (e.g., How does my current level of health in this area affect my day-to-day activities?). After the open-ended questions, participants are asked two additional closed-ended questions for each of the six categories: (a) their self-perceived level of competence in carrying out day-to-day activities related to each area of health and (b) their level of interest in improving that area of health and occupational participation. These two questions provide crucial information to therapists who want to be change agents for health behavior because they immediately identify the client's commitment to change and self-perceived competence in health-related participation.

The PHWA has qualitative and quantitative elements. It determines occupational and health goals and corresponding interventions that promote health and prevent disability. Previous research examined the face and content validity of the PHWA (Pizzi, 2001); however, investigation is needed to determine its reliability and responsiveness to reflect changes in response to an intervention.

The PHWA provides valuable information that is not captured by existing assessments. This study examined the reliability of the PHWA in addition to its responsiveness to change over time. The study addressed two research questions: (a) What is the reliability of the PHWA? (b) What is the responsiveness of the PHWA?

Methods

Participants

The study used a convenience sample of students enrolled in master of occupational therapy programs at a Midwestern university and an East Coast university during the summer of 2018. The study received dual institutional review board approval from the study institutions.

Study Design and Procedures

The study used a repeated measures design. Students completed the PHWA and shared demographic information on age, gender, ethnicity, and previous education via a 30-minute Qualtrics survey. They provided this information during the first class of the course, the second week of the course, and the eighth week of the course. Students received information about the study via a college classroom data collection form that outlined the purpose and process of the study and disclosed the nature of voluntary participation. Students entered identification numbers rather than names on the survey. Any students who did not want their results to be included in the research analysis could make this request by contacting a specific faculty member who was not affiliated with the course. Course instructors were blind to student participation.

Students used the results of the PHWA assessment in coursework to create a personal wellness intervention. The self-designed wellness intervention served as the process to examine the responsiveness of the PHWA assessment to change from the initial assessment to the 8-week assessment. Students began the wellness intervention after they completed the second round of the PHWA to allow for analysis of reliability.

Student Wellness Interventions

Students who were enrolled in a graduate occupational therapy program in the Midwest took an 8-week summer course, Health and Wellness Promotion for Occupational Therapy. The wellness intervention was a course assignment. Students completed the PHWA and an occupational profile in week 1 and then reflected on the results to set goals. Students were encouraged to strive for a balance of occupations to promote health and to match their interests and values. Students received the following directions to start this process.

The purpose of the Personal Wellness Journal is to guide you in promoting your own health and wellness. Each week you will have a new prompt to guide your journaling process. You will start with a personal health assessment and reflection, identify specific wellness goals, create an intervention plan guided by occupational therapy theory, and track your progress throughout the course. This journal is private; only you and the instructor can view your journal entries. Week 1: Assess your own wellness by completing an occupational profile using the American Occupational Therapy Association Occupational Profile Template and a Pizzi Health and Wellness Assessment (PHWA). Record the results in your journal and reflect on the following questions. How do your occupations of work, play, rest, and sleep balance? Does this balance promote or hinder your ability to complete occupations that match your interests and values? Based on your assessment results and reflection on these results, what are your goals to promote your wellness in the next few weeks? Write one to three measurable long-term goals that you can meet by the end of the course. Then write one to two short-term goals as a first step for each long-term goal.

In week 2, students selected an occupation-based theory to guide their personal wellness interventions. Students used their journals to reflect on the results of their assessments, goals, intervention plans, and progress each week for the next 7 weeks. The journal used by the students included prompts for reflection on their progress and goals and prompts to update their goals, modify their goals as needed, and set new goals. Journal prompts encouraged students to apply concepts learned in the course to their personal wellness interventions, including identifying psychosocial and environmental barriers to meeting their goals and making plans to overcome these barriers. In week 8, students completed a reassessment using the American Occupational Therapy Association Occupational Profile and the PHWA.

Students who were enrolled in a graduate occupational therapy program on the East Coast participated in a weekend hybrid program that included a summer course entitled Advanced Practice. Similar to the Midwest program, students were instructed to complete the PHWA and identify specific wellness goals. Using their PHWA data, students selected a health promotion theory to guide the development of a personal wellness intervention. They then implemented the self-selected interventions for 8 weeks. Interventions were based on the areas that the students wanted to improve. Students were encouraged to record reflections on their interventions, including challenges to implementation. The course assignment was a final report on the role of occupational therapy in health and wellness, the theory underlying their interventions, analysis of pre- and posttest results, and a discussion of the results. The students also wrote a reflective report similar to that of the Midwestern students, where they examined supports and barriers to their personal well-being.

The interventions were similar across student cohorts. Selected interventions included gardening, shopping with family, engaging in leisure pursuits, beginning an exercise routine, and socializing with friends and family. All students set goals and performed reflective journaling. The results of the PHWA were available for analysis only after the students had completed the course.

Data Analysis

The PHWA provides a scale for measurement of quantitative and qualitative data. The researchers did not use qualitative data in this study. This study used the three quantitative aspects of the PHWA: (a) an overall rating of self-perceived health in six domains (physical, social, family, occupational, mental/emotional, and spiritual) in which participants were asked to evaluate their level of health in each area; (b) a rating of self-perceived level of competence in carrying out day-to-day activities in each of the six domains of health; and (c) their level of interest in improving each of the six domains of health. The study analyzed quantitative data with SPSS, version 25. The analysis included only participants who completed all three assessments. The total sample size was 48 (26 from the Midwestern university and 22 from the East Coast university). Two-sample t tests were performed on the quantitative outcomes for the six overall domain ratings between the two groups to ensure that they were similar. The groups had no significant differences, allowing the participant data to be combined for analysis. In addition, histograms were examined for each item to ensure normality of the data for parametric statistical analysis.

The first six quantitative questions were analyzed with intraclass correlation coefficient (ICC) values to assess the reliability of the instrument. The quantitative portion of the instrument asks users to use a scale of 1 to 10 to rate their perceived levels of health in the following areas: social, physical, family, occupational, mental/emotional, and spiritual. With use of a test-retest approach within a very short time frame (maximum of 1 week), the ICC was calculated for each corresponding item on the instrument. This step was done exclusively to test the reliability of the instrument.

Descriptive statistics were calculated for every quantitative item on the instrument. We examined PHWA responsiveness with pretest-posttest analysis. We were interested in whether participants' self-ratings changed from the beginning of the course (before the personal wellness intervention) to the end of the course (after the personal wellness intervention). We performed paired t tests for the six key measurements of the PHWA to examine the responsiveness of the assessment to change from week 1 to week 8 of the course as the students executed their self-designed wellness interventions. The researchers used a Bonferroni correction to correct for simultaneous error rate and conducted paired t tests to examine change over time as a result of the intervention. The t tests examined the responsiveness of the assessment and were not associated with reliability.

Results

No differences were found in the demographic features of participants between the Midwestern and East Coast groups. Table 1 shows student demographic features.

Participant Demographic Features (N = 48)

Table 1:

Participant Demographic Features (N = 48)

Reliability of the instrument was high, as indicated by significant ICC values of 0.7 or higher (with the exception of social health, which had an ICC of 0.69, p < .001) between each item on the assessment from the initial assessment to week 2. The results of the ICC analysis for the six key domains were as follows: social health, ICC = 0.69, n = 48, p = .000; physical health, ICC = 0.73, n = 48, p = .000; family health, ICC = 0.74, n = 48, p = .000; occupational health, ICC = 0.75, n = 48, p = .000; mental/emotional health, ICC = 0.71, n = 48, p = .000; and spiritual health, ICC = 0.7, n = 48, p = .000.

Descriptive statistics for each item on the PHWA are shown in Table 2. Paired t tests were conducted for the six domains of the PHWA. As seen in Table 2, the findings for three of the six items were significant (p < .05). The three items that showed significant change from the beginning of the course to the end of the course were physical health, family health, and spiritual health. The three items that did not show significant change were social health, occupational health, and mental/emotional health.

Pizzi Health and Wellness Assessment: Overall Domain Rating of Self-Perceived Health, Paired t Test, Week 1 to Week 8 (N = 48)

Table 2:

Pizzi Health and Wellness Assessment: Overall Domain Rating of Self-Perceived Health, Paired t Test, Week 1 to Week 8 (N = 48)

In addition, data were analyzed from the two questions on changes in health behavior in each area: self-perceived level of competence in carrying out day-to-day activities and level of interest in improving health and occupational participation. Table 3 and Table 4 show these results.

Pizzi Health and Wellness Assessment: Self-Perceived Level of Competence, Paired t Test, Week 1 to Week 8

Table 3:

Pizzi Health and Wellness Assessment: Self-Perceived Level of Competence, Paired t Test, Week 1 to Week 8

Pizzi Health and Wellness Assessment: Level of Interest in Improving Rating, Paired t Test, Week 1 to Week 8

Table 4:

Pizzi Health and Wellness Assessment: Level of Interest in Improving Rating, Paired t Test, Week 1 to Week 8

Discussion

This study examined the reliability of the PHWA, with an ICC analysis for each domain, as well as the responsiveness of the PHWA over time for 48 participants who completed all three assessments. The finding of significant strong to moderate ICC between week 1 and week 2 for all PHWA categories showed that the PHWA had good reliability. We believe that this first assessment of the reliability of the PHWA contributes to the psychometric properties of the instrument. The PHWA is unique in that it examines the client's perception of occupational participation, health, well-being, and quality of life, which limits the ability to compare it with existing assessments. However, the reliability scores are consistent with those of other client-centered, occupation-based assessments, such as the OSA and the Canadian Occupational Performance Measures. Murad, Farnworth, and O'Brien (2012) examined the test-retest reliability of the OSA in a population of workers with a musculoskeletal injury. They found ICC values of 0.413 to 0.844 between initial assessment and a second assessment 7 to 14 days later (Murad et al., 2012). Eyssen, Beelen, Dedding, Cardol, and Dekker (2005) examined the test-retest reliability of the Canadian Occupational Performance Measures, comparing an initial assessment with a second assessment 4 to 14 days later, and found significant ICC values of 0.67 and 0.69 for mean performance and satisfaction scores, respectively, in a population of 95 adults who sought outpatient occupational therapy services for a variety of conditions.

Paired t tests from week 1 to week 8 showed significant changes in three of the six categories (physical, family, and spiritual health), demonstrating that those categories, over time, are responsive to changes. Two categories (social health and mental/emotional health) of the competence ratings and no categories of the interest ratings showed a significant change from week 1 to week 8. The question on level of interest for each section of the PHWA is based on the Stages of Change Theory (Prochaska & DiClemente, 1982). Although it has not been significant, it has practical significance when considering interventions for motivation and interest in changing a behavior. In addition, a person who has a low perception of health also may have a low sense of competence in implementing changes in behavior. This is different from one's motivation to change a behavior and is worthy of further research.

Participants rated social health the lowest of the six domains in their level of interest ratings, yet showed significant improvement from pretest to posttest in self-perceived level of competence for completing activities related to social health, but not for overall domain ratings of self-perceived social health. However, level of interest ratings were all very high, with a range of 7.85 to 9.17. In addition, no domain on this rating scale had changes from pretest to posttest.

Both the overall domain rating of self-perceived health and the self-perceived level of competence showed significant changes from pretest to posttest; however, different domains of health showed changes in each domain. The overall domain rating of self-perceived health showed significant change from pretest to posttest for physical, family, and spiritual health. Self-perceived level of competence ratings showed significant changes from pretest to posttest for social and mental/emotional health.

No significant changes in level of interest ratings were reported in any category; however, increasing interest in health was not the target of the intervention. The objective was improving overall health and wellness through occupational balance. The occupational therapy students who participated in this study are likely to have a high interest in promoting health and wellness, as reflected in their high initial mean scores for each category in level of interest, with a range of mean scores. This set of questions had the highest ratings. Lack of significant change from week 1 to week 8 on level of interest ratings may reflect a ceiling effect for this group of participants.

Implications for Practice

The PHWA has good psychometric properties and focuses on health, well-being, and quality of life. Unlike current assessments used in occupational therapy, data from the assessment are primarily concerned with the client's perception of health and well-being and its influence on occupational participation versus performance of tasks or components. It also focuses on goals and changes in self-selected client health behavior to improve participation. The PHWA directly links occupational participation with the health and well-being needs of clients. In addition, it is focused on occupation and the client, with information gathered through occupational questions related to each area of health. This study of the PHWA, involving a healthy population, indicated that the assessment has good test-retest reliability and that it can detect changes in response to an intervention. The PHWA assists in helping people to make the health behavior and occupational changes that are necessary for sustained health, well-being, and quality of life. The PHWA should be used in conjunction with other assessments to assist clients in making lifestyle choices to promote health and well-being.

Limitations and Future Research

This study was limited by a small sample size and a population of occupational therapy students. The data in this study showed good test-retest reliability between week 1 and week 2. Analysis of data from week 1 to week 8 was used to show changes, assuming that reliability was already established with the ICC values from week 1 to week 2. Future studies could use a larger sample size and other populations to confirm the reliability first seen in this study.

Future studies are needed to examine the responsiveness of the assessment to detect changes in various clinical populations. Further research is needed with various populations to best determine the importance of this assessment for research and clinical use. Although the six health domains have proven to be relevant, further studies may show other information that can be used to improve the tool. Future research should also examine the assessment in both clinical and general populations to determine whether there are any ceiling effects.

International research is needed to examine potential cultural differences in assessment responses. In addition, further qualitative and mixed methods studies can examine the use of information obtained from open-ended items.

Conclusion

To adapt to the changes in health care systems around the world, occupational therapists must be authentic practitioners who are in tune with the core value of occupation (Lamb, 2016). This aligns with the same concept emphasized by the early leaders of occupational therapy 50 years ago. Adapting to changing health care needs includes changing assessment methods and the process of assessment and developing occupation-focused, client-centered health assessments that help to validate the role of occupational therapy in supporting the well-being and quality of life of individuals, communities, and populations. This study provides evidence that the PHWA shows good reliability using ICC analysis and is responsive to changes. The assessment can help adult clients to make a direct connection between occupational participation and their health, well-being, and quality of life.

References

  • Baron, K., Kielhofner, G., Iyender, A., Goldhammer, V. & Wolenski, J. (2006). A user's manual for the occupational self assessment (OSA) [Version 2.2]. Chicago, IL: Model of Human Occupational Clearinghouse, Department of Occupational Therapy, College of Applied Health Sciences, University of Illinois at Chicago.
  • Eyssen, I. C., Beelen, A., Dedding, C., Cardol, M. & Dekker, J. (2005). The reproducibility of the Canadian occupational performance measure. Clinical Rehabilitation, 19(8), 888–894. doi:10.1191/0269215505cr883oa [CrossRef]16323388
  • Gillen, G. (2013). A fork in the road: An occupational hazard?American Journal of Occupational Therapy, 67, 641–652. doi:10.5014/ajot.2013.676002 [CrossRef]24195898
  • Lamb, A. J. (2016). The power of authenticity. American Journal of Occupational Therapy, 70, 7006130010. doi:10.5014/ajot.2016.706002 [CrossRef]27767937
  • Meyer, A. (1922). The philosophy of occupation therapy. Archives of Occupational Therapy, 1, 1–10.
  • Murad, M. S., Farnworth, L. & O'Brien, L. (2012). Psychometric properties of occupational self-assessment for injured workers with musculoskeletal disorders. Social and Behavioral Sciences, 42, 507–517. doi:10.1016/j.sbspro.2012.04.217 [CrossRef]
  • Pender, N. J. (1996). Health promotion in nursing practice (3rd ed.). Stamford, CT: Appleton & Lange.
  • Pizzi, M. (2001). The Pizzi holistic wellness assessment. Occupational Therapy in Health Care, 13(3–4), 51–56. doi:10.1080/J003v13n03_06 [CrossRef]23944261
  • Pizzi, M. A. & Richards, L. G. (2017). Promoting health, well-being, and quality of life in occupational therapy: A commitment to a paradigm shift for the next 100 years. American Journal of Occupational Therapy, 71(4), 7104170010. doi:10.5014/ajot.2017.028456 [CrossRef]28661379
  • Prochaska, J. O. & DiClemente, C. C. (1982). Transtheoretical therapy: Toward a more integrative model of change. Psychotherapy, 19(3), 276–288. doi:10.1037/h0088437 [CrossRef]
  • Scaffa, M.E., Reitz, S. M. & Pizzi, M. A. (2010). Assessments for health promotion practice. In Occupational therapy in the promotion of health and wellness (pp. 173–194). Philadelphia, PA: F. A. Davis.
  • von Bertalanffy, L. (1968). General system theory. New York, NY: George Braziller.
  • World Health Organization. (n.d.). WHOQOL: Measuring quality of life. Retrieved from http://www.who.int/healthinfo/survey/whoqol-qualityoflife/en

Participant Demographic Features (N = 48)

Characteristicn (%)
Gender
  Female47 (97.9)
  Male1 (2.1)
Ethnicity
  White39 (81.2)
  Asian4 (8.3)
  African American3 (6.3)
  Other2 (4.2)
Education level
  Associate's degree2 (4.2)
  Bachelor's degree43 (89.5)
  Master's degree3 (6.3)
College region
  Midwest26 (54.2)
  East Coast22 (45.8)

Pizzi Health and Wellness Assessment: Overall Domain Rating of Self-Perceived Health, Paired t Test, Week 1 to Week 8 (N = 48)

Pizzi Health and Wellness Assessment categoryPretestPosttesttp
MSDMSD
Social health7.791.1298.021.604−0.876.386
Physical health6.901.4777.631.496−3.746.000*
Family health7.501.6638.211.487−3.125.003*
Occupational health7.791.3688.041.557−1.107.274
Mental/emotional health6.981.4367.561.583−2.280.027
Spiritual health7.151.5718.081.499−4.953.000*

Pizzi Health and Wellness Assessment: Self-Perceived Level of Competence, Paired t Test, Week 1 to Week 8

Pizzi Health and Wellness Assessment categorynPretestPosttesttp
MSDMSD
Social health477.681.1818.090.996−3.163.003*
Physical health477.551.7047.681.617−0.724.473
Family health477.621.6237.451.5300.814.420
Occupational health477.601.5277.811.345−1.258.215
Mental/emotional health476.891.5647.621.636−4.038.000*
Spiritual health467.851.6607.911.684−0.322.749

Pizzi Health and Wellness Assessment: Level of Interest in Improving Rating, Paired t Test, Week 1 to Week 8

Pizzi Health and Wellness Assessment categorynPretestPosttesttp
MSDMSD
Social health477.851.9457.981.984−0.643.523
Physical health479.041.3989.171.049−0.924.360
Family health478.771.6188.491.8401.265.212
Occupational health478.771.5358.911.412−0.730.469
Mental/emotional health479.171.2399.091.0600.628.533
Spiritual health468.391.9948.831.554−1.511.138
Authors

Dr. Serwe is Associate Professor, Department of Occupational Therapy, Occupational Therapy Program, Concordia University Wisconsin, Mequon, Wisconsin. Dr. Walmsley is Professor of Mathematics and Research Methodologist, Concordia University Wisconsin, Mequon, Wisconsin. Dr. Pizzi is Associate Professor, Department of Occupational Therapy, Dominican College, Orangeburg, New York.

The authors have no relevant financial relationships to disclose.

The authors are thankful for the support of the occupational therapy students who agreed to participate in this research.

Address correspondence to Katrina M. Serwe, PhD, OTR/L, Associate Professor, Department of Occupational Therapy, Occupational Therapy Program, Concordia University Wisconsin, 12800 N. Lake Shore Drive, Mequon, WI 53097; e-mail: katrina.serwe@cuw.edu. All inquiries regarding use of the PHWA should be sent to Dr. Michael Pizzi at phwa58@gmail.com.

Received: April 30, 2019
Accepted: July 23, 2019
Posted Online: September 23, 2019

10.3928/24761222-20190910-04

Sign up to receive

Journal E-contents