Health care in New Zealand comprises both government (public) and private systems and services across primary, tertiary, and secondary care. Like citizens in other industrialized countries, New Zealanders have poor health literacy skills, and Māori (the indigenous population of New Zealand comprising approximately 15% of New Zealand's total population) are particularly affected (Ministry of Health, 2010), contributing to significant inequities in health outcomes. Addressing health literacy is, therefore, an important government and organizational priority for health and care services in New Zealand and globally.
In health care, our understanding of health literacy has evolved from being focused on consumer capacity (i.e., consumer knowledge and skills) to a more systemic model that considers the social and organizational factors that affect a person's capacity “to find, interpret and use information and health services to make effective decisions for health and wellbeing” (Ministry of Health, 2015, p. 1).
D'Eath et al. (2012) highlight the paucity of relevant and high-quality research that clearly delineates which health literacy interventions are most effective at reducing disparity and inequity in health. However, what is clearly understood is that concerted effort across all sectors is required to improve health literacy by reducing health literacy demands, better supporting how consumers access and navigate services, and improving our communication of health information (Ministry of Health, 2010). This demands a focus on the health system as wells as organizational change (Ministry of Health, 2015).
The Rauemi Atawhai (RA) (Māori words meaning “resources” [Rauemi] that are “supportive and enable care” [Atawhai]) Program, based on the Ministry of Health's Rauemi Atawhai Framework (Ministry of Health, 2012) and delivered by Health Literacy New Zealand, is a professional development program that was delivered to health care professionals at a New Zealand district health board. This is 1 of 20 boards responsible for the funding and provision of health care services in New Zealand within defined geographical boundaries. This program was delivered at Counties Manukau Health (CM Health). It aims to develop the capability of health care professionals to recognize and develop health literate, culturally competent health education resources (henceforth referred to as “consumer resources”) and help build supportive systems and processes for ensuring the availability and accessibility of resources within health and care services. The program was delivered to help CM Health deliver on their strategic aim to become a health literate organization (Counties Manukau Health, 2015).
Existing literature on similar programs was limited to describing perceived value in training and changes in the quality of consumer resources (for example, Demir et al., 2009). Although such research recognizes that supporting health care professionals to prepare consumer resources is a useful practice, it has not considered or measured outcomes related to the organizational design or refinement of systems or processes for consumer resource development.
This article intends to extend the current evidence regarding program or training to support health literacy by taking a dual focus on the programmatic learning to support both consumer resource development and building organizational systems and processes.
This evaluation aimed to determine the effectiveness of the RA program with a focus on (1) learning outcomes of program participants, (2) systems and process change on wards/services, and (3) future opportunities to improve health literacy, with attention to organizational leadership and management approaches to address health literacy.
Context and Intervention
The RA Program consisted of three workshops and ongoing support and feedback delivered over a 3-month period from March to May 2017. Participants attended with consumer resources that they redesigned over the course of the program. Sessions focused on system requirements for better resource development, benefits and limitations of written resources, guiding principles and processes for resource development, and more.
The training aimed to support staff in (1) identifying and reviewing the key health education resources currently used in their services; (2) developing a system for developing, reviewing, disseminating, and evaluating written health education resources for their service, (3) using the process described in “Rauemi Atawhai - A guide to developing health education resources in New Zealand” (Ministry of Health, 2012); and (4) developing team and service approaches for engaging patients and families in relation to written health education resources.
A mixed-methods pre- and post-test with follow-up evaluation design was applied, involving three participant groups: program coordinators, program participants, and senior leaders.
The measures used in this study are shown in Table 1.
Summary of Evaluation Measures and Tools Used in the Rauemi Atawhai Program
An intelligent transcript of each interview or focus group was developed. The transcripts were thematically analyzed. Pre- and post-scaled survey responses were tested for difference in proportion of responses, and whether these differences were statistically significant using the McNemar's exact test. The analysis was carried out on complete cases for the paired nominal data.
This evaluation was approved by the New Zealand Ethics Committee.
Program participants were selected for their demonstrated interest and experience in developing consumer resources within their service. Further information about the demographics is provided in Table 2.
Participants consolidated their understanding, or built a more comprehensive understanding, of health literacy as a result of program participation. Five of seven (71%) post-program evaluation participants self-reported an improvement in their understanding of health literacy. Post-program participants more often emphasized systemic understandings of health literacy:
I know it very much [that health literacy] depends on the interaction between the individual and the system … it's really the quality of the interaction is how I see it, and that's why we've tried to turn it around from focused on the individual, you know the level of capability of the individual, to the way the system deals with individuals, so we're talking about health literate organisations and systems as opposed to judging someone as having no health literacy or whatever level of health literacy
Participants also reported an enhanced recognition and awareness of health literacy in their work. They gained a better understanding of the importance of consumer or community feedback, perspectives, and insights when developing consumer resources:
So one of the real key points that surfaced in this learning was that it's easy for us to assume that we really know what's going on with our whānau [a Māori language term describing a kinship group that includes family but may also extend further to include friends. Whānau is considered the primary economic and familial unit in traditional Māori society] and it's easy for us to start believing that we can really have, speak and have their voice . . . But we are still a part of the institution, we are still a part of the problem realistically. We can't assume that we have that voice irrespective of whatever commonalities we have . . . My voice is not the voice that needs to be heard
Due to small sample size, qualitative findings were not reflected in quantitative pre- and post-health literacy survey responses, none of which reached statistical significance. The RA Program was an impetus for the continued socialization of health literacy approaches and the RA framework, goal setting around individual or service improvements, and role-modelling behaviors that support consumer health literacy.
System and Service Changes
Overall, the RA Program had limited influence on the design and refinement of systems for developing, reviewing, disseminating, and evaluating written health education resources for their service, and on further approaches for engaging patients and family in design and review during the evaluation period.
Before the RA Program, 50% of evaluation participants reported that they did not have an existing process or system for the development of consumer resources in their service. Further, 5 of 8 participants (63%) reported having no established system or process for selecting, accessing, and storing consumer resources. As described by one participant:
[The resources], they were literally just in the drawers here. We couldn't find them on the computer system so I don't even know where they came from, who made them… nothing.
At 3 to 4 months after the RA Program, 1 participant (of 8) had made changes to the system or process for developing consumer resources within their team or service. The following two barriers to action prevented participants from making changes: (1) securing time to undertake the work, and (2) lack of awareness and/or prioritization of health literacy within their team or service.
Beyond RA Program actions, broader organizational barriers to becoming a health literate organization were also identified by participants and program coordinators. These included inadequate funding and resourcing of health literacy work, poor organizational accountability for health literacy, lack of awareness and prioritzation of health literacy, lack of consistency in processes and systems, and having no identification system for recognizing good and poor quality resources.
This evaluation shows that the RA Program offers staff great learning and the provision of a framework for consumer resource development; however, several barriers to systems and process change identified by participants highlight the importance of addressing systemic and organizational factors as well as staff skills.
Recommended future improvements highlight both programmatic and organizational changes to address health literacy. Programmatically, we recommend reviewing the scope and focus of the program, which requires improved alignment with targeted program participants, specifically to ensure relevance and inclusion of staff who are able to influence systems and processes at an organizational level. We recommend champion models be incorporated in the program design to enhance participant actions around socializing the approach, role-modelling, and goal-setting. There is also a need to increase the relevance and quantity of the RA Program outputs through improved enforcement of program eligibility requirements (i.e., needing an identified resource for [re]development) to ensure practical application of the framework to a relevant consumer resource. Finally, we recommend enhancing participant networking, support pathways, and long-term participant follow-up to understand long-term changes to systems or processes within services.
Many of the necessary changes to systems and processes that are fundamental to becoming a health-literate organization are beyond the current resource, scope, and influence of the RA Program and its participants. This evaluation points to the need for increased resourcing of our organizational strategies for building health literate systems; improved organizational accountability for health literacy; and supportive information technology platforms to ensure ease of accessibility to consumer resources, ease of identification of quality consumer resources, and consistent approaches for the development of consumer resources.
The RA Program demonstrates value in extending participant knowledge and awareness of health literacy and implications of health literacy in their daily clinical or managerial practice. Further, the RA Program also demonstrates value in enabling staff to think systemically about health literacy, but it was limited in leveraging systems or process change within the evaluation period. Critically, leaders and managers need to participate in capability building and discussions to create conditions (e.g., resource and authorization) for change in the environments in which staff work.
- Abrams, M. A., Kurtz-Rossi, S., Riffenburgh, A. & Savage, B. A. (2014). Building health literate organizations: A guidebook to achieving organizational change. UnityPoint Health. https://www.unitypoint.org/filesimages/Literacy/Health%20Literacy%20Guidebook.pdf
- Counties Manukau Health. (2015). Healthy together: Strategic plan 2015–2020. http://www.countiesmanukau.health.nz/assets/Uploads/CM-Health-Strategic-Plan-April-2016.pdf
- D'Eath, M., Barry, M. M. & Sixsmith, J. (2012). A rapid evidence review of interventions for improving health literacy. European Center for Disease Prevention and Control. https://ecdc.europa.eu/sites/portal/files/media/en/publications/Publications/1205-TER-Improving-Health-Literacy.pdf
- Demir, S. G., Bulut, H. & Dal, Ü. (2009). Student's experience with developing patient education materials. Procedia: Social and Behavioral Sciences, 1(1), 2828–2831 doi:10.1016/j.sbspro.2009.01.503 [CrossRef]
- French, M. & Hernandez, L. M. (2013). Organizational change to improve health literacy: Workshop summary. National Societies of Sciences, Engineering, and Medicine. https://www.nap.edu/catalog/18378/organizational-change-to-improve-health-literacy-workshop-summary
- Lambert, M., Luke, J., Downey, B., Crengle, S., Kelaher, M., Reid, S. & Smylie, J. (2014). Health literacy: Health professionals' understandings and their perceptions of barriers that Indigenous patients encounter. BMC Health Services Research, 14(1), 614 doi:10.1186/s12913-014-0614-1 [CrossRef] PMID:25471387
- Ministry of Health. (2010). Kōrero Mārama: Health literacy and Māori results from the 2006 Adult Literacy and Life Skills Survey. https://www.health.govt.nz/publication/korero-marama-health-literacy-and-maori-results-2006-adult-literacy-and-life-skills-survey
- Ministry of Health (2012). Rauemi Atawhai – A guide to developing health education resources in New Zealand. http://www.health.govt.nz/publication/rauemi-atawhai-guide-developing-health-education-resources-new-zealand
- Ministry of Health. (2015). A framework for health literacy. https://www.health.govt.nz/publication/framework-health-literacy
- Rootman, I. & Gordon-El-Bihbety, D. (2008). A vision for a health literate Canada. Canadian Public Health Association. https://swselfmanagement.ca/uploads/ResourceDocuments/CPHA%20(2008)%20A%20Vision%20for%20a%20Health%20Literate%20Canada.pdf
- Shoemaker, S. J., Staub-DeLong, L., Wasserman, M. & Spranca, M. (2013). Factors affecting adoption and implementation of AHRQ health literacy tools in pharmacies. Research in Social & Administrative Pharmacy, 9(5), 553–563 doi:10.1016/j.sapharm.2013.05.003 [CrossRef] PMID:23759672
- Shoemaker, S. J., Wolf, M. S. & Brach, C. (2014) The Patient Education Materials Assessment Tool (PEMAT) and user's guide. Agency for Healthcare Research and Quality. https://www.ahrq.gov/ncepcr/tools/self-mgmt/pemat.html
- Weaver, N. L., Wray, R. J., Zellin, S., Gautam, K. & Jupka, K. (2012). Advancing organizational health literacy in health care organizations serving high-needs populations: A case study. Journal of Health Communication, 17(Suppl. 3), 55–66 doi:10.1080/10810730.2012.714442 [CrossRef]
Summary of Evaluation Measures and Tools Used in the Rauemi Atawhai Program
|Measure||Before program||After program||Purpose|
|Patient Education Materials Assessment Tool(Shoemaker et al., 2014)||Yes||Yes||To assess the readability and actionability of consumer health resources|
|Ministry of Health literacy survey (condensed) (Ministry of Health, 2015)||Yes||Yes||To assess changes in program participant knowledge and understanding of health literacy|
|Semi-structured interviews (participants)||Yes||Yes||To explore key learning and program experiences|
|Semi-structured interviews (senior leaders, N = 2)||No||Yes||To gain leadership perspectives on organizational approaches to creating health literate services|
|Focus group (program coordinators (N = 3)||No||Yes||To contribute to the discussion around future opportunities to improve organizational health literacy and identify barriers to improved health literacy|
|Participation area||N||%||Further details|
|Total RA program participants||15||100||Participants came from various roles, such as community midwives, nurse specialists, occupational therapists, technicians, and three staff in management positions (across seven services at CM Health).|
|Agreed to participate||9||60||RA program participants who agreed to participate in the evaluation|
|Completed pre-program interview/survey||8||53||Identified their pre-program understanding of health literacy and consumer resources|
|Completed post-program interview/survey||7||46||Provided evaluation feedback on their program experience and learning outcomes|