HLRP: Health Literacy Research and Practice

Original Research-Qualitative Open Access

Understanding and Advancing Organizational Health Literacy Within a Public Health Setting

Natalie Kružliaková, PhD, RD, LD; Kathleen Porter, PhD, RD; Pamela A. Ray, DVM, MBA, MPH; Valisa Hedrick, PhD, RD; Donna Jean Brock, MS; Jamie Zoellner, PhD, RD

  • HLRP: Health Literacy Research and Practice. 2021;5(1):e35-e48
  • https://doi.org/10.3928/24748307-20210114-01
  • Posted February 13, 2021

Abstract

Background:

Organizational health literacy (OHL) within the public health setting is lacking.

Objective:

The aim of this study was to form a health literacy (HL) improvement team consisting of university researchers and Virginia Department of Health (VDH) district directors and staff to assess and improve OHL practices of VDH staff in four medically underserved health districts in southwest Virginia.

Methods:

The Agency for Healthcare Research and Quality Health Literacy Universal Precautions Toolkit guided this mixed-methods needs assessment and improvement plan. VDH staff completed a 44-item survey adapted from this Toolkit and a roundtable discussion to indicate their perceptions of current OHL practices. VDH clients completed a survey including seven items measuring perceptions of staff OHL practices and three items measuring subjective HL.

Key Results:

About one-half of VDH staff (n = 252, 88% female, average age 49 ± 12 years, 23% ≤ high school education [HS]) reported “doing well” across all OHL domains. Staff survey and roundtable discussion revealed the need to strengthen the written communication domain. Among 185 VDH clients (82% female, average age 33 ± 14 years, 40% ≤ HS), perceptions of staff OHL practices were high, ranging from 3.07 to 3.64 (scale of 1–4). Client HL status was significantly positively correlated (p < .01–.05) with 5 of 7 OHL practices. Findings aided development and initial implementation of an OHL improvement plan, including e-newsletters and in-person workshops. On average, 60% of staff opened quarterly e-newsletters. Staff ratings of the Clear Communication Index workshop were high in terms of utility and applicability of content.

Conclusions:

Results reflected notable strengths and weaknesses in current OHL practices from staff and client perspectives, with the greatest need identified in written communication. E-newsletter series and in-person workshops on the Clear Communication Index helped lay groundwork for additional HL improvement activities for VDH staff. Limitations and future recommendations for public health settings are discussed. [HLRP: Health Literacy Research and Practice. 2021;5(1):e35–e48.]

Plain Language Summary:

This study describes use of the Agency for Healthcare Research and Quality Health Literacy Universal Precautions Toolkit to conduct an organizational health literacy needs assessment and improvement plan in a public health setting, the Virginia Department of Health. Assessment of staff and clients revealed strengths and weaknesses in organizational health literacy practices. Feedback guided efforts to improve organizational health literacy capacity.

Abstract

Background:

Organizational health literacy (OHL) within the public health setting is lacking.

Objective:

The aim of this study was to form a health literacy (HL) improvement team consisting of university researchers and Virginia Department of Health (VDH) district directors and staff to assess and improve OHL practices of VDH staff in four medically underserved health districts in southwest Virginia.

Methods:

The Agency for Healthcare Research and Quality Health Literacy Universal Precautions Toolkit guided this mixed-methods needs assessment and improvement plan. VDH staff completed a 44-item survey adapted from this Toolkit and a roundtable discussion to indicate their perceptions of current OHL practices. VDH clients completed a survey including seven items measuring perceptions of staff OHL practices and three items measuring subjective HL.

Key Results:

About one-half of VDH staff (n = 252, 88% female, average age 49 ± 12 years, 23% ≤ high school education [HS]) reported “doing well” across all OHL domains. Staff survey and roundtable discussion revealed the need to strengthen the written communication domain. Among 185 VDH clients (82% female, average age 33 ± 14 years, 40% ≤ HS), perceptions of staff OHL practices were high, ranging from 3.07 to 3.64 (scale of 1–4). Client HL status was significantly positively correlated (p < .01–.05) with 5 of 7 OHL practices. Findings aided development and initial implementation of an OHL improvement plan, including e-newsletters and in-person workshops. On average, 60% of staff opened quarterly e-newsletters. Staff ratings of the Clear Communication Index workshop were high in terms of utility and applicability of content.

Conclusions:

Results reflected notable strengths and weaknesses in current OHL practices from staff and client perspectives, with the greatest need identified in written communication. E-newsletter series and in-person workshops on the Clear Communication Index helped lay groundwork for additional HL improvement activities for VDH staff. Limitations and future recommendations for public health settings are discussed. [HLRP: Health Literacy Research and Practice. 2021;5(1):e35–e48.]

Plain Language Summary:

This study describes use of the Agency for Healthcare Research and Quality Health Literacy Universal Precautions Toolkit to conduct an organizational health literacy needs assessment and improvement plan in a public health setting, the Virginia Department of Health. Assessment of staff and clients revealed strengths and weaknesses in organizational health literacy practices. Feedback guided efforts to improve organizational health literacy capacity.

Health literacy (HL) is the degree to which people have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions (Ratzan & Parker, 2000). Low HL affects more than 36% of adults in the United States and is consistently associated with poorer self-reported health status, less use of preventive services, and inability to manage chronic conditions (Kutner, 2006). The complexity of the U.S. health care system and its processes and procedures pose significant challenges for patients, which can be further exacerbated by low HL (Baker, 2006; Koh et al., 2013). Consequently, HL was pinpointed as a national priority area at the turn of the 21st century (Adams & Corrigan, 2003). Over the past decade, the need for systems-level efforts to alleviate the burdens of low HL has been recognized (Brach et al., 2012; Kindig et al., 2004).

The term organizational health literacy (OHL) is defined as an organization-wide effort to make it easier for individuals to navigate, understand, and use information and services to better care for their health (Brach, 2017; Brach et al., 2012). Health care systems that embody OHL characteristics may have greater influence on patient health behaviors and outcomes, regardless of HL status (Kaphingst et al., 2014). Over 20 OHL toolkits and guides are available, and many have been used over the past decade to increase OHL within health care organizations of various settings. However, guides and OHL implementation specific to the public health setting are lacking (Farmanova et al., 2018).

One such guide, the Agency for Healthcare Research and Quality (AHRQ) Health Literacy Universal Precautions (HLUP) Toolkit was developed and found to aid primary care settings in reducing the complexity of the health care system by applying a universal precautions approach (i.e., treating all patients as if they have risk factors of not understanding health information) (DeWalt et al., 2011). The AHRQ-HLUP Toolkit includes 21 evidence-based tools that represent at least 1 of the 4 domains of HL: oral communication, written communication, self-management and empowerment, and supportive systems (DeWalt et al., 2011). This toolkit is one of few that addresses each of the ten attributes of health literate health care organizations, embodies key characteristics of quality improvement, and demonstrates adaptability across settings (Brach et al., 2012; Callahan et al., 2013; Farmanova et al., 2018; Kripalani et al., 2014). Given this, the AHRQ-HLUP Toolkit was the optimal choice for this study.

This study took place within the Virginia Department of Health (VDH). VDH provides essential services including health education and promotion, preventive health care, occupational and environmental health, and emergency preparedness. OHL principles are broadly applicable within this local health district setting and should be incorporated within each discipline. The challenge for public health settings exists in increasing the capacity for organizations to provide care that addresses HL needs in a manner suitable across disciplines (Rudd et al., 2013).

This study included four VDH districts (Lenowisco, Cumberland Plateau, Mount Rogers, New River) in Appalachian southwest Virginia (SWVA). This region is prone to significant health disparities and socioeconomic disadvantages, lower education and literacy levels, and poorer Health Opportunity Index (HOI) scores compared to other Virginia areas (Pollard, 2016; Virginia Department of Health, Office of Health Equity, n.d.; U.S. Department of Health & Human Services, n.d.; U.S. National Agricultural Library, 2014). This study demonstrates a partnership among VDH district directors and staff and an academic research team from the University of Virginia (UVA) and Virginia Tech (VT). The aims were to (1) perform a needs assessment using adapted AHRQ-HLUP Toolkit instruments to identify perceptions in OHL practices from both VDH staff and clients across the four domains of OHL and (2) describe how needs assessment findings were used to collaboratively develop and implement an OHL improvement plan. In turn, these aims supported an overarching goal of this project: to use strengthened OHL capacity of VDH to support implementation of SIPsmartER, a 6-month intervention developed using HL strategies and guided by the Theory of Planned Behavior consisting of small-group classes, one live Teach-Back call, and 11 interactive voice response calls proven effective at reducing sugar-sweetened beverage consumption in adults living in southwest Virginia (Zoellner et al., 2016). SIPsmartER effectiveness and implementation findings have been published elsewhere (Zoellner et al., 2016; Porter et al., 2019).

Methods

Study Design

This study details a mixed-methods needs assessment and multifaceted OHL improvement plan. The needs assessment included a (1) staff survey, (2) staff roundtable discussion, and (3) client survey. The improvement process included (1) professional development kick-off training, (2) an e-newsletter series, and (3) in-person workshops on the Clear Communication Index (CCI). Figure 1 details the study timeline. The study protocol was approved by the UVA, VT, and VDH Institutional Review Boards; VDH staff and clients provided consent prior to participation.

Study timeline and components.

Figure 1.

Study timeline and components.

Formation of the Health Literacy Improvement Team

The partnership between VDH and the academic research team was born from a shared desire to reduce health disparities by building OHL capacity of VDH and expanding evidence-based programming. To ensure key stakeholders were on board, the research team engaged organizational decision-makers (i.e., health district directors) and front-line staff (e.g., health educators, nurses). Framed by AHRQ-HLUP Toolkit Tool 1 “Form a Team,” the focus of initial meetings was to establish an HL improvement team, determine best uses of the HLUP Tool-kit, and collaboratively adapt needs assessment instruments. Adapted staff and client surveys were revised per feedback from VDH district directors to generalize questions to ensure applicability to all VDH units or disciplines (i.e., both clinical and nonclinical staff). For example, language was modified (e.g., “patient” was changed to “client,” “clinician” changed to “staff,” and “medical attention” was changed to “services”). Guided by AHRQ-HLUP Toolkit 2 “Create a Health Literacy Improvement Plan,” later meetings focused on interpretation of needs assessment findings and development of improvement plan activities.

Aim 1: Needs Assessment

Staff survey. Staff perceptions of OHL practices were assessed in April 2016 using the adapted 44-item AHRQ Primary Care Health Literacy Assessment Tool (see https://figshare.com/s/7a248ea9eb48780c3017 for Appendix 1). This survey included (1) seven individual-level questions that asked staff to rank how often they participate in certain OHL practices on the job and (2) 37 unit-level questions that asked staff to rate how well their unit engages in OHL practices. Demographic characteristics were self-reported. The survey was delivered through Training Finder Real-time Affiliate Integrated Network (TRAIN) Virginia, a web-based learning portal used by VDH staff to complete online trainings and surveys. Inclusion criteria required respondents to be at least 18 years old and English-speaking. Staff were required by their VDH district director to complete the survey (but chose whether to consent to sharing their responses for research purposes).

Staff roundtable discussion. At the kick-off training in May 2016, additional staff perceptions of OHL practices and training needs were gathered. Attendees were grouped by job unit and seated eight per table. A modified World Café approach was used to stimulate collaborative dialogue around purposeful questions (The World Cafe, 2017). Staff answered four open-ended questions in groups (Table 1), recorded their discussions on poster-size paper, and reported out to the audience. Posters were photographed for documentation.

VDH Staff Responses from the Kick-Off Training Roundtable DiscussionaVDH Staff Responses from the Kick-Off Training Roundtable DiscussionaVDH Staff Responses from the Kick-Off Training Roundtable Discussiona

Table 1:

VDH Staff Responses from the Kick-Off Training Roundtable Discussion

Client Survey. To assess client perceptions of VDH OHL practices, clients were guided by trained VDH staff from each district to complete a paper-based survey between July and September of 2016 (see https://figshare.com/s/8488c6cf2e55bcc084e3 for Appendix 2). Clients completed surveys in multiple VDH localities (e.g., clinics, work sites, reception areas, Special Supplemental Nutrition Program for Women, Infants, & Children [WIC] office). Seven questions (see https://figshare.com/s/8488c6cf2e55bcc084e3 for Appendix 2, questions 7–12) were adapted from the Health Literacy Responsiveness of Primary Care Practices screener and the AHRQ Health Literacy Patient Survey (Altin et al., 2015; Brega, 2015). Subjective HL status was also assessed using three questions (Jeppesen et al., 2009; Morris et al., 2006; Wallace et al., 2006) (see https://figshare.com/s/8488c6cf2e55bcc084e3 for Appendix 2, questions 3–5). Inclusion criteria required that clients be at least 18 years old, English-speaking, and to have received VDH services within the past year.

Aim 2: Improvement Process

Professional development kick-off training. In May 2016, staff from all four VDH districts participated in a 5-hour staff kick-off training. The purpose of this researcher-led training, guided by HLUP Toolkit Tool 3 “Raise Awareness,” was to raise awareness of the importance of OHL, inform staff of implementing a universal precautions approach when interacting with clients, and prepare for future improvement activities.

E-newsletter series. The quarterly email e-newsletter series entitled Health Literate SWVA was developed using tools from the AHRQ-HLUP Toolkit (Figure 1) resources (e.g., Centers for Disease Control and Prevention, 2016 [CDC]; Health Resources & Services Administration, 2019; the National Library of Medicine, 2019). VDH staff received an e-newsletter delivered through an email marketing service. Launched in October 2016, the e-newsletters used needs assessment results to provide HL techniques staff could use for improving client experiences and outcomes. While researchers developed the newsletters, the HL improvement team reviewed newsletters prior to release. The email listserv was continually updated to include new hires.

Clear communication workshop. The CCI is a tool with four open-ended and 20 quantitative questions used to assess, develop, and evaluate public communication materials. The CDC developed the CCI to help users and distributors of materials comply with the Plain Language Act and the National Action Plan to Improve Health Literacy. As part of the HL improvement plan, the research team designed and implemented a 90-minute in-person staff development CCI workshop for each VDH district. The intent of this workshop was to help staff incorporate easy-to-implement OHL strategies to improve client communication and comprehension. The first 30 minutes were didactic, and the final hour was interactive. VDH district leaders provided frequently used written communication materials to ensure workshop content was relatable. To work with existing organizational structure and decrease staff burden, each workshop was held during an already-planned staff development day in each district. At the conclusion of each workshop, staff completed a paper-based survey developed by the research team to assess the utility of the workshop as part of the OHL improvement plan (see https://figshare.com/s/a7bc3936d0ecf67f4533 for Appendix 3).

Statistical Analysis

Data were analyzed with SPSS Statistics (Version 24) and Microsoft Excel was used for quantitative analyses. Descriptive statistics were used to summarize staff and client responses and demographics.

Staff and Client Surveys

Reliability analysis (Cronbach's alpha) was conducted to determine the internal consistency of the four subscales of the adapted AHRQ staff needs assessment. Spearman's correlations were conducted to measure the association between client perceptions of OHL practices and client self-reported HL status.

Staff Roundtable Discussion

Each table's posters for the four open-ended questions were photographed using a digital camera and transcribed verbatim by a research assistant into Microsoft Excel. Content analysis was used to categorize staff responses (Downe-Wamboldt, 1992). Two researchers independently used an inductive approach (i.e., developing codes based on observation of collected data) to generate initial codes. A third researcher helped resolve discrepancies, develop final codes, and create a code book with definitions. Finally, researchers independently assigned meaning units from each table's responses into the appropriate code. When overlap was noted between codes, codes were collapsed. Using Microsoft Excel, code counts were tabulated, and totals were summarized descriptively. Interrater reliability was calculated (Armstrong et al., 1997), and disagreements were discussed and resolved. Efforts were made to triangulate quantitative and qualitative data from the staff needs assessment and roundtable discussion responses to strengthen findings (Creswell et al., 2003).

E-Newsletter Series

For each newsletter, analytics from the email marketing service server were recorded and analyzed descriptively in Microsoft Excel. Analysis included the number and percentage of staff that opened each email newsletter and the number and percentage of staff that clicked on embedded hyperlinks.

Clear Communication Workshop

Descriptive statistics were used to summarize survey responses from workshop participants. A research assistant transcribed this data verbatim into Microsoft Excel. Content analysis was used to analyze qualitative responses (Downe-Wamboldt, 1992). Using SPSS, code counts were tabulated and totals for each code were summarized descriptively.

Key Results

Demographics

Of the 279 staff who attended the kick-off training and completed the needs assessment, 252 (90%) consented to sharing their data. There were 185 clients who completed their respective assessment. Tables 2 and 3 includes details of staff and client demographics.

Baseline Demographics for VDH Staff (N = 252)Baseline Demographics for VDH Staff (N = 252)

Table 2:

Baseline Demographics for VDH Staff (N = 252)

Baseline Demographics for VDH Clients (N = 185)

Table 3:

Baseline Demographics for VDH Clients (N = 185)

Aim 1: Needs Assessment

Staff survey. All four subscales of the adapted staff survey demonstrated acceptable internal consistency (Cronbach's alpha values = 0.79–0.85) (Nunnaly, 1978). For both unit-and individual-based questions, leadership and staff survey responses revealed that the area in need of most improvement was written communication (Figure 2A–D). Most of the staff reported frequently engaging in OHL practices while also acknowledging room for greater consistency in practice (Figure 2E–J).

(A–D) Staff self-perceptions of unit-based organizational health literacy (OHL) practices; (E–J) staff self-perceptions of their individual OHL practices, N = 252.

Figure 2.

(A–D) Staff self-perceptions of unit-based organizational health literacy (OHL) practices; (E–J) staff self-perceptions of their individual OHL practices, N = 252.

Staff roundtable discussion. For all four questions, interrater reliability between the two coders was high (agreement range 76.1%–80.2%; average 77.2%). Self-perceptions of practice from the staff roundtable discussions supported and enriched the findings from the quantitative staff needs assessment across all four domains of HL. In 3 of 4 questions (questions #1, #2b, and #3), communication emerged as the most discussed theme, which demonstrated that the domains of HL most in need of strengthening were written and oral communication (Table 1). Additionally, staff shared valuable insight on specific training needs related to OHL practices.

Client survey. Overall, client perceptions of whether VDH staff use OHL practices were moderate to high, ranging from 3.1 to 3.6 on a 4-point scale (Table 4). For 5 of 7 OHL practices, there was a statistically significant positive correlation with client self-reported HL status (i.e., the higher the client self-reported HL status, the higher their perception of staff OHL practices). These correlations were strongest for written OHL practices. Client subjective HL scores were 12.8 on average (on a scale of 3 [low HL] to 14 [high HL]; SD = 1.7).

Client Perceptions of Whether VDH Staff Uses OHL Practices and Correlations with Client Subjective Health Literacya (N = 185)

Table 4:

Client Perceptions of Whether VDH Staff Uses OHL Practices and Correlations with Client Subjective Health Literacy (N = 185)

Aim 2: Improvement Process

Professional development kick-off training. There were 279 VDH staff who attended the successfully delivered kick-off training.

E-newsletter series. E-newsletters were delivered to approximately 300 staff members approximately every 3 months. The staff open rate was 83% (251/303), 75% (222/297), 71% (210/297), and 9% (30/321) for e-newsletters 1–4, respectively. Staff clicked on at least one link in the e-newsletters at a rate of 9% (27/303), 6% (18/297), 3% (18/297), and 1% (4/321) for e-newsletters 1–4, respectively.

Clear communication workshop. The New River (n = 50), Lenowisco (n = 49), and Cumberland Plateau (n = 31) district workshops included staff from all units, and the Mount Rogers district included staff from clinical and WIC units (n = 40). Overall staff ratings of the CCI training were moderately high (5-point scale, 5 = highest/best rating), including indication that staff learned new information (M = 4.1, SD = 0.68), the workshop information could improve the level of service VDH provides to clients (M = 4.0, SD = 0.71), training was delivered in an engaging manner (M = 4.0, SD = 0.80), training provided useful content and skills (M = 3.9, SD = 0.82), and training content could help job performance (M = 3.7, SD = 0.81). When asked what they liked about the workshop, staff most often reported the applicability of activities (i.e., using relevant written materials to learn the CCI, appreciation for dialogue about the importance of improving communication, and the interactive design of the workshop). The most frequent complaint was that more time was needed to complete the workshop. Also, some staff voiced a desire for tips on using the CCI with social media content and in conjunction with oral communication strategies. When asked what new information or skills were learned during the workshop, staff most often mentioned they learned how to assess, evaluate, or improve the use of written communication materials and that their awareness was raised when considering the understandability of materials and clarity of interactions with clients.

Although most of the staff mentioned ways in which they would like to use information gleaned from this workshop, many identified barriers. Commonly cited barriers were (1) being mandated to use certain forms or education materials, (2) needing approval from upper management to change materials, and (3) not having enough time to review or develop new materials.

Discussion

With the overarching aim of building capacity for OHL, conducting an organizational needs assessment can be an important first step to becoming a health literate health care organization (Brach et al., 2012; Kripalani et al., 2014; U.S. Department of Health & Human Services, Office of Disease Prevention and Health Promotion, 2010). This needs assessment guided the efforts of the HL improvement team by identifying strengths and weaknesses in OHL practices from both staff and client perspectives and understanding specific staff training needs. In addition, client self-reported HL status was significantly correlated with client perceptions of staff OHL practices, which justifies the need for OHL improvement.

This is the first known study to publish an application of the AHRQ-HLUP Toolkit in a local health district setting and contributes unique insight. At VDH, local health district staff play a key role in preventing and controlling chronic disease and reducing health care disparities. When developing the HL improvement plan, it was challenging to develop training content to be beneficial and engaging for all health department units. Ultimately, this strengthened the approach by thinking beyond primary care. The adaptability of the AHRQ-HLUP Toolkit pushed researchers and VDH staff to think outside of the box while still aligning with the AHRQ-HLUP Toolkit's purpose and structure, enabling use of the AHRQ-HLUP Toolkit in a manner beneficial to this local health district setting. Although specific needs of each district were considered, the HL improvement team developed a uniform needs assessment and improvement plan for all four districts. As recommended by others who have tested the AHRQ-HLUP Toolkit, future iterations of this project could involve each of the districts, and even respective units within each discipline (e.g., nursing, emergency preparedness) testing their own approach (i.e., choosing and implementing certain tools) to further address specific needs and capacity and increase involvement and engagement of district staff (DeWalt et al., 2011; Mabachi et al., 2016; Shoemaker et al., 2013).

This study used quantitative assessment of heath literacy practices at both the leadership, staff, and client levels, and qualitative assessment at the staff level. Triangulation of quantitative and qualitative data strengthened the OHL needs assessment approach (Kripalani et al., 2014). The staff needs assessment was bolstered by including perceptions of both individual and unit OHL practices. Individual perceptions of practice mirrored perceptions of unit-based practices, and the congruence between the two sets of results helped prioritize certain domains of HL for the initial improvement plan activities. The staff roundtable discussion provided an opportunity for VDH staff to engage with their peers and provide in-depth reflection on OHL practices and training needs. These tangible findings supported and enriched the results that expanded upon the quantitative staff survey and laid the groundwork not only for initial improvement plan activities but future improvement efforts as well. The roundtable discussion also raised awareness of the importance of engaging in health literate practices. Public health settings should consider initial and ongoing assessment to gather rich insight about the facilitators and barriers to OHL improvement efforts (Brach et al., 2014).

The client survey revealed a snapshot of client perceptions of VDH staff practices and the data were used to inform content of improvement plan activities. The finding that clients' perceptions of staff OHL practices were correlated with client HL status has meaningful implications. The correlation is most evident in the written communication domain, which closely aligns with findings from the staff assessment and reinforces importance of training on written communication practices and strategies. The alignment of lower client self-reported HL and lower client perceptions of staff OHL practices gives further justification for OHL improvement in these VDH districts; these findings show that those who subjectively rate their HL as low perceive the health care system as more complex. Although the subjective HL scores were somewhat higher than expected given the needs of this medically underserved population, scores still significantly correlated with perception of staff OHL practices. This study used valid, effective measures of subjective HL (Jeppesen et al., 2009; Morris et al., 2006; Wallace et al., 2006). However, it is important to recognize the limitations of nonresponse bias and self-report data. People tend to overreport their reading abilities (Williams et al., 1995). Clients present at VDH voluntarily completed surveys, and the impact of nonresponse bias was not assessed. This could have inflated the results of the subjective HL measure (i.e., clients with low HL may not have been present to take the survey, or may have declined to take the survey).

Improvement plan activities included delivery of workshops on the Clear Communication Index (CCI) and an e-newsletter series. Given the need to focus on the written communication domain, the initial professional development workshop series was based on the CCI. The HL improvement team made efforts to ensure each district's workshop met identified training needs of staff from all units (i.e., workshop addressed needs of both nurses and environmental health specialists alike). The workshops were enriched for VDH staff by offering the opportunity to work hands-on with familiar written communication materials, which increased the likelihood that staff could successfully use their new knowledge to improve client materials. Although certain content was likely more relevant for some units compared to others, overall feedback from staff was encouraging. Additionally, the workshops were held during planned staff development days to reduce burden on staff and leadership by making best use of time and resources. Although e-newsletter viewing and click rates progressively declined during implementation, the fourth e-newsletter reflected strikingly low rates. This could be due to this newsletter being sent 1 week before the Thanksgiving holiday (also a hectic time for many staff due to flu season) or simply that staff had become inured to the newsletters. Given these results, distribution of e-newsletters was not an optimal way to engage with VDH staff. Other avenues for disseminating this information will be considered in future improvement efforts (e.g., hands-on workshops).

Overall, the HL improvement team was successful in accomplishing a staff and client needs assessment and developing and implementing initial improvement plan activities based upon these findings. There were a few notable challenges, such as turnover with health district directors, which led to continual emphasis on partnership development and revisiting of the HL improvement plan. Additionally, VDH staff expressed lack of time to incorporate additional roles or practices on top of their usual job responsibilities. Whereas many strategies suggested during the roundtable discussion and workshops were immediately feasible (e.g., using plain language, emphasizing individualized interactions with clients), others required additional time and resources for implementation (e.g., assessing and improving written materials and providing training on communication and cultural competence). As a result, improvement efforts were largely driven by the research team and health district directors, and engagement among nonleadership staff during the planning and implementation process was limited. Ideally, front-line staff should be involved in OHL improvement efforts as these are the people who carry out day-to-day work and boast invaluable perspective (Caroulis & Howe, 2016; DeWalt et al., 2011; Huppelschoten et al., 2013; Jangland & Gunningberg, 2017; Taylor & Groene, 2015). OHL improvement is often initiated by HL change champions, informal advocates, or staff hired to solely address OHL improvement (Brach, 2017). In future assessment and improvement efforts undertaken by VDH and other public health settings, further opportunities should be made available for front-line staff from various disciplines to join the HL improvement team.

Study Limitations

There are three limitations of this study that should be acknowledged. First, it is ideal to conduct a comprehensive OHL assessment that includes capturing perceptions of leadership, providers, patients/clients, and an objective review of an organization's policies, procedures, and structure (Kripalani et al., 2014). However, time and resources prevented the team from conducting an objective review. Second, the staff survey included response categories “not sure” and “not applicable for role.” The interpretation of these responses is somewhat limited. Future research should explore if staff who select these response categories may have trouble or may fail to correctly perceive the applicability of an OHL practice in their role. The third limitation is that most of the staff and clients were White. Although this sample is representative of the southwest Virginia population, application of findings may not be generalizable to more diverse populations.

Moving forward, the partnership between the researchers and VDH staff and OHL improvement efforts will be maintained. The projected plan may include a continuation of 1 to 2 district-wide professional development trainings per year, hands-on workshops specific to certain units, and when appropriate, a reassessment of OHL practice perceptions and needs to determine next steps of the improvement process.

Conclusion

This study represents one of the first uses of the AHRQ-HLUP Toolkit in a local health district setting to conduct an organizational HL needs assessment and develop an OHL improvement plan, which included e-newsletters and in-person workshops. Findings reveal strengths and weaknesses in current OHL practices from both staff and client perspectives, with the greatest needs identified in the written communication domain. Although there is a trend toward adopting a universal precautions approach, it is important to recognize a system's unique barriers to providing care that addresses HL needs of patients and clients and implement training accordingly. Future OHL research and quality improvement projects should consider the challenges revealed by this study, especially the importance of yet difficulty in engaging busy front-line staff in strategic planning and improvement efforts. Finally, future work could expand on this study by considering proposed modifications and suggestions when using the AHRQ-HLUP Toolkit adapted for public health settings.

References

  • Adams, K. & Corrigan, J. M. (Eds.). (2003). Priority areas for national action: transforming health care quality. National Academies Press. doi:10.17226/10593 [CrossRef].
  • Altin, S. V., Lorrek, K. & Stock, S. (2015). Development and validation of a brief screener to measure the health literacy responsiveness of primary care practices (HLPC). BMC Family Practice, 16, 122 doi:10.1186/s12875-015-0336-4 [CrossRef] PMID:26362669
  • Armstrong, D., Gosling, A., Weinman, J. & Martaeu, T. (1997). The place of inter-rater reliability in qualitative research: An empirical study. Sociology, 31(3), 597–606 doi:10.1177/0038038597031003015 [CrossRef]
  • Baker, D. W. (2006). The meaning and the measure of health literacy. Journal of General Internal Medicine, 21(8), 878–883 doi:10.1111/j.1525-1497.2006.00540.x [CrossRef] PMID:16881951
  • Brach, C. (2017). The journey to become a health literate organization: A snapshot of health system improvement. Studies in Health Technology and Informatics, 240, 203–237 PMID:28972519
  • Brach, C., Dreyer, B. P. & Schillinger, D. (2014). Physicians' roles in creating health literate organizations: A call to action. Journal of General Internal Medicine, 29(2), 273–275 doi:10.1007/s11606-013-2619-6 [CrossRef] PMID:24113805
  • Brach, C., Keller, D., Hernandez, L. M., Baur, C., Parker, R., Dreyer, B. & Schillinger, D. (2012). Ten attributes of health literate health care organizations. National Academy of Medicine. doi:10.31478/201206a [CrossRef]
  • Brega, A. G., Barnard, J., Mabachi, N. M., Weiss, B. D., DeWalt, D. A., Brach, C., Cifuentes, M., Albright, K. & West, D.R. (2015). AHRQ Health Literacy Universal Precautions Toolkit, second edition. Agency for Healthcare Research and Quality. https://www.ahrq.gov/sites/default/files/publications/files/healthlittoolkit2_3.pdf
  • Callahan, L. F., Hawk, V., Rudd, R., Hackney, B., Bhandari, S., Prizer, L. P., Bauer, T. K., Jonas, B., Mendys, P. & DeWalt, D. (2013). Adaptation of the Health Literacy Universal Precautions Toolkit for rheumatology and cardiology - applications for pharmacy professionals to improve self-management and outcomes in patients with chronic disease. Research in Social & Administrative Pharmacy, 9(5), 597–608 doi:10.1016/j.sapharm.2013.04.016 [CrossRef] PMID:23867753
  • Caroulis, E. & Howe, C. J. (2016). Assessing the health literacy environment in a pediatric clinic. Boston University Medical Campus. https://www.bumc.bu.edu/healthliteracyconference/files/2016/06/Carol_Howe.pdf
  • Centers for Disease Control and Prevention. (2016) The CDC Clear Communication Index. https://www.cdc.gov/ccindex/index.html
  • Creswell, J. W., Plano Clark, V. L., Gutmann, M. L. & Hanson, W. E. (2003). Advanced mixed methods research designs. Handbook on mixed methods in the behavioral and social sciences ( Tashakkori, A. & Teddlie, C., Eds.). Sage Publications.
  • DeWalt, D. A., Broucksou, K. A., Hawk, V., Brach, C., Hink, A., Rudd, R. & Callahan, L. (2011). Developing and testing the Health Literacy Universal Precautions Toolkit. Nursing Outlook, 59(2), 85–94 doi:10.1016/j.outlook.2010.12.002 [CrossRef] PMID:21402204
  • Downe-Wamboldt, B. (1992). Content analysis: Method, applications, and issues. Health Care for Women International, 13(3), 313–321 doi:10.1080/07399339209516006 [CrossRef] PMID:1399871
  • Farmanova, E., Bonneville, L. & Bouchard, L. (2018). Organizational health literacy: Review of theories, frameworks, guides, and implementation issues. Inquiry, 55, 46958018757848 doi:10.1177/0046958018757848 [CrossRef] PMID:29569968
  • Health Resources & Services Administration. (2019). Health literacy. https://www.hrsa.gov/about/organization/bureaus/ohe/health-literacy/index.html
  • Huppelschoten, A. G., Aarts, J. W., van Empel, I. W., Cohlen, B. J., Kremer, J. A. & Nelen, W. L. (2013). Feedback to professionals on patient-centered fertility care is insufficient for improvement: A mixed-method study. Fertility and Sterility, 99(5), 1419–1427 doi:10.1016/j.fertnstert.2012.12.024 [CrossRef] PMID:23357454
  • Jangland, E. & Gunningberg, L. (2017). Improving patient participation in a challenging context: A 2-year evaluation study of an implementation project. Journal of Nursing Management, 25(4), 266–275 doi:10.1111/jonm.12459 [CrossRef] PMID:28164406
  • Jeppesen, K. M., Coyle, J. D. & Miser, W. F. (2009). Screening questions to predict limited health literacy: A cross-sectional study of patients with diabetes mellitus. Annals of Family Medicine, 7(1), 24–31 doi:10.1370/afm.919 [CrossRef] PMID:19139446
  • Kaphingst, K. A., Weaver, N. L., Wray, R. J., Brown, M. L., Buskirk, T. & Kreuter, M. W. (2014). Effects of patient health literacy, patient engagement and a system-level health literacy attribute on patient-reported outcomes: A representative statewide survey. BMC Health Services Research, 14, 475 doi:10.1186/1472-6963-14-475 [CrossRef] PMID:25288179
  • Kindig, D. A., Panzer, A. M. & Nielsen-Bohlman, L (Eds.). (2004). Health literacy: A prescription to end confusion. National Academies Press. https://www.nap.edu/catalog/10883/health-literacy-a-prescription-to-end-confusion
  • Koh, H. K., Brach, C., Harris, L. M. & Parchman, M. L. (2013). A proposed ‘health literate care model’ would constitute a systems approach to improving patients' engagement in care. Health Affairs (Project Hope), 32(2), 357–367 doi:10.1377/hlthaff.2012.1205 [CrossRef] PMID:23381529
  • Kripalani, S., Wallston, K., Cavanaugh, K. L., Osborn, C. Y., Mulvaney, S., Scott, A. M. & Rothman, R. (2014). Measures to assess a health literate organization. Vanderbilt Center for Effective Health Communication. https://www.researchgate.net/publication/268333875_MEASURES_TO_ASSESS_A_HEALTH-LITERATE_ORGANIZATION_Vanderbilt_Center_for_Effective_Health_Communication
  • Kutner, M. A. (2006). The health literacy of America's adults: results from the 2003 National Assessment of Adult Literacy. National Center for Education Statistics. http://nces.ed.gov/pubs2006/2006483.pdf
  • Mabachi, N. M., Cifuentes, M., Barnard, J., Brega, A. G., Albright, K., Weiss, B. D., Brach, C. & West, D. (2016). Demonstration of the Health Literacy Universal Precautions Toolkit: Lessons for quality improvement. The Journal of Ambulatory Care Management, 39(3), 199–208 doi:10.1097/JAC.0000000000000102 [CrossRef] PMID:27232681
  • Morris, N. S., MacLean, C. D., Chew, L. D. & Littenberg, B. (2006). The single item literacy screener: Evaluation of a brief instrument to identify limited reading ability. BMC Family Practice, 7, 21 doi:10.1186/1471-2296-7-21 [CrossRef] PMID:16563164
  • National Library of Medicine. (2019). Health literacy. https://nnlm.gov/initiatives/topics/health-literacy
  • Nunnaly, J. (1978). Psychometric theory. McGraw-Hill.
  • Pollard, K. & Jacobsen, L. A. (2016). The Appalachian Region: A data overview from the 2010–2014 American Community Survey. Appalachian Regional Commission. http://www.prb.org/pdf16/DataOverviewfrom2010to2014ACS.pdf
  • Porter, K. J., Brock, D. J., Estabrooks, P. A., Perzynski, K. M., Hecht, E. R., Ray, P., Kruzliakova, N., Cantrell, E. S. & Zoellner, J. M. (2019). SIPsmartER delivered through rural, local health districts: Adoption and implementation outcomes. BMC Public Health, 19(1), 1273 doi:10.1186/s12889-019-7567-6 [CrossRef]
  • Ratzan, SC & Parker, RM. (2000). Introduction. In Selden, C. R., Zorn, M., Ratzan, S. C. & Parker, R. M. (Eds.). National Library of Medicine Current Bibliographies in Medicine: Health Literacy. National Institutes of Health, U.S. Department of Health and Human Services.
  • Rudd, R. E., Groene, O. R. & Navarro-Rubio, M. D. (2013). On health literacy and health outcomes: Background, impact, and future directions. Revista de Calidad Asistencial : Organo de la Sociedad Espanola de Calidad Asistencial, 28(3), 188–192 doi:10.1016/j.cali.2013.03.003 [CrossRef] PMID:23684050
  • Shoemaker, S. J., Staub-DeLong, L., Wasserman, M. & Spranca, M. (2013). Factors affecting adoption and implementation of AHRQ health literacy tools in pharmacies. Research in Social & Administrative Pharmacy, 9(5), 553–563 doi:10.1016/j.sapharm.2013.05.003 [CrossRef] PMID:23759672
  • Taylor, A. & Groene, O. (2015). European hospital managers' perceptions of patient-centered care. Journal of Health Organization and Management, 29(6), 711–728 doi:10.1108/JHOM-11-2013-0261 [CrossRef] PMID:26394254
  • The World Cafe. (2017). World cafe method. http://www.theworldcafe.com/key-concepts-resources/world-cafe-method/
  • U.S. Department of Health & Human Services, Office of Disease Prevention and Health Promotion. (2010). National action plan to improve health literacy. https://health.gov/our-work/health-literacy/national-action-plan-improve-health-literacy
  • U.S. Department of Health & Human Services. (n.d.). MUA find. https://data.hrsa.gov/tools/shortage-area/mua-find
  • U.S. National Agricultural Library. (2014). What is rural? https://www.nal.usda.gov/ric/what-is-rural
  • Virginia Department of Health, Office of Health Equity. (n.d.) Virginia Health Opportunity Index. Counties. The health opportunity landscape. https://apps.vdh.virginia.gov/omhhe/hoi/dashboards/counties
  • Wallace, L. S., Rogers, E. S., Roskos, S. E., Holiday, D. B. & Weiss, B. D. (2006). Brief report: Screening items to identify patients with limited health literacy skills. Journal of General Internal Medicine, 21(8), 874–877 doi:10.1111/j.1525-1497.2006.00532.x [CrossRef] PMID:16881950
  • Williams, M. V., Parker, R. M., Baker, D. W., Parikh, N. S., Pitkin, K., Coates, W. C. & Nurss, J. R. (1995). Inadequate functional health literacy among patients at two public hospitals. Journal of the American Medical Association, 274(21), 1677–1682 doi:10.1001/jama.1995.03530210031026 [CrossRef] PMID:7474271
  • Zoellner, J. M., Hedrick, V. E., You, W., Chen, Y., Davy, B. M., Porter, K. J., Bailey, A., Lane, H., Alexander, R. & Estabrooks, P. A. (2016). Effects of a behavioral and health literacy intervention to reduce sugar-sweetened beverages: A randomized-controlled trial. The International Journal of Behavioral Nutrition and Physical Activity, 13, 38 doi:10.1186/s12966-016-0362-1 [CrossRef] PMID:27000402

VDH Staff Responses from the Kick-Off Training Roundtable Discussiona

Codeband No. of Tables (%) Examples
Question 1: “Think about a typical low health literate client that is served by your unit. What are some specific practices that your unit could do to strengthen the VDH experience of a low health literacy client?”
Oral communication
Decrease language barriers; 21 (57%) “need to translate materials in different languages” “use interpreters or language line”
Use plain language/simplify message; 20 (54%) “limit medical terminology or technical terms and speak in layman's terms” “make conversation with patient informative, short, and to the point”
Ask open-ended questions; 19 (51%) “ask clients ‘what questions do you have' rather than 'do you have any questions to encourage feedback’” “ask open-ended questions to determine comprehension”
Use Teach-Back; 17 (46%) “use Teach-Back technique to make sure you communicated effectively” “have clients repeat information back to assess understanding”
Awareness/cultural competence
Offer assistance; 25 (68%) “take the time to explain and educate about forms and procedures” “offer every client help reading and completing forms”
Reduce judgement/build trust/demonstrate cultural competence; 22 (59%) “be aware of cultural differences; place yourself in client's place and make them feel at ease” “create a welcoming non-judgmental environment”
Awareness of low health literacy signs/do not assume understanding; 17 (46%) “check for understanding and make adjustments” “be alert to physical and verbal cues that patient needs help with paperwork”
Written/audiovisual communication
Provide written or audiovisual materials for reference; 26 (70%) “provide videos to explain environmental health processes and permits” “put brochures and posters in waiting areas, and use health education videos in waiting room”
Modify/explain written material; 22 (59%) “simplify language on printed forms, avoiding jargon and acronyms” “provide appropriate paperwork that is sensitive to literacy needs and explain purpose of each form”
Supportive systems
Refer to proper services; 12 (32%) “make sure patients are aware of other services they may qualify for” “empower clients by pointing them to resources outside of our agency”
Hire more staff/provide staff training; 12 (32%) “need oral communication education” “need more employees to enable more time for appointments with each client”
Increase access/reduce distractions; 8 (22%) “more client focused time versus paperwork time” “provide private area to discuss issues”
Question 2a: “What are the challenges that you or your unit face to improve the VDH experiences of a low health literacy client?”
Supportive systems barriers
Limited time with client; 19 (51%) “having enough time to spend with client to explain everything” “too much required material to cover”
Understaffed/undertrained staff; 12 (32%) “staff pulled too many directions” “undertrained staff”
Unreliable access/contact with clients; 11 (30%) “getting in touch with clients for follow-up” “(lack of) transportation to clinic”
Limited financial resources/technology; 9 (24%) “difficult to navigate VDH website to obtain handouts and information” “lack of funding”
Communication barriers
Complex (written) materials for clients; 20 (54%) “not having visual aids/handouts at appropriate literacy levels” “paperwork and educational materials are at higher reading level than clients ability”
Language barriers; 15 (41%) “patients with limited English skills” “not enough translators”
Knowledge and cultural barriers
Cultural/trust barriers between staff and client; 19 (51%) “perception that environmental health is ‘the bad guy’” “clients can't understand due to cultural differences”
Client lack of awareness; 4 (11%) “patients that are not aware of what the health department actually does” “lack of awareness concerning own insurance benefits”
Question 2b: “How can your unit work with the health literacy team to help address these challenges?”
Communication solutions
Simplified procedures and materials; 20 (54%) “pick most important education needs” “simplify materials, make them have narrow focus”
Use/update written material; 17 (46%) “follow up verbal information with written information” “use simple literature; more pictures less words”
Translators/language line; 14 (38%) “need more interpretation services; phone services can be slow and ineffective” “need easier access to interpreter to help deliver message”
Reliable access/contact with clients; 9 (24%) “offer home visits and school-based clinics” “collect multiple ways to contact clients”
Supportive systems solutions
Electronic solutions & technology improvements; 18 (49%) “use electronic medical records to reduce paperwork” “evaluate and simplify phone system so client can reach a person”
More staff/staff training; 17 (46%) “cross train staff to fill in when short-staffed” “training in communication skills to address language barriers and education levels and cultural sensitivity”
Collaboration of health professionals; 14 (38%) “communicate between disciplines” “work in a team and ask for suggestions on how to better relate”
Outreach/referral to proper services; 7 (19%) “outreach to local dentists and mobile dental/medical unit” “use incentives to bring clients in”
Communication training
Oral communication strategies; 18 (49%) “need refreshers on Teach-Back and related skills” “need better ways for us to simplify our counseling”
Management of language barriers; 17 (46%) “training to better communicate with people who do not speak English” “training on how to use language line (interpreter phone line)”
Use of/simplification of written materials; 15 (41%) “need to know how to simplify information for patients” “educate employees to evaluate reading level of educational materials”
Knowledge and cultural training
Training on cultural competence; 19 (51%) “more cultural sensitivity training due to diverse populations” “refresher course on recognizing low health literacy”
Supportive systems training
Client education strategies; 6 (16%) “need ways to encourage participation” “need different learning strategies for limited reading comprehension of clients”

Baseline Demographics for VDH Staff (N = 252)

Characteristic n (%)
Sex
  Female 213 (85)
  Male 28 (11)
  Not reported 11 (4)

Age (years)
  18–24 3 (1)
  25–44 75 (34)
  45–64 128 (58)
  ≥ 65 13 (5)
  Not reported 33 (13)

Race/ethnicity
  White 240 (95)
  Black 2 (1)
  Not reported 10 (4)

Education
  High school degree/GED 59 (23)
  Associate degree 65 (26)
  Bachelor degree 94 (37)
  Graduate degree   34 (14)

District
  Mount Rogers 110 (44)
  Lenowisco 54 (21)
  Cumberland Plateau 48 (19)
  New River 40 (16)

Unit
  Nursing 86 (34)
  Administrative 55 (22)
  Environmental health 38 (15)
  WIC 35 (14)
  Health education 11 (4)
  Epidemiology and emergency response 10 (4)
  Other 17 (7)

Years in staff position
  0–10 140 (56)
  11–20 41 (16)
  21–30 36 (14)
  31–40 19 (8)
  >40 14 (6)
  Not reported 2 (1)

Years at VDH
  0–10 112 (45)
  11–20 51 (20)
  21–30 52 (21)
  31–40 21 (8)
  ≥ 41 15 (6)
  Not reported 1 (1)

Baseline Demographics for VDH Clients (N = 185)

Characteristic n (%)
Sex
  Female 153 (83)
  Male 32 (17)

Age (years)
  17–24 57 (30)
  25–44 90 (49)
  45–64 27 (15)
  ≥65 8 (4)
  Not reported 3 (2)

Race/ethnicitya
  White 171 (93)
  Black 12 (6)
  Hispanic/Latinx 6 (3)
  Not reported 10 (5)

Education
  ≤ High school degree 73 (39)
  Some college 54 (30)
  College degree 37 (20)
  Graduate degree 21 (11)

District
  Cumberland Plateau 72 (39)
  New River 56 (30)
  Mount Rogers 30 (16)
  Lenowisco 27 (15)

Client Perceptions of Whether VDH Staff Uses OHL Practices and Correlations with Client Subjective Health Literacya (N = 185)

VDH Client Questions M (SD)b Spearman's r (p value)
VDH asks about following instructions 3.1 (1.0) 0.15 (.04)
VDH refers to personal history 3.3 (0.8) 0.16 (.04)
VDH forms easy to understand 3.4 (0.7) 0.50 (<.01)
VDH written materials easy to understand 3.5 (0.7) 0.48 (<.01)
VDH staff spends enough time 3.5 (0.7) 0.13 (.08)
VDH encourages questions 3.5 (0.7) 0.13 (.09)
VDH explains services 3.6 (0.6) 0.26 (<.01)
Authors

Natalie Kružliaková, PhD, RD, LD, is an Assistant Professor, Department of Nutrition and Health Science, Ball State University. Kathleen Porter, PhD, RD, is an Assistant Professor of Research, School of Medicine, Department of Public Health Sciences, University of Virginia. Pamela A. Ray, DVM, MBA, MPH, is the Population Health Community Coordinator, Virginia Department of Health, New River Health District. Valisa Hedrick, PhD, RD, is an Assistant Professor, Department of Human Nutrition, Foods, and Exercise, Virginia Tech. Donna Jean Brock, MS, is a Research Program Officer, School of Medicine, Department of Public Health Sciences, University of Virginia. Jamie Zoellner, PhD, RD, is a Professor, School of Medicine, Department of Public Health Sciences, University of Virginia.

Address correspondence to Natalie Kružliaková, PhD, RD, LD, Department of Nutrition and Health Science, Ball State University, 1613 W. Riverside Avenue, Muncie, IN 47306; email: kruzliakova18@gmail.com.

Grant: The study was funded by a grant (R21CA202013) from the National Institutes of Health/National Cancer Institute.

Disclosure: The authors have no relevant financial relationships to disclose.

The authors thank Darian Carter, MS, Erin Hecht, MS, and Katelynn Perzynski, MS, all from Virginia Tech, for their contributions to the manuscript. They are also thankful for the assistance of the Cumberland Plateau, New River, Lenowisco, and Mount Rogers Health Districts, in particular current and former Health District Directors Drs. Sue Cantrell, Molly O'Dell, Melody Counts, Noelle Bissell, and Karen Shelton.

This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (https://creativecommons.org/licenses/by-nc/4.0). This license allows users to copy and distribute, to remix, transform, and build upon the article non-commercially, provided the author is attributed and the new work is non-commercial.
Received: October 24, 2019
Accepted: February 27, 2020

10.3928/24748307-20210114-01

Sign up to receive

Journal E-contents