Research in Gerontological Nursing

Instrument Development 

Diffusion of Innovations in Long-Term Care Measurement Battery

Eleanor S. McConnell, PhD, RN, GCNS-BC; Kirsten N. Corazzini, PhD; Deborah Lekan, MSN, RN, C; Donald E. Bailey, PhD, RN; Richard Sloane, MPH; Lawrence R. Landerman, PhD; Mary T. Champagne, PhD, RN, FAAN

Abstract

Poor understanding of factors influencing integration of new practices into long-term care (LTC) hinders timely implementation of evidence-based practices (EBPs). Using the Diffusion of Innovations (DOI) framework, a new instrument measuring staff perceptions of an EBP was developed as part of a DOI-LTC measurement battery and tested in a cross-sectional survey of North Carolina LTC nursing personnel. Valid questionnaires were received from 95 licensed nurses and 102 certified nursing assistants (CNAs). Internal consistency reliability for five of seven subscales was acceptable (Cronbach’s alpha coefficient = 0.77 to 0.95). Perception of innovation attributes was associated with intention to adopt the new practice (Spearman rho correlation: licensed nurses = 0.41 to 0.68, p < 0.0001; CNAs = 0.26 to 0.54, p = 0.05 to <0.0001). The DOI-LTC measurement battery represents a promising new approach to studying implementation of EBPs in LTC. Future work should examine its responsiveness to interventions that facilitate implementation of EBPs in LTC.

Abstract

Poor understanding of factors influencing integration of new practices into long-term care (LTC) hinders timely implementation of evidence-based practices (EBPs). Using the Diffusion of Innovations (DOI) framework, a new instrument measuring staff perceptions of an EBP was developed as part of a DOI-LTC measurement battery and tested in a cross-sectional survey of North Carolina LTC nursing personnel. Valid questionnaires were received from 95 licensed nurses and 102 certified nursing assistants (CNAs). Internal consistency reliability for five of seven subscales was acceptable (Cronbach’s alpha coefficient = 0.77 to 0.95). Perception of innovation attributes was associated with intention to adopt the new practice (Spearman rho correlation: licensed nurses = 0.41 to 0.68, p < 0.0001; CNAs = 0.26 to 0.54, p = 0.05 to <0.0001). The DOI-LTC measurement battery represents a promising new approach to studying implementation of EBPs in LTC. Future work should examine its responsiveness to interventions that facilitate implementation of EBPs in LTC.


Across the continuum of care, significant lags exist between discovery of effective clinical approaches and their consistent integration into daily care routines, resulting in suboptimal care outcomes (Berwick, 2003; Roe et al., 2004; Titler, 2004). Evidence-based practices (EBPs) that have not been integrated into routine care can be conceptualized as innovations, which are behaviors, routines, and ways of working, communicated through certain channels over time among members of a social system (Rogers, 2003). Over the past 4 decades, the Diffusion of Innovation (DOI) framework has been successfully applied to explain technology transfer in a variety of fields, including agriculture, public health, and clinical care. DOI theory has guided studies of adoption of evidence-based approaches to geriatric problems in acute care settings (Titler et al., 2009), in primary care (Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004), and in adoption of new programs of service in long-term care (LTC, Castle, 2001; Kovach, Morgan, Noonan, & Brondino, 2008). The DOI framework therefore holds promise as an approach for exploring the process of implementing evidence-based approaches in the LTC setting, an area of growing emphasis in health care (Berta et al., 2010; Miles, 2010).

Research on how innovations are adopted suggests that several factors influence the rate of adoption of new care practices, including the adopter’s perception of key attributes of the innovation, the process of communicating about the innovation, and characteristics of the social system in which the new behaviors occur. If the DOI framework can be shown to identify factors that inhibit or facilitate successful implementation of EBPs in LTC, then the scientific basis for proposing intervention trials to accelerate the process of translating EBPs across geriatric care settings would be strengthened. A key barrier to using DOI in implementation research in LTC is the lack of reliable and valid instruments to measure key constructs from the DOI framework. This article reports the development and preliminary testing of a set of measures, the DOI-LTC measurement battery, designed to assess three domains within the DOI framework among licensed and unlicensed nursing staff: (a) perception of the attributes of the innovation, (b) leadership quality, and (c) communication patterns. The ultimate goal of the DOI-LTC measurement battery is to study factors that influence readiness to adopt new practices in LTC so that clinicians, administrators, or investigators can tailor implementation approaches and track progress in enhancing staff or organizational readiness to adopt new practices.

Background

Key Domains Within the DOI Framework

The DOI framework takes into account factors that are both external and internal to the system where new practices will be adopted (Greenhalgh et al., 2004). Although factors external to the system where an innovation is adopted have been shown to influence uptake of new care practices, our focus has been on factors internal to the system, since internal factors are more amenable to influence by managers and clinicians. Key individual and organizational level influences on adoption of innovations are shown in the Figure. Each domain contains variables that could be systematically manipulated to promote the rate of adoption of an innovation, such as integrating EBPs into LTC.

Key Diffusion of Innovation model domains and constructs from the perspective of internal context (adapted from Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004). The focus of the study instrument development is highlighted in gray.

Figure. Key Diffusion of Innovation model domains and constructs from the perspective of internal context (adapted from Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004). The focus of the study instrument development is highlighted in gray.

Adopter Characteristics. Numerous studies have shown that the adopter’s perception of attributes of an innovation consistently influences likelihood of adopting an innovation. According to an integrative review of innovations across diverse fields (Rogers, 2003), between 49% to 87% of the variance in the rate of adoption can be explained by the adopter’s perception of five attributes of the innovation: relative advantage, compatibility, complexity, trialability, and observability. Relative advantage refers to the degree to which an innovation is perceived as being better than the status quo. Innovations that are not perceived as superior to current practice are unlikely to be integrated into routine care. Compatibility captures the degree to which an innovation is perceived as consistent with existing norms, values, or practices in the workplace. Innovations that can be linked with past similar experiences are more likely to be implemented than those that are less consistent with existing practice. Complexity indicates the degree to which an innovation is perceived as relatively difficult to understand or to implement. Less complex innovations are more readily adopted than more complex ones. Observability is the degree to which the outcomes or results of an innovation are easily seen and understood. Observability of the innovation enhances adoption by decreasing uncertainty about the implications of using the innovation. Trialability is the degree to which an innovation is perceived as testable on a small scale. Innovations that can be pilot tested are more likely to be implemented. Two other attributes have been shown to influence the adoption decision of information technology in the workplace: the effect of the new practice on the adopter’s image, and the voluntariness of the adoption decision (Moore & Benbasat, 1991). New practices that enhance the adopter’s image with respect to other workers are more likely to be adopted. Likewise, innovations that are perceived to be required by supervisors are more likely to be implemented.

Although it is appealing to think about the innovation attributes described above as fixed properties of the innovation, most investigators have focused on end-user perception of the innovation’s attributes because rate of adoption is thought to be a function of the interactions among the innovation, the intended adopters, and the particular context in which the innovation was adopted (Stetler, 2003). While these attributes have frequently been associated with adoption of innovations in fields outside of health care (Moore & Benbasat, 1991), limited attention has been paid to development of reliable and valid measures of worker perception of innovations in health care.

Organizational Characteristics. A broad array of characteristics of the organization has been shown to be related to likelihood of adopting new practices. Three domains have been described as particularly influential: (a) system antecedents for change, including structural variables such as size and formalization, the ability to integrate new knowledge from outside the organization, and leadership quality; (b) system readiness for a specific innovation, including variables that would foster or inhibit uptake of new practices; and (c) implementation processes, including internal communication patterns (Greenhalgh et al., 2004). Of the constructs identified in these domains, two of the constructs lend themselves to measurement among LTC staff, as reliable and valid measures have been successfully applied among this workforce: leadership quality and communication patterns (Anderson, Corazzini, & McDaniel, 2004).

Adoption Decision. Rogers (2003) emphasized that the adoption process occurs over time, and therefore attention has been devoted to various stages of the adoption decision, including the intention to adopt, as well as behaviors that reflect adoption of new practices. Intention to adopt has been measured among health care professionals (Gagnon et al., 2003) and has been associated with subsequent innovation adoption behavior in health care settings (Mesters & Meertens, 1999). Measures of intention to adopt a new behavior exist in both the individual health behavior literature (e.g., Hughes, Keely, Fagerstrom, & Callas, 2005), as well as in the diffusion literature (Gagnon et al., 2003) and demonstrate that intention can be measured with acceptable reliability. For example, Gagnon et al. (2003) measured 519 physicians’ intention to adopt the use of telemedicine, using a 3-item instrument (Cronbach’s alpha coefficient = 0.84), where intention to adopt was expressed as the mean of the responses to three items, scored on a 7-point Likert scale: “I estimate that my chances of using telemedicine in my practice are (very high to very low)”; “If I have the opportunity, I will use telemedicine in my practice” (strongly agree to strongly disagree); and “I intend to use telemedicine in my practice” (strongly agree to strongly disagree).

Consequences of Implementation. Consequences of implementing the innovation may influence the sustainability of the original adoption decision. As adopters gain experience with the new work practices and the outcomes associated with those practices, their perceptions of those innovation attributes may change. Likewise, experience with a new care practice may ultimately influence organizational characteristics, particularly in the area of system readiness.

Method

We conducted instrument and testing in three stages: (a) questionnaire development, which occurred from February through October 2006; (b) field testing, which occurred in November 2006; and (c) a mailed survey, which was conducted from July through October 2007.

Human Subjects Protection

This study was reviewed and approved by the University Medical Center Institutional Review Board. We obtained written informed consent for interviews of participants in the field trial of the instrument; however, in the case of mailed surveys, a waiver of written consent was granted since the only risk to participants was loss of confidentiality and the only document that would contain participant identifying information following return of the mailed survey was the consent form, since mailing lists were destroyed after the survey mailings were completed. To ensure that participants were informed of the risks and benefits of the study, prior to sending the questionnaires, we sent an advance letter that included a summary of the study purpose, risks, and benefits, and an option to contact the investigators to opt out of the study. For participants who did not opt out, when we mailed the survey, we included in a cover letter key study information, including risks, benefits, and the participant’s rights to withdraw from the study, emphasizing that return of the questionnaire implied consent to participate.

Questionnaire Development

Item Pool Generation. To develop an item pool for the DOI-LTC measurement battery, we first conducted a systematic literature review guided by the following question: “In relation to diffusion of innovation or adoption of evidence-based or research-based health care practices, what instruments exist to identify and measure attributes of the innovations themselves, or factors related to organizations or personnel, that affect likelihood of diffusion or adoption?” Under the guidance of a medical librarian skilled at conducting searches in PubMed, CINAHL, and PsycInfo databases, we obtained 606 articles for review, from which we selected 51 with instruments that could potentially contribute items to the DOI-LTC measurement battery. No instrument was identified that was specific to implementation of new practices in LTC. Only two subscales were sufficiently close enough to DOI constructs of interest that they could be used intact: leadership scales from the Baldrige National Quality Program (2011) and a communication scale that measured openness, accuracy, and timeliness of communication (Roberts & O’Reilly, 1974). From the articles identified in the systematic literature review, we entered 1,307 items from existing instruments into a relational database, mapped each questionnaire item to a specific construct within the DOI framework, and then, focusing on those items that mapped to “perception of the innovation,” modified items for possible use in LTC.

Item Refinement. To refine items identified from existing instruments to measure perceptions of an innovations attribute on LTC staff, we first developed a concrete example of an EBP that was likely to be implemented in LTC: the use of a “pain thermometer” to assess pain severity (Herr, Bjoro, Steffensmeier, & Rakel, 2006). We selected pain management as our specific referent, as this is an area with a documented evidence-practice gap that influences important LTC outcomes (Cadogan et al., 2005), and pain management in LTC is currently receiving considerable national attention.

Following methods that had been used successfully to study the process of innovation adoption in information technology among a diverse workforce (Moore & Benbasat, 1991), we first considered the context and time frame of the EBP implementation (new pain management practices), the range of behaviors currently involved in pain management practice, and the target behaviors involved in implementing the new evidence-based pain management practice. In describing the proposed practice implementation, we focused on the ward or unit level practices, rather than the level of individual practice. We then considered the influence of new practice on a given nursing staff member’s role; we used the nursing process as an organizing framework to distinguish licensed nurse (RN and licensed practical nurse [LPN]) and certified nursing assistant (CNA) work expectations. Lastly, we considered potential influences of new practice on care outcomes or work routines.

Since our aim was to develop a measure that would include items pertinent to all levels of LTC nursing staff, we developed parallel forms of the instrument that contained items that were role appropriate for different levels of nursing staff found in all nursing homes: RNs, LPNs, and CNAs. Although advanced practice nurses are an increasing part of the LTC staffing mix, they were not included because their use is not universal (Bakerjian, 2008). As candidate items were refined, we considered whether the new practice called for different target behaviors based on role. If so, we developed a different item for each level of nursing staff. If target behaviors were the same regardless of role, the same item was used for both licensed and unlicensed nursing personnel. For example, licensed nurses are accountable for assessing patients’ pain and administering treatments; in contrast, CNAs are responsible for identifying and reporting occurrences of behaviors that may indicate pain. Thus, to address the construct of relative advantage, the item specific to the licensed nurse role read: “Using the new pain management program helps me know when residents need pain medicine”; whereas the item tapping the same construct for the CNA role read: “Using the new pain management program will help me learn quickly when my residents are in pain.” Where possible, we used item stems directly from established measures; however, we also considered reading level issues in the refinements. When refining items, we used everyday language rather than technical terms and included concrete or vivid ways to describe the new work practice, method, or outcome that was consistent with a direct care worker’s logic, language, or experience (Doak, Doak, & Root, 1996).

Content Validity. We asked a panel of experts with extensive LTC experience (4 RNs and 2 CNAs) to provide feedback on each of the potential items and to eliminate items that had confusing wording or that were not consistent with the role of licensed nurse or CNA. We also asked them to tell us their preferences regarding items with similar wording to identify items that best addressed the key constructs. Based on that feedback, we produced a 40-item battery for the CNAs and a 50-item battery for licensed nurses. The difference in length between the two versions was due to the more complex role of licensed nurses, which necessitated more items to fully address their perceptions of the new work practices. To avoid response bias, a total of 25 items on the licensed nurse questionnaire and 19 items on the CNA questionnaire were reverse coded. The Flesch-Kincaid reading levels were 6.9 for the CNA questionnaire and 7.1 for the licensed nurse questionnaire.

Subscale Development and Scoring. All items were rated on a 6-point Likert scale, anchored by 1 (strongly disagree) and 6 (strongly agree). I don’t know was also a response option, separated from the other checkboxes on the questionnaire by a gray line. Items were assigned a priori by the investigators to constructs from the DOI framework. Subscales were scored by summing responses to items assigned to each subscale after inverting the rating on the items that were reverse coded. Thus, the instrument is composed of seven subscale scores and does not provide an overall score or rating of the EBP. In all instances, higher scores reflect a more favorable attitude toward the EBP.

The relative advantage subscale consists of 12 items for licensed nurses and 10 items for CNAs. The total possible score ranges from 18 to 72 for the licensed nurse version and 14 to 60 for the CNA version. The compatibility, observability, image, and voluntariness subscales contain 6 items in the licensed nurse version and 5 items in the CNA version. Scores can range from 6 to 36 and 5 to 30, respectively. The complexity subscale contains 8 items in the licensed nurse version and 6 items in the CNA version; scores can range from 8 to 48 and 6 to 36, respectively. The trialability subscale contains 6 items for the licensed nurse version and 4 items for the CNA version, resulting in scores that range from 6 to 36 and 4 to 24, respectively.

The survey also included two established measures of organizational characteristics of the participant’s workplace: one that assessed communication patterns and another that tapped leadership quality. The communications measure asked staff to rate their level of agreement with 14 statements concerning the openness, accuracy, and timeliness of communication patterns in their workplace (Roberts & O’Reilly, 1974, applied to LTC by Anderson et al., 2004). A sample item was “It is easy for me to talk openly with all staff in this facility.” The leadership measure was a 7-item instrument derived from the Baldrige National Quality Program (2011). A sample item was, “My senior leaders create a work environment that helps me do my job.” For both instruments, ratings are made on a 5-point Likert scale anchored by 1 (strongly disagree) and 5 (strongly agree). The potential range of each subscale is: communication openness, 5 to 25; communication accuracy, 5 to 25; communication timeliness, 4 to 20; and leadership, 7 to 35. Higher scores denote more positive communication patterns and leadership quality.

Finally, in addition to the questionnaire items described above, a single item—intention to adopt the new EBP—was included, phrased as “I intend to use the new pain care practices with my patients.” This item was rated using the same format as the perceptions of the EBP scale, where 1 = strongly disagree and 6 = strongly agree.

Field Testing of Items

We submitted the items selected during the questionnaire development phase to a field test using a modified cognitive interviewing technique (Willis, 1999; Willis, Royston, & Bercini, 1991). We interviewed 6 RNs in supervisory roles (100% women, 16.5% non-White, 66% with highest educational attainment as “some college” but less than a bachelor’s degree, mean age of 52, mean time in position of 6.5 years) and 21 staff in direct care roles (1 RN, 2 LPNs, and 18 CNAs: 84% women; 89% non-White; 26% with a high school education or less; mean age of 36.2; mean time in position of 5.6 years). In each instance, the participant was given a questionnaire, but an interviewer asked each question aloud. After completing the questionnaire, each participant was asked if any of the questions were difficult or confusing to answer. For any items identified as such, the participant was asked to identify the item and then discuss what was difficult and what would have made responding easier. For questions where the participants chose I don’t know, interviewers probed in more depth for the reasons why the participant was uncertain, using questions such as “Can you tell me in your own words what the question means?” and “What about the question keeps you from being able to answer?” The interviewer also asked about additional information that might have made the question answerable. For example, when asked about whether their supervisors would expect them to use the new pain practices, those who responded I don’t know said they truly did not know how their supervisors would respond. For any questions that were skipped, the interviewer was instructed to ask the participants why they skipped the item and whether any additional information could have been given so they would have known how to answer the question.

We then asked if the participants found it interesting to answer the questions and to specify why they did or did not find answering the questions interesting. We closed by asking three final questions:

  • “Thinking about your experiences in implementing new care routines, are there questions that you were expecting that we did not ask? If yes, what were they?”
  • “What did you think about the length of the survey?”, with responses (a) just right (b) too long, and (c) could have answered more questions.
  • “Is there anything else you’d like to tell us about what it was like to answer the survey?”

All individuals who participated in the field test said they found the items easy to understand and that the items covered important areas of work practice in LTC. In general, participants reported being pleased to be asked about their new care practices, and all reported that they thought the questionnaire was “just right” in length. One issue raised by the participants concerned the image construct in the DOI model. Several CNA participants stated that image was not an important consideration to them in the work they performed. A second area of concern raised by the participants was in the voluntariness scale, where some respondents found it difficult to predict what their supervisors would think of the new EBP. Based on that feedback, minor editorial changes were made to the items, but no items were deleted.

Mailed Survey of Hypothetical Change in Pain Care Practice

To evaluate the psychometric properties of the DOI-LTC measurement battery, we conducted a cross-sectional survey of a simple random sample of licensed nurses (RNs and LPNs) and CNAs in North Carolina (NC) who identified geriatrics or LTC as their practice focus. We obtained mailing lists from the NC Board of Nursing (NC-BoN) for RNs (n = 4,522) and LPNs (n = 6,252), and from the NC Nurse Aide Registry for CNAs (n = 65,742). The NC-BoN registry contains information about the specialty practice and practice location of RNs and LPNs, including fields that identify geriatrics and LTC as an area of practice. Using the random number generation function in SAS version 9.1, we associated unique random numbers with each of the individuals listed on each mailing list. We then sorted the list by the random number and drew a sample of the first 200 RNs, 200 LPNs, and 400 CNAs from these re-ordered lists.

Following a modified Dillman (2000) survey administration method, we mailed letters in advance of the questionnaire, explaining the purpose of the study and requesting the nursing staff member’s help with the study. Participants were offered the option to contact the study office to opt out of the study. Two weeks later, we sent the questionnaire along with a cover letter re-introducing the study, accompanied by a pen to facilitate completion of the scannable forms, and a sample laminated pain assessment card, which they were allowed to keep, as the hypothetical practice they were to consider when completing the questionnaire. The pain assessment card was selected from the Medicare Quality Improvement Community (MedQIC) website, a clearinghouse for EBP tools sponsored by the Centers for Medicare & Medicaid Services ( http://www.qualitynet.org/dcs/ContentServer?pagename=Medqic/MQPage/Homepage).

Data Analysis

We first calculated descriptive statistics for each item and examined these for missing data, range restrictions, and violations of the assumptions of parametric statistical analyses. We then formed instrument subscales by summing items that were assigned, a priori, to specific constructs and examined the distribution of these subscales. To assess internal consistency reliability of each subscale, we calculated Cronbach’s alpha coefficients. We then proposed two research questions to guide the assessment of construct validity of the DOI perceptions of the innovation scale:

  • What is the relationship between LTC staff perception of attributes of an evidence-based clinical practice innovation, perception of leadership, and perception of communication patterns and LTC-staff reported intention to adopt the clinical practice?
  • How do these relationships vary by staff type?

To examine the first question, we computed Spearman correlations between each subscale and the response to the intention-to-adopt item scaled as both an ordinal measure, and performed the Wilcoxon rank-sum test on the relationship with the intention-to-adopt item scaled dichotomously. Nonparametric tests were used since the intention-to-adopt item was not normally distributed. We then compared these relationships between two staff types: licensed nurses and CNAs.

Findings

Ninety-five licensed nurses (RNs and LPNs) and 102 CNAs responded to the survey, a response rate of 24% among licensed nurses, and 26% among CNAs. Demographic characteristics of the respondents are shown in Table 1. Of the RNs, 36.4% were 50 and older, compared with 34.4% of LPNs, and only 18.1% of CNAs. Participants were predominantly women (91% to 96%); however, their minority status varied by nursing staff type, with 6% of RN participants reporting their race as non-White, compared with 35.6% of LPNs and 47% of CNAs. Educational level also varied by group, with 78% of RNs having completed a 2-year degree or higher, compared with only 38.6% of LPNs and 23.5% of CNAs. Eighty-one percent (n = 77) of the licensed nurses worked in a skilled nursing or assisted living facility, and 32% of the CNAs (n = 33) worked in a skilled nursing or assisted living facility. Other settings of care represented were hospital-based LTC units, dialysis units, and home care. The majority of RNs had been in their current position for more than 5 years, while the majority of LPNs and CNAs had been in their position 5 years or less.

Characteristics of Respondents to Mailed Survey (N = 197)

Table 1: Characteristics of Respondents to Mailed Survey (N = 197)

These characteristics are consistent with the demographic and educational characteristics of the licensed nursing workforce in NC, except for the minority status of the RNs. In 2006, 15% of RNs and 29% of LPNs were non-White (North Carolina Center for Nursing, 2007; Probst, Baek, & Laditka, 2009). According to National Nursing Home Survey data, 46% of CNAs nationally are non-White, and 24% of CNAs nationally have preparation at the community college or trade school level or are college graduates (Probst et al., 2009; Squillace et al., 2009).

Table 2 displays key descriptive statistics and reliability coefficients for each subscale by nursing staff type. In general, the pattern of results across staff was similar: All levels of staff viewed the new pain practice favorably. Participants used nearly the full range of responses available, although a modest range restriction was observed in the lower ranges of the subscales. This tendency was most pronounced in the relative advantage subscale, where LPNs tended to make less use of the lower ranges of the scales than CNAs or RNs. The inter-item mean scores adjusted for number of items were 4.7 for RNs, 4.3 for LPNs, and 5.0 for CNAs.

Key Descriptive Statistics and Reliability Coefficients for DOI-LTC Measurement Battery Subscales By Nursing Staff Type

Table 2: Key Descriptive Statistics and Reliability Coefficients for DOI-LTC Measurement Battery Subscales By Nursing Staff Type

With respect to differences across staff types in terms of magnitude of ratings, CNAs tended to rate the new practices as more advantageous, but less easy to use, less compatible with their work practices, and less favorable to their image than did the licensed nurses. CNAs and licensed nurses each viewed use of the new pain card as equally observable, trialable, and voluntary. Licensed nurses and CNAs had similar perceptions of leadership and communication patterns, including openness, accuracy, and timeliness.

The reliability estimates for each subscale were similar among the staff types except for two subscales: compatibility and trialability. For these two subscales, the reliability was substantially higher for licensed nurses than for CNAs (Cronbach’s alpha coefficients: compatibility = 0.78 for licensed nurses versus 0.56 for CNAs; trialability = 0.50 for licensed nurses versus 0.40 for CNAs).

To assess the construct validity of the new scale, we examined the correlation between the various DOI subscales and a single-item 6-level rating of intention to adopt the new practice. Means for intent to adopt was 4.66 (SD = 1.10) for CNAs and 4.47 (SD = 1.20) for licensed nurses. When the intention-to-adopt measure was dichotomized into those who did not intend to adopt (ratings 1 to 3) and those who did intend to adopt (ratings 4 to 6), the majority of participants reported an intention to adopt the new practice, with 75 (73.5%) of the CNAs intending to adopt, and 78 (82.1%) of the licensed nurses indicating an intention to adopt.

Table 3 displays the correlations between the intention-to-adopt measure and the perceptions of the new pain practices, leadership, and communication patterns, using both the 6-point ordinal scale and the dichotomous scoring. When intention to adopt was scaled as an ordinal variable, it was significantly associated with all but one of the new subscales measuring perception of attributes of the innovation and with perceptions of leadership and communication openness; however, it was not consistently associated with perception of the voluntariness of using the new practice, nor with communication timeliness or accuracy. When the analysis was performed with the intention-to-adopt item dichotomized, the pattern of results was similar, with two exceptions. Among the CNAs, perceptions of compatibility and trialability were no longer significantly associated with intention to adopt. For both licensed nurses and CNAs, the association with leadership perceptions was no longer significant. The strength of the associations between perception of the new practice and intention to adopt was moderately strong. The strongest associations were observed with relative advantage (Spearman rho correlations = 0.68 for licensed nurses, 0.54 for CNAs). The next strongest associations were observed in the complexity subscales (Spearman rho correlations = 0.60 for licensed nurses, 0.44 for CNAs), followed by the observability subscales (Spearman rho correlations = 0.45 for licensed nurses, 0.51 for CNAs). The leadership and communication subscales demonstrated the weakest associations with intention to adopt.

Correlations Between the Intention-To-Adopt Measure and DOI-LTC Measurement Battery Subscale, Using Ordinal and Dichotomous Scoring

Table 3: Correlations Between the Intention-To-Adopt Measure and DOI-LTC Measurement Battery Subscale, Using Ordinal and Dichotomous Scoring

To address the second research question, we rank ordered the size of the relationship between perceptions of the innovation’s attributes and the intention to adopt, and compared them across staff types. Among both licensed nurses and CNAs, the associations were strongest for relative advantage, complexity, and observability; however, the relative importance of complexity and observability differed between these groups. Complexity had the next strongest association with intention to adopt for licensed nurses; for CNAs, observability had the next strongest association.

Discussion

We have described the development and initial testing of a new measurement battery designed to examine factors influencing the likelihood of adoption of an EBP in LTC. Participants were selected by random sampling of nursing personnel from NC who identified geriatrics or LTC as their primary practice setting. Ten of the 12 subscales in the DOI-LTC measurement battery showed acceptable to excellent levels of internal consistency reliability. In this sample, we have demonstrated that among nursing staff who work in LTC, their perceptions of the attributes of a new pain EBP were more strongly associated with their intention to adopt the new practice than their perceptions of organizational variables such as leadership and communication, which also may affect successful implementation of new care practices.

These findings are consistent with previous studies conducted within the DOI framework in other fields. DOI theory suggests that end-user perceptions of innovation attributes strongly influence how quickly an innovation is adopted and put into practice. Relative advantage has consistently been shown to be one of the most important attributes that affect rate of adoption of innovations (Greenhalgh et al., 2005; Tornatzky & Klein, 1982). In this study, relative advantage was most strongly associated with intention to adopt the new pain care practice.

Also consistent with other studies cited by Rogers (2003), in this study, LTC nursing staff’s perceptions of observability, complexity (measured as ease of use), and image were positively associated with intention to adopt. The complexity subscale was second most strongly associated with intention to adopt a new practice among licensed nurses but was less strongly associated with intention to adopt among CNAs. Thus, although similarities exist between LTC staff types with respect to the general pattern of association between perceived attributes of evidence-based care practices and intention to adopt, some differences exist in the magnitude of the relationship on how perceptions of a new pain EBP are related to intention to adopt that may influence how leaders approach facilitating the implementation process.

Some attributes of the innovation were less strongly associated with intention to adopt the new pain care practice. Contrary to our expectations, compatibility of the new pain care practices with existing work practices was not strongly associated with CNAs’ intention to adopt (Spearman rho correlation = 0.26), although it was moderately associated with licensed nurses’ intention to adopt (Spearman rho correlation = 0.41). Trialability and voluntariness were not consistently related to intention to adopt; however, since these subscales had low reliability, we cannot tell whether that is due to a true lack of relationship or to an instrument that inadequately represents these constructs.

Current research on adoption of EBPs among health care providers suggests that several aspects of the care environment influence adoption of new care practices, including patient, social system, and unit-level factors (Greenhalgh et al., 2004; Titler et al., 2009). Since the DOI-LTC Measurement Battery captures multiple constructs contained in the DOI model, it should advance LTC investigators’ ability to systematically probe the influence of nursing staff-related factors, such as their perceptions of EBPs, at the same time that unit level factors, such as leadership and communication patterns, are measured. To our knowledge, this is the first such measure to tap multiple DOI constructs in LTC, a setting with many new opportunities to implement changes in care practice as the knowledge base for care of older adults expands.

Limitations

This study has several limitations that must be considered when interpreting its findings. The first limitation concerns the nature of the respondents to the survey. The study was conducted in only one state, and the response rate from the survey was low. Participants may differ in important ways from LTC nursing staff nationally; therefore, the findings regarding differences in how staff view a given care practice may not generalize to other LTC nursing staff members. Additionally, the instruments were only tested with nursing staff. Although nursing staff are the most numerous staff in LTC, current workforce trends suggest that the LTC workforce will become more diverse with respect to background and training (Institute of Medicine, 2008), and for this reason future work should incorporate, where possible, perceptions of the remainder of LTC staff.

A second limitation concerns the reliability of some of the instrument subscales. Although the majority of the subscales demonstrated acceptable internal consistency reliability, three subscales were problematic. The trialability and voluntariness scales showed unacceptable internal consistency in both licensed nurses and CNAs, and the compatibility subscale was below standard for the CNA workgroup. The low reliability of these subscales should raise caution with respect to how effectively these constructs were tapped by the measurement battery. Strategies for improving the reliability of these subscales include obtaining additional feedback from nursing staff working in LTC to reword existing items and developing new items that better tap the constructs of complexity, trialability, and voluntariness from the CNA perspective. Using larger numbers of CNAs on subsequent expert panels may also improve the quality of the item pool or item revisions. Once these approaches are taken, techniques such as factor analysis should be used to further evaluate the instruments’ underlying structure and construct validity. Factor analysis also holds the potential to identify opportunities to reduce the time needed to complete the instrument, thereby increasing the feasibility of its use with busy LTC staff. The instrument’s current length may be a deterrent to its use.

A third limitation concerns our testing of the instrument with only one EBP (a pain thermometer), at one point in time, to examine the relationships among DOI constructs. The DOI-LTC measurement battery was developed with the intention of being able to substitute different types of EBPs into the questionnaire items, in a manner analogous to how self-efficacy questionnaires are used. For the DOI-LTC measurement battery to serve this purpose, its psychometric properties will need to be evaluated using other EBPs and should be tested during actual implementation of an EBP. Although we demonstrated moderately strong associations between staff perceptions of an EBP and their reported intention to adopt the EBP, we do not know the relationship between intention to adopt and actual behavior among nursing staff in LTC. Given the early stage of work in developing measures that predict readiness to adopt EBP innovations, we believe that intention to adopt an evidence-based care innovation was a reasonable first criterion against which to assess the construct validity of the DOI-LTC measurement battery. However, these measures will require further testing to see whether they are equally or even more useful in predicting adoption behavior. Studies that examine the relationships between perceptions of a new care practice and actual implementation would provide a much stronger test of the relationship between perceptions of the attributes of an evidence-based care practice and its rate of adoption.

Conclusion

The DOI-LTC measurement battery holds promise as an important new method in understanding the process of EBP implementation in LTC. The measures contained in this new battery are feasible for use with LTC staff, and the majority of the subscales in the measure show acceptable reliability. Consistent with prior DOI research, staff perceptions of EBPs have a moderately strong association with intention to adopt a new care practice, and these perceptions are more strongly associated with intention to adopt than are organizational characteristics such as leadership quality and communication patterns. The DOI-LTC measurement battery can be used to promote development of a more robust scientific basis for understanding practice change in this growing sector of health care as systematic insights from the DOI model are developed for use in LTC.

References

  • Anderson, R.A., Corazzini, K.N. & McDaniel, R.R. Jr.. (2004). Complexity science and the dynamics of climate and communication: Reducing nursing home turnover. The Gerontologist, 44, 378–388. doi:10.1093/geront/44.3.378 [CrossRef]
  • Bakerjian, D. (2008). Care of nursing home residents by advanced practice nurses: A review of the literature. Research in Gerontological Nursing, 1, 177–185. doi:10.3928/00220124-20091301-01 [CrossRef]
  • Baldrige National Quality Program. (2011). Are we making progress? Retrieved from the National Institute of Standards and Technology website: http://www.nist.gov/baldrige/publications/progress.cfm
  • Berta, W., Teare, G.F., Gilbart, E., Ginsburg, L.S., Lemieux-Charles, L., Davis, D. & Rappolt, S. (2010). Spanning the know-do gap: Understanding knowledge application and capacity in long-term care homes. Social Science & Medicine, 70, 1326–1334. doi:10.1016/j.socscimed.2009.11.028 [CrossRef]
  • Berwick, D.M. (2003). Disseminating innovations in health care. Journal of the American Medical Association, 289, 1969–1975. doi:10.1001/jama.289.15.1969 [CrossRef]
  • Cadogan, M.P., Schnelle, J.F., Al-Sammarrai, N.R., Yamamoto-Mitani, N., Cabrera, G., Osterweil, D. & Simmons, S.F. (2005). A standardized quality assessment system to evaluate pain detection and management in the nursing home. Journal of the American Medical Directors Association, 6, 1–9. doi:10.1016/j.jamda.2005.12.011 [CrossRef]
  • Castle, N.G. (2001). Innovation in nursing homes: Which facilities are the early adopters?The Gerontologist, 41, 161–172. doi:10.1093/geront/41.2.161 [CrossRef]
  • Dillman, D.A. (2000). Constructing the questionnaire. In Mail and internet surveys: The tailored design method (2nd ed., pp. 79–148). New York: Wiley and Sons.
  • Doak, C.C., Doak, L.G. & Root, J.H. (1996). Teaching patients with low literacy skills (2nd ed.). Philadelphia: Lippincott Williams & Wilkins.
  • Gagnon, M.-P., Godin, G., Gagné, C., Fortin, J.-P., Lamothe, L., Reinharz, D. & Cloutier, A. (2003). An adaptation of the theory of interpersonal behaviour to the study of telemedicine adoption by physicians. International Journal of Medical Informatics, 71, 103–115. doi:10.1016/S1386-5056(03)00094-7 [CrossRef]
  • Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P. & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly, 82, 581–629. doi:10.1111/j.0887-378X.2004.00325.x [CrossRef]
  • Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., Kyriakidou, O. & Peacock, R. (2005). Storylines of research in diffusion of innovation: A meta-narrative approach to systematic review. Social Science & Medicine, 61, 417–430. doi:10.1016/j.socscimed.2004.12.001 [CrossRef]
  • Herr, K., Bjoro, K., Steffensmeier, J. & Rakel, B. (2006). Acute pain management in older adults. Retrieved from the National Guideline Clearinghouse website: http://www.guideline.gov/content.aspx?id=10198
  • Hughes, J.R., Keely, J.P., Fagerstrom, K.O. & Callas, P.W. (2005). Intentions to quit smoking change over short periods of time. Addictive Behaviors, 30, 653–662. doi:10.1016/j.addbeh.2004.08.011 [CrossRef]
  • Institute of Medicine (2008). Retooling for an aging America: Building the health care workforce. Retrieved from the National Academies Press website: http://books.nap.edu/openbook.php?record_id=12089
  • Kovach, C.R., Morgan, S., Noonan, P.E. & Brondino, M. (2008). Using principles of diffusion of innovation to improve nursing home care. Journal of Nursing Care Quality, 23, 132–139. doi:10.1097/01.NCQ.0000313762.79396.ec [CrossRef]
  • Mesters, I. & Meertens, R.M. (1999). Monitoring the dissemination of an educational protocol on pediatric asthma in family practice: A test of associations between dissemination variables. Health Education & Behavior, 26, 103–120. doi:10.1177/109019819902600109 [CrossRef]
  • Miles, R.W. (2010). Cognitive bias and planning error: Nullification of evidence-based medicine in the nursing home. Journal of the American Medical Directors Association, 11, 194–203. doi:10.1016/j.jamda.2009.08.007 [CrossRef]
  • Moore, G.C. & Benbasat, I. (1991). Development of an instrument to measure the perceptions of adopting an information technology innovation. Information Systems Research, 2, 192–222. doi:10.1287/isre.2.3.192 [CrossRef]
  • North Carolina Center for Nursing. (2007). North Carolina trends in nursing: 1987–2006. RN and LPN workforce demographics. Retrieved from the Health Workforce Information Center website: http://www.hwic.org/pdf/demographic_trends.pdf
  • Probst, J.C., Baek, J.-D. & Laditka, S.B. (2009). Characteristics and recruitment paths of certified nursing assistants in rural and urban nursing homes. Journal of Rural Health, 25, 267–275. doi:10.1111/j.1748-0361.2009.00229.x [CrossRef]
  • Roberts, K.H. & O’Reilly, C.A. (1974). Measuring organizational communication. Journal of Applied Psychology, 59, 321–326. doi:10.1037/h0036660 [CrossRef]
  • Roe, B., Watson, N.M., Palmer, M.H., Mueller, C., Vinsnes, A.G. & Wells, M. (2004). Translating research on incontinence into practice. Nursing Research, 53(6 Suppl.), S56–S60. Retrieved from http://www.nursingcenter.com/pdf.asp?AID=535028 doi:10.1097/00006199-200411006-00009 [CrossRef]
  • Rogers, E.M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.
  • Squillace, M.R., Remsburg, R.E., Harris-Kojetin, L.D., Bercovitz, A., Rosenoff, E. & Han, B. (2009). The National Nursing Assistant Survey: Improving the evidence base for policy initiatives to strengthen the certified nursing assistant workforce. The Gerontologist, 49, 185–197. doi:10.1093/geront/gnp024 [CrossRef]
  • Stetler, C.B. (2003). Role of the organization in translating research into evidence-based practice. Outcomes Management, 7, 97–103.
  • Titler, M.G. (2004). Methods in translation science. Worldviews on Evidence-Based Nursing, 1, 38–48. doi:10.1111/j.1741-6787.2004.04008.x [CrossRef]
  • Titler, M.G., Herr, K., Brooks, J.M., Xie, X.-J., Ardery, G., Schilling, M.L. & Clarke, W.R.,… (2009). Translating research into practice intervention improves management of acute pain in older hip fracture patients. Health Services Research, 44, 264–287. doi:10.1111/j.1475-6773.2008.00913.x [CrossRef]
  • Tornatzky, L.G. & Klein, K.J. (1982). Innovation characteristics and innovation adoption-implementation: A meta-analysis of findings. IEEE Transactions on Engineering Management, 29, 28–45.
  • Willis, G.B. (1999). Cognitive interviewing: A “how to” guide. Retrieved from the National Cancer Institute website: http://appliedresearch.cancer.gov/areas/cognitive/interview.pdf
  • Willis, G.B., Royston, P. & Bercini, D. (1991). The use of verbal report methods in the development and testing of survey questionnaires. Applied Cognitive Psychology, 5, 251–267. doi:10.1002/acp.2350050307 [CrossRef]

Characteristics of Respondents to Mailed Survey (N = 197)

RNs (n= 50) LPNs (n= 45) CNAs (n= 102)
Characteristic n(%) n(%) n(%)
Age
  <50 19 (38) 17 (37.8) 48 (47)
  ⩾50 23 (46) 22 (48.9) 33 (32.4)
  Missing 8 (16) 6 (13.3) 21 (20.6)
Race
  Caucasian 46 (92) 26 (57.8) 54 (52.9)
  African American 3 (6) 15 (33.3) 40 (39.2)
  Asian 0 (0) 0 (0) 4 (3.9)
  Other 0 (0) 1 (2.2) 4 (3.9)
  Missing 1 (2) 3 (6.7) 0 (0)
Gender
  Women 48 (96) 41 (91.1) 98 (96.1)
  Men 2 (4) 4 (8.9) 4 (3.9)
Educational level
  Junior high school 0 (0) 0 (0) 1 (1)
  High school or equivalent 0 (0) 3 (6.7) 22 (21.6)
  Some college 2 (4) 20 (44.4) 42 (41.2)
  2-year college 22 (44) 14 (31.1) 20 (19.6)
  4-year college 12 (24) 3 (6.7) 4 (3.9)
  Other 13 (26) 4 (8.9) 7 (6.9)
  Missing 1 (2) 1 (2.2) 6 (5.9)
Length of time in position
  1 year or less 7 (14) 4 (8.9) 22 (21.6)
  >1 to 5 years 13 (26) 21 (46.7) 42 (41.2)
  5 to 10 years 17 (34) 10 (22.2) 13 (12.7)
  10 or more years 12 (24) 10 (22.2) 13 (12.7)
  Missing 1 (2) 0 (0) 12 (11.8)

Key Descriptive Statistics and Reliability Coefficients for DOI-LTC Measurement Battery Subscales By Nursing Staff Type

Licensed Nurses Unlicensed Staff
RNs (n= 50) LPNs (n= 45) CNAs (n= 102)
Subscale Sample Item Number of Subscale Items Cronbach’s Alpha Coefficient Mean (SD), Range Mean (SD), Range Number of Subscale Items Cronbach’s Alpha Coefficient Mean (SD), Range
Relative advantage Using the new pain care practices will result in improved resident comfort. 12 0.95 56.2 (10.5), 18 to 72 51.9 (13.3), 26 to 70 10 0.91 50.1 (8.3), 14 to 60
Compatibility Using the new pain care practices will fit in with how I like to do my work. 6 0.78 28.6 (3.9), 15 to 34 26.2 (5.1), 14 to 35 5 0.56 20.9 (4.2), 11 to 30
Complexity (ease of use)a Using the new pain care practices will require a lot of mental effort. 8 0.77 39.6 (4.9), 23 to 48 37.4 (5.4), 25 to 47 6 0.79 28.5 (4.5), 17 to 36
Observability Using the new pain care practices will show families and visitors we care about treating resident pain. 6 0.87 25.6 (5.5), 9 to 36 24.1 (6.6), 8 to 33 5 0.78 21.4 (5.3), 7 to 30
Image Using the new pain care practices will help coworkers see me as a more valuable team member. 6 0.86 25.0 (5.5), 11 to 34 23.6 (6.2), 11 to 33 5 0.85 17.4 (4.8), 6 to 24
Trialability There are enough people in my work area to help me try out the new practices. 6 0.50 22.6 (3.6), 10 to 29 21.0 (3.4), 13 to 27 4 0.40 17.1 (2.9), 9 to 23
Voluntariness I would probably get into trouble if I didn’t use the new pain care practices. 6 0.65 21.2 (3.3), 13 to 27 20.8 (3.7), 11 to 29 5 0.69 17.7 (3.9), 10 to 26
Leadershipb I know my facility’s mission (i.e., what it is trying to accomplish). 7 0.91 26.0 (5.7), 7 to 35 24.6 (5.6), 10 to 35 7 0.90 26.1 (5.7), 7 to 35
Communication openness It is easy for me to talk openly with all staff in this facility. 5 0.89 17.8 (3.9), 8 to 25 16.2 (4.2), 7 to 25 5 0.92 17.5 (4.6), 5 to 25
Communication accuracy The information I receive is often inaccurate. 5 0.90 15.6 (3.7), 9 to 23 15.7 (4.6), 6 to 25 5 0.86 16.4 (4.2), 5 to 25
Communication timeliness I get information on the status of residents when I need it. 4 0.70 14.7 (2.5), 10 to 20 14.9 (2.9), 8 to 20 4 0.69 13.8 (3.0), 5 to 20

Correlations Between the Intention-To-Adopt Measure and DOI-LTC Measurement Battery Subscale, Using Ordinal and Dichotomous Scoring

Licensed Staff’s (RNs and LPNs,n= 95) Intention to Adopt CNAs’ (n= 102) Intention to Adopt
Subscale Rank Order Spearman Rho Correlation (pValue)a pValue from Wilcoxon Rank-Sum Testb Rank Order Spearman Rho Correlation (pValue)a p Value from Wilcoxon Rank-Sum Testb
Relative advantage 1 0.68 (<0.0001) <0.0001 1 0.54 (<0.0001) 0.0004
Complexity 2 0.60 (<0.0001) 0.0004 3 0.44 (<0.0001) 0.0004
Observability 3 0.45 (<0.0001) <0.0001 2 0.51 (<0.0001) 0.0006
Image 4 0.45 (<0.0001) 0.001 4 0.41 (<0.0001) 0.018
Compatibility 5 0.41 (0.0001) 0.007 9 0.26 (0.02) 0.14
Trialability 6 0.27 (0.01) 0.002 5 0.38 (0.0004) 0.06
Voluntariness 7 0.29 (0.01) 0.28 NS 0.12 (0.29) 0.19
Leadership 8 0.28 (0.01) 0.35 6 0.36 (0.0007) 0.08
Communication openness 9 0.23 (0.03) 0.16 7 0.32 (0.003) 0.07
Communication accuracy NS 0.08 (0.49) 0.76 NS 0.08 (0.45) 0.25
Communication timeliness NS 0.08 (0.46) 0.73 8 0.30 (0.005) 0.06
Authors

Dr. McConnell, Dr. Corazzini, and Dr. Bailey are Associate Professors, Ms. Lekan is Clinical Associate, Mr. Sloane is Statistician, Duke University School of Nursing, Dr. Landerman is Associate Professor, Division of Geriatric Medicine, Duke University School of Medicine, and Dr. Champagne is Laurel Chadwick Professor of Nursing and Professor, Duke University School of Nursing and Department of Community and Family Medicine, Durham, North Carolina. In addition, Dr. Corazzini, Dr. Bailey, and Dr. Champagne are Senior Fellows, Mr. Sloane is Statistician, and Dr. Landerman is Associate Professor, Duke Center for the Study of Aging and Human Development, Durham, North Carolina. Dr. McConnell is also Clinical Nurse Researcher, Durham Veterans Affairs Medical Center, Geriatric Research, Education and Clinical Center, Durham, North Carolina.

The authors disclose that they have no significant financial interests in any product or class of products discussed directly or indirectly in this activity. The authors acknowledge funding from the following sources: Duke University (National Institutes of Health [NIH] 1 P20-NR07795-01), Trajectories of Aging and Care in Nursing Science (Eleanor S. McConnell & Elizabeth C. Clipp, PI); University of Iowa (NIH P30-NR03979) (Toni Tripp-Reimer, PI) and the University of Iowa John A. Hartford Center of Geriatric Nursing Excellence (Kathleen C. Buckwalter, PI); and an Alzheimer’s Association Medical and Scientific Division Investigator Initiated Research Grant 05-14332 (Sheryl Zimmerman, PI).

Address correspondence to Eleanor S. McConnell, PhD, RN, GCNS-BC, Associate Professor, Duke University School of Nursing, Box 3322 DUMC, Durham, NC 27710; e-mail: eleanor.mcconnell@duke.edu.

Received: December 31, 2009
Accepted: December 07, 2010
Posted Online: June 29, 2011

10.3928/19404921-20110602-04

Sign up to receive

Journal E-contents