The Journal of Continuing Education in Nursing

Administrative Angles 

Measuring Change as a Result of Participation in Educational Activities

Jennifer Graebe, MSN, RN, NEA-BC

Abstract

This article addresses how nurse planners and nursing professional development practitioners can meet educational design process criterion 7 in the American Nurses Credentialing Center primary accreditation criteria that also correspond with the Association for Nursing Professional Development Scope and Standards of Practice standard 6.

J Contin Educ Nurs. 2018;49(8):340–342.

Abstract

This article addresses how nurse planners and nursing professional development practitioners can meet educational design process criterion 7 in the American Nurses Credentialing Center primary accreditation criteria that also correspond with the Association for Nursing Professional Development Scope and Standards of Practice standard 6.

J Contin Educ Nurs. 2018;49(8):340–342.

This article explores the American Nurses Credentialing Center (ANCC) primary accreditation educational design process criterion 7 (EDP 7) that corresponds with the Association for Nursing Professional Development (ANPD) Scope and Standards of Practice standard 6 (Harper & Maloney, 2016). According to Dickerson and Graebe (2018), “utilization of the criteria and standards is recommended when planning any educational interventions as they provide a sequential process for content development” (p. 4), including evaluating and measuring for change at the level of educational need identified (knowledge, skill, and/or practice).

First Things First

The ANCC primary accreditation criterion EDP 7 requires that “the Nurse Planner measures change in knowledge, skills, and/or practices of the target audience that are expected to occur as a result of participation in the educational activity” (ANCC, 2015, p. 40). The ANPD scopes and standards of practice standard 6 require that “the nursing professional development practitioner evaluates progress toward attainment of outcomes” (Harper & Maloney, 2016, p. 41). Both the ANCC EDP 7 criterion and the ANPD standard 6 require that change is measured as a result of RNs and/or members of the health care team participating in the educational activity. To fully understand the effects of measuring change on nursing professional development patient and health outcomes, we must review the key components of the educational design process that make measuring change effective.

As mentioned in previous articles in this Administrative Angles series, the first step in the educational design process (EDP 1) is the identification of the practice gap, which is the “distance between where things are now and where they could or should be” (Dickerson & Graebe, 2018). The process of identifying a professional practice gap is strategic and not operational. It requires the nurse planner (NP) or nursing professional development (NPD) practitioner to identify the opportunity or issue at the beginning of the educational activity planning process. Educational activities are to be focused on a preferred or desired state. By drilling down to the root cause of an issue or opportunity, educational activities are designed to close an identified professional practice gap.

After the professional practice gap is identified, the NP or NPD practitioner analyzes the practice gap to determine where the underlying educational need exists—knowledge, skill, and/or practice. A gap analysis is completed to determine the underlying educational need after the professional practice gap is identified. As per Moyer and Graebe (2018), “EDP 2 is the identification of the appropriate educational need (knowledge, skill and/or practice) that validates the established practice gap” (p. 52). By identifying the underlying educational need, the NP or NPD practitioner “focuses on what is required to fill the distance between what is occurring and what should be occurring related to practice and/or patient outcomes” (Moyer & Graebe, 2018, p. 52).

How to Measure Change?

Why is identifying the professional practice gap and underlying educational need important when measuring change as a result of learners participating in an educational activity? According to Dickerson and Graebe (2018), “there is no realistic way to measure the value of the educational intervention” (p. 4) if the professional practice gap and underlying educational need are not identified.

To measure change, the NP or NPD practitioner must evaluate the activity at the level of educational need identified: knowledge, skill, and/or practice. The NP or NPD practitioner must ensure that measuring change in knowledge, skill, and/or practice is directly related to the identified professional practice gap and the desired learner outcomes.

The goal in measuring change is to answer what would or should be expected of the learners to know, know how to do, or do because of participating in the educational activity. Key questions must be answered to effectively evaluate where change has occurred and how it will be measured:

  • What is the desired change and where will that occur?
  • How will you determine if the activity was effective?
  • How will you know change occurred?
  • What methods will be used to measure change?

Asking these questions is a critical component of the ANCC primary accreditation criterion EDP 7 and ANPD standard 6. Determining what and how change will be evaluated is critically significant in measuring whether the educational activity achieved its desired outcome.

Methods of Evaluating Change

The NP or NPD practitioner is responsible for choosing the method that best aligns with the desired change and underlying educational need. Although there is no prescribed method for evaluating change, the activity must be evaluated for change at the level of the underlying need identified. When assessing knowledge, skill, and/or practice, it is important to ensure that the method to evaluate and measure change is in parallel with the educational need identified. For example, if you are evaluating impact to practice, then the underlying need must be at the practice level.

The ANPD Scope and Standards of Practice standard 6 encourages that educational programming is evaluated using valid and reliable methods to ascertain outcomes achievement (Harper & Maloney, 2016). Methods to evaluate educational activities include but are not limited to: learner self-report related to knowledge attainment and/or intent to change practice, case study analysis, simulation, evidence of performance, and/or proxy measures such as quality, safety and performance data (ANCC, 2015). Methods to evaluate change may be formative and embedded in the educational activity or evaluated longitudinally. The NP or NPD practitioner must verify the ability to access identified methods for evaluating change prior to providing the activity. For example, access to patient data and safety reporting systems.

Applying the Principles

If we refer to the case study example provided in the second series of these articles by Moyer and Graebe (2018), the initial request for an educational activity was focused on a “perceived” educational need of knowledge related to nurses not implementing falls precautions after the emergency department experienced an increase in patients falling. Through a strategic gap analysis and environmental scan, the NP or NPD practitioner was able to determine a specific professional practice gap related to chemically impaired patients that were falling in the emergency department. The underlying educational need identified was nursing staff's lack of knowledge related to fall risks in this patient population affecting the ability to apply fall risk standards. The NP or NPD practitioner developed a knowledge-based program related to chemical impairment and fall risk, including methods to decrease the risk. The NP or NPD measured a change as a result of the nurses participating in the activity by incorporating the following methods:

  1. Change in knowledge was evaluated and measured by:

    • Pre- and posttests.
    • Formative assessment and evaluation in the learning environment through participant discussion and question and answer.

  2. Change in practice was evaluated and measured by:

    • Formative evaluation of intent to change practice, and if so, what?
    • Direct observation of application of the fall risk standards.
    • Longitudinally measuring a decrease in fall rates in the emergency department, with a measurable goal.

ANCC Implications

The provider unit must have a clearly defined process for meeting EDP 7 that can be tailored to an individual learner or teams of learners. The defined process is operationalized to ensure that the effectiveness of an educational activity is evaluated and that those results, in aggregate, are used to measure the desired change. The NP is responsible for ensuring that the evaluation method to measure change correlates with the identified professional practice gap and the underlying educational need. The results of measuring change are then synthesized and captured in the summative evaluation (EDP 6). It is important to note that “change in knowledge, skill, and/or practice may not occur for a variety of reasons” (ANCC, 2015, p. 26). In this case, the NP would identify the barriers or reasons for not achieving the desired measurable outcome.

References

  • American Nurses Credentialing Center. (2015). ANCC primary accreditation provider application manual. Silver Spring, MD: Author.
  • Dickerson, P.S. & Graebe, J. (2018). Analyzing gaps to design educational interventions. The Journal for Continuing Education in Nursing, 49, 4–6. doi:10.3928/00220124-20180102-02 [CrossRef]
  • Harper, M. & Maloney, P. (2016). Nursing professional development: Scope and standards of practice. Chicago, IL: Association for Nursing Professional Development.
  • Moyer, A. & Graebe, J. (2018). Identifying the underlying educational needs that contribute to the professional practice gap. The Journal for Continuing Education in Nursing, 49, 52–54. doi:10.3928/00220124-20180116-02 [CrossRef]
Authors

Ms. Graebe is Director of Primary and Joint Accreditation, American Nurses Credentialing Center, Silver Spring, Maryland.

The author has disclosed no potential conflicts of interest, financial or otherwise.

Author correspondence to Jennifer Graebe, MSN, RN, NEA-BC, Director of Primary and Joint Accreditation, American Nurses Credentialing Center, 8515 Georgia Avenue, Suite 400, Silver Spring, MD 20910; e-mail: jennifer.graebe@ana.org.

10.3928/00220124-20180718-02

Sign up to receive

Journal E-contents