Accreditation requirements (American Nurses Credentialing Center, 2015) and nursing professional development practice standards (Harper & Maloney, 2016) address evaluation at two levels: the success of educational activities in addressing identified practice gaps, and the overall effectiveness of the department or provider unit in contributing to the mission and goals of the organization.
Activity outcomes can only be measured in context with the identified gap or problem in practice that precipitated the need for the educational activity. As noted by Dickerson and Graebe (2016), “Without clear identification of the gap at the beginning of activity planning, there is no realistic way to measure the value of the educational intervention” (p. 4). This problem, or opportunity for improvement, can be identified as point A, whereas the desired outcome is point B. The desired outcome shows what the learner will know or be able to do by the end of the activity. The purpose of the activity is to narrow or close the practice gap, represented by the distance between point A and point B (Figure 1). The ability to measure this gap closure hinges on accurate identification of the problem and determination of the appropriate level of educational intervention. If evidence shows that the problem is caused by a lack of knowledge, then the focus of the education should be on improvement of knowledge. If evidence shows that the problem is related to lack of or improper performance of a skill, the focus of the education should be on acquisition of skill. Often, education is designed incrementally to first address knowledge, then improve skill, and ultimately change or reinforce practice (Baughman, 2019).
Measurement of gap closure.
Measurable activity outcomes are developed relating to the identified problem and are congruent with the level of educational need for learners. The evaluation plan for the activity should reflect how data will be collected to measure the outcome. Examples of measurable outcomes at the activity level are provided in Table 1.
Activity Level Outcomes
Department or Provider Unit Outcomes
Similarly, departmental or provider unit outcomes are based on identified gaps and are designed to measure progress in closure of those gaps over time. For departments or provider units that are part of a larger organization, alignment of the department or provider unit outcomes with the strategic initiatives of the organization helps to validate the importance of the department to the organization. Scheller (2016) recommended asking three key questions when determining outcome measures for a department or provider unit:
- What is important?
- What can be measured to demonstrate accomplishments?
- What measurable targets will provide evidence?
Setting goals and establishing outcome measures at least annually helps the department or provider unit establish an action plan for achievement of those outcomes through implementation of educational activities and other initiatives throughout the year. Without this plan in place, activity development can become fragmented and isolated. Lack of a cohesive plan impedes the ability of the department to demonstrate its value to the organization.
One helpful approach is to look at the organization's strategic goals when selecting the outcome measures for the provider unit. What gaps currently exist between the organization's goals and the knowledge, skills, and abilities of nurses and other members of the health care team? What can the provider unit do, over the course of the year, to close those gaps? Some gaps might be related to how the provider unit conducts its work; others relate to how the provider unit plans to measure how its work has improved the practice of nursing or the professional development of nurses. For example, the organization has purchased a learning management system (LMS), which has not been well used by nurses. An identified provider unit gap might be that currently only five activities are available to nurses in the organization's LMS, but learners are asking for more activities and a greater variety of topics. Using Scheller's (2016) questions, it is important that the LMS be effectively used to meet the needs of nurse learners. What can be measured over the course of a year might be the number of activities available on the LMS and/or the variety of activities available. A realistic measure might be that 10 activities, representing at least four different topics, will be available on the LMS within the next 12 months. After this outcome measure has been identified, the provider unit will have a clear road map to success. Not only will the provider unit be able to measure success to meet its own goals, but evidence will clearly show that the provider unit has supported the organization's goal of improving use of the LMS.
From the professional practice perspective, the organization may have a goal of decreasing the number of avoidable readmissions to the medical–surgical units. What is important to the provider unit in helping the organization achieve this goal? Numerous initiatives can be undertaken depending on the gaps that have been identified as the cause of higher than desired avoidable re-admissions. Examples of gaps might be lack of interprofessional collaboration in developing realistic discharge plans, lack of education for patients and families related to posthospitalization adherence to plans of care, and ineffective communication between the hospital's nursing unit and long-term care or home health providers who receive the patient post-hospitalization. On the basis of this information, the provider unit might develop a strategy of providing education to nurses to improve patient and family teaching, collaborate with other departments to develop interprofessional education focused on discharge planning, and work with long-term care and home health agencies to jointly provide education to improve continuity of care. Each of these activities will have its own measurable outcome, but collectively the provider unit will need to determine what can be measured to determine how it has contributed to the organizational goal.
Measuring both activity and provider unit/department outcomes is important. At both levels, the process starts with identifying gaps and identifying measurable goals. Working toward closure of gaps and measuring outcomes provides evidence of the value of the department in supporting the strategic initiatives of the organization.
- American Nurses Credentialing Center. (2015). ANCC primary accreditation provider application manual. Silver Spring, MD: Author.
- Baughman, C. (2019). Achieving incremental outcomes to reach goals. The Journal of Continuing Education in Nursing, 50, 193–195. doi:10.3928/00220124-20190416-02 [CrossRef]
- Dickerson, P. & Graebe, J. (2016). Analyzing gaps to design educational interventions. The Journal of Continuing Education in Nursing, 49, 4–6.
- Harper, M. & Maloney, P. (2016). Nursing professional development: Scope and standards of practice. Chicago, IL: Association for Nursing Professional Development.
- Scheller, M.K. (2016). Developing outcome measures for a continuing nursing education provider unit. The Journal of Continuing Education in Nursing, 47, 293–295. doi:10.3928/00220124-20160616-02 [CrossRef]
Activity Level Outcomes
|Sample 1||Sample 2||Sample 3|
|Problem||Learners have inconsistent levels of knowledge about caring for hospitalized patients with diabetes.||New faculty do not know how to provide constructive feedback to students in the clinical setting.||Members of the health care team are not communicating clearly and effectively in emergency situations.|
|Level of educational need||Knowledge||Knowledge||Skill|
|Outcome||100% of learners will pass a posttest with a score of 90% or better, demonstrating consistency in level of knowledge.||Given case scenarios with students practicing on a medical–surgical unit, 90% of new faculty will develop appropriate feedback comments to facilitate student growth.||In a simulation setting, all health care team members will communicate effectively as measured by a standardized team-based communication skills checklist.|