Thomas A. Edison said, "If there is a way to do it better, find it" (cited in Davis, 1984). Orientation programs for new nursing employees are repetitive. They are driven by the policies and procedures unique to the employing institution. This challenges staff development educators to find effective methods to present these programs. To be effective, teaching methods must be compatible with the employees' learning needs. A formative evaluation can help staff development educators assess how well they are meeting these needs. Formative evaluation, as described by Tyler in Sarnecky's article (1990), ". . . is a continual gathering of information throughout an education program for purposes of diagnosis and adaptation of the program" (p. 26).
Staff development managers can use formative evaluation to monitor elements of a program. Formative evaluation aids the manager in revising program components. This article reports the use of a formative program evaluation process with a nursing orientation program at a 350-bed Department of Veteran Affairs Medical Center.
Worthen and Sanders (1987) state that "Formative evaluation is conducted during the operation of a program to provide program directors evaluative information useful in improving the program" (p. 34). A program evaluation tool designed for this purpose can ease the process. In this study, the idea for the tool was inspired by the Massachusetts General Hospital Staff Education Manual (Stetler, 1982). An evaluation tool was designed to meet this purpose.
In 1985 the nurse employee orientation program underwent a major revision. Conversion of core program content to Self Learning Modules (SLMs) decreased the amount of lecture-discussion time. The revised format allowed instructors to provide individualized and flexible orientation schedules. The SLMs ranged from single modules to modules combined with audiovisual programs. Skill practice sessions and guest speakers provided variety in the program format. The formative evaluation process assessed the effectiveness of the revised program format.
REVIEW OF THE LITERATURE
Self-learning modules, as documented in the literature, are a suitable teaching strategy for nurses in the work setting (Cochenour, 1992; Miller, 1989; Rosenthal, 1989). Coleman, Dracup, and Moser (1991) found support for the modular instruction in CPR training; retention of knowledge and skills equaled the didactic method of teaching CPR. Other major advantages cited are cost effectiveness, flexibility of scheduling, and better use of instructor time (Leidy, 1992; Miller, 1989; Schmidt & Fisher, 1992). Prociuk (1990) studied nurses' perceptions of self-directed learning after experience with a unit-specific competency-based orientation program. Prociuk found that most nurses preferred self-directed learning.
Figure 1. Mean years of work experience.
SLMs incorporate principles of adult learning. They aiiow the learner to be self-directive, to apply knowledge from past experiences, and to relate content to the work setting. Formative program evaluation that occurs during the learning experience can help the educator adjust content or methods to meet learners' needs.
PURPOSE OF THE SURVEY
The formative evaluation process surveyed nursing employees' responses to the revised orientation program. The purposes of the survey were to determine the preferred teaching method (SLMs or lecturediscussion), and to provide a basis for program refinement. Based on the design of the orientation program evaluation tool, the survey questions were:
1. What is the preferred learning method for nursing orientees?
2. What is the preferred learning method based on education level? (Diploma, associate degree, baccalaureate, master's, or licensed practical nurse?)
3. Are there significant differences of preferred learning styles between the education levels?
4. Are there significant differences of preferred learning styles between years of experience?
5. Did the modules allow the orientees to proceed at their own pace?
6. Did the modules hold the orientees' interest?
The variables measured in the survey were education level, years of nursing experience, pacing, learning method, and interest.
Nursing instructors directed new employees regarding the completion and collection of the orientation evaluation tool. Orientees were to return the tools at the end of their unit orientation. The orientation program consisted of two weeks in the classroom and two weeks on the assigned unit. The head nurse and /or a preceptor managed the unit orientation. The collection of the tools at the end of the four-week period created problems. Despite reminders by the instructors, some orientées were more cooperative than others in returning the tools. Evaluation tools collected between 1986 and 1989 were analyzed- A total of 312 new employees completed the orientation program, and 126 (40%) of the tools were retrievable.
Selected portions of the evaluation tool yielded data for the survey. The demographic data recorded date of orientation, type and degree of nursing programs, year graduated, and date of last work experience. Specific questions evaluated the preferred teaching method - whether the modules allowed self-pacing and whether the modules held the employees' interest. Three openended questions assessed the orientees' perceptions of the classroom and unit orientation milieu.
Descriptive statistical procedures examined the frequency distributions of the variables and crosstabulation of preferred learning styles as a function of education levels and years of nursing experience. Content analysis of the open-ended questions for themes and issues provided a basis for program refinement.
The demographic profile of employees' levels of education preparation were: 41 licensed practical nurses (32.5%); 32 baccalaureate (25.4%); 25 associate degree (19.8%); 23 diploma (18.3%); and 5 master's (4.0%).
Figure 1 displays the mean number of years worked by each education group. Subtraction of the year graduated from the year of orientation gave the number of years worked. The author verified the accuracy of the work history. The diploma group had the highest mean years of work experience (24 years). The mean number of years of work experience for all employees was 11.47 (standard deviation [SD] ±9.77).
Study Question 1. "What is the preferred learning method for the majority of nursing orientées?" The orientées preferred the SLMs over the lecturediscussion method as displayed in Figure 2. "Both" was not an option on the evaluation tool, but 15.6% of the respondents wrote it in.
Study Question 2. "What is the preferred learning method based on education level?" Figure 3 displays how each teaching method rated within the educational groups. The diploma and master's groups preferred the SLMs. The baccalaureate, associates, and LPNs preferred the lecture-discussion method.
Study Question 3. "Are there significant differences of preferred learning styles between the education levels?" Application of the chi-square statistic revealed no significant differences (p = .01, df= 8, ? = 122) between preferred learning styles and education level.
Figure 3. Preferred learning methods between educational levels.
Study Question 4. "Are there significant differences of preferred learning styles between years of experience?" Again, as measured by chi-square statistic, there were no significant differences between preferred learning styles and years of experience, even when examined by educational level.
Study Question 5. "Did the modules allow the orientees to proceed at their own pace?" An overwhelming majority of the orientees (98.4%) indicated that the SLMs allowed them to master the content at their own pace.
Study Question 6. "Did the modules hold the orientees' interest?" Most of the new employees (79%) responded that the modules held their interest. A few (9.7%) stated the modules did not hold their interest, and 11.3% stated the modules held their interest only some of the time.
Content analysis of the three open-ended questions regarding expectations of the classroom and unit orientation indicated problems with transition from classroom to the unit. Employees felt the need for more patient care, more preceptor time, and more orientation to unit routine before functioning independently.
The majority of nursing orientees confirmed support for SLMs as the preferred method. However, 15.6% of the nurses wrote in "both" as a response. The reviewed literature supports using a variety of teaching strategies to motivate adult learners (Cochenour, 1992; Linares, 1989; Prociuk, 1990; Rosenthal, 1989).
The unexpected response that the BSN, ADN, and LPNs preferred the lecture-discussion method posed more questions. Why do diploma graduates like selfstudy, and why do BSN graduates prefer lecturediscussion? A further review of the literature did not reveal definitive correlations between learning styles and education level.
Linares (1989) studied the learning characteristics of RN and generic students in a baccalaureate program. The majority of the RN students had initial ADN degrees. There were no significant differences in preferences for self-directed learning and learning styles between the two groups. Linares found that age significantly influenced the readiness of students to engage in self-directed learning, i.e., older students were more ready to engage in self-directed learning. Age was not a demographic variable in the present survey. However, the diploma group preferred the self-study modules. They represented an older population with the highest mean years of work experience (24 years and SD ± 10.80) and a longer span from formal education experience (1963; SD ±10.66). As a result, they may be more comfortable pursuing learning at their own pace. They may not enjoy an environment where they are expected to participate in discussions.
Wells and Higgs (1990) studied learning styles and learning preferences in first and fourth semester baccalaureate nursing students. Using the Gregore style delineator, they found no significant changes in learning style scores from first to fourth semester. Their assessment of preferred learning methods ranked independent study (57.4%), workbooks (51.1%) and slide/ filmstrips (57.4%). Drill and practice, television/ movies, group discussion, and short lectures with questions and answers ranked highest.
Without further study, these findings cannot be generalized. Lecture-discussion is a prevalent method for delivering content in the academic setting. Perhaps the BSN and ADN graduates are more comfortable with this approach. The mean year of graduation for the BSN and ADN group was 1980 (SD ±7.64 and ±5.04). The mean year of graduation for the LPN group was 1977 (SD ±7.5). The mean year of graduation for the diploma was 1963 (SD ±10.66). The BSN and ADN group had more recent experience in a formal education setting.
While 79% responded that the modules held interest, 21% selected either a negative or "sometimes" response. Perhaps the information lacked clarity and ease of comprehension. It is difficult to keep modules current with changes in policies and procedures. A scheduled review of the modules can help ensure clarity and currency. Handwritten notations can update the modules until revised.
Written comments to open-ended questions revealed problems in transition from classroom to unit orientation. The evaluation tool format was not sensitive to specific unit orientation issues. A different type of evaluation is required to assess these needs. As Mast and Van Atta (1986) reported, it is important that instructors link modular learning experiences to clinical problem solving. The instructors must be vested in the self-learning process to articulate the program objectives to the orientees.
This study's formative evaluation process provided a basis for program refinement. Some of the implemented changes include:
* Separation of classroom and unit orientation into two distinct phases of evaluation.
* Addition of a clinical competency component and instructor to the unit orientation phase. This links classroom content with clinical routines.
* Consolidation and reduction of the number of SLMs.
* Time with SLMs alternated with other teaching methods, e.g., short lecture-discussion, clinical labs.
* Collection of evaluation tools at the end of the classroom portion.
* Periodic review of individual orientee files for completion.
* Modules reviewed and revised on a regular schedule.
Success of the formative evaluation survey and program changes is often revealed in written comments on recent evaluation tools. A sampling of comments include: "The best orientation I have ever experienced"; "a thorough and applicable orientation"; "this is the most informative orientation I ever had"; "it touched on pertinent/ different issues"; "the information was reviewed in the modules and reviewed again through demonstration"; "a nice balance between hands-on and self-paced learning''; "SLMs were effective modes"; "the orientation was above and beyond what I expected."
Challenges remain for staff development educators to conduct studies to analyze preferred learning styles and appropriate teaching strategies in the work environment. Formative program evaluation can provide clues for program refinement as well as learner responses to teaching methodologies.
- Cochenour, C. (1992). Self-learning packages in staff development. Journal of Nursing Staff Development, 8, 123-127.
- Coleman, S., Dracup, K., & Moser, D.K. (1991). Comparing methods of cardiopulmonary resuscitation instruction on learning and retention, journal of Staff Development, 7, 82-87.
- Davis, W. (1984). The best of success: A handbook of success ideas. Lombard, IL: Great Quotations, Inc.
- Leidy, K. (1992). Effective screening and orientation of independent contract nurses. The Journal of Continuing Education in Nursing, 23, 64-68.
- Linares, A.Z. (1989). A comparative study of learning characteristics of RN and generic students. Journal of Nursing Education, 28, 354-360.
- Mast, M.E., & Van Atta, MJ. (1986). Applying adult learning principles in instruction design. Nurse Educator, 1 1, 35-39.
- Miller, P.J. (1989). Developing self-learning packages, journal of Nursing Staff Development , 3, 73-77.
- Prociuk, J.L. (1990). Self-directed learning and nursing orientation programs: Are they compatible? The Journal of Continuing Education in Nursing, 21, 252-256.
- Rosenthal, K.A. (1989). Multimethod teaching modules. Dimensions of Critical Care Nursing, 8, 310-316.
- Sarnecky, M.T. (1990). Program evaluation part 1: Four generations of theory. Nurse Educator, 25(5), 25-28.
- Schmidt, K.L., & Fisher, J.C. (1992). Effective development and utilization of seif-ieaming modules. The Journal of Continuing Education in Nursing, 23, 54-59.
- Stetler, C.B. (Ed.) (1982) Massachusetts General Hospital staff education manual. Englewood Cliffs, NJ: Reston Publishing Co.
- Wells, D., & Higgs Z.R. (1990) Learning styles and learning preferences of first and fourth semester baccalaureate degree nursing students, journal of Nursing Education, 29, 385-390.
- Worthen, B., & Sanders, J.R. (1987). Educational evaluation. New York, NY: Longman.