Journal of Nursing Education

The articles prior to January 2012 are part of the back file collection and are not available with a current paid subscription. To access the article, you may purchase it or purchase the complete back file collection here

The Critical Incident Technique Applied to the Evaluation of the Clinical Practicum Setting

Christy Z Dachelet, MS; Mary F Wemett, RN, MS; E Jean Garling, RN, MS; Kathleen Craig-Kuhn, RN, MS; Nancy Kent, RN, MSN; Harriet J Kitzman, RN, MS

Abstract

However, even with these precautions, the technique proved less than ideal. There was uneven enthusiasm, interest and commitment among the reporters. This led to variation both in the number and in the extent of specificity of the incidents reported. At times it was apparent that some reporters related incidents without having made a clear judgment as to why it was productive or counter-productive to their learning or teaching objective. The small number of reporters, each contacted repeatedly over the nine-week period, made the study more of a burden to the reporters than it might have been, had there been a larger group from which incidents could have been solicited. This problem was more apparent the second year when there were only half as many students.

The subjective nature of the data was not unexpected, but nevertheless it did prove cumbersome to data analysis and interpretation. Furthermore, there always remained an uneasiness that the meaning of reported frequencies on the incidents is uncertain. Bates speculated on the meaning of the frequencies of the various categories of incidents in herstudy on nurse-physician teamwork.5 She postulated four factors that might influence the frequency distribution of incidents among categories: "(l) the frequency of the actual behaviors themselves, (2) the importance of the behaviors, (3) the emotional impact of the behaviors on the (reporter), and (4) some discrepancy between standards or expectations of performance and the behaviors itself." She went on to speculate that these factors might also influence the proportion of "good" and "bad" incidents.

Although the technique was not without its shortcomings, it did have noteworthy strengths.

The incidents proved helpful to the faculty as they evaluated their teaching techniques and the clinic operations. Faculty found the incidents allowed and encouraged them to focus attention on specific clinic activities and teaching methods in a substantive way. At the same time, the data offered a holistic perspective on the clinical experience. Whereas some methodologies focus narrowly on single, isolated aspects of the clinical practicum, the critical incident technique provided a broad picture of what activities were occurring in the clinic, and how these were perceived by those learning and teaching there.

The critical incident technique also provided an interesting picture of the students and preceptors experiences and reactions to the clinical experience over time. The incidents reflected the progress of the student through the clinical experience. The initial incidents reported by the students reflected the mixed eagerness and apprehension over the first patient visit. The student expressed concern over how to present herself to the patient. There was an anxiety about how the patient would respond to her as a provider. The first two to three weeks were exciting for the student, and the students often had a very detailed recall of their first patient encounters. This was reflected in the number and specificity of the incidents reported. An element of frustration was recorded at this time in the incidents due to the high patient "no show" rate. The students eager to try out their new skills were disappointed when "their" patients did not show up.

The next series of incidents noted were those reported as the students began to see their first new patient. The experience was reported as positive when the patient presented with a single problem or was essentially healthy; for example, the patient seeking a pre-employment physical exam. The experience was often reported as "overwhelming" when the new patient was a multi-problem patient.

About the fifth week a routine had begun to develop. Students and preceptors were finding it more difficult to recall "critical" incidents or even recent patient encounters.…

Introduction

Nurse educators have long recognized the importance of the clinical practicum component of the nursing curriculum. While the didactic component can convey fact and theory, it remains the function of the clinical component to cultivate clinical skills. The clinical component takes on increasing importance in the curricula of nurse practitionerprograms. Theexpanded role skills in health history taking and physical assessment, essential elements of the NP role, are developed and polished during the clinical experience.

The considerable literature to be found on the evaluation of the clinical performance of students is evidence of the interest and concern surrounding the subject. Woolley provided a very readable historical overview of clinical performance evaluation, focused on evaluating students' performance within the clinical setting.1 However, few studies have attempted to identify what and how specific setting characteristics, situations and teaching approaches influence the effectiveness of a given practice setting as a clinical training site. The study reported here was designed to examine the getting itself, its specific characteristics and the perceptor-student interactions occurring there, in terms of its function as a clinical classroom. In other words, this study was not designed to evaluate the clinical performance of students, but rather to examine the conditions under which their clinical learning occurred.

Background - Objectives of the Study

In 1973 the Interdisciplinary Group Practice (IDG) was established within the general medical clinic of Strong Memorial Hospital of the University of Rochester. A primary/ impetus COT the establishment of the IDG was the need for a clinical retting in which nurse clinician (NC) students could develop expanded clinical assessment skills under the close preceptorship of nurses and physicians. The IDG is an interdisciplinary practice of nurse clinicians and physicians, The IDG operates year round, with three OT four half-day sessions a week. Nurse and medical faculty members provide continuous primary care to approximately 700 adult patients. From January through April, the IDG serves as the clinical practicum site for first year students in the older family track of the masters level Family Health Nurse Clinician (FHNC) curriculum. The IDG experience is the first of a sequence of three clinical courses completed by the FHNC students during the two-year curriculum. In this first practicum experience, the primary emphasis is on developing basic physical assessment and health history skills. It provides the foundation for the second and third clinical practicum in which broader nursing aspects of the expanded primary care nurse role are emphasized. During the several months of this first clinical experience, the students have access to the patients in the practice caseloads of the nurse faculty. The students are precepted by the nurse and physician faculty.

Inasmuch as a primary impetus for the establishment of the IDG was its utility as clinical learning site, this study was undertaken to examine the conditions and characteristics of the setting that contributed to producing an effective or ineffective learning environment there. The specific objectives for this study were:

First, to identify the factors, conditions, and opportunities deemed to be of significant influence to the students' learning experience in the IDG practice setting;

Second, to compare the factors, conditions, and opportunities deemed to be of significant influence to the learning experience in the IDG practice setting by faculty (nurse and physician preceptors) and by the NC students; and

Third, to determine the viability of the critical incident technique as a means for evaluating the clinical practice setting in terms of its usefulness as a learning environment.

Methodology

The critical incident technique was selected over the other methodologies considered for this study. This technique was originally described by Flanagan.2 The technique consists of collecting a comprehensive list of observed behaviors from individuals who are most competent and in the best position to make judgments about the particular activity being investigated. The data, i.e., critical incidents, a re collected through partially structured interviews. An analysis of the critical incidents collected allows one to begin to formulate the critical requirements or elements of an activity. Flanagan defined an incident as "critical" if it made a significant contribution, either positively or negatively to the general aim of the activity; the definition of "significant" being dependent on the nature of the activity. Flanagan described the technique in detail and carefully outlined the specifications regarding the observations; i.e., requirements of the persons making the observations, the groups to be observed and reported. He also outlined the method for analyzing and reporting the data. The critical incident technique is not unknown in the nursing literature. It has been adapted and applied to evaluating nursing students' performance, identifying behavioral criteria of nursing effectiveness, and studying nurse-physician teamwork.2-4

In the present study, data in the form of critical incidents were collected during two 9-week periods when students were in the IDG practice. Each 9-week period followed four weeks of introductory physical assessment training in the student laboratory. The first 9-week data collection period occurred between January and April, 1977.

The second 9-week data collection period occurred between January and April, 1978. Preliminary findings from analysis of data from the first data collection period led to specific recommendations for changes in the clinical experience. These changes were implemented before the second data collection period.

The natural "experts" selected to report incidents were the NC students, the nurse preceptors and physician preceptors in the IDG. These individuals were considered to be in the best positions to make competent judgments about the learning and teaching conditions and activities occurring in the practice setting.

There were twelve NC students, five nurse preceptors and four physician preceptors reporting incidents during the first data collection period. In the second data collection period, six NC students, three nurse preceptors and two physician preceptors reported incidents. One physician preceptor did not participate in the second collection period. Of the four physician preceptors from the first data collection period, only one served as a preceptor during the second collection period. Three of the five nurse preceptors from the first data collection period served as preceptors during the second collection period.

The principal investigator contacted each nurse and physician preceptor individually prior to the study to explain the rationale for the study, describe the methodology, and explain what would be asked of each in the course of the study period. The NC students received the same information as a group at their clinic orientation meeting.

The investigator taped semi-structured focused interviews with students on three clinic days each week. These interviews were not allowed to interfere with patient care or clinic operations. Each student was contacted at least once a week and was asked to relate incidents that had occurred over the past two clinic sessions that she* considered to be productive or counterproductive to the learning experience. The reporters were asked to relate all positive and all negative experiences. The nurse and physician preceptors were interviewed by the investigator as their schedule permitted. Two physicians chose to dictate their own incidents immediately after their clinic session.

The semi-structured interviews lasted from 5 to 30 minutes. The investigator applied certain criteria as specified by Flanagan to the incidents as they were being collected: (a) was the actual behavior reported and not "trait" names; (b) was it observed by the reporter; (c) were all relevant factors in the situation given; (d) had the observer/reporter made a definite judgment as to the criticalness of the behavior; and, (e) had the observer/reporter made it clear why he believed the behavior is critical.

Within several hours of each taped interview, the investigator transcribed each incident reported to a separate 5x8 note card. Each card was labeled as to name of reporter, date of incident, and whether it was a positive or negative incident. Also, a sequential identification number was assigned to each incident.

Analysis

The following examples are provided to familiarize the reader with the nature of the data collected and analyzed in this study. Positive incident reported by a NC student:

"On Friday I had difficulty trying to elicit deep tendon reflexes on an obese patient. The nurse preceptor came in to the examining room and demonstrated the reflexes and how to elicit them."

Positive incident reported by physician preceptor:

"The student approached me with the chart of a patient with a chronic urinary tract infection and a positive urine culture. The student questioned whether or not the patient should be called in for treatment. I felt it was a good teaching opportunity for me with this student because the entire chart had to be reviewed. We went through the importance of understanding the relationship of one urine culture to others, as well as to symptoms, abnormal urine sediment, past history, and its importance to structural and/or functional disease of the kidney. We decided on a treatment plan based on these considerations."

Negative incident reported by NC student:

"I felt really incompetent in clinic today. I was unfamiliar with the pathophysiology of the condition my patient had. I couldn't begin to answer the questions the physician preceptor asked me about the problem. I was really primed for learning something about the condition but the physician preceptor just said to go and read up on it. He just didn't seem to have any inclination to teach. He just said to 'go read up on it.'"

Data analysis involved examining the discrete incidents for similarities or common elements. The objective was to summarize and describe the data in a concise, efficient manner, thereby increasing the usefulness of the data while sacrificing as little as possible of its comprehensiveness, specificity, and validity. Flanagan described the process of "category formulation."" Bailey summarized the steps involved in category formulation as follows: (I) studying the incidents and behaviors carefully, analytically in accordance with the purpose of the research; (2) formulating a rough classification system which would seem to encompass the major area of incidents and behaviors; (3) reclassifying the behaviors according to the major areas formulated and revising the areas if necessary; and (4) studying the behaviors in the major areas, grouping similar behaviors together to formulate sub-areas, and writing specific descriptive statements to cover similarities of the incidents.3

Because decisions made in this process required subjective judgment, the tentative categories and subcategories into which the incidents were placed were submitted to an associate for review. This associate was helpful in analyzing incidents which involved interpretation or judgment.

Three categories and 18 subcategories were identified (Appendix) The main problem with the classification scheme was that in some cases these divisions were not always mutually exclusive. However, the attempt was to arrive at categories that would result in only a minimum of overlap.

A total of 332 critical incidents were recorded over the two data collection periods. Of these, 12 (3.6%) were found not to meet the criteria of usable incidents. These incidents were excluded either because they were generalizations rather than specific incidents, or because they were not related to the general aim of the investigation. There were 21 reporters in 1977 compared to 11 in 1978. This was due to the larger class size in 1977, 12 students in 1977 compared to 6 students in 1978. There were five nurse preceptors and four physician preceptors in 1977 compared to three nurse and two physician preceptors in 1978. Of the 320 usable incidents, 202 (63.1%) were collected in 1977 and 118 (36.9%) in 1978.

Table 1 shows the number of incidents reported for the combined data collection periods. The number of incidents provided by the three reporter types was in proportion to their number in the study group. The physician preceptors reported slightly fewer incidents. The physician preceptors participating in 1978 reported a somewhat greater number of incidents than those physicians participating in 1977. The two physicians reporting incidents in 1978 were third-year residents in a general internal medicine program. Three of the four physicians reporting in 1977 were older. University-based internal medicine faculty.

Table

TABLE 1NUMBER OF INCIDENTS BY REPORTERS OF INCIDENTS

TABLE 1

NUMBER OF INCIDENTS BY REPORTERS OF INCIDENTS

Table

TABLE 2TYPE OF INCIDENT BY REPORTER OF INCIDENT

TABLE 2

TYPE OF INCIDENT BY REPORTER OF INCIDENT

Of the 320 incidents reported, 59.4% were positive and 40.6% were negative. As shown in Table 2, with the data from the two collection periods combined, the NC students and nurse preceptors reported the same percentage of positive and negative incidents. In both data collection periods, the physician preceptors reported a greater percentage of positive incidents (75.0% in 1977 and 69.2% in 1978). In 1978 the NC students reported a somewhat greater percentage of positive incidents (65.4% in 1978 compared to 54.7% in 1977). The nurse preceptors reported an identical percentage of positive and negative incidents for the two years.

The incidents collected in each data collection period were analyzed in relationship to the three categories and eighteen subcategories. These categories and subcategories are defined and examples of each offered in the Appendix. These categories and subcategories were formulated on the basis of the 202 incidents collected during the first data collection period. When the 118 incidents collected during the second data collection were categorized, no new categories emerged. This suggested that the number of incidents collected in the sample represented the gamut of critical behaviors.*

The distribution of the incidents among the categories and subcategories were compared over the two data collection periods. Because more similarities than differences were noted, the data reported here combines the incidents from the two collection periods. Where differences were found, these will be noted.

The 320 incidents were distributed among three major categories: Category I (student-preceptor interaction); Category II (student-patient interaction); and Category III (clinic operations). Table 3 shows the number of positive and negative incidents in each category. Fifty percent of the incidents involved student-preceptor interactions.

Over 75% of these incidents were positive; that is, they facilitated learning in the clinic situation. Forty percent of the incidents involved clinic operations. Over 70% of these incidents had a negative effect on clinical learning. Ten percent of the incidents involved student-patient interactions. Over 90% of these were positive to the learning objective of the clinic.

Table

TABLE 3CATEGORY OF INCIDENT BY TYPE OF INCIDENT

TABLE 3

CATEGORY OF INCIDENT BY TYPE OF INCIDENT

Table

TABLE 4CATEGORY OF INCIDENT BY REPORTER OR INCIDENT

TABLE 4

CATEGORY OF INCIDENT BY REPORTER OR INCIDENT

Comparing the data from the two data collection periods, there was a somewhat greater percentage of incidents involving student-preceptor interactions in 1978 (44.0% in 1977 compared to 60.2% in 1978). The percentage of positive and negative incidents was nearly the same for the two years. There was a somewhat lesser percentage of incidents involving clinic operations in 1978 (45.6% in 1977 compared to 30.5% in 1978). However, the percentage of negative incidents in this category increased over 10% between the two years (67.4% negative in 1977 compared to 80. 6% negative in 1978).

Table 4 shows the type, number, and percent of incidents reported by the students, nurse preceptors and physician preceptors. The mix of incidents by type as reported by the students and nurse preceptors was similar. The mix of incidents reported by physicians was skewed toward more incidents involving student-preceptor interactions. About three fourths of the incidents reported by the physician preceptors involved such interactions. The other one-fourth involved clinic operations. Table 5 combines the information on report er, type of incident and nature of the incident.

Students and nurse preceptors individually reported on the same incident 20 times. In 19 of these pairs of incidents, the student and nurse preceptor interpreted the incident in the same way. In the remaining pair one interpreted it as a positive incident and the other as a negative incident. There were six instances in which a student and physician preceptor reported the same incident, five instances in which the nurse preceptor and physician reported the same incident, two instances in which two nurse preceptors reported the same incident, and two instances in which a student, nurse preceptor, and physician preceptor each reported the same incident.

With classification of the incidents refined to the subcategory level, further analysis was done. Table 6 shows the frequency and percent distribution of the 190 positive and 130 negative incidents over the 18 subcategories. Category I, student-preceptor interaction, had 160 incidents distributed among lOsubcategories. Nearly fifty percent of the incidents involving student-preceptor interactions were in three of these 10 subcategories; namely, teaching approach assessed, preceptor demonstrates a technique or procedure, and preceptor guides the student through case evaluation and case management. The 32 incidents offering an assessment of the teaching approach were equally divided between positive and negative. Of the 42 incidents in the latter two subcategories only one was negative.

Table

TABLE 5CATEGORY OF INCIDENT BY REPORTER AND TYPE OF INCIDENT

TABLE 5

CATEGORY OF INCIDENT BY REPORTER AND TYPE OF INCIDENT

Table

TABLE 6CATEGORY BY TYPE OF INCIDENT BY POSITIVE AND NEGATIVE NATURE OF INCIDENT

TABLE 6

CATEGORY BY TYPE OF INCIDENT BY POSITIVE AND NEGATIVE NATURE OF INCIDENT

Of the 160 incidents involving studentpreceptor interactions, 45.6% (73) named the nurse preceptor and 54.4% (87) named the physician. Half of the incidents that described a preceptor demonstrating a technique or procedure named the nurse perceptor and half named the physician preceptor. All incidents describing the preceptor guiding the student through case evaluation and case management named the physician preceptor. The nurse and physician preceptors were named equally as frequently in incidents in which the teaching method was assessed.

Twelve incidents, all positive, described the preceptor observing or assisting the student with a technique or procedure. Seventy-five percent of these incidents involved the nurse preceptor. And, in 8 of the 12 incidents, it was a pelvic examination with which the preceptor assisted.

Seventeen incidents described a discussion between the student and preceptor concerning how the student handled a specific patient encounter. Nearly 90% of these incidents were positive. The nurse or physician preceptor were equally as likely to be the preceptor involved.

Sixteen incidents addressed the general quality of the interaction between student and preceptor. Thirteen (80%) of these incidents referred to an interaction between the student and physician. Over 60% of the incidents in the subcategory were seen as counter-productive to the learning objective.

Category H, student-patient interaction, had 32 incidents, 30 positive and 2 negative, distributed between two subcategories. Eighteen incidents (56.2%) showed the students demonstrating competence in handling a patient encounter. Theincidents in this subcategory typically included such phrases as, "I felt so successful today," and "I felt I was able to present myself as a competent provider." Fourteen incidents (43.8%) reported the student noting progress in a patient as a result of her intervention.

The 128 incidents in Category III, clinic operations, accounted for 40.0% of the total number of incidents reported. Of the six subcategories under clinic operations, three, appropriateness of patient, availability of preceptors and adequacy of the scheduling system, accounted for over 70% of the incidents reported. Over 85% of the incidents citing the availability of preceptor and adequacy of the scheduling system were negative.

Thirty incidents related to the availability of the preceptors to the students. Of these, 20 were negative. The incidents in this subgroup were further divided as to the reason for the unavailability of the preceptor. That is, whether the preceptor was not present in the clinic or whether the preceptor was present but busy with other students or patients when needed by another. Seven incidents, reported on times when the preceptor was later in arriving in clinic, and some problem arose in the preceptor's absence. In the other 21 negative incidents reported, the preceptors were present but busy with other students or patients when needed by another. Students reported patients waiting from 15 minutes to over 1 ½ hours while the preceptor and student interpreted findings, performed stat laboratory procedures, discussed management and decided on medications. Most of these incidents occurred during the Friday session which students and preceptors noted as the most hectic clinic day.

The adequacy of the scheduling system was reflected in the incidents collected. Of the 31 incidents in this group, 27 were negative. It must be noted that some of the incidents assigned to this subgroup reflect recognized scheduling system deficiencies that to date have proved unavoidable and unsoivable. Some of the problems with the scheduling system inferred from the incidents were over- and under-scheduling, not allowing enough time for new patient visits, "no show" patients, and scheduling a patient to the wrong provider on a followup visit.

Thirty-three incidents referred to the appropriateness of the patient for student learners. Of all 18 subcategories, this subgroup contained the most incidents, 10.3% of the total. Two-thirds of these incidents were positive, one-third negative. The majority of incidents concerned the appropriateness of the patients for the student learner by type or complexity of problem. Others concerned the appropriateness of the patient by personal characteristics. That is how articulate, coherent and cooperative the patient was. Students, nurse preceptors and physician preceptors supplied incidents to this subgroup in a frequency about proportional to their number. A review of the incidents indicated that students and preceptors essentially agreed on what characterized the patient appropriate for the student learner. These characteristics inferred from the incidents were as follows:

(a) the patient was alert, coherent, cooperative, could verbalize his problem, present a concise history;

(b) the patient presented with a clear set of symptoms or a single specific complaint, a "textbook flu" case, or a urinary tract infection, for example;

(c) the patient did not have numerous complex medical problems; and

(d) the patient did not have overwhelming social, family, or economic problems overlaying his medical problems.

The positive incidents frequently related an incident which involved a new patient examination of an essentially healthy adult (as for example, a patient in clinic for a preemployment physical). Often the students commented that they were still relatively insecure even with the physical examination on "normals." To have patients with numerous abnormal findings created anxiety and loss of confidence in themselves. In 1977, over 20% of the incidents in the clinic operations category referred to the availability of patients. Three-fourths of these incidents cited the problem of frequent "no-show" patients. However, in 1978, only 8% of the incidents cited this problem.

Table 7 provides an overall summary of the data. This table presents the frequency of incidents by category, reporter and type of incident.

Discussion

One objective of this study was to determine the viability of the critical incident technique as a method for studying the factors influencing the learning and teaching milieu in the IDG practice. Several shortcomings of this technique were recognized before the study was undertaken.4 The retrospective nature of the data made it subject to the reporter's possible lack of recall of details. Because of the sensitive nature of some of the incidents some respondents might alter facts; or, in the instance of self-reports unintended bias might alter the account. This technique does not catalogue or describe all relevant behaviors; the more common, but less spectacular behaviors or incidents are likely to be underrepresented. And, while the procedure for collecting the data is objectively defined, the procedure for classification of the critical incidents is subjective.

Recognizing these shortcomings of the technique prior to undertaking the study, the investigators made every effort to limit their impact. The reporters were encouraged to write down critical incidents as they occurred for reporting to the investigator later. At intervals during the data collection period, the reporters were reminded of the criteria to be used to define a critical incident. Reporters were assured that the incidents related would be kept in strict confidence.

Table

TABLE 7CATEGORY BY REPORTED OF INCIDENT BY POSITIVE AND NEGATIVE NATURE OF INCIDENT

TABLE 7

CATEGORY BY REPORTED OF INCIDENT BY POSITIVE AND NEGATIVE NATURE OF INCIDENT

However, even with these precautions, the technique proved less than ideal. There was uneven enthusiasm, interest and commitment among the reporters. This led to variation both in the number and in the extent of specificity of the incidents reported. At times it was apparent that some reporters related incidents without having made a clear judgment as to why it was productive or counter-productive to their learning or teaching objective. The small number of reporters, each contacted repeatedly over the nine-week period, made the study more of a burden to the reporters than it might have been, had there been a larger group from which incidents could have been solicited. This problem was more apparent the second year when there were only half as many students.

The subjective nature of the data was not unexpected, but nevertheless it did prove cumbersome to data analysis and interpretation. Furthermore, there always remained an uneasiness that the meaning of reported frequencies on the incidents is uncertain. Bates speculated on the meaning of the frequencies of the various categories of incidents in herstudy on nurse-physician teamwork.5 She postulated four factors that might influence the frequency distribution of incidents among categories: "(l) the frequency of the actual behaviors themselves, (2) the importance of the behaviors, (3) the emotional impact of the behaviors on the (reporter), and (4) some discrepancy between standards or expectations of performance and the behaviors itself." She went on to speculate that these factors might also influence the proportion of "good" and "bad" incidents.

Although the technique was not without its shortcomings, it did have noteworthy strengths.

The incidents proved helpful to the faculty as they evaluated their teaching techniques and the clinic operations. Faculty found the incidents allowed and encouraged them to focus attention on specific clinic activities and teaching methods in a substantive way. At the same time, the data offered a holistic perspective on the clinical experience. Whereas some methodologies focus narrowly on single, isolated aspects of the clinical practicum, the critical incident technique provided a broad picture of what activities were occurring in the clinic, and how these were perceived by those learning and teaching there.

The critical incident technique also provided an interesting picture of the students and preceptors experiences and reactions to the clinical experience over time. The incidents reflected the progress of the student through the clinical experience. The initial incidents reported by the students reflected the mixed eagerness and apprehension over the first patient visit. The student expressed concern over how to present herself to the patient. There was an anxiety about how the patient would respond to her as a provider. The first two to three weeks were exciting for the student, and the students often had a very detailed recall of their first patient encounters. This was reflected in the number and specificity of the incidents reported. An element of frustration was recorded at this time in the incidents due to the high patient "no show" rate. The students eager to try out their new skills were disappointed when "their" patients did not show up.

The next series of incidents noted were those reported as the students began to see their first new patient. The experience was reported as positive when the patient presented with a single problem or was essentially healthy; for example, the patient seeking a pre-employment physical exam. The experience was often reported as "overwhelming" when the new patient was a multi-problem patient.

About the fifth week a routine had begun to develop. Students and preceptors were finding it more difficult to recall "critical" incidents or even recent patient encounters.

About week six, students were expressing through incidents and through casual conversation with the investigator an anxiety about how they were performing. Many expressed insecurity about the completeness of the examinations and the technical accuracy of the procedures they were doing. A recurrent concern of the students was that because the preceptors seldom actually observed them as they took the patients' history or did the physical examination they might be making some mistake repeatedly without knowing it. After the Spring recess, some degree of self-confidence and relaxation was noted.

The conduct of the study also provided indirect benefits. It served to demonstrate the combination of research with practice. It offered the students and preceptors the opportunity to vent their frustrations or to share their excitement with the clinic experience. It demonstrated the concern of the faculty that the full potential of the clinic be realized for the patients' and students' benefit.

Based on findings from the initial data collection period, four specific changes were introduced into the clinical setting prior to the second study period. One, nurse preceptors agreed to arrive in the IDG clinic by 8:00 A.M. to be more available to the students prior to the patients' arrivals. Two, the students had the opportunity to do complete history and physical examinations on two volunteer "patients" who were, incidentally, members of the nursing alumni association. Three, prior to the first clinical session, each student accompanied and observed a nurse faculty during a clinical session. This was done to acquaint the students with the substance of a "typical" nurse clinician patient encounter. And four, the two participating physician preceptors conducted pre-clinic conferences prior to the morning's sessions.

The rationale for introducing these changes was twofold. First, based on the incidents reported the first year, it seemed that these changes would improve the clinical experience. Second, this was seen as an opportunity to ascertain whether the critical incident technique would be sensitive enough to gauge the impact of these changes. The results were inconclusive. In 1977, seven incidents referred to problems that arose as a result of a nurse preceptor not being present in the IDG when the students arrived. In 1978, with the preceptors making deliberate effort to arrive in the IDG at 8:00 A.M., no such incidents were reported. However, the impact of the other three changes implemented in the IDG; i.e., the observation of the nurse preceptor, the opportunity to practice on volunteer "patients," and the pre-session conferences were not reflected in the incidents reported in 1978.

The study did offer some insights into specific features, characteristics, situations and interactions that contribute to the effectiveness of the teaching and learning environment in the IDG practice. Ninety percent of the incidents considered to influence the effectiveness of the IDG as a clinical laboratory were found to involve student-preceptor interactions or clinic operations. The students' needs for immediate validation and feedback of their findings on history taking and physical examination was documented. Many incidents reflected the student's lack of confidence in the completeness or accuracy of their findings. Students recurrently expressed the fear that they might be missing an important sign or be repeatedly doing a procedure incorrectly. It was clear that students want to be closely observed during their earliest patient encounters to be reassured they are performing correctly.

Students reported benefitting from demonstration of techniques and procedures. The data showed that in student-preceptor interactions the physician preceptor tended to be involved in demonstrating techniques or procedures. Nurse preceptors' interactions with students on the other hand tended to find the nurse preceptor assisting or observing as the student performed the procedure or practiced the technique. The physician preceptor was shown through the incidents to be especially helpful in guiding students through the thought and action required for evaluating and managing the disease process. Both preceptor types were found to be involved in discussions with the student about specific patient conditions or patient attitudes. The nurse preceptor was seen as helpful in intervening in clinic operations on behalf of the students' learning needs.

The study also provided information on the characteristics of patients appropriate for student learners. The incidents reported by preceptors and students indicated the fact that patients with multiple, complex medical problems and severe neuroses should be turned away from the student at this stage of her learning. Generally, it seemed better not to confront the student with this type of patient when her energies are so focused on acquiring the new basic assessment skills that are to become part of her expanded role capabilities.

Incidents grouped in the category Clinic Operations documented problems that for the most part heretofore had not gone unrecognized. The high patient "no show" rate, the large portion of patients inappropriate to the student learner (i.e., patients with complicated medical problems often further complicated by neurosis), the demands placed on preceptors' time and energy, and the problems with the scheduling system were substantiated through the incidents reported.

Conclusion

The application of the critical incident technique to the evaluation of the IDG clinic as a clinical site proved viable. Conduct of the study in only one clinical site limits the generalizability of the findings. However, the study does describe an approach to identifying the specific factors, characteristics, and situations that affect the learning and teaching milieu in the clinical setting.

Acknowledgment

The authors would like to acknowledge the valuable contributions made by Judith A. Sullivan, R. N., Ed. D., Assistant Professor of Nursing and of Preventive Medicine and Community Health, and Clinical Chief of Community Health Nursing.

This research was made possible by a grant from the Robert Wood Johnson Foundation. The research was conducted at the University of Rochester, School of Nursing.

References

  • 1. Woolley, AS: The long and tortured history of clinical evaluation. Nun Outhok. 25:308-315, May 1977.
  • 2. Flanagan JC: The critical incident technique. Psychoh Bull 51:327-358, July 1954.
  • 3. Bailey JT: The critical incident technique in identifying behavioral criteria of professional nursing effectiveness. Nurs Res 5:52-64, October 1956.
  • 4. Sanazaro PJ, Williamson JW: A classification system of physician performance in internal medicine. J Med Educ 43:389-397, March 1968.
  • 5. Bates B: Nurse-physician teamwork. Med Cart, 4:69-80, 1966.

TABLE 1

NUMBER OF INCIDENTS BY REPORTERS OF INCIDENTS

TABLE 2

TYPE OF INCIDENT BY REPORTER OF INCIDENT

TABLE 3

CATEGORY OF INCIDENT BY TYPE OF INCIDENT

TABLE 4

CATEGORY OF INCIDENT BY REPORTER OR INCIDENT

TABLE 5

CATEGORY OF INCIDENT BY REPORTER AND TYPE OF INCIDENT

TABLE 6

CATEGORY BY TYPE OF INCIDENT BY POSITIVE AND NEGATIVE NATURE OF INCIDENT

TABLE 7

CATEGORY BY REPORTED OF INCIDENT BY POSITIVE AND NEGATIVE NATURE OF INCIDENT

10.3928/0148-4834-19811001-04

Sign up to receive

Journal E-contents