Journal of Nursing Education

The articles prior to January 2012 are part of the back file collection and are not available with a current paid subscription. To access the article, you may purchase it or purchase the complete back file collection here

BRIEFS 

Evaluating Borderline Students

Sally Brozenec, MS, RN; Julie R Marshall, MS, RN; Charlene Thomas, MSN, RN; Marian Walsh, MS, RN

Abstract

Introduction

Throughout the history of nursing education, clinical evaluation has been a controversial topic (Woolley, 1977). The debate continues over the amount of emphasis appropriately placed on clinical time; the relationship of the clinical experience to reality, the extent to which error is tolerable; and the best process for grading the experience.

Direct observaton is the primary means of clinical evaluation, and is an inherently subjective technique. To further complicate the process, students in the clinical setting must be evaluated in all three domains of learning: cognitive, psychomotor and affective. While it is usually easy to determine mastery of skills and slightly more difficult to assess cognitive abilities, it is extremely difficult to objectively evaluate affective behavior no matter how sophisticated the evaluation instrument.

Another dilemma in clinical evaluation is in the determination of the difference between evaluation time and learning time. No one has yet established a standard for the number of chances a student should be given to effect a behavior change that indicates learning. When does the learning experience end and the evaluation process begin? Generally, these difficulties are minimized when a student does very well or very poorly in the clinical setting. More problematic is the "borderline" student who does not clearly fall into a pass or fail pattern.

Discussion of the overall problem of clinical nursing evaluation has appeared in the literature as early as the 1920s (Gilman, 1921), with many articles focusing on the problems of failing a student. However, a recent search of the literature revealed few references addressing the especially difficult problem of evaluating a "borderline" student. The literature does suggest that inter-rater reliability and faculty consensus may be strengthened by exploration and in depth discussion of this problem (Hayter, 1973; Loustau, Lentz, Lee, McKenna, Hirako, Walker & Goldsmith, 1980; Richards, Jones, Richardson, Riley & Swinson, 19811.

In order to stimulate such discussion, members of the faculty development committee developed a half-day workshop which was attended by approximately 30 faculty members. The participants included both novice and experienced clinical instructors. A student was "created" in this borderline category for the group to evaluate through a case study. The activities of a quarter were conveyed by the following methodologies: 1) role plays depicting both student^patient and student/faculty interactions, and 2 1 the distribution of a hypothetical instructor's weekly anecdotal notes. The hypothetical student named "Anne," exhibited both passing and failing behaviors and in doing so was intended to pose an evaluation dilemma for her clinical instructor. The following is an overview of the case study:

Case Study Overview

Anne was a first quarter senior nursing student enrolled in her fourth sequential nursing course which introduces normal behavioral science theories. Her junior year curriculum had focused primarily upon principles of the biological sciences. Since this nursing program utilizes an integrated approach it is assumed that principles of either the behavioral or biological sciences can be applied to any clinical setting as long as the student has been instructed in the content. The case study was structured as such to address the especially difficult task of evaluating intangible characteristics.

In Anne's case, her patient care activities and written care plans were strong in the physical aspects but weak in terms of the behavioral inter- relationships. In addition, she seemed to lack insight into her failure to meet her patients' psychosocial needs. Though Anne improved throughout the quarter, her behavior was inconsistent.

Pass or Fail?

After collectively viewing the case presentation and accompanying documentation, faculty were asked to independently decide if Anne should pass or fail this clinical experience using the standard criterion referenced evaluation tool…

Introduction

Throughout the history of nursing education, clinical evaluation has been a controversial topic (Woolley, 1977). The debate continues over the amount of emphasis appropriately placed on clinical time; the relationship of the clinical experience to reality, the extent to which error is tolerable; and the best process for grading the experience.

Direct observaton is the primary means of clinical evaluation, and is an inherently subjective technique. To further complicate the process, students in the clinical setting must be evaluated in all three domains of learning: cognitive, psychomotor and affective. While it is usually easy to determine mastery of skills and slightly more difficult to assess cognitive abilities, it is extremely difficult to objectively evaluate affective behavior no matter how sophisticated the evaluation instrument.

Another dilemma in clinical evaluation is in the determination of the difference between evaluation time and learning time. No one has yet established a standard for the number of chances a student should be given to effect a behavior change that indicates learning. When does the learning experience end and the evaluation process begin? Generally, these difficulties are minimized when a student does very well or very poorly in the clinical setting. More problematic is the "borderline" student who does not clearly fall into a pass or fail pattern.

Discussion of the overall problem of clinical nursing evaluation has appeared in the literature as early as the 1920s (Gilman, 1921), with many articles focusing on the problems of failing a student. However, a recent search of the literature revealed few references addressing the especially difficult problem of evaluating a "borderline" student. The literature does suggest that inter-rater reliability and faculty consensus may be strengthened by exploration and in depth discussion of this problem (Hayter, 1973; Loustau, Lentz, Lee, McKenna, Hirako, Walker & Goldsmith, 1980; Richards, Jones, Richardson, Riley & Swinson, 19811.

In order to stimulate such discussion, members of the faculty development committee developed a half-day workshop which was attended by approximately 30 faculty members. The participants included both novice and experienced clinical instructors. A student was "created" in this borderline category for the group to evaluate through a case study. The activities of a quarter were conveyed by the following methodologies: 1) role plays depicting both student^patient and student/faculty interactions, and 2 1 the distribution of a hypothetical instructor's weekly anecdotal notes. The hypothetical student named "Anne," exhibited both passing and failing behaviors and in doing so was intended to pose an evaluation dilemma for her clinical instructor. The following is an overview of the case study:

Case Study Overview

Anne was a first quarter senior nursing student enrolled in her fourth sequential nursing course which introduces normal behavioral science theories. Her junior year curriculum had focused primarily upon principles of the biological sciences. Since this nursing program utilizes an integrated approach it is assumed that principles of either the behavioral or biological sciences can be applied to any clinical setting as long as the student has been instructed in the content. The case study was structured as such to address the especially difficult task of evaluating intangible characteristics.

In Anne's case, her patient care activities and written care plans were strong in the physical aspects but weak in terms of the behavioral inter- relationships. In addition, she seemed to lack insight into her failure to meet her patients' psychosocial needs. Though Anne improved throughout the quarter, her behavior was inconsistent.

Pass or Fail?

After collectively viewing the case presentation and accompanying documentation, faculty were asked to independently decide if Anne should pass or fail this clinical experience using the standard criterion referenced evaluation tool with which all participants were very familiar. As differences of opinion became apparent, a lively and informative discussion ensued. Faculty who were in favor of failing Anne considered her communication deficits alone to be serious enough to warrant a clinical failure. Since the objectives for her current quarter centered upon behavioral science concepts, they felt justified in making this decision. To further validate their opinion they noted Anne's inability to realistically evaluate herself in term of strengths and limitatons as evidenced by her inconsistent performance.

Opposing arguments presented by faculty voting to pass Anne placed more importance on her undisputed ability to provide safe patient care. They reasoned that her behavior was perhaps not inappropriate for a first quarter senior student. In addition, since she had shown some improvement since the early weeks of the quarter perhaps one should not assume that she was incapable of developing further in the future. Some faculty questioned whether or not the hypothetical instructor had provided adequate opportunity for "Anne" to successfully meet the course objectives through planned clinical activities.

Discussion

Overall, the immediate faculty response to this simulated experience was strongly positive despite the fact that an uncomfortable issue was raised. That issue involves the subjective nature of clinical evaluation. Given the same course objectives, evaluation criteria, and identical data, the competent nurse educator would like to be able to assume that any other competent nurse educator would ultimately reach the same conclusion.

It is somewhat unsettling to confront the fact that opinions do vary even among equally competent educators. With regard to the "borderline" student, then, progression or dismissal may actually depend upon their assignment to a particular clinical instructor. Though subjectivity is undoubtedly a factor, a sense of fairness brings about the realization that this is a less than ideal situation.

The presence of subjectivity in clinical evaluation has long been recognized as a topic for investigation. In a classic study, Hayter (19731 found that after group discussion of videotaped student performances and the objectives for the experience, a significant increase in agreement among the faculty occurred.

In a later study by Bondy (1984), which also used videotaped student behavior, results indicated that the use of specific criteria attached to a rating scale significantly increases evaluation accuracy and reliability. The experience of the faculty member was not shown to be a factor which increased rater accuracy and reliability. Instead, the effect of training in the use of evaluation tool was considered the most important variable. Another study of a quality monitoring tool suggested that personal characteristics and education may affect rater reliability, though training in the use of the evaluation tool may assist in decreasing the effect of these variables (Hegyvary & Haussmann, 19751.

In the case of our faculty, the postsimulation discussion afforded the opportunity to clarify established course objectives, review critical elements of the evaluation mechanism and share various strategies for the evaluation of "borderline" performance. It was hoped a methodology that more closely approximates a standard was utilized by faculty in subsequent actual clinical situations due to a combination of the "training effect" in the use of the evaluation tool itself as well as clarifying passing performance criteria.

One- Year Follow-Up

To validate our assumptions regarding the workshop's impact upon subsequent clinical evaluations, we invited faculty members who had participated in our workshop to an informal follow-up discussion one year later. There was overall agreement that clarification of objectives and the exchange of ideas had influenced evaluations over the past year. Several faculty members cited specific situations with "borderline" students during which they had utilized suggestions that had been discussed at the workshop. They also revealed that they had since been much more aware of the effect of their personal perceptions upon student evaluations.

WHAT CAN BE DONE?

Faculty generally agreed that certain factors which affect evaluation of clinical performance are difficult to overcome. One of these factors is subjectivity, which is inherent in the evaluation process. The one-year follow-up workshop ended with a discussion of how to overcome the obstacles to ensure fair clinical evaluation of the "borderline" student.

Evaluation Methodology

Ample evidence exists to support the superiority of criterion-referenced rather than norm- re fere need tools in fostering objectivity and in increasing reliability and validity (Krumme, 19751. An important aspect of the learning tool is that it should be able to measure the learning of the critical content as identified in the conceptual framework and course objectives (Carpenito & Duespohl, 1985). The tool should contain behavioral objectives that reflect critical concepts of the curriculum.

Too often, faculty develop lengthy tools in the hope that subjectivity will be eliminated. Loustau et al. (19801 recommends decreasing the number of points on the rating scale to improve validity and reliability. This has resulted in an improvement in reliability among raters as demonstrated in several studies. Also, the specific items on the evaluation tooJ should be discussed and clarified among faculty members to help improve inter-rater reliability

Another key issue cited by Carpenito and DuespohJ (1985 > is that the method of evaluation should be appropriate for the learning objectives being measured. Methods other than direct observation of the student in the clinical laboratory should be employed in the evaluation process. These methods might include clinical simulations, use of videotape, self-evaluations, and written assignments. The use of written care plans and process recordings which are graded help assess the cognitive ability of the student. By increasing the various methods of evaluation, the instructor is better able to evaluate the student and focus in on problem areas. For example, many students can write a comprehensive, superior care plan but are unable to perform clinically.

Anecdotal Notes

There is general agreement about the understated importance of good recordkeeping in the form of anecdotal notes. Without documentation of clinical incidences, it is difficult to recall specific events at evaluation time. Records should also be kept of evaluations (midterm and final!, and any other counseling conferences. Notations should be made as soon as possible after the clinical day. Tape recording notes might be useful for the busy clinical instructor to be transcribed later Í Carpenito & Duespohl, 1985).

Multifaculty Input

Often a clinical instructor suspects a problem with a student but cannot isolate the unsatisfactory behavior. Having another faculty member assist in clinical observation and evaluation may help identify the problem and bring objectivity to the situation. This demonstrates to the student that there is a genuine interest in giving a fair evaluation. Input about the student's performance may also be obtained from nurses working on the clinical unit. The head nurse, staff nurses, and even house medical staff may have observed the student in various clinical situations. Observation by others, including faculty superiors, also provides support and direction to the instructor, especially the novice.

Counseling the Student

Once the instructor informs the student of unsatisfactory performance, the student may respond with guilt, anger or hostility. Since failing is a real threat to the students self-esteem, the instructor should also be certain that positive aspects of the student's performance are cited. When the clinical instructor is unable to focus upon positive aspects in the student, it is especially important to get another instructor involved (Carpenito, 1983). In the case study, Anne refused to believe that her performance was unsatisfactory, and the clinical instructor needed to gather more data to justify her position. Providing frequent feedback is a must. Errors which are pointed out promptly prevent surprise in later evaluations (Meisenhelder, 1982). Weekly self-evaluations by the student in a clinical log may help the instructor see the individual's perceptions of progress and performance. When the individual's perception is different from the instructor's, a conference should be held to explore the disparity (Carpenito & Duespohl, 1985).

Extending Clinical Time

The discussion of whether or not to pass Anne brought out many different viewpoints among faculty members. One clinical instructor who was in favor of passing Anne stated that the instructor had not provided the appropriate situations to allow her to demonstrate clinical competency in certain key behaviors. This is a very important point. While it seems obvious that the instructors should select patient situations which allow performance of behavioral cues, many students need more than "one chance" before they can exhibit competency. Our faculty felt very strongly about opting for an extension of clinical hours if more time and observation was needed to make a decision about a "borderline" student. In addition, the extra time may alleviate the uneasiness of deciding to pass or fail the "borderline" student.

Summary

While it was difficult for our faculty to face the disparity which arose in deciding whether or not to pass Anne, many important points were raised which strengthened the evaluation process particularly in the case of the borderline student. Newer faculty members related uneasiness in being specific and directive with "borderline" students. Experienced faculty offered suggestions on overcoming this fear and finding support among peers.

Open discussion provided a forum for the exchange of ideas and philosophies. Though no definitive solutions were uncovered we judged this workshop to be a valuable learning experience with broad appeal across a very diverse faculty.

References

  • Bondy, K. (1984). Clinical evaluation of student performance: The effects of criteria on accuracy and reliability. Research in Nursing and Health, 7, 25-33.
  • Carpenito, L. (1983). The failing or unsatisfactory student. Nurse Educator. #4), 32-33.
  • Carpenito, L. & Duespohl, T. (19851. Evaluation and clinical instruction. A Guide for Effective Clinical Instruction. Rockwell: Aspen Systems Corporation.
  • Gil man, A. (1921). Hospital and Training School Administration. American Journal of Nursing. 21, 478-481.
  • Hayter, J. (1973). An approach to laboratory evaluation. The Journal of Nursing Education. 12. 17-23.
  • Loustau. A., Lentz, M., Lee, K.. McKenna, M., Hirako, S., Walker, WF & Goldsmith, J.W. (1980). Evaluating student's clinical performance: Using videotape to establish rater reliability. Journal of Nursing Education. 7917), 10-17.
  • Meisenhelder, J. (1982). Clinical evaluation - An instructor's dilemma. Nursing Outlook. 30. 340-351.
  • Krumme, U. (1975). The case for criterion -referenced measurements. Nursing Outlook. 23(12}, 764-770.
  • Richards, A., Jones, A.. Nichols. K., Richardson, F., Riley, B. & Swinson, R. (1981). Videotape as an evaluation tool. Nursing Outlook. 29. 35-38.
  • Hegyvary, S. & Haussmann, R.K. (1975). Monitoring nursing care quality. Journal of Nursing Administration 5. 17-26.
  • Woolley, A. (1977). The long and tortured history of clinical evaluation. Nursing Outlook. 25(5), 308-315.

10.3928/0148-4834-19870101-11

Sign up to receive

Journal E-contents