Journal of Nursing Education

The articles prior to January 2012 are part of the back file collection and are not available with a current paid subscription. To access the article, you may purchase it or purchase the complete back file collection here

Revamping the Peer Review Process

Claire N Brannigan, RN, MS; Janet Z Burson, RD, MS

Abstract

Faculty evaluation is a guaranteed part of university life. While it should be a growth potentiating process for the individual, in reality the opposite is often true. At the same time the individual is experiencing the evaluation process, usually with a considerable degree of anxiety, the peer review committee is experiencing the process from a very different perspective. The peer review committee must have an equitable process that both reduces the anxiety of the faculty member and provides a reliable evaluation for the university. Developing such a process was our challenge.

The Process of Review:

Identifying the Problem The peer review process is an integral part of university life. It is the method used to measure faculty teaching ability, contributions to the university, and the community at large. An additional purpose is to promote the professional growth of each faculty member.

The peer review committee is faced with a review process of singular, often anxietyladen, significance to the individual. It is charged with arriving at a valuation of each faculty member's net worth while maintaining a process that will be positive and promote professional growth and development. Realistically, peer review is more than this. It is the process by which jobs are kept and rewards are given.

Implied in the process of evaluation is the element of power. Power means having control, in particular, control over the distribution of rewards. Power in a school of nursing usually rests with the administrator - the Dean. Yet, a peer review committee which makes its recommendation directly to the Dean also has a great deal of influence. The process of peer review helps to diffuse the power by spreading it along a broade. spectrum. If this is so, the peer review committee must be backed by valid and reliable information.

A paradox is encountered in relation to the desire to know how one is doing, versus the fear of being evaluated. The anxiety produced by the process of evaluation limits the potential for professional growth and satisfaction. Faculty fear injustice, feeling that the evaluation will be based on personality traits, or that it may be an avenue for retaliation from angry students or peers. Most faculty are also aware of their own limitations and faults and feel pushed to excel in all the required areas of evaluation. For the majority, however, the most anxiety-producing aspects of the evaluation process are the high stakes - job security, rank, salary, and one's sense of worth and self-esteem. Realistically, it is difficult for faculty to perceive the growth potential which is also inherent in the evaluation process, when the power of the committee is perceived to be so great. An equitable process that is fair, makes sense, and is as objective as possible is absolutely necessary to reduce faculty anxiety to a reasonable degree and allow the growth potential of the evaluation to come through.

Student evaluation of faculty is another aspect of the peer review process which warrants careful attention. The problems inherent in student evaluation of teaching effectiveness are well documented (McKeachie, 1979; Soper, 1980; Stafford, 1978). So many things, other than factual knowledge and teaching effectiveness, affect student evaluation of faculty that their validity is heatedly argued (Machlup, 1979; Raskin, 1979). The tremendous weight given to student evaluations, adds to faculty anxiety.

It is certainly possible that student evaluations are a reliable source of information.

They are an incomplete source, however, by themselves. Students experience teaching from a different, more limited perspective. They are, in fact, learning. They are not colleagues, nor can it be ignored that students are evaluating those with a…

Faculty evaluation is a guaranteed part of university life. While it should be a growth potentiating process for the individual, in reality the opposite is often true. At the same time the individual is experiencing the evaluation process, usually with a considerable degree of anxiety, the peer review committee is experiencing the process from a very different perspective. The peer review committee must have an equitable process that both reduces the anxiety of the faculty member and provides a reliable evaluation for the university. Developing such a process was our challenge.

The Process of Review:

Identifying the Problem The peer review process is an integral part of university life. It is the method used to measure faculty teaching ability, contributions to the university, and the community at large. An additional purpose is to promote the professional growth of each faculty member.

The peer review committee is faced with a review process of singular, often anxietyladen, significance to the individual. It is charged with arriving at a valuation of each faculty member's net worth while maintaining a process that will be positive and promote professional growth and development. Realistically, peer review is more than this. It is the process by which jobs are kept and rewards are given.

Implied in the process of evaluation is the element of power. Power means having control, in particular, control over the distribution of rewards. Power in a school of nursing usually rests with the administrator - the Dean. Yet, a peer review committee which makes its recommendation directly to the Dean also has a great deal of influence. The process of peer review helps to diffuse the power by spreading it along a broade. spectrum. If this is so, the peer review committee must be backed by valid and reliable information.

A paradox is encountered in relation to the desire to know how one is doing, versus the fear of being evaluated. The anxiety produced by the process of evaluation limits the potential for professional growth and satisfaction. Faculty fear injustice, feeling that the evaluation will be based on personality traits, or that it may be an avenue for retaliation from angry students or peers. Most faculty are also aware of their own limitations and faults and feel pushed to excel in all the required areas of evaluation. For the majority, however, the most anxiety-producing aspects of the evaluation process are the high stakes - job security, rank, salary, and one's sense of worth and self-esteem. Realistically, it is difficult for faculty to perceive the growth potential which is also inherent in the evaluation process, when the power of the committee is perceived to be so great. An equitable process that is fair, makes sense, and is as objective as possible is absolutely necessary to reduce faculty anxiety to a reasonable degree and allow the growth potential of the evaluation to come through.

Student evaluation of faculty is another aspect of the peer review process which warrants careful attention. The problems inherent in student evaluation of teaching effectiveness are well documented (McKeachie, 1979; Soper, 1980; Stafford, 1978). So many things, other than factual knowledge and teaching effectiveness, affect student evaluation of faculty that their validity is heatedly argued (Machlup, 1979; Raskin, 1979). The tremendous weight given to student evaluations, adds to faculty anxiety.

It is certainly possible that student evaluations are a reliable source of information.

They are an incomplete source, however, by themselves. Students experience teaching from a different, more limited perspective. They are, in fact, learning. They are not colleagues, nor can it be ignored that students are evaluating those with a power that may ultimately influence the student's career.

The Experience of Peer Review Revision

The personnel policy at our university requires that each faculty member be reviewed each year by a peer review committee which makes recommendations regarding reappointment, promotion, tenure, and sabbaticals directly to the dean of the school of nursing. The four areas of evaluation include teaching effectiveness, service to the university, service to the community, and scholarly achievement. However, a clear priority is given to excellence in teaching effectiveness.

In the past, the peer review process utilized at our school of nursing placed heavy emphasis on student evaluations of faculty. Peer evaluation was minimal, selfevaluation was allowed to be at the discretion of the faculty member. The result was a lopsided, skewed evaluation.

It was during the process of trying to meet our charge as a committee that members began to think that our entire process needed revamping. This was based on the difficulties encountered in the review process due to inconsistent and/or inadequate data and the justifiably high anxiety level of the faculty concerning the process. First, the student teacher evaluation tool was revised by an ad hoc committee composed of faculty and students. Second, members of the peer review committee decided to develop a tool to collect data from peers. Third, the guidelines used by faculty to prepare materials for review were revised. When being reviewed, thé individual is asked to include her own perspective, that is, a self-evaluation, as well as the evaluations of students and peers. It was decided that in order for the process to be equitable, none of the three components should stand alone, but in combination with the other two. Input from all three perspectives should produce a more reliable result than in the past when the student evaluations had been so heavily weighted.

Problems in Designing A Peer Evaluation Tool

The formulating of a reliable peer evaluation tool is not without problems. First, there was the question of the areas to be evaluated. After much discussion by the peer review committee a consensus was reached regarding the areas to evaluate for the school of nursing faculty: team membership, classroom teaching, and clinical teaching. Then our attention was turned to developing a meaningful scale.

Initially a five-point scale was proposed. However, as various qualifiers or descriptive adjectives were tried as a means of differentiating between the points on the scale, it became obvious that none could be defined to the satisfaction of all the faculty. It was eventually decided to use a scale with three simple points and no descriptive terminology. The points skillful, adequate, and unsatisfactory were utilized; and although they could be open to interpretation, it was believed that meaningful differentiation existed between them. For those faculty who may be more than "adequate," but considered less than "skillful," the comments area provided an outlet for distinguishing differences and explaining discrepancies.

Obtaining adequate data on teaching in the clinical setting is a well-recognized and generally unresolved problem in peer evaluation. In the future, as we implement our integrated curriculum and have faculty working together in clinical areas, it is believed that peer evaluation of clinical teaching will become a more meaningful component of peer review.

Classroom observation of faculty teaching is another problematic area in peer review. In our experience the anxiety level of faculty members was considerably high over this question as many imagined a firing squad marching into their classroom, announced or unannounced, to make an "official" observation. It was generally believed that an artificial situation would be created whereby the faculty member actually teaching might feel too intimidated to demonstrate her best teaching qualities. While many schools of nursing do use this method, it was not thought appropriate to our needs. We believe that working as part of an integrated team should provide faculty with enough exposure to each other's classroom abilities to serve as a basis for evaluation.

The last issue in designing a peer evaluation tool was the hotly debated topic of the procedure itself. Ideally, all evaluations done by peers should be randomly selected to assure fairness. However, the faculty's obvious fear of loss of control and the need for self-protection had to be considered and eventually took precedence. Even though there is considerable potential for skewed results, the approved procedure as it stands now includes the random selection of three peer reviewers by the peer review committee and the selection of four peer reviewers by the faculty member being reviewed.

Description of the Tool

The peer review process takes on additional importance when a school of nursing implements an integrated curriculum since this requires more group planning and implementation. It is paramount that the nursing faculty member work effectively as a member of a team. The faculty member must respect and tolerate individual differences of other faculty, have compassion, and be able to lend support to colleagues. Sensitivity and responsiveness to other's ideas is essential for the team to function comfortably and productively in their effort to meet team goals.

Our tool (Figure) was designed to use particularly in evaluating nursing faculty who teach in an integrated curriculum. The format consists of three major sections which address the specific roles of a nurse educator.

FIGURE

FIGURE

The behaviors listed in section A, team membership, reflect the emphasis on faculty group work in our nursing program. When working as a team, the faculty member should establish spontaneous, cooperative working relationships and be partially accountable for team results. Inherent in this team approach is the need to communicate in order to avoid fragmentation, repetition and/or ommissions. All nursing faculty need feedback from peers regarding their performance in order to provide data based on concrete evidence for the peer review process.

Faculty in a school of nursing are expected to assume a variety of roles. Item 3, section A, addresses the need for communicating these varied roles to colleagues; thus individual faculty growth and development can be fostered and supported by peers. In the broadest sense, item 5 suggests supporting a colleague by giving active aid, and positive support. Item 10 does not exclude individual creativity but implies that consensus at some point is mandatory since team effort is diminished in effectiveness when a team member separates themselves from the rest of the group. The remaining questions in this section are designed to elicit data considered vital to the teacher's role as a team member.

Section B, which rates behaviors specific to classroom teaching, is designed to elicit information that is identical to that which is obtained from student evaluation of the teacher. Thus the peer review committee can compare and contrast this input from two different sources and perspectives.

Items 6 and 7 (section B) require that the teacher be attentive to the needs of students, allow them to ask for clarification, and offer their ideas as a contribution to the classroom learning. Students' rights prevent faculty from evaluating students arbitrarily. An accepted criteria communicated to students at the outset of the course is imperative. A sort of self-policing is implied in item 8 to assure students' rights. If a faculty member is unavailable, either because of the lack of establishing office hours or too much outside activity, the burden of student guidance and/or advice falls on the shoulders of other faculty members who are more available. Item 9 allows opportunity for peers to examine this aspect of peer responsibility and accountability.

Section C of the tool is perhaps the most difficult for peers to complete because of the current lack of opportunity for actual observation. Items 1, 2, and 4 appear on the student's evaluation of the teacher as well, again allowing peers or the peer review committee to compare and contrast results.

In each section the evaluator can select number 4 "no basis for evaluation" as an option. This flexibility in the tool allows input from individuals who work with a peer in only one area. Our expectation is that most evaluators will be able to complete the first section plus one of the other two sections.

Summary

The time has come for peers to establish criteria to evaluate each other. No longer can this vital activity be left to the students alone. This is not to say that student evaluation has no place but rather to identify the need for peer review based on mutually determined and acceptable criteria.

With the change to an integrated curriculum, in which there is emphasis on team work in both planning and execution of the program, the need for a tool to document areas previously considered to be too subjective becomes obvious. Faculty must be able to collaborate, cooperate, communicate, and be mutually supportive to each other in order to affect a successful operation of the program. Some faculty feel that these behaviors are an inherent part of being responsible, accountable and professional, and therefore do not need to be measured. Yet these are the same behaviors that we stress repeatedly and measure with students.

We believe that our tool provides nurse educators with a method of peer evaluation that is both equitable and easy to use. We believe that it will also help the faculty member determine his or her goals based on an evaluation process that is broader as a result of input from three levels: students, peers, and self.

Lastly, we believe that this tool meets the needs of the Peer Review Committee by providing valid input from peers to round out the evaluation procedure.

The procedure for using the tool is designed to help reduce the fears and anxieties of the faculty concerning the evaluation process as well as to produce a more all encompassing and equitable peer review.

References

  • Machlup, F. Poor learning from good teachers. Academe. October 1979, pp. 376-380.
  • McKeachie, W. Student ratings of faculty: A reprise. Academe. October 1979, pp. 384-397.
  • Raskin, B., and Plante, P. The student devaluation of teachers. Academe, October 1979, pp. 381-383.
  • Soper, B. Can students evaluate teaching effectiveness? National Forum, 1980, 60 (2), 27.
  • Stafford, L., and Graves, C. Same problems in evaluating teaching effectiveness. Nursing Outlook, 1978, 26 (8), 484-487.

10.3928/0148-4834-19830901-06

Sign up to receive

Journal E-contents