External examining is a system of quality assurance m nursing education that is quite foreign to most nurses in the United States. A search of the ERIC documents data base in 1998 uncovered 42 articles on the topic; not one of which was published in an American nursing journal. This article attempts to address the omission by providing American nurses with a definition of the concept, criteria for appointment, and roles and functions of an external examiner including the four stages of the process. Conclusions about the experience are drawn by the authors, African-American nurses who served as external examiners for a baccalaureate degree program in the department of nursing at the University of Botswana in southern Africa.
The concept of an external examiner- a visiting assessor of high academic standing and possessed of integrity and objectivity- supports a notion of the universality of educational standards and justice for the individual student (Walter et al,, 1995). External, in this context, means that the examiner is from another institution and is involved in teaching courses similar to those that s/he examines. At the most basic level, external examiners bring a combination of knowledge of the area examined and a detachment from the context in which a course is delivered (Sheehan, 1994). Indeed in England, at the university level where external examining is a long-standing tradition, external examiners are considered to be the major guarantors of quality and equality in higher education. They have power to influence the decisions made in the examining process and in some instances the ultimate décision may rest with them as to whether or not to award a degree. It was not until 1985, however, that nursing education was included in this system of quality assurance in education. At that time the English National Board of Nursing, Midwifery and Health Visiting (1993) published the policy requiring the appointment of external examiners for the final examination in National Health Service-based nurse education as well as guidelines for the external examiner. The new system was intended to ensure that internal examiners would be expected to assume greater responsibility for the maintenance of academic and professional standards (Bradshaw & Moore, 1988).
Sheehan (1994) identified three crucial issues in the external examining process. First, there is the vast and sometimes overwhelming mass of evidence in various forms that need to be assessed. Second, the time between receiving the exam materials for preview and the actual examination is usually short, and because external examiners typically come from the ranks of those who are currently employed, they also have their own job related work to complete. Unfortunately, there is a strong reluctance to employ retired people on the grounds that they Eire no longer current. Third, there is the challenge of coining to a decision that is fair to all concerned.
A landlocked country, the Republic of Botswana lies in the southern African region, sharing borders with South Africa in the east and south, Namibia in the west, and both Zambia and Zimbabwe in the northeast. The eastern part of the country contains the major cities and the bulk of the 1.5 minion population. The Kgalagadi Desert covers 84% of the land area (Botswana Visitors Guide, 1994).
Slightly more than 20 years old, the University of Botswana, the only university in the country, was founded in the capital city of Gaborone in 1976. This was nearly ten years after independence was granted to this former British Protectorate known then as Bechuanaland. Instruction is in English at all levels of the University.
Created in 1980, the department of nursing education is a department in the faculty of education at the University of Botswana. In 1978, the first university education for nurses began as a program in nursing and one of the tracks in the science education department in the faculty of education. This initial nursing program consisted of two nursing staff and eight post-basic Botswana student nurses.
The program in nursing became a department of nursing education in 1980. In the 18 years since its creation, the department has graduated more than 300 nurses with post-basic baccalaureate degrees not only from Botswana but other countries such as Swaziland, Lesotho, Zimbabwe, Malawi, Kenya, Uganda, South Africa, and Namibia.
CRITERIA FOR APPOINTMENT
Typically, external examiners in African universities have been British educators or British-educated Africans. The appointment of the authors departed radically from this tradition since neither is a British educator nor a British-educated African; both are African-American and American educated. Together the two authors have served five years as external examiners for nursing education in Botswana in Southern Africa.
Potential external examiners are appointed primarily for a period of one to three years through the informal social and professional networks within educational institutions of the host country. The host institution's primary concern is likely to be the appointee's level of formal academic qualification (Shanley, 1990). Typically, the candidate is identified by the departmental chairperson, confirmed by departmental faculty, and a formal letter of appointment is sent by the university's vice chancellor (president).
There are six basic criteria for appointment:
1. Proposed external examiner should not have been a member of staff, governor, or a student in the institution in the last 5 years.
2. Expertise and experience teaching the subject to be examined.
3. Competence in assessing student knowledge.
4. Equivalence between the level of the subject to be examined and the qualifications of the potential external examiner.
5. Experience as an external examiner (preferred).
6. Compensation package determined by the university (Sheehan, 1994).
In America external examinations such as the National Council Licensure Examination (NCLEX), state boards of nursing, and National League for Nursing (NLN) accreditation procedures are preferred methods for evaluating academic achievement of students as well as curriculum quality in nursing education programs. Public accountability, a very important concept in America, has led American nurse educators to believe that nationally based examinations inspire more public confidence local ones.
ROLE AND FUNCTIONS OF AN EXTERNAL EXAMINER
Under the British system, no degree is awarded the procedure has been overseen by an external examiner (Campbell, 1991). The primary role of the external examiner is to ensure that examination procedures are conducted according to university regulations governing the degree and that the examination questions reflect the aims and objectives for the course(s) examined. For the examining process to operate effectively the following conditions are required:
* Timely and full information on objectives of the examination and curriculum.
* Willingness to function within local circumstances and procedures.
* Consultation during the development of the examination and an invitation to comment on the final draft.
* Involvement in the assessment of student performance by reviewing continuous assessment materials and selected final and midterm examinations- usually those at the higher and lower ends of the grading scale as well as those in the middle range.
* Dual role of ensuring justice for the individual student and maintenance of institutional standards for awarding of the degree (Walters et al., p. 1095).
THE EXTERNAL EXAMINING PROCESS: FOUR STAGES
Stage One: Curriculum Review
The departmental chairperson, for the course(s) to be examined, provides the external examiner with a packet of materials for review and evaluation as the first step in the process. The packet includes all course syllabi with specifics of assignments, special projects and all examinations including oral, written, and clinical/practicals. The external examiner reviews each syllabus for consistency between program and course objectives, content, and desired skill level. S/he is expected to judge whether the content is at the appropriate level and quality for the degree/diploma to be awarded (Sheehan, 1994).
In addition, the accompanying final examination questions and scoring/grading strategies must be evaluated. Specific critical areas to consider during the review:
1. Do the examination questions address the aims and objectives of the course?
2. Do the questions adequately cover the content and skills identified in the syllabus?
3. Are the examination points or weighting allocated appropriately for the content and skills covered? A note should be made of any errors in question stems, distracters, or answer key.
4. Are the questions clearly stated and do they include sufficient background information to enable the student to answer the question fully?
5. Are all expected answers shown so that the external examiner can determine if the question will elicit the desired response?
6. Do the syllabi, tests, and grading fit together consistently (Partington, 1993)?
The external examiner summarizes the findings from this review and sends the results to the specific departmental chairperson. Recommendations are made to the department regarding clarity of the examination questions and consistency of the grading scheme. Any recommended changes are noted at this time. This written report must be returned promptly to allow time for the department to make changes prior to the final examination time.
During the final examination period, departmental faculty administer and grade the examinations. Some institutions place student scores/grades on the examination booklets while others do not disclose the internal examination scores. Such decisions are left to the discretion of the institution.
Stage Two: The On-Site Visit
The on-site visit, the second stage of the external examining process, begins when the external examiner arrives at the institution for a period of five working days. All course-related materials for the academic year, from each graduating student, are presented to the external examiner at this time. Meetings are arranged with departmental faculty to review the curriculum and further clarify specifics of each course and can include other areas of concern that faculty wish the external examiner to address during the review process. An assessment of course workloads for the student is an example of another area of concern. The most time-consuming aspect of the process, the actual grading of course examinations, is then completed.
Since the analysis, evaluation, and grading of examinations can be a monumental task because of the large numbers of examinations to be graded, it is helpful to establish a systematic plan prior to beginning (Partington et al., 1993; Walters et al., 1995). How the examiner proceeds depends on personal preference. According to Bradshaw & Moore (1988), a majority of external examiners assess the examinations on a sampling basis. Walters et al. (1995) recommended the sampling to include upper and lower ends of the grading scale plus a sample of those in the middle. In addition, Sheehan (1994) endorsed sampling the examinations of all first year, third year, and failing students.
Grading clinical/practical examinations in nursing requires explicit guidelines. Since the clinical/practical examination must be evaluated and graded in the presence of both the internal and external examiner, it is scheduled during the on-site visit. This process is also rather lengthy and time-consuming. To facilitate this assessment process, an observation guide with the grading scale clearly delineated is a necessity. Of equal importance, before the examination begins, is to inform the student of performance expectations, how s/he is to be evaluated including time limitations. Guidelines must also include regulations that cover verbal communication between student and faculty, opportunities for repeating any portion of the examination, steps to be taken if the student performs poorly at the beginning of the examination, and which behaviors are mandatory for passing. Jinks & Morrison (1997) commented that "none of the established approaches to assessing clinical practice are ideal. The process of assessing clinical is still evolving and a great deal of work needs to be done to ensure assessment methods developed measure up to the task" (p. 409). Finally, it is necessary to determine whether the two examiners will discuss the student's performance and reach a mutually agreeable grade or if each one arrives at a grade independently.
Stage Three: Consolidating Internal and External Assessments
The grades from the external examiner are recorded on a standard form along with those of the faculty or internal examiners. The results for each course are verified by both examiners; the standardized form is then signed by the external examiner, the course faculty member, and the departmental chairperson.
All grade reports are submitted for review and approval to the departmental curriculum committee before forwarding them to the University Academic Board where, based on student grades, the final decision on which degree will be awarded to the student is determined: Fu-st or Second Class Pass; Pass; Supplemental Fail; Fail and Repeat; or Fail and Discontinue.
Stage Four: The External Examiner's Reports
Two reports are required. The first is an oral report given to the departmental faculty at the conclusion of the site visit. This report should include information on student performance in comparison to their peers in comparable programs, strengths and weaknesses of students, and quality of the knowledge and skills demonstrated by students. At this time faculty may make comments and raise questions for the external examiner related to the oral report.
The second report, due one month following the on-site visit, is written and addressed to the vice chancellor of the university. The structure, design, organization of the curriculum, grading of course assessment materials, quality of teaching as indicated by student achievement, syllabi, teaching strategies, human and instructional resources, and evaluation procedures can be components of the written report.
Finally the written report may also include recommendations for improvement or alteration in the program of study (Sheehan, 1994). Individual external examiner reports vary widely in structure, content, and length (Shanley, 1990). If possible the report should be written prior to departure, thereby ensuring that the one-month deadline for submitting this final report is met.
Perhaps because of distance and absence from the setting, personal experience has shown that the hoped for impact of the written report can be difficult to assess. Recommendations relating to programmatic, curriculum, and examination question revisions often do not appear to have influenced departmental decisions. Identified problems continued to be present when reviewed upon the return of the external examiner to the university during the second year of appointment.
The experience can be both emotionally challenging and physically draining. It is emotionally challenging because the external examiners' decisions can significantly alter a nurse's career path by determining whether it will move forward, will be interrupted, delayed, or ended. Although some institutions vary their interpretation of the English National Board (ENB, 1993) guidelines, according to the guidelines the responsibility for the final grading decision rests with the external examiner.
The physical drain results from the rigors of the 15hour flight from the United States, the need to function immediately, effectively, and efficiently in a different climate and time zone, and then coping with the stress of fulfilling requirements within the 5-day period allocated for the appointment. The process becomes extremely rushed and seems overwhelming, especially when the appointee is unfamiliar with the system and the process. Adding to the anxiety is the possibility that educational requirements for baccalaureate education for nurses in the United States may be inappropriate in parts of the developing world.
It is questionable whether an academic clinician visiting very briefly from a developed country can really appreciate differing expectations related to role, performance, and responsibilities of the graduate nurse- expectations that are often unrelated to educational preparation. For example, the graduate may be expected to assume responsibilities far beyond the level of the preparation provided by the host country's nursing education program. Therefore, the judgments of external examiners may not be informed by the same set of considerations as the internal examiners who teach the course(s). A large measure of respect, therefore, is due for judgments of nurse educators in the host country who have the most indepth, insightful, and comprehensive understanding of the educational and performance needs for graduate nurses at all educational levels. Although internal examiners in the course(s) examined may have been formally prepared as expert clinicians and not as teachers, they may lack some of the expected pedagogical skills held by the appointee. Therefore, persons who have difficulty accepting nursing education practices that differ from their own are perhaps better advised not to accept this responsibility.
THE SYSTEM: FUTURE PREDICTIONS
Around the world there is much divergence of opinion about the value and future of external examiners. In Canada, for example, they are largely a thing of the past; in the United States they are not used at all. Nevertheless, in England they remain a highly valued means for ensuring comparable standards for first college degrees at all schools in the country (Walters et al., 1995).
In other countries where the system is in use, such as those in Australia, the trend is toward reduction in utilization. However, m former British-administered countries of Africa and Asia external examiners are still extensively used. Nevertheless, the growing dissatisfaction with the external examiner system in those countries continuing to use it and the lack of objective data to support its efficacy, suggests that in the future it will have a diminishing place. An additional factor that will mitigate against the system is its high cost at a time when universities are experiencing hard financial times (Walters et al., 1995). In the final analysis, as Sheehan (1994) observed "... it is possible to run an educational system and award qualifications without the involvement of external examiners" (p. 948).
Although predictions forecast elimination of the external examination system, it remains highly respected in a number of countries. Given increased mobility, the globalization of all aspects of modern life, and more nursing programs offering course experiences in other countries, it is likely that other nurses in the United States may be invited to become external examiners. Information about this process can also be significant to nurse educators who serve as site reviewers to accrediting bodies in the United States. Moreover, some knowledge of an educator's role and function in a unique system can be very helpful. With such insights, the potential appointees, as Walters et al. (1995) observed, ". . . could sometimes contribute more than they do" (p. 1094).
Contributions while on site could include, but is not limited to, such activities as providing a workshop for departmental faculty on creative teaching strategies, writing for publication, clinical teaching methods, and principles of evaluation. Bilateral nursing research projects-educational and clinical- could be discussed. Jinks and Morrison (1997) reported that "a great deal of research is urgently needed to explore the process of assessing clinical practice" (p. 412).
Since many of the health care problems in the developing world are largely preventable and nursing curricula remain primarily illness-focused and cure-oriented, serious dialogue could be initiated in a faculty forum. The discussion, planned to coincide with the on-site visit, could center on a curriculum shift to community-based primary health care, additional general education content, patient-centered nursing care, and ethical dilemmas in practice.
Nurses in the United States, who are arguably the best educated in the world, have much to share as well as learn from our colleagues who continue to use the external examination system. The role and functions of an appointee permit far more of value than traveling abroad to review and grade examination papers. The experience can be an enriching one for both appointee and host institution!
Some critics of the concept view the system as hardly more than a means of free travel for senior academic faculty whose contribution to the host educational institution is quantitative rather than qualitative. These critics believe an internal examiner could provide the same service at much less cost.
Walters et al., (1995) observed that carefully chosen external examiners can provide the appropriate balance between the important interrelated issues of academic rigor and contextual relevance; who could on occasion, contribute much more than they do. However, without appropriate orientation and experience with requirements for the role and function of an examiner, what they can and do accomplish is limited. The Handbook for External Examiners in Higher Education (1993) would have been extremely helpful to one author had it been available for the initial appointment in 1991. This document is strongly recommended as must reading for all first-time external examiners, especially American. This requirement would contribute immeasurably to achieving the desired role function goals.
Without question, at the time of the actual visit the external examiner must be prepared to listen, observe, share, and learn. Too often Americans give the impression that they have the best system, know everything there is to know, and are inclined to tell others how to solve their problems!
Sheehan (1994) has summarized the three historic roles of the external examiner as: (1) ensuring comparability between institutions; (2) providing a safeguard for fairness to students; and (3) guaranteeing that institutional procedures in course examinations and assigned grades are upheld.
Clearly the role of the external examiner is an extremely complex one with many facets and many issues relating to the extent of their powers, upholding standards, exercising impartial judgment, and legitimizing examination results. It is important for the appointee to recognize his or her own motivation for agreeing to become an external examiner for nursing education programs. If the decision will be driven by the desire to travel to an exotic location, or merely to include the experience on a resume, then the intended value to the host country and the appointee may well be completely lost.
- Bradshaw, P., & Moore, D. (1988). External examiners' role perceptions and opinions. Nursing Times, 84, 50.
- Campbell, P. (1991). Some thoughts on the role of external examiners. Biochemical Education, 19, 13-14.
- Department of Information & Broadcasting. (1994). Botswana visitor's guide. Gaborone, Botswana: Department of Printing and Publishing.
- English National Board for Nursing, Midwifery and Health Visiting. (1993). Regulations and guidelines for the approval of institutions and courses. London: Author.
- Jinks, A,, & Morrison, P. (1997). The role of the external examiner in the assessment of clinical practice. Nurse Education lbday, 17, 408-412.
- Partington, J., Brown, G-, & Gordon, G. (1993). Handbook for external examiners in higher education. Sheffield, England: The Universities of Kent and Leeds.
- Shanley, E. (1990). External examiner system in traditional nurse education. Nurse Education Today, 10, 136-139.
- Sheehan, J. (1994). External examiners: Roles and issues. Journal of Advanced Nursing, 20, 943-949.
- Walters, W., Sivanesaratnam, V., & Hamilton, J. (1995). External examiners. Lancet, 345, 1093-1095.