Journal of Nursing Education

The articles prior to January 2012 are part of the back file collection and are not available with a current paid subscription. To access the article, you may purchase it or purchase the complete back file collection here

TIPS ON TEST CONSTRUCTION

Suzanne MacAvoy, RN, EdD; Lynne B Welch, RN, EdD

Abstract

An integral component of the educational system in the United States is that of testing. From the time young children enter the system until the time they leave, they are faced with test-taking, as frequently the sole indicator of their degree of success in this system. In professional education, this focus on achievement assumes even greater significance because of its presumed correlation to the potential and/or actual ability of the learner. This article will provide suggestions for nursing educators to consider in the construction of meaningful testing instruments. The factors affecting testing which will be considered are: classifications of testing, the learner, testing by objective, writing objectives, content, faculty expectations, selection of items, evaluation of items, and environment for testing.

Classification of Testing

Prior to actual test construction, faculty need to consider which type of testing they will use, criterion-referenced or normreferenced. Criterion-referenced testing evaluates the student against an absolute standard. This kind of testing determines what a student knows based upon the objectives that have been mastered. Normreferenced testing distributes scores, so that students are measured against a . relative standard which reflects how their knowledge relates to the knowledge other students possess. High achievement in a norm-referenced test does not necessarily indicate high mastery of the content tested. This is important to keep in mind when planning approaches to testing and when constructing tests, particularly in light of accountability and consumerism.

The Learner

There are many factors that affect the construction of a good examination but probably the most significant of these is the learner. Who are the learners? What are their backgrounds, their goals? What knowledge do they bring with them to our courses, both experiential and theoretical? What supportive courses have they completed? Are they concrete thinkers, abstract thinkers, or somewhere in between?

Concrete thinkers handle factual material more easily and may have difficulty with transfer of knowledge from one situation to another; for example, the application of the nursing process to a variety of patient situations. They will do better on factual knowledge questions and simple recall. Since learning concepts requires multiple examples, testing of conceptual materials must come after enough time has been provided for this learning to take place. Initial testing should relate to the facts basic to the concept. For example, "define the steps of the nursing process," rather than, "evaluate the nursing process utilized in the following situations."

It is essential that we be familiar with our students in these terms so that we can utilize the knowledge and thinking ability they possess to its greatest advantage. These factors affect the vocabulary used in test items, the amount of nursing and medical terminology incorporated, the degree of sophistication expected in their answers, and the degree of scholarship considered appropriate. Faculty frequently come upon this information intuitively. Obtaining these data about our learners purposely and systematically, allows us to respond to them more appropriately and, certainly, more comfortably.

Testing by Objectives

One technique which is helpful in writing appropriate test items is that of formulating definitive objectives for each area of class content and then writing items that test each objective. The key here is in writing a good objective. If the objective, for example, indicates a learner goal of application, the test item, in order to be congruent, must be one which tests this level of knowledge. An example would be the following:

Objective: Applies the steps of the nursing process to nursing situations.

Item: Which of the following statements is an example of nursing assessment?

a. How does your foot feel today?

b. I'm going to rub your back now.…

An integral component of the educational system in the United States is that of testing. From the time young children enter the system until the time they leave, they are faced with test-taking, as frequently the sole indicator of their degree of success in this system. In professional education, this focus on achievement assumes even greater significance because of its presumed correlation to the potential and/or actual ability of the learner. This article will provide suggestions for nursing educators to consider in the construction of meaningful testing instruments. The factors affecting testing which will be considered are: classifications of testing, the learner, testing by objective, writing objectives, content, faculty expectations, selection of items, evaluation of items, and environment for testing.

Classification of Testing

Prior to actual test construction, faculty need to consider which type of testing they will use, criterion-referenced or normreferenced. Criterion-referenced testing evaluates the student against an absolute standard. This kind of testing determines what a student knows based upon the objectives that have been mastered. Normreferenced testing distributes scores, so that students are measured against a . relative standard which reflects how their knowledge relates to the knowledge other students possess. High achievement in a norm-referenced test does not necessarily indicate high mastery of the content tested. This is important to keep in mind when planning approaches to testing and when constructing tests, particularly in light of accountability and consumerism.

The Learner

There are many factors that affect the construction of a good examination but probably the most significant of these is the learner. Who are the learners? What are their backgrounds, their goals? What knowledge do they bring with them to our courses, both experiential and theoretical? What supportive courses have they completed? Are they concrete thinkers, abstract thinkers, or somewhere in between?

Concrete thinkers handle factual material more easily and may have difficulty with transfer of knowledge from one situation to another; for example, the application of the nursing process to a variety of patient situations. They will do better on factual knowledge questions and simple recall. Since learning concepts requires multiple examples, testing of conceptual materials must come after enough time has been provided for this learning to take place. Initial testing should relate to the facts basic to the concept. For example, "define the steps of the nursing process," rather than, "evaluate the nursing process utilized in the following situations."

It is essential that we be familiar with our students in these terms so that we can utilize the knowledge and thinking ability they possess to its greatest advantage. These factors affect the vocabulary used in test items, the amount of nursing and medical terminology incorporated, the degree of sophistication expected in their answers, and the degree of scholarship considered appropriate. Faculty frequently come upon this information intuitively. Obtaining these data about our learners purposely and systematically, allows us to respond to them more appropriately and, certainly, more comfortably.

Testing by Objectives

One technique which is helpful in writing appropriate test items is that of formulating definitive objectives for each area of class content and then writing items that test each objective. The key here is in writing a good objective. If the objective, for example, indicates a learner goal of application, the test item, in order to be congruent, must be one which tests this level of knowledge. An example would be the following:

Objective: Applies the steps of the nursing process to nursing situations.

Item: Which of the following statements is an example of nursing assessment?

a. How does your foot feel today?

b. I'm going to rub your back now.

c. His anxiety is due to fear of surgery.

d. He should be turned every two hours.

In this item, the student must apply her knowledge of what constitutes a nursing assessment to the statements listed in the options.

Writing good objectives and appropriate test items takes practice but ultimately makes the job of test construction much simpler. When students receive the content objectives, they know precisely what is expected of them in the testing situation. For example, will they have to "list" the information, "explain" the information, "analyze" the situation, or whatever. Their study, therefore, is more productive. They will know what, and how, to study, because they will know which level of knowledge is being tested. How often have we heard, "What are we supposed to study for this test?" or variations on that theme?

Writing Objectives

In order for objectives to be meaningful to both students and faculty, they should encompass three areas: the conditions under which the behavior will occur, the act they are to perform, and the degree of performance or criterion of accepted performance. For example, "given the nursing process (conditions), list each of the four steps (act), with complete accuracy (degree of performance)." In some objectives, the conditions and the degree of performance are implied. For example, the objective stated above could also be worded as "list each of the four steps of the nursing process." With other objectives, more explicit wording is necessary. The precision with which the objectives are written is the crucial factor in making them meaningful.

When objectives are written in behavioral terms, the expected action is stated and, therefore, the measurement technique is identified. If the action verb is "define" or "list," the measurement technique is the definition or the listing. Although not all content is equally adaptable to this approach, behavioral objectives communicate their intent to both faculty and students better than more general objectives. In general objectives, the action verb frequently is not precise enough to communicate its meaning and how it would be measured.

Content

Another factor critical to this discussion is the content which we are testing. If it is complex and sequential we must be sure that the basic material is understood before testing the more sophisticated information. Assessment or entry tests are appropriate to ascertain basic understanding, while independent study modules with post-tests are one effective means of students' acquiring basic knowledge. However, even simple factual material needs to be assessed in terms of the level of learning desired. Do we want the students to simply recall the information, or do we want them to demonstrate comprehension of the material by explaining it or making predictions? Bloom's taxonomy, which classifies cognitive learning into components of knowledge, comprehension, application, analysis, synthesis and evaluation, is an invaluable aid in helping faculty be precise in their testing. In nursing programs, with concepts being frequently threaded throughout the curriculum, this taxonomy can be quite helpful. For example, sophomore students might be tested at the recall and comprehension levels for certain information, while juniors may be tested at the application level for that same information, and seniors at the higher levels of analysis, synthesis and evaluation. This again demonstrates how important it is to establish precise objectives so that the tests constructed actually measure the content and are congruent with faculty expectations.

Faculty Expectations

It is crucial for the faculty to be consistent in their expectations of the students and in what they are teaching and testing. It is not unusual to find faculty teaching and testing at the recall level in the classroom and simultaneously expecting application and synthesis in clinical laboratory situations. Then we wonder why the students have trouble "correla ting theory to practice."

It is important for faculty to look critically at their test items and ask themselves if they are testing what they are teaching. Team teaching has facilitated this process in those schools where faculty on teaching teams critique each others' test items for clarity, pertinence, ambiguity, accuracy, and consistency with learner objectives. For this process to be most effective, faculty need to be comfortable in their roles as teachers and function in a collégial atmosphere where such critiques will be given and received as constructive measures to improve test effectiveness.

Selection of Test Items

It is also important to select the appropriate type of test question for the objectives being tested. For example, objectives which relate to the affective domain, such as attitudes and values, are more difficult to evaluate in the traditional paper-andpencil types of tests commonly used in nursing. Objectives which relate to the higher levels of the cognitive domain, such as analysis and synthesis, may be more effectively tested by essay questions rather than objective tests. When writing objective test items, there are some basic tenets to follow. Be sure the stem and the options are congruent in content, context, structure and grammar. All options should be of similar length. Correct options should be varied in placement, i.e., be aware of your pattern of correct options, such as generally placing the correct answer first or second. Once your items are arranged in the preferred sequence, again check the pattern of correct answers and adjust them as indicated. When ordering your items, it is helpful to the students to have easy items first, followed by more difficult questions or more conceptual material. The length of the test should be appropriate for the type and amount of material being tested. Test taking at the recall and comprehension levels generally involves less time than at the analysis and synthesis levels. The tests should be long enough to adequately sample the material taught, but short enough, and frequent enough, to prevent fatigue. Learner fatigue can be a deterrent to optimal test performance.

Evaluation of Items

Following testing, an item analysis is particularly valuable in helping faculty evaluate the items for retention, elimination, or restructuring. Many computers in colleges and universities are, or can be, programmed to perform this service. Analysis of the questions in terms of the difficulty and discrimination indices, provides a rational basis for item critiques by faculty. Critique of test items by teaching teams improves the overall quality of the testing instrument. As items aredeveloped, tested, refined and retested, a file of valid and reliable test items can be collected and used as a resource by faculty. When the test bank, or file, becomes extensive enough, matched tests can be developed. One of the authors has found it helpful to identify, on the item analysis card for each item, the number of the content objective on the item tests. The objectives tested are also kept filed with the test items. Items should also be periodically evaluated to determine if they are still relevant to the content objectives or if they reflect content preferences of faculty which may be interesting but extraneous.

Environment for Testing

Another factor which affects the testing situation is the environment in which the test is taken. Maintenance of quiet by closing doors/windows as needed and by not talking with other faculty or students is important. Distractions of this nature can be very disturbing to students and model a lack of consideration of others. There should be adequate light, heat and ventilation, and, if possible, enough space between students so that cheating is not a temptation. Where adequate space is not possible, matched tests can be used.

Before beginning the test, be sure the directions are clear. State if you will allow questions to be asked during the test. Exams should be proofread before being reproduced; however, if any corrections need to be made, writing them on the board and drawing the students' attention to them before testing is begun is preferable to reading them aloud or interrupting the students once the test has started. Reading corrections aloud can cause confusion resulting in errors, while interruptions interfere with concentration. Facilitating test taking by environmental control is an important faculty responsibility.

Being a nursing educator in today's society is a position for which we are accountable. We guide the students through our programs and monitor their progress. We, therefore, need to systematically plan and evaluate this progress. One way to do this is by developing wellconstructed tests.

Bibliography

  • Bloom BS (ed): Taxonomy of Educational Objectives: Handbook I, Cognitive Domain. New York, David McKay, 1956.
  • Gagne RM, Briggs LJ: Principici of Instructional Design. New York, Holt, Rinehart and Winston, lnc, 1974.
  • Hug WE: instructional Design and the Media Program. Chicago, American Library Association, 1975.
  • Krathwohl DR: Taxonomy of Educational Objectives: Handbook II, Affective Domain. New York, David McKay, 1964.
  • Mager RF: Goal Analysis. Belmont, California, Fearon Publishers, Lear Siegler, Inc, 1972.
  • Mager RF: Preparing Instructional Objectives. Palo Alto, California, Fearon Publishers, 1962.
  • Reilly DE: Behavioral Objectives in Nursing: Evaluation of Learner Attainment. New York, Appleton-CenturyCrofts, 1975.

10.3928/0148-4834-19810301-04

Sign up to receive

Journal E-contents