The Journal of Continuing Education in Nursing

Computer Assisted Evaluation of Problem Solving Skills of Primary Health Care Providers

Patricia A Woodbury, RN, MSN, CPNA

Abstract

The challenge to the modern educator is to find a means for handling the information explosion. The store of today's scientific and technical knowledge is so vast that it is impossible to instruct students on any but a small portion1 (Figure 1). The teaching-learning approach must give way to an easier approach to encourage the development of problem-solving skills which will enhance the student's use of knowledge.

To some degree individuals do possess an ability to problem solve. Since it is also a learned behavior some persons utilize the skill better than others. Factual knowledge is a must in the problem-solving process in order to recognize concepts and make judgments to complete the steps. In the educational program how the learner gets to the solution is considered more important than the end product. In a study of the characteristics of successful and unsuccessful problem solvers it was found that the successful ones were more systematic.2 Using a computer is very supportive to a systematic, step- by-step approach to learning. In fact, students are prevented from looking ahead, being distracted, or losing their way by being required to answer each question in sequence.

The health care student in particular must be assured of competency in the art of clinical problem solving or decision making. The problems presented in clinical practice necessitate action based on sound knowledge as well as a skillful decision-making process. The traditional training of the health science student is conducted in actual clinical or hospital situations with real patients. Frequently these clinical encounters are limited by the number and variety of patients as well as complete observations of the patient from beginning to end. With the involvement of the supervising preceptor the students seldom are allowed to experience the consequences of their own clinical judgments.

Case management simulation is one method of developing and evaluating the students' problem-solving skills. This method, either on paper or by some audiovisual or interactive means, seems to provide an approximate to the realistic clinical situation in which the students can try out their decision making without the influence of the clinical authority and with no risk to a patient.

Training in health care seems to be a natural fit for the technology of the computer. The science of health consists of factual content as well as an organized cognitive process to utilize the content. The mind can gather a larger amount of information than it can use. With a limited capacity for short-term memory, the mind often loses some of the information.3 The computer makes up for this limitation by storing large amounts of information which can be recalled at any time in the same form that it was received. In addition, the computer encourages, through programming schemes, orderly and logical processing of information which ultimately can assist in decision making.

The current lifestyle of the graduate level health science student indicates a desire for self-paced, individualized, interactive instruction. The varied backgrounds and levels of abilities of the students demand different educational and learning needs. These students often require flexibility in the educational curriculum due to part-time jobs. The computer is extremely suited to meet many of these needs. For example, a single time-sharing system with multiple terminals can replace a large number of personal tutors.

CASE MANAGEMENT SIMULATION- PNA 1

Upon accessing the program, the students are given a few brief instructions about the program and the specific commands they will need to know. A few demographic questions are asked to identify the participant by health care' discipline, educational status, and years of experience in clinical practice. This information will describe…

The challenge to the modern educator is to find a means for handling the information explosion. The store of today's scientific and technical knowledge is so vast that it is impossible to instruct students on any but a small portion1 (Figure 1). The teaching-learning approach must give way to an easier approach to encourage the development of problem-solving skills which will enhance the student's use of knowledge.

To some degree individuals do possess an ability to problem solve. Since it is also a learned behavior some persons utilize the skill better than others. Factual knowledge is a must in the problem-solving process in order to recognize concepts and make judgments to complete the steps. In the educational program how the learner gets to the solution is considered more important than the end product. In a study of the characteristics of successful and unsuccessful problem solvers it was found that the successful ones were more systematic.2 Using a computer is very supportive to a systematic, step- by-step approach to learning. In fact, students are prevented from looking ahead, being distracted, or losing their way by being required to answer each question in sequence.

The health care student in particular must be assured of competency in the art of clinical problem solving or decision making. The problems presented in clinical practice necessitate action based on sound knowledge as well as a skillful decision-making process. The traditional training of the health science student is conducted in actual clinical or hospital situations with real patients. Frequently these clinical encounters are limited by the number and variety of patients as well as complete observations of the patient from beginning to end. With the involvement of the supervising preceptor the students seldom are allowed to experience the consequences of their own clinical judgments.

Case management simulation is one method of developing and evaluating the students' problem-solving skills. This method, either on paper or by some audiovisual or interactive means, seems to provide an approximate to the realistic clinical situation in which the students can try out their decision making without the influence of the clinical authority and with no risk to a patient.

Training in health care seems to be a natural fit for the technology of the computer. The science of health consists of factual content as well as an organized cognitive process to utilize the content. The mind can gather a larger amount of information than it can use. With a limited capacity for short-term memory, the mind often loses some of the information.3 The computer makes up for this limitation by storing large amounts of information which can be recalled at any time in the same form that it was received. In addition, the computer encourages, through programming schemes, orderly and logical processing of information which ultimately can assist in decision making.

The current lifestyle of the graduate level health science student indicates a desire for self-paced, individualized, interactive instruction. The varied backgrounds and levels of abilities of the students demand different educational and learning needs. These students often require flexibility in the educational curriculum due to part-time jobs. The computer is extremely suited to meet many of these needs. For example, a single time-sharing system with multiple terminals can replace a large number of personal tutors.

Figure 1. An innovative means of handling the current information explosion.

Figure 1. An innovative means of handling the current information explosion.

Figure 2. The programming techniques or algorithm of program development.

Figure 2. The programming techniques or algorithm of program development.

In order for an interested educator to pursue the development of a computer program, some exposure to the world of computer technology is necessary. The educator needs to become familiar with different computer systems, their multiple uses, and the available hardware and software. Some understanding and facility in the use of the terminology in this scientific field might best be achieved through a review of the literature and/or enrollment in computer courses.

Computer programs of patient management simulations are not readily available. The time required for development and the cost of programming and maintaining such materials have outweighed the support and encouragement given to faculty to pursue these teaching and evaluating approaches. Programs must be developed so that more valid measures of student clinical problem solving skills can be available to today's educators.

METHODOLOGY

In order to address the problem of a reliable simulation to evaluate the clinical problem-solving skills of the students, we used a minor illness encounter associated with that part of the curriculum which addressed the diagnosis and management of acute disease in children. The students would be exposed to the program following classroom instruction in acute disease. The program assumed prior knowledge and therefore its goal was to evaluate the students' skill in data collection, diagnosis and management of illness, and integration of related knowledge areas.

Establishing the overall and specific goals of the program is important in order to maintain a focus as the program develops and to determine when the program reaches its goal. The specific goals for the program are to first simulate the real minor illness encounter as much as possible within the constraints of the computer. Then the student should be able to:

1. determine what kind and how much data to collect;

2. analyze the data and make appropriate judgments; and,

3. prescribe a correlated plan of treatment.

From the students' responses the educator would be able to evaluate their understanding of the approach to minor illness, knowledge of child/family development, pathology of disease, and the contribution of diagnostic procedures.

The length of the program should be consistent with the typical time frame of a clinic or office visit. The complexity and length of the program may help to determine the choice of hardware or type of computer system. More often the choices may be based on the cost or availability of an existing system. At the University of Minnesota there is a timesharing system for student education. This includes microcomputer terminals in the student learning resource center equipped with either hardcopy printers or cathode ray tube, video display units. The central computer system is accessed through the hard-wired or telephone connected terminals. For this program, the hardcopy equipped terminals were preferred because of the length of the program and for purposes of research and evaluation.

The greatest challenge to program development is determining the programming techniques or algorithm. An algorithm is the step-by-step procedure or rules for making the diagnostic decision. The chosen process will vary among authors. Basically there are two models: statistical or logical. The logical algorithm generally simulates the human diagnostic process. It includes flow charts, sequential questions, and branching or decision tree approaches. This latter approach consists of questions with several alternative responses that will lead each student in selecting a response. For this reason, the author must consider all possible responses to questions, appropriate reciprocal responses, and the direction to the next question or branching (Figure 2).

Consultation with colleagues with related content expertise will facilitate consideration of many more potential responses. An experienced programmer will be invaluable not only in converting the program into machine language, but also in recognizing such problems as branching and sequencing pitfalls, lack of logical flow, and redundancy. It is important for educational authors to know that a good computer programmer cannot be a substitute for a welldeveloped program design. The following is the description of the case management computer simulation program developed by this author.

Figure 3. Determining the approach or sequence of data collection.

Figure 3. Determining the approach or sequence of data collection.

CASE MANAGEMENT SIMULATION- PNA 1

Upon accessing the program, the students are given a few brief instructions about the program and the specific commands they will need to know. A few demographic questions are asked to identify the participant by health care' discipline, educational status, and years of experience in clinical practice. This information will describe the sample and provide correlative information with selected responses. Following these questions the students are given the basic identifying information about the minor illness situation for which they will be responsible for making an assessment, diagnosis, and plan. Simulated cases can be derived from textbooks, excerpts or medical records. However, data developed from subjective or hypothetical probabilities perform as well as actuarial probabilities at less time and cost for development.3

PNA 1 is divided into four problem-solving areas common to most health care providers:

1. Subjective or historical data collection

2. Objective data collection - physical examination and laboratory tests

3. Diagnosis

4. Plan of care, education and follow-up.

The content for each section is developed according to principles of child/family development, data collection process, and diagnosis and management of acute disease.

In the first two problem-solving areas the student is given menus to select the categories of data they wish to pursue relative to the database. The history question responses appear in terms similar to what the patient or parent might state in a real encounter. The results of the physical examination reflect the typical report or recording of a health care practitioner. In each case, the students determine the approach or sequence of data collection and when they have obtained sufficient data (Figure 3). The student is given feedback for the choices either in the form of actual clinical information or a reason for choosing another category at this point in the sequence. History and physical examination information must be integrated by the student in order to make the correct decision about necessary laboratory or diagnostic procedures. An obvious limitation in this simulation is the lack of visual and auditory information common to the real clinical encounter. Another programming option, not included in this simulation, is the opportunity to pursue a symptom or comment further as well as asking for additional clarification for specific clinical information. Instead, there are opportunities to provide reasoning for their responses at points where the author thought there might be a difference of opinion. These student comments will be used to revise the program later.

The climax of the program is when the students must make an independent judgment based on their integration of knowledge and analysis of the database, as to the diagnosis or diagnoses of the simulated patient. The program technique employed here is based on the method of instruction used in the nurse practitioner program. The technique consists of three steps:

1. The student must determine if the condition of the child is life threatening.

2. The student must determine if the presenting concern is a valid one based on the data collection.

3. The student must make a diagnosis.

The students may make as many diagnoses as they feel apply to the situation, and for each one they must include the etiology. Feedback to the participant in this section follows each diagnosis entered as to probability or correctness of judgment, need for more specific diagnosis and/ or etiology, and additional clinical information relative to this condition in pediatrics.

In order to proceed with the final area (the Plan) the student is advised of the correct diagnosis. In this area there are no right or wrong answers. Frequently in health care there are many equally good ways to manage a minor illness situation. Much depends on the individual's training and the comprehensiveness of the assessment. The technique applied in this area is a sequence of five questions with menus from which the student selects the preferred plan for treatment, patient education, and follow-up. The computer summarizes the chosen aspects of the plan and allows for any additions or changes. In reviewing the plan the student has the opportunity to verify the consistency of the plan in correlation with the diagnosis and the database. The method applied in this final area of the program provides for participant input, supports variation in management styles, and allows for self-evaluation.

PRELIMINARY TEST AND EVALUATION

It has been estimated that it takes approximately 100 to 200 hours of preparation time to one hour of instruction time to develop a computer program. In this experience the original writing of the program took less time to develop than the subsequent continuous debugging and correction process. It is anticipated that the current testing of the program will lead to additional hours of revisions and reprogramming.

Four samples of participants are being recruited to test the simulation program. These participants consist of current pediatric nurse associate (graduate school) students, pediatric nurse practitioners, family practice residents, and pediatric residents. The preliminary testing results are being stored in a separate computer file which is collecting basic descriptive data and frequency of responses. The responses of the participants will be helpful in determining the satisfaction of this program as an approximate simulation of the real clinical encounter. In addition these responses will help to make needed changes and adaptations in the program.

The PNA students' responses on the computer program will be compared to didactic testing on similar content used in their regular curriculum. Hölzerner et al showed that performance on a multiple-choice cognitive examination of similar content areas had predictive value as to performance on the simulation.4 A separate questionnaire will determine the students' response to the program relative to enjoyment, usability, benefits, helpfulness, self-evaluation, reflection of knowledge and performance, and as a testing tool in their coursework.

APPLICATION

This computer management simulation program is useful in the health professions for teaching and evaluating problem-solving skills in a specific minor illness encounter. It can be synthesized to a shorter form (currently it takes approximately 45 minutes to complete) which can be reprogrammed on a diskette and used on a microcomputer with only a CRT or videoscreen. It could also be expanded to incorporate audio and video components which would interface with the interactive computer.

The potential for using this program as a research tool has just begun to develop into several possible studies. The preliminary test thus far has given some indication as to potential pathways or patterns of decision making that can be identified among the participants. By documenting these patterns, another sample of participants could be compared to see what variables in the sample population correlate with the chosen pattern of problem solving.

Another study has been proposed to answer whether this computer simulation is a better method of clinical evaluation than the traditional model of on-site faculty observation of performance. The study would attempt to confirm that the computer simulation provides the opportunity for more independent decision making and eliminates the question of inter-rater reliability associated with the traditional clinical performance evaluation.

CONCLUSION

The author was challenged by this venture into a futuristic method of student education and evaluation. The participant responses to this first computer management simulation program has been positive and encouraging toward development of additional computer programs. The survival of this learning approach depends on well-documented research studies to validate the contribution of computer technology in student education.

References

  • 1. Yeaw EMJ: Problem solving as a method of teaching strategies in classroom and clinical teaching. J Nurs Educ 1980; 18(7):16-22.
  • 2. Hudgins BB: Problem Solving in the Classroom. New York, MacMillan Company, 1966, pp 15-19.
  • 3. Rogers W, Ryack B, Moeller G: Computer-aided medical diagnosis: A literature review. Int J Biomed Comput 1979; 10:267-287.
  • 4. Hölzerner WL, Schieutermann JA, Farrand LL, et al: A validation study: Simulations as a measure of nurse practitioners' problem-solving skills. Nurs Res 1981; 30(3):139-144.

10.3928/0022-0124-19840901-08

Sign up to receive

Journal E-contents