The main fundamental goal in nursing education is to learn to care for the human person integrally as well as individually. Evaluation of the quality of learning forms a part of the assessment of how far this goal has been achieved.
This study is concerned with the evaluation of the quality of learning. Answers are sought to questions concerning the mode of information processing that the student manifests in her learning outcomes, at the final stages of education. The mode is expressed in the way the students conceive the material and the cognitive structure. The research material consists of the final exam answers of surgical and psychiatric student nurses in Finland in 1985.
This study has its roots in the reform of nursing education which was initiated in Finland in 1984. In the development of methods for the evaluation of student performance, the main emphasis has been on qualitative evaluation.
It is useful to start by defining the concept of qualitative evaluation, especially in relation to quantitative evaluation. Two points of view are helpful. First, we can study the logic of the conclusions drawn by the evaluator. The logic of quantitative evaluation consists of classification and serialization, and output of numerical comparisons. The logic of qualitative evaluation is direct comparison, output in the form of new approaches, and new classifications.
Second, we can examine the way in which the evaluator relates to reality. In quantitative evaluation, the knowledge that is easily observable and empirically verifiable is regarded as the most characteristic feature of reality. Qualitative evaluation, then, focuses not only on the observable features of the phenomenon, but also on its significant relationships (Willis, 1978).
In the evaluation of the quality of learning, the aim is to acquire information on how much the student knows about the subject-matter. Qualitative evaluation reflects the depth of the caring structures and explanations adopted by the student in each particular field. Furthermore, it aims to determine the explanatory power and adaptability of the student's knowledge and problem-solving capacity.
As far as the evaluator is concerned, this means that the emphasis must be on the evaluation of the mastery of the central concepts within each field of knowledge (in the context of nursing, concepts such as person, health, environment, and nursing acts; of Fawcett, 1984), the principles of problem-solving and the theoretical knowledge behind the phenomena. Therefore, in the evaluation of exam answers for instance, attention must be paid not only to the content areas of knowledge mentioned by the student, but also to the way in which these are combined, and to the general structure of the answer. In the training of nurses, this means that theoretical and practical knowledge must be evaluated as a whole.
THE LEVELS OF INFORMATION PROCESSING (MARTON & SÄUÖ 1984, MARTON & SVENSSON 1982)
During the last two decades there has been a significant change of emphasis in the study of learning, as the behavioristic school has been losing ground to the cognitive approach. The advantage of the cognitive tradition is that the learning outcome depends to a great extent on the student's intentions, interpretations, structurings, and his/ her active knowledge - construction. The student constructs reality with the help of knowledge in the course of the learning process. By these constructions the person develops inner images of reality, i.e., cognitive structures. The construction process represents the core object of the evaluation of the quality of learning.
In knowledge processing, it is possible to distinguish between different levels which correspond to different cognitive structures. Marton and his research group differentiate between surface and deep level information processing, which involve qualitatively different kinds of activity (see Table 1).
Marton uses the term atomistic to describe surface level information processing; the corresponding term for deep level processing is holistic. The information processing and the outcome are intricately intertwined. New information is selected and interpreted on the basis of what already exists. The efficiency and depth of interpretation depends on the extent, integration, and conceptual level of previous informational structures. As the students conceptual structures become better organized, changes occur in his/ her action. Therefore, the nature of the process of knowledge acquisition is in turn reflected in the structure of the knowledge that is stored in the memory. At the end of his/ her education, the student nurse has formed a certain structure of caring knowledge.
Products indicating learning outcome (e.g. answers in a test paper) may be evaluated on the basis of the complexity of their cognitive structures. Cognitive complexity refers to individual differences in interpreting the world in an integrated, multidimensional, and discriminating way. The complexity of the product's structure can be determined using the concepts of differentiation, discrimination, and integration.
Differentiation in this context means the number of basis dimensions present in the structure of knowledge, discrimination of the level of accuracy in specifying these dimensions, and integration of the kind of conclusions drawn. A complex knowledge structure consists of many different dimensions. It is fine and flexibly integrated. The dimensions then form an integrated whole which adapts to new situations and is capable of forming new construction systems or alternative views (Mandi & Huber, 1978).
The goal in the training of nurses is to learn caring principles and their application in various situations. This requires the student to recognize the basic dimensions of caring and the ability to combine them into new individual entities. He/she should have access to alternative solutions and be able to cope with all kinds of situations in a flexible way. Therefore, training should aim to enhance the student's cognitive complexity.
Biggs & Collis (1982) have developed a classification for the purpose of evaluating cognitive complexity, especially the quality of learning outcome. They study the development of the structure of learning outcome in relation to Piaget's stages of cognitive development. From this basis, they attempt to find appropriate paths of student guidance. The levels of Biggs' and Collis' classification reflect the development of the student's cognitive structure. At the lower levels, the student's ability of discrimination and integration is illogical and random. At the higher levels, the student is capable of complex, holistic action.
Biggs & Collis call their classification the 'SOLO-taxonomy' (Structure of the Observed Learning Outcome). Its key concepts are the student's working capacity, the relating operation, and consistency. Working capacity reflects the number of cues and relevant items of knowledge that the individual is able to deal with simultaneously. The relating operation denotes the mode of interrelating the question and the answer. It also manifests the degree of generalization.
Consistency comprises the need to bring the action to an end. In striving for consistency, two kinds of needs can be seen in the student: the need to reach any conclusion and the need to reach a stable conclusion free from contradictions. Increasing structural complexity leads to a growing need for consistency and stability.
The data from this study consists of the final exam answers (written in the form of an essay) of surgical and psychiatric nursing students. The exam is taken after 2.5 years of education. These two branches of nursing were chosen for this investigation because they were supposed to emphasize different fields in the mastery of knowledge. The emphasis of surgical nursing was presumed to be more on the mastery of nursing acts, and psychiatric nursing's emphasis was rather on the holistic understanding of a person's mind. The aim of this comparison was to clarify the possible connection between the subject matter to be learned and the information processing.
The data was collected in 1985 by asking two groups of students to send in their marked answers by post. The number of answers received in both groups was 209, giving a total of 418 ( = N) answers. All of the students contacted participated, and all of the answers received had been graded pass.
The data was analyzed using the content analysis method. Two types of investigation, associative and discursive, can be distinguished in content analysis (Pietilä, 1973). The associative pattern of thought is based on the idea that, in principle, a linguistic document, e.g. a text, can be broken up into its basic parts and these parts can again be classified into content classes. The occurrence of the classes together and separately in the texts is studied statistically. The discursive pattern of thought, for its part, is based on the idea of the text as a whole that cannot be broken up into its basic parts in the way associati /e thinking suggests, because a text is not a statistical, but a logical whole.
In this study, questions and answers are wanted to be kept as logical wholes with the intention of clarifying the view on the quality of learning. In the background of these patterns of thought, classification of words, clauses, paragraphs, etc., is not the way to reach the objective of this investigation. Results of the investigation are indicated in frequency- and percentage-distributions and verbal descriptions.
Responses were analyzed twice at an approximate 2month interval. Actual parallel classifiers were not used. The two main concerns in the analysis were: 1) the student's mode of conceiving the material, and 2) the knowledge structure of the answer. Factors influencing the students' mode of conceiving and knowledge structure of students were not sought at this point.
The following categorization was used in the analysis of the mode of conceiving:
Category A: Atomistic approach. The student concentrates on facts and details. He/she does not attempt to achieve an integral picture of the subject-matter. Details or facts are not combined with reference to main theme. The outcome remains narrow and restricted (surface level).
Category B: Meristic approach. The student concentrates on certain parts of the whole and conceives the material concerned as partial wholes. Within these he/she analyzes and combines items, but does not organize interrelations between the partial wholes, nor connections or developmental trends (surface level).
Category C: Holistic approach. The student concentrates on wholes. He/she attempts to create an integral whole, organized around a central theme. An integral whole is formed by organizing the relations between the parts of the whole and by utilizing details to clarify and support the main theme (deep level).
The student's knowledge structure was analyzed using the following categories:
Category 1: Prestructural answer. The student combines items in an arbitrary way and lists separate items without applying any consistent criteria of selection (surface level).
Category 2: Unilevel answer. The student deals with the matter from one single aspect without being able to name any even closely connected items (surface level).
Category 3: Multilevel answer. The student deals with the matter from several aspects combining items, e.g. in a lateral or serial way. However, no conclusions are drawn on the basis of how the different aspects are related; there seems to be no recognition of interrelating aspects (surface level).
Category 4: Relating answer. The student deals with the matter with the help of several interrelating aspects, displaying a hierarchic knowledge structure (deep level).
The categorization of the data proved to be relatively reliable. On the conceiving dimension 95% and on the structure dimension 93% of the answers were classified in the same categories both times (in content analysis anything above 90% can be considered reliable). The high reliability is partly explained by the short-time interval between the categorizations.
STUDENTS' MODE OF CONCEIVING
In considering the validity of content analysis, there are two relevant problems. Does an item placed in a certain category actually correspond to that category, and does that category properly represent the phenomenon it is supposed to represent? In the present context, the question we must ask is whether the final exam answers adequately reflect the way in which the student processes information in practical nursing situations, or merely the strategy she has found most appropriate in the context of nursing education.
Secondly, it must of course also be asked whether the structure of the answer is a valid measure of the student's strategy of processing information. As far as the caring profession is concerned, the analytical categories of mode conceiving and knowledge structure are valid, in the sense that the qualitative differences in nursing are most clearly reflected in whether or not the student is capable of encountering the patient as an integral individual.
Mode of conceiving
The distributions by mode of perceiving are presented in Table 2. On the whole, the atomistic mode of conceiving was clearly predominant. By subject-matter, the tendency towards the atomistic mode of conceiving the material was more evident in surgical than in psychiatric nursing. There were no difficulties in recognizing the atomistic mode: the student listed various items in his/her answers without any logical connections between them.
Classification of the meristic mode of conceiving was more problematic, especially in the cases where the student tended to divide the matter into partial wholes (by giving titles to the parts, for example), but then within these entities used the atomistic mode. These answers were classified in the category of atomistic answers. Students whose approach was classified as meristic showed that they had recognized the parts of the whole, e.g. by underlining the key words or sentences in their answers.
The holistic approach was clearly underrepresented. In typical cases, the student conceived a certain main theme and pursued it throughout the answer. The answers in this category varied considerably, especially in their details. The number of details played no part in the classification; the answer was classified as holistic if it showed a recognizable main theme.
The multilevel category (category 3) was clearly predominant in the students' knowledge structure. This suggests that the students recognize factors related to nursing situations from several different aspects. However, it is important to note that this category represents the surfacelevel method of information processing.
The proportion of answers classified in this category was higher in surgical nursing. This trend is further supported by the low number of answers falling under the relating category of deep level (category 4) responses. It seems that in psychiatric nursing, there is a slightly clearer trend towards deep level, relating answer structures. Prestructural answers (category 1) typically consisted of short lists, and the items in these lists were not discussed at all (see Table 3).
The proportion of prestructural and unilevel answers was very low. The typical answer in the latter category dealt with the question from one single point of view. For instance, when asked how she would care for a patient with a plaster cast, the student described the more technical aspect of handling the plaster, but omitted all other caring aspects. Those answers, in which the student proved he/she had understood the matter from several different aspects, were classified as multilevel.
In this category there was variation between the answers in how extensively and profoundly they dealt with these various aspects. In a relating-level answer, the student usually started by determining the main goal and then pursued various items related to the main and sub-goals throughout the answer. A relating answer ended with a conclusion, a summary, or was left partly open.
STUDENTS' KNOWLEDGE STRUCTURE
Knowledge structure and mode of conceiving: summary.
The student's mode of conceiving was atomistic in all the prestructural and unilevel answers. The rest of the atomistic answers fell into the category of multilevel structures. In other words, the students whose mode of conceiving was atomistic, used in their answers completely unstructured, unilevel, or multilevel structures.
We can see clear qualitative differences in the answers with multilevel structures: a multilevel structure could appear through an atomistic or a meristic mode of conceiving. By character, the atomistic multilevel answers were such that they merely tended to state the facts, whereas the meristic multilevel answers contained logical parallel parts of the whole. In both, there is no attempt to relate the parts to each other.
All of the answers reflecting a holistic mode of conceiving fell into the category of relational answering structures. In addition, meristic responses, in which the student had shown an ability to relate in some narrow aspect of her answer, also fell in the category of relating answers, even if the answer as a whole could not be considered holistic.
In nursing education the primary objective is to learn to care for the human person. On the student's part, this requires a holistic orientation to training and teaching materials and integral application of one's knowledge. In this study, both of these dimensions are considered by analyzing the final exam answers of student nurses. The results are not very encouraging. Students tended to conceive the material in an atomistic way, and the structure of their knowledge can be described as multilevel. Both of these results suggest that even after his/her 2.5 year training, the nursing student still tends to process information at a rather superficial level.
It seems that in surgical nursing the students are more inclined to superficial information processing. However, the differences are small and a connection between the branch of nursing and the level of information processing cannot be found on this basis. One objective of this study was to find methods for analyzing exam answers on a qualitative basis. The categories employed in the analysis were found to be suitable for the present context of nursing education, but nevertheless it would be useful to develop more precise tools for evaluation.
The classification categories utilized made possible the qualitative description of the structure of the responses. The classification was relatively easy and the reliability was high. The use of parallel classifiers would have been expethent for testing the reliability. However, a kind of comprehensive analysis of the responses is in question, so special attention must be paid on the validity.
In a test it is always possible that the student answers in the way he/she has found to be the most profitable, but which is not necessarily his/her own strategy on the whole. To improve the validity, the investigation could have been complemented by an interview, a learning strategy measure, or additional exercises besides the final exam answers.
Any generalizations from the results presented above must be made with caution. However, what every nursing teacher should see here is that at the time they graduate, nursing students still tend to have a highly atomistic and fragmentary understanding of their professional knowledge. In the present study we were not concerned to seek relevant background factors. Therefore, at this point, we can offer no explanation for the results.
So, the question for further research is: what are the factors in the curriculum, in teaching, and in evaluation practices that support this kind of thinking? How could these factors be eliminated and how could we develop the kind of evaluation methods that would facilitate the adoption of a holistic approach to learning? In order to answer these challenging questions, it is obviously necessary to do more research in the field of evaluation of the quality of learning.
- Biggs, J.B. & Collis, K.E (1982). Evaluating the quality of learning. The SOLO -taxonomy. New York: Academic Press.
- Fawcett, J. ( 1984 ). Analysis and Evaluation of Conceptual Models of Nursing. Philadelphia: EA. Davis.
- Mandi, H. & Huber, G.L. (1978). Kognitive Komplexität. Göttningen: Verlag fur Psychologie.
- Marton, E & Svensson, L. (1982). Towards a phenomenography of learning II. Gothenburg: Department of Education, University of Gothenburg.
- Marton, F. & Säljö, R. (1984). Approaches to Learning. In: Marton, F. et al. (1984). The Experience of Learning. Edinburgh: Scottish Academic Press.
- Pietila, V. (1973). Sisällön erittely Helsinki: Gaudeamus.
- Willis, G. (Ed. ) (1978). Qualitative evaluation, concepts and cases in curriculum criticism. Berkeley: McCutchan.
THE LEVELS OF INFORMATION PROCESSING (MARTON & SÄUÖ 1984, MARTON & SVENSSON 1982)
STUDENTS' MODE OF CONCEIVING
STUDENTS' KNOWLEDGE STRUCTURE