Litwak, Line and Bower (1985) define evaluation as the "continuing process of assessing individual knowledge, competencies, and behavior. Although it frequently involves a series of measurements, evaluation always rests on professional judgment based on sets of predetermined criteria" (p. 3). The concept of evaluating students' knowledge, skills, and abilities is not new in nursing education. Most of the evaluation, however, has been that of individual student achievement or examination of different aspects of the nursing program as part of an overall program evaluation plan. The accreditation criteria, and therefore the systematic evaluation plan, included sections on structure and governance, material resources, policies, faculty, and the curriculum (NLN, 1983).
In 1989, the Council for Baccalaureate and Higher Degree Programs in Nursing approved changes in the criteria for accreditation. Included in the revision were five required outcome criteria: critical thinking, communication, therapeutic nursing interventions, graduation rates, and patterns of employment (NLN, 1992). The first three of the five criteria require nursing programs to use student achievement data for program evaluation purposes. Programs must evaluate the critical thinking abilities of individual students and use that outcome data in the development, maintenance, and revision of programs. Recognition of the dual purpose of the NLN outcome criteria requires a modified approach to evaluation that is different from the traditional single purpose approach to either student achievement or program evaluation. Though certainly related to one another, student achievement and program evaluation have been viewed as distinctly different and separate, both in terms of process and product.
The model for evaluating critical thinking that is proposed was developed using the curriculum development process described by Torres and Stanton (1982), principles of program evaluation (Litwak, Line, & Bower, 1985), and the use of the critical thinking process as conceived by Brookfield (1987) and Paul (1993). For discussion purposes, the implementation process is presented in four phases, and each phase addresses a component of the model. The entire model is depicted in the Figure. The solid lines down the center of the model represent the unidirectional progress of the curriculum development process. The dotted lines indicate feedback loops used during curriculum development. Finally, the broken lines depict the program evaluation process with the potential to evaluate and revise any or all of the components of critical thinking. This process guides the faculty in the evaluation and revision of the curriculum.
Torres and Stanton (1982) describe the process of curriculum development beginning with discussion of philosophy, definition of terms, and a description of the characteristics of the graduate. Then curriculum objectives, course objectives, and specific content are identified. The entire curriculum is examined, though the emphasis is placed on courses in the nursing major. During the next phase of curriculum design, the philosophy is translated into action. An overall plan for implementation, including testing and teaching strategies, is developed. A significant component at this point is the examination for congruence from the philosophy to the teaching and testing methods and processes. Finally, judgments are made about the degree to which the graduates have attained the characteristics described in the initial phase of curriculum design.
Figure. Model for critical thinking.
The process of curriculum development is not unfamiliar to faculty in nursing programs. Often curriculum development centers around nursing content, i.e., theory and clinical practice settings, roles of the professional nurse, an identified nursing model or theorist, or an integrated approach to the nursing curriculum. The construct of critical tWnking is not a set body of knowledge, but is a process or way of thinking about theory, practice, roles, and so forth. Brookfield (1987) writes that "critical thinking is a process not an outcome" (p. 6). While that is true about critical thinking, per se, it is the challenge for nurse educators to describe the outcome or product of the critical thinking process in order to teach it and evaluate its use relevant to nursing practice.
The proposed model has several advantages for nurse educators. First, it provides a format for development and implementation that is based on a familiar format, that of curriculum development. The model also demonstrates the relationship among the necessary components of critical thinking and its infusion throughout the curriculum which should be particularly useful in striving for congruence among the definition, expectations of the graduate, methods of evaluation, and so forth. The components of the model organize the critical thinking process and identify the decision points that occur during the process. Lastly, the model acknowledges the similarities and overlap between the evaluation of individual student achievement and program evaluation, while making the necessary distinction between those two evaluation targets. The author believes this last advantage addresses a primary source of difficulty for nursing programs with respect to the NLN critical thinking outcome criterion (Videbeck, 1997).
USING THE MODEL
The process for implementation or use of the model includes questions to be addressed by faculty and assumptions that need to be examined based in part from relevant literature and research on critical thinking. The emphasis is placed on how to think about critical thinking, and less on what to think about it. In essence, it is the application of critical thinking processes to be used by nurse educators when addressing the construct and its application in nursing education. Hence the first assumption for examination: All nurse educators have the ability to think critically, and do so equally well. Hartley and Aukamp (1994) found that nurse educators had a significantly higher level of critical thinking ability than did nursing students, but Saarmann, Freitas, Rapps, and Riegel (1992) found that the critical thinking ability of faculty was not significantly higher than that of nursing students when the influence of age was controlled statistically.
PHASE ONE: DESCRIBING CRITICAL THINKING
The initial phase of the development process for critical thinking includes describing critical thinking in a global sense, followed by operationalizing and contextualizing the global definition. The solid lines in the Figure represent the essentially unidirectional steps in this initial phase of development, and the dotted lines indicate feedback loops to be used during the curriculum development discussions as questions are answered and assumptions are examined.
Whether the global definition is adopted or adapted from a recognized author, or constructed from a variety of sources, it is essential to have a definition that is used consistently throughout the process. Specifically, if both cognitive abilities and affective qualities are included in the conception of critical thinking, then both domains must be addressed when identifying expectations of graduates, curriculum and course objectives, teaching and development activities, as well as stated outcomes and measurement of those outcomes. It can be tempting to include attributes such as a spirit of inquiry, fair and open-mindedness, or intellectual curiosity simply because they fit the ideal of critical thinking. Educators would like their graduates to possess such qualities but find affective attributes difficult to operationalize, teach and evaluate. However, if these qualities are included in global and/or operational definitions, then they must be taught and evaluated.
Questions for nurse educators include:
What does the program mean by critical thinking?
How is critical thinking operationalized?
What behaviors indicate use of critical thinking abilities?
What affective qualities should graduates possess? Can it (they) be described in behavioral terms? Can it (they) be measured?
Can faculty agree on the operational definition(s), at least to the point that students get a clear and consistent message?
Another important aspect during the initial phase involves contextualizing the operational definitions) of critical thinking. Nurse educators often view problem-solving, decision-making, and use of the nursing process as evidence of critical thinking in a global way with little discrimination among those activities (Jones & Brown, 1991; Tanner, 1993; Videbeck, 1995). Kramer (1981) suggests that providing care, simple teaching, and use of predictable interventions for patients with common and recurring problems can be carried out by persons with technical background and preparation. These activities may not necessitate use of critical thinking abilities. Some nurse educators surveyed (Videbeck, 1995) indicated that whether or not activities such as problem-solving or use of the nursing process required critical thinking would depend on the situation or context surrounding the activity. Contextualizing critical thinking can be guided by the following questions:
What factors distinguish common, recurring situations from those that are non-routine?
What variables, such as culture or personal beliefs, will alter the decision or judgment than might otherwise be indicated?
What factors make a situation, or the care of a client, complex in nature?
What assumptions might interfere with viewing a situation from multiple perspectives?
Assumptions related to, and possibly interfering with, contextualizing critical thinking are:
The most complex situations involve the sickest patients, or those in certain clinical areas, i.e., intensive care units.
All nursing activities require critical thinking.
Nursing activities that do not require critical thinking are not important, and therefore, should not be the responsibility of the professional nurse.
Evidence of critical thinking in nursing practice includes some of the decisions regarding context, particularly with respect to clinical judgments in the care of patients with complex problems. It also necessarily includes discussion of nursing practice activities in a variety of roles. Activities related to conducting research, supervising and counseling staff, and making complicated budgetary decisions may well be considered by nurse educators when identifying evidence of critical thinking in nursing practice. Doubtless, some activities of this nature will be described as requiring critical thinking abilities. Once nursing practice activities are identified, it is important to consider the level of advanced education and/or experience needed to fulfill these types of responsibilities. In other words, they undoubtedly require critical thinking abilities, but is it reasonable to expect that newly graduated nurses with an undergraduate degree and little experience could or should occupy roles commensurate with these responsibilities? This leads to a series of questions appropriate to this phase of the critical thinking planning process:
What expectations are reasonable for the graduate upon completion of the undergraduate nursing curriculum?
What responsibilities require advanced education and/or experience?
What foundations for advanced roles and responsibilities are necessary in the undergraduate curriculum?
What does providing foundation knowledge and/or abilities mean?
Accompanying assumptions to examine during this phase of decision-making are:
Students benefit from at least a brief introduction to the activities of all roles and responsibilities that are possible in professional nursing.
The undergraduate curriculum can (or should) prepare the graduate for all future practice.
Once the expectations for baccalaureate graduates have been determined, they form the basis for both curriculum objectives and operationalized outcomes related to critical thinking in the next phase of development.
PHASE TWO: IDENTIFICATION OF OUTCOMES
It is important to describe critical thinking abilities at the curriculum objective level of the curriculum since these abilities develop over time and require attention to their development throughout the nursing major. Shortterm efforts or occasional assignments and exercises will produce few and disappointing results.
This is one of the points in development where checking for congruence is crucial. If the definition of critical thinking and the expectations for graduates are based heavily on making clinical judgments, then the stated outcomes need to reflect that emphasis. If critical thinking abilities have been contextualized to require consideration of complex patient care needs or differing cultures or personal values, the outcomes need to be stated in terms of those contexts.
An essential component of the dual function of the model, namely individual student achievement and program evaluation, is directly addressed in the designation of critical thinking outcomes. Litwak, Line, and Bower (1985) distinguish between mandatory and desirable criteria. Mandatory criteria are those students must meet to continue in the program, while desirable criteria refers to all the material that faculty would like students to master. In the same way, a program outcome in critical thinking should be stated in terms of what it can be reasonable to expect from all graduates of the nursing program. A baseline expectation is established, and data for program evaluation purposes are reported in terms of those graduates meeting, exceeding, or failing to meet the outcome, in aggregate form. Over time, if data showed that too many graduates clustered in the "fails to meet" or "exceeds" category, program revision could be indicated. One of the possible problems is that the outcome may be too minimal or too stringent. The identification of desired outcomes for individual student achievement can represent the broader range of abilities that exist among students, so that all students can be challenged to develop to their fullest potential. Pertinent questions that might be helpful are:
What are the cognitive skills and/or affective qualities that represent the minimum achievement to be expected from graduates?
What are the full range of abilities and qualities that faculty would like to see graduates possess?
Are the outcomes for critical thinking integrated with other nursing practice outcomes rather than isolated or separate outcomes?
The identification of course objectives relevant to critical thinking may actually involve revision of existing objectives or construction of new ones. This is particularly true if the conception of critical thinking contains or is built upon activities such as problem-solving or decisionmaking. Selecting or grouping existing course objectives related to other processes will not necessarily result in critical thinking objectives or abilities. Course objectives need to identify the components and/or behavioral evidence of critical thinking in order to maintain congruence within the construct of critical thinking, and to provide needed direction to faculty and students in both classroom and clinical settings.
It is also necessary to identify a hierarchical order for the concepts and behaviors of critical thinking that have been identified by the faculty (Workman & Allcorn, 1985). The principle of building from simple to complex knowledge and skills applies to critical thinking as it does to many concepts in nursing as well as other fields. For example, it would seem logical to determine whether students could distinguish significant or relevant data before proceeding to making any kind of clinical judgment based on that relevant data. Likewise, students must accurately and clearly define an issue before being able to take a defensible position on that issue. Too often, clinical judgment or the elements of defending one's position on an issue are introduced in their entirety with too little consideration of the progression of the knowledge and skills involved. Questions at this stage of the model include:
How can the knowledge, skills, and abilities involved in critical thinking be arranged in a hierarchical manner?
How will the sequence of nursing courses (and relevant course objectives) use this hierarchical progression?
What kind of collaboration among faculty in different courses will be needed to maximize student learning/achievement?
Relevant assumptions to examine are:
After students learn the foundations of critical thinking, they will begin to use those abilities when an appropriate situation occurs.
If the theoretical components of critical thinking are presented in the first nursing course, then students apply those abilities in subsequent courses.
PHASE THREE: PLANNING FOR TEACHING/LEARNING
This phase of the model involves the selection of teaching strategies, learning experiences (both classroom and clinical), and formative evaluation processes. There are many references in the higher education and nursing literature that describe teaching/learning strategies designed to promote critical thinking. Two themes are apparent in most of these references: increased use of varied classroom strategies; and use of collaborative teaching techniques, that is, collaboration among students.
The lecture continues to be the most frequent format for the classroom, despite numerous articles and recommendations to the contrary. The lecture, no matter how pertinent, relevant, or well delivered, will not foster critical thinking abilities of students. Nursing, as well as other disciplines, laments that there is so much content that has to be covered, there is simply no other method as efficient as the classroom lecture, there is not enough time to incorporate other activities. It is true that alternatives to the lecture take more "teacher time" to prepare, are less predictable in terms of the "content" that is covered, and need student participation to provide the most benefit to the learner. But if nurse educators do not begin to use some of the alternative, more interactive classroom activities, students' opportunities to think critically will continue to be limited. Lectures need not be totally abandoned, for a good lecture is a suitable format for certain types of material, and is certainly efficient (Brookfield, 1990). But the lecture does not provide practice in using critical thinking skills.
The other theme that emerges in the literature regarding learning activities is a need to use collaborative or group learning situations more effectively. Looking back at the ways in which critical thinking is evidenced in nursing practice, it could be said that few of these activities take place in isolation. Collaboration among colleagues within nursing as well as with members of other disciplines is frequently a desired goal in nursing practice, and therefore, nursing education. Nurse educators emphasize the need for collaboration and have course and/or curriculum objectives that relate to the collaborative role or function. However, most of the "instruction" and evaluation of students is done on an individual basis, involving teacher and student. For students to value collaboration and become skilled in working collaboratively with others, they must have significantly more experience doing so in the educational setting. If a professional nurse had a very complex patient care situation, or a complicated problem to solve, wouldn't it be an expectation that the nurse would seek assistance from colleagues? It is doubtful that it would be wise or practical for the nurse to struggle with the situation all alone. Again, if collaboration is an expectation in practice, the educational experience should provide opportunities to develop needed abilities in similar circumstances.
Formative evaluation strategies are included with teaching strategies and are distinct from summative evaluation. Litwak, Line, and Bower (1985) define formative evaluation as data-gathering throughout the educational process. Data from formative evaluation may be used to modify teaching strategies while a course is in progress (Brookfield, 1990) without disrupting the entire course. Formative evaluation is also essential to guide student learning in the form of constructive feedback. That feedback is most helpful to the student when it is related to standards of achievement (Paul, 1993), such as accuracy, depth, relevancy, clarity, and so forth. For example, if students are assigned to write a paper on a current issue in health care, typical guidelines for the assignment usually include length, format, and essential components for the content, i.e., description of the issue, the student's position, and rationale for the position. A student may complete the assignment, including all the elements, yet produce a paper of questionable quality. However, if the essential components include standards, such as in-depth description of the issue, a clearly stated position on the issue, and logical, defensible rationale, it is much easier to provide corrective feedback. The other advantage of using standards is providing the student with more information about what the assignment entails, or what kind of thinking is required.
Some of the relevant questions at this point are:
What learning experiences will lead to the desired student outcomes?
What sequence of learning experiences will foster development of critical thinking skills?
Since experiences in the clinical setting cannot be predicted, what teaching strategies can supplement clinical learning?
How closely do expectations of students mirror expectations in practice?
Assumptions to examine include:
It is essential to include all of the "content" of nursing, that is, students must have at least a brief exposure to all content.
Observational experiences are valuable if they expose students to clinical areas where actual experience is not possible.
Students readily transfer knowledge and skills from the classroom to the clinical setting.
Experiences that promote critical thinking abilities can be generalized from one setting or situation to another.
If students can solve problems, make decisions, or think critically on an individual basis, they will be able to do so in a group situation.
PHASE FOUR: SUMMATIVE EVALUATION
In planning for the evaluation of students' critical thinking abilities, it is helpful to reexamine the expectations and outcomes. The selected methods for evaluation need to be closely related to the established outcomes. For example, if an outcome describes the student's ability to make a clinical judgment about patient care in a complex situation, then the selected method needs to evaluate the student's ability to do so. A written standardized multiple-choice test would not provide the data needed to evaluate the outcome.
Data collected to evaluate individual student achievement may be similar to data collected for program evaluation purposes, or in some instances perhaps the same method will be used to generate data for both purposes. A primary difference will be the way in which the data are used to make the two different decisions that are required. Data for program evaluation purposes are reported in terms of the students/graduates' relationship to the established minimum achievement required, in aggregate form. Often this is reported as those meeting the standard(s), exceeding the standard(s), or failing to meet the standard(s).
Data related to individual student achievement have been used throughout the actual implementation of the "critical thinking curriculum" to guide and foster individual achievement as students progressed through the nursing major. From a summative evaluation perspective, the range of student achievement data can assist faculty in evaluating instructional processes, student expectations, and so forth as indicated by the broken lines in the Figure. If for example, the majority of students "excelled" in their individual achievement of critical thinking abilities, faculty might want to examine whether or not individual expectations are too minimal, that is, students might be capable of achieving more than was expected in terms of critical thinking. Or perhaps the selected methods did not measure what was intended.
To date, there have been no tests or methods developed that "measure" or evaluate critical thinking in nursing. Faculty developed assessment or measurement methods would probably best reflect the program's conception of critical thinking in nursing practice. Standardized paperand-pencil tests are often chosen because reliability and validity have been established, sometimes including normative data specifically using registered nurse groups. Since these tests usually report data on abilities of inductive and deductive reasoning, identification of assumptions, and so forth in a non-nursing context, the results might be useful to determine students' baseline abilities upon entering the major, if the tests are to be used at all.
Relevant questions for this phase of the model include:
Which measures can be selected to best evaluate students' critical thinking abilities?
Does the method of measure closely resemble the established outcome, that is, does the measure mirror the outcome in terms of content, conditions, and so forth?
Should the measures be directly related to a nursing course, or would a capstone project provide more accurate and useful data?
Examination of assumptions at this point will involve questioning some of the evaluation methods that have long been used in nursing education. If, in fact, nursing is attempting to produce graduates who are able to think critically in order to keep pace with the rapid rate of change in health care, then traditional evaluation processes must be seriously questioned. Though not inclusive, some assumptions that have been made are:
Performance in clinical practice, particularly clinical judgment skills, can be evaluated through written work submitted by the student, i.e., the written work reflects actual clinical performance (Tanner, 1986, p. 8).
The written nursing care plan, often developed after the care of the patient has been given, is an effective teaching tool, and/or provide pertinent data for evaluating students' thinking skills.
Critical thinking skills were used to develop NANDA nursing diagnoses, therefore, use of these diagnoses requires critical thieving abilities.
The most accurate evaluation of critical thinking focuses on the solution to the problem or the decision that is made.
As with any model that includes feedback loops, coming to the end places one back at the beginning. The data from summative evaluation that describe the output of the model provide input as the ongoing cycle of planning, implementing, and evaluating continues.
The model for critical thinking has been presented in an effort to clarify and delineate the dual evaluation aspects of NLN Required Outcome Criterion 1: Critical Thinking. Since the model is based on the curriculum development process and program evaluation principles that are familiar to faculty, it can be incorporated into existing development and evaluation activities. Because the model identifies the similarities and differences in the two types of evaluation, it is hoped that nurse educators wül be assisted to address both individual student achievement and program evaluation with respect to critical thinking more clearly.
As with any newly proposed model, its true usefulness and its continued development will only occur as it is used by nurse educators. The sample of questions to ask and the assumptions to examine are not inclusive, but only a beginning. As with any critical thinking process, more and better questions are generated by those involved in its use.
- Brookfield, S.D. (1987). Developing critical thinkers: Challenging adults to explore alternative ways of thinking and acting. San Francisco, CA: Jossey-Bass.
- Brookfield, S.D. (1990). The skillful teacher. San Francisco, CA Jossey-Bass.
- Hartley, D., & Aukamp, V. (1994). Critical thinking ability of nurse educators and nursing students. Journal of Nursing Education, 33(1), 34-35.
- Jones, S.A, & Brown, L.N. (1991). Critical thinking: Impact on nursing education. Journal of Advanced Nursing, 16, 529-533.
- Kramer, M. (1981). Philosophical foundations of baccalaureate nursing education. Nursing Outlook, 29(4), 224-228.
- Litwak, L., Line, L., & Bower, D. (1985). Evaluation in nursing: Principles and practice. New York, NY: National League for Nursing.
- National League for Nursing. (1983). Criteria for baccalaureate and higher degree programs in nursing. New York, NY: Author.
- National League for Nursing. (1992). Criteria and guidelines far the evaluation of baccalaureate and higher degree programs of nursing. New York, NY: Author.
- Paul, R. W. (1993). Critical thinking: How to prepare students for a rapidly changing world. Santa Rosa, CA Foundation for Critical Thinking.
- Saarmann, L., Freitas, L., Rapps, J, & Riegel, B. (1992). The relationship of education to critical thinking ability and values among nurses: Socialization into professional nursing. Journal of Professional Nursing, 8(1), 26-34.
- Tanner, CA. (1986). The nursing care plan as a teaching method: Reason or ritual? Nurse Educator, 11(4), 8-9.
- Tanner, CA. (1993). Thinking about critical thinking. Journal of Nursing Education, 32(3), 99-100.
- Torres, G., & Stanton, M. (1982). Curriculum process in nursing: A guide to curriculum development. Englewood Cliffs, NJ: Prentice-Hall.
- Videbeck, S.L. (1995). Critical thinking as an outcome measure in baccalaureate schools of nursing. Dissertation Abstracts International, 9531798.
- Videbeck, S.L. (1997). Critical thinking: Prevailing practice in baccalaureate schools of nursing. Journal of Nursing Education, 36(1), 5-10.
- Workman, L.L., & Allcorn, S. (1985). A new look at nursing school program development: Self- and external evaluation. Journal of Nursing Education, 24(5), 187-191.