Journal of Nursing Education

Major Article 

Advancing the Science of Research in Nursing Education: Contributions of the Critical Decision Method

Angela M. McNelis, PhD, RN, ANEF, CNE; Pamela M. Ironside, PhD, RN, ANEF, FAAN; Sarah E. Zvonar, BSN, RN; Patricia R. Ebright, PhD, RN, FAAN

Abstract

Advancing the science of nursing education will require the discipline to conduct research that investigates complex phenomena, such as students’ clinical thinking and decision-making skills, using multiple methods. The research methods developed in other disciplines can provide nursing education researchers with new ways to investigate clinical teaching and learning in nursing. The critical decision method (CDM), derived from psychology and human factors engineering, is a technique by which researchers elicit experts’ thinking and the cognitive work informing decision making in the context of practice. This article describes how the CDM was adapted to study nursing students’ situation awareness, cues for action, and pattern recognition during clinical experiences. The CDM is a promising method for investigators to use to conduct research in nursing education and to inform the design of clinical experiences to promote these critical abilities. [J Nurs Educ. 2014;53(2);61–64.]

Dr. McNelis is Associate Professor, Dr. Ironside is Professor, Ms. Zvonar is a graduate student, and Dr. Ebright is Associate Professor, Indiana University School of Nursing, Indianapolis, Indiana.

This study was supported by Indiana University School of Nursing and the National Council of State Boards of Nursing.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

Address correspondence to Angela M. McNelis, PhD, RN, ANEF, CNE, Associate Professor, Indiana University School of Nursing, 1111 Middle Drive, NU E435, Indianapolis, IN 46202; e-mail: ammcneli@iu.edu.

Received: January 02, 2013
Accepted: September 18, 2013
Posted Online: January 22, 2014

Abstract

Advancing the science of nursing education will require the discipline to conduct research that investigates complex phenomena, such as students’ clinical thinking and decision-making skills, using multiple methods. The research methods developed in other disciplines can provide nursing education researchers with new ways to investigate clinical teaching and learning in nursing. The critical decision method (CDM), derived from psychology and human factors engineering, is a technique by which researchers elicit experts’ thinking and the cognitive work informing decision making in the context of practice. This article describes how the CDM was adapted to study nursing students’ situation awareness, cues for action, and pattern recognition during clinical experiences. The CDM is a promising method for investigators to use to conduct research in nursing education and to inform the design of clinical experiences to promote these critical abilities. [J Nurs Educ. 2014;53(2);61–64.]

Dr. McNelis is Associate Professor, Dr. Ironside is Professor, Ms. Zvonar is a graduate student, and Dr. Ebright is Associate Professor, Indiana University School of Nursing, Indianapolis, Indiana.

This study was supported by Indiana University School of Nursing and the National Council of State Boards of Nursing.

The authors have disclosed no potential conflicts of interest, financial or otherwise.

Address correspondence to Angela M. McNelis, PhD, RN, ANEF, CNE, Associate Professor, Indiana University School of Nursing, 1111 Middle Drive, NU E435, Indianapolis, IN 46202; e-mail: ammcneli@iu.edu.

Received: January 02, 2013
Accepted: September 18, 2013
Posted Online: January 22, 2014

Calls for advancing the science of nursing education are plentiful and reflect the discipline’s urgent need for a body of research to guide pedagogical decision making (Institute of Medicine, 2010). However, concerns persist about the methodological rigor, the types of methods used, and the sophistication of published studies (Broome, Ironside, & McNelis, 2012). One way to move forward the science of nursing education is to adapt the methods developed and tested in other disciplines to study nursing phenomena. The critical decision method (CDM; Crandall, Klein, & Hoffman, 2006; Klein, 1998), developed in psychology and human factors engineering, has been successfully used to study new and expert nurses’ cognitive work and decision making in the context of practice (Gazarian, 2013); however, it has not yet been used to study nursing students’ thinking and cognitive work. This article describes the use of CDM as a research method for exploring nursing students’ decision making during clinical learning experiences and the strengths, limitations, and implications of this method for advancing the science of nursing education.

A New Method for Nursing Education Research

The CDM is derived from a broad family of cognitive task analysis methods that uncover human cognition and decision-making processes in environments that are characterized by high stakes, high time pressure, or high complexity. The methods are focused on the outcomes that people want to achieve, using particular procedures for identifying important cognitive activities associated with tasks leading to desired outcomes (Crandall et al., 2006). The CDM is based on the premise that in a given situation, individuals will pick up on cues and indicators that allow them to recognize patterns and then choose a single course of action they believe will best achieve the desired outcomes (Klein, 1998). This method enables researchers to elicit descriptions of practice situations that reveal participants’ complex cognitive functions, such as decision making and sense making (Crandall et al., 2006), relative to an event or experience that may be difficult for participants to articulate. Focusing on nonroutine events, researchers interview participants (e.g., decision makers), using the CDM, and, using cognitive probes, ask them to describe recent events during which a decision was made or an action was taken (Klein, Calderwood, & MacGregor, 1989). Cognitive probes are designed to require participants to reflect on their thinking, their cues and goals for action, what they were seeing and hearing at the time, and their selection of strategies or decisions. Because expert practice is seamless and intuitive (Benner, Tanner, & Chesla, 1998), the use of nonroutine events more readily brings specific situations to the minds of the participants so they can be further explored by the investigator during the study.

Cognitive task analysis methods, including the CDM, have been widely used in health care and other industries, including aviation, firefighting, nuclear power, the military, and national security, as a way to better understand the thinking and cognitive work involved in practice and decision making. In health care, researchers have used the CDM to investigate the assessment skills of neonatal intensive care nurses (Crandall & Calderwood, 1989; Militello & Lim, 1995), nurses’ use of computer-based medical devices (Obradovich & Woods, 1996), situation awareness in emergency medical dispatch personnel (Blandford & Wong, 2004), and the cues nurses use to identify and intervene during a potentially preventable cardiopulmonary arrest (Gazarian, Henneman, & Chandler, 2010). In four studies, Pryor (2004, 2005, 2006, 2007) used the CDM to examine nurses’ cues and decision making while providing care to patients with brain injury and aggressive behavior. Recently, Cioffi (2012) used the CDM to explore nurse midwives’ decisions about whether to use sutures in patients after childbirth, focusing on the cues, goals, and typical ways of responding to arrive at a course of action.

Although the CDM is most commonly used with experts in other fields, Ebright et al. (2003, 2004) used it to understand the complexity of RN work and novice nurses’ decisions about near-miss or adverse event situations in acute care settings. These studies, which led to the identification of “RN stacking,” or how nurses manage complex clinical situations (Ebright et al., 2003), and to the description of common patterns surrounding new nurse graduate errors and near-miss situations (Ebright et al., 2004), demonstrated that the CDM could be useful in broader contexts. The strength of the CDM is that it is contextually rich—meaning, it offers researchers a way to examine participants’ thinking and cognition in naturally occurring and complex settings. It provides a technique for investigators to make visible the tacit knowledge and decisions participants use in managing care. The CDM allows participants to describe factors that influenced their judgments (Crandall & Getchell-Reiter, 1993), what they expected to occur, and what they learned from the situation. The CDM is a helpful method for studying experts because their experience is so contextual and nuanced that it is often difficult to articulate discrete aspects of the situation, cues, and patterns (Benner et al., 1998). Combining observations of actual work, followed by interviews during which investigators probe this cognitive work, can shed light on how decisions are made in practice.

In seeking to better understand clinical education in nursing, the authors adapted the CDM for use with student participants. Although students have not been included in studies that use the CDM, the authors believed that because students are often prompted by faculty to reflect on their experiences, the CDM would be a familiar approach to draw out students’ descriptions of their work. By combining observation with interviews (using cognitive probes), important data could be collected about the cues students observed (and those they missed), what they noticed (or missed), what outcomes they expected or goals they were trying to achieve, what they learned, and how they understood what was happening during the experience. This knowledge is crucial if the discipline is to remain committed to making evidence-based pedagogical decisions.

The CDM for Studying Clinical Education Experiences

The authors completed a series of pilot studies, composed of direct observation and follow-up interviews, to assess the utility of the CDM as a research method to collect data on students’ thinking and cognitive work in the clinical setting (the outcomes of these studies will be published elsewhere). As described herein, our adaptation of the CDM is accomplished by using a series of six steps.

Step One

An institutional review board approval and inform participants about the study purpose, risks, and goals should be obtained. When students are studied, it is important to assure them that the purpose of the study is not to evaluate their clinical practice and that no data collected via observation or interview will be shared with their clinical instructor. This step minimizes the pressure for students to perform, do everything right, or provide responses their teachers would expect. Students should also be informed that their participation is completely voluntary.

Step Two

A 3-hour direct observation of each student in the clinical unit should be completed. An investigator observes one student and documents all observable behavior (actions), including interactions with patients and families, preceptors, peers, faculty, staff nurses, and other personnel in the hospital. Observation data are collected manually, with the investigator listing each student action and the interaction observed. For example, one might note the following interactions:

  • Asking the charge RN about pain management for a patient.
  • Describing to the charge RN that a patient’s pain is so bad that the patient’s leg is shaking.
  • Listening to the charge RN’s instructions on how to care for the patient’s wound.

Step Three

A semistructured interview should be conducted and audiorecorded with the student as soon as possible after the observation is completed. Depending on the student’s schedule, the interview may occur immediately after the shift or within a few days of the observation. Each student is interviewed individually by the observing investigator. The interview is organized around one or two specific observed situations that required action, and the student is guided in the recall of the incident through information-gathering probes. First, time-line verification is used to structure the incident into meaningful and progressive segments. The investigator describes the observed situation to the student and asks him or her to verify that he or she remembers the situation and that the description was accurate. Following verification, the student is asked to describe the situation, starting at the beginning, when he or she first noticed that action was required, and to describe what was “going through his or her head” as the situation unfolded. As the student describes his or her experience in the situation, the investigator uses cognitive probes to elicit the student’s perceptions, expectations, cues, information, experience, options, and decision making (Crandall et al., 2006). This approach leads to a contextually rich description of the situation. The investigator should also use “what-if” questions to explore alternative decisions or actions the student could have made.

Step Four

All observation notes and recorded interviews are to be transcribed verbatim by an experienced transcriptionist. The investigator verifies the accuracy of the transcription and inserts pseudonyms as needed to protect the confidentiality of the participant, other people, and sites.

Step Five

The research team reads each transcript to develop an overall understanding of the account and then writes a brief summary of each observation and interview. The sharing of these summaries assists team members to develop a “start list” of codes (Sandelowski, 2000) to facilitate analysis. Using this start list, team members code interviews independently and meet regularly to discuss and compare code assignments. Repeated iterative discussion among team members fosters consensus on common and contrasting code assignments across students’ accounts. As data analysis progresses, new codes are added to the code list if investigators note rich contextual aspects of students’ thinking and cognitive work that were not reflected in the initial start list codes.

Step Six

The research team reviews coded data to identify themes. A theme describes common and recurring experiences and depicts students’ thinking and cognitive work. The team presents themes using excerpts from the data, the interpretive commentary, and the current literature to explicate students’ thinking and cognitive work while engaged in clinical situations.

Advantages of Using the CDM in Nursing Education Research

On the basis of the successful use of the CDM in the authors’ studies, they assert that the CDM is a promising method for educational researchers to collect data examining students’ thinking and decision making in clinical practice. A particular strength of this approach is that it overcomes the reliance on participant self-report (common in the nursing education literature) by focusing on what actually occurred, how students understood the encounter, and the factors that informed their decisions and actions. Because student participants describe their thinking, what they were noticing, and what that meant to them, investigators can better understand their decisions and how the curriculum is (and is not) developing these skills and abilities. Importantly, the CDM provides a rich description of how students think through clinical situations and how they make decisions amid the complexity and competing demands of clinical practice. Because the method focuses on both observed behavior and students’ descriptions of their thinking, investigators can expose the contributions and the oversights or shortcomings of current learning experiences. The CDM can uncover missed opportunities for teaching, reinforcing, and clarifying what the student is thinking and how that thinking leads to appropriate decisions, which may not be discovered through the more routine practices, such as self-report or clinical examinations, that are currently used.

Limitations of Using the CDM in Nursing Education Research

Despite its advantages, the CDM has several limitations. First, data collection is time consuming, with investigators expending at least 4 hours per student. Further, investigators often require practice in data collection methods to become skilled at the observation and interview techniques. Second, as with any research, observations are time limited and cannot include the wide range of situations students encounter while in a clinical experience. Third, although students are assured that their practice is not being evaluated, the potential impact of the perceived power differential between investigators and students cannot be ignored. A related issue, particularly with students, is that participation must be voluntary, and these volunteers are likely to be students who are comfortable about someone watching their performance; therefore, it may not be a representative sample. Finally, and most importantly, students’ ability to describe their thinking to investigators, even with probes, may be limited, and their ability to recall may be affected because the interview frequently occurs some time after the observation.

Conclusion

Faculty members in schools of nursing across the country are struggling with the question of how to maximize learning in the face of an evolving and highly complex health care system. Empirical evidence to guide pedagogical decisions in clinical education is lacking, and new methods for generating data are needed. The CDM is an innovative research approach to generate insight into how students are making care decisions through exploration of their thinking, the cues they pick up on, what they are noticing, and how they understand clinical situations in the midst of clinical practice. It overcomes the overreliance on self-report and acontextual testing, as well as not particularly meaningful outcomes, such as satisfaction and confidence, that are common in the nursing education literature (Valiga & Ironside, 2012).

Although the CDM was developed to elicit expert thinking in a practice domain, it was found to be an effective method to use with nursing students. Utilized in actual clinical practice settings, this method provided the authors access to students’ situation awareness, cues for action, and pattern recognition surrounding decision making. The importance of understanding how these complex cognitive aspects of nursing practice are (and are not) developing in clinical education makes the CDM a promising research method for conducting pedagogical investigations and contributing to the science of nursing education.

References

  • Benner, P.A., Tanner, C.A. & Chesla, C.A. (1996). Expertise in nursing practice: Caring, clinical judgment, and ethics. New York, NY: Springer.
  • Blandford, A. & Wong, B.L.W. (2004). Situation awareness in emergency medical dispatch. International Journal of Human–Computer Studies, 61, 421–452. doi:10.1016/j.ijhcs.2003.12.012 [CrossRef]
  • Broome, M.E., Ironside, P.M. & McNelis, A.M. (2012). Research in nursing education: State of the science. Journal of Nursing Education, 51, 521–524. doi:10.3928/01484834-20120820-10 [CrossRef]
  • Cioffi, J. (2012). Expanding the scope of decision-making research for nursing and midwifery practice. International Journal of Nursing Studies, 49, 481–489. doi:10.1016/j.ijnurstu.2011.10.015 [CrossRef]
  • Crandall, B. & Calderwood, R. (1989). Clinical assessment skills of experienced neonatal intensive care nurses. Fairborn, OH: Klein Associates.
  • Crandall, B. & Getchell-Reiter, K. (1993). Critical decision method: A technique for eliciting concrete assessment indicators from the intuition of NICU nurses. Advances in Nursing Sciences, 16(1), 42–51. doi:10.1097/00012272-199309000-00006 [CrossRef]
  • Crandall, B., Klein, G. & Hoffman, R.R. (2006). Working minds: A practitioner’s guide to cognitive task analysis. Cambridge, MA: MIT Press.
  • Ebright, P.R., Patterson, E.S., Chalko, B.A. & Render, M.L. (2003). Understanding the complexity of registered nurse work in acute care settings. Journal of Nursing Administration, 33, 630–638. doi:10.1097/00005110-200312000-00004 [CrossRef]
  • Ebright, P.R., Urden, L., Patterson, E. & Chalko, B. (2004). Themes surrounding novice nurse near-miss and adverse event situations. Journal of Nursing Administration, 34, 531–538. doi:10.1097/00005110-200411000-00010 [CrossRef]
  • Gazarian, P.K. (2013). Use of the critical decision method in nursing research: An integrative review. Advances in Nursing Science, 36, 106–117. doi:10.1097/ANS.0b013e3182901f8d [CrossRef]
  • Gazarian, P.K., Henneman, E.A. & Chandler, G.E. (2010). Nurse decision making in the prearrest period. Clinical Nursing Research, 19, 21–37. doi:10.1177/1054773809353161 [CrossRef]
  • Institute of Medicine. (2010). The future of nursing: Leading change, advancing health. Washington, DC: National Academies Press.
  • Klein, G. (1998). Sources of power: How people make decisions. Cambridge, MA: MIT Press.
  • Klein, G.A., Calderwood, R. & MacGregor, D. (1989). Critical decision method for eliciting knowledge. IEEE Transactions on Systems, Man and Cybernetics, 19, 462–472. doi:10.1109/21.31053 [CrossRef]
  • Militello, L. & Lim, L. (1995). Patient assessment skills: Assessing early cues of necrotizing enterocolitis. The Journal of Perinatal & Neonatal Nursing, 9(2), 42–52. doi:10.1097/00005237-199509000-00007 [CrossRef]
  • Obradovich, J.H. & Woods, D.D. (1996). Users as designers: How people cope with poor HCI design in computer-based medical devices. Journal of Human Factors and Ergonomics Society, 38, 574–592. doi:10.1518/001872096778827251 [CrossRef]
  • Pryor, J. (2004). What environmental factors irritate people with acquired brain injury?Disability and Rehabilitation, 26, 974–980. doi:10.1080/09638280410001702397 [CrossRef]
  • Pryor, J. (2005). What cues do nurses use to predict aggression in people with acquired brain injury?Journal of Neuroscience Nursing, 37, 117–121. doi:10.1097/01376517-200504000-00010 [CrossRef]
  • Pryor, J. (2006). What do nurses do in response to their predictions of aggression?Journal of Neuroscience Nursing, 38, 177–182. doi:10.1097/01376517-200606000-00007 [CrossRef]
  • Pryor, J. (2007). Characteristics of staff who do, and do not, work well with people with acquired brain injury. Journal of the Australasian Rehabilitation Nurses’ Association, 10, 15–21.
  • Sandelowski, M. (2000). Combining qualitative and quantitative sampling, data collection, and analysis techniques in mixed-method studies. Research in Nursing & Health, 23, 246–255. doi:10.1002/1098-240X(200006)23:3<246::AID-NUR9>3.0.CO;2-H [CrossRef]
  • Valiga, T.M. & Ironside, P.M. (2012). Crafting a national agenda for nursing education research. Journal of Nursing Education, 51, 3, 6. doi:10.3928/01484834-20111213-01 [CrossRef]

10.3928/01484834-20140122-05

Sign up to receive

Journal E-contents