Techniques used in university teaching have evolved in the past few years due to the incorporation of new information and communication technology (Brady, 2013). Likewise, nurse educators have been looking for new and efficient teaching tools that would allow students to achieve higher quality learning and obtain better results (Tanabe & Kobayashi, 2013). In contrast to traditional teaching methods, these new educational strategies aim to increase knowledge while actively involving students in their learning process (Chi & Wylie, 2014).
University students, many of whom belong to the Millennial generation, live immersed in a predominantly digital world. They often spend their free time playing games based in technology, which allows them to compete with each other and interact socially with others (White & Shellenbarger, 2018). On the other hand, it is possible that they have become discouraged by traditional teaching methods, such as those based on lecture classes with slide projections (Levine & Dean, 2012). Due to this fact, the implementation of games in an academic setting may be perceived as an attractive learning method that improves on traditional education, making classes more interactive and innovative (Brigham, 2015; Brull & Finlayson, 2016; Solís de Ovando, Rodríguez, & Hullin, 2018). Furthermore, using gamification may be a way to reduce costs, increase the quality of teaching, and increase methodological efficiency (Kapp, 2012).
Gamification is defined as the use of mechanisms that attract, stimulate to action, and foster critical thinking in users, with the goal of promoting learning and problem solving (Kapp, 2012). This new concept, which has been in the spotlight only during the past few years, uses game elements in settings that are not actually games, which generates new experiences and allows students to actively participate in their own learning, thereby generating feelings of dominance and increased autonomy (Huotari & Hamari, 2017). These feelings are important because, as the theoretical framework known as PAD (Pleasure, Arousal, and Dominance) points out, different emotional dimensions are represented in the applied context of gamification (Hammedi, Leclerq, & Van Riel, 2017; Harwood & Garry, 2015). More precisely, domination is defined as having feelings of control and influence over others and over your environment. These emotions make up a fundamental part of the emotional experiences players may undergo when playing a game (Lim & Lee, 2009; Ravaja, Saari, Salminen, Laarni, & Kallinen, 2009).
Several studies have investigated the influence of gaming on nursing students' learning (Gallegos, Tesar, Connor, & Martz, 2017; Gómez-Urquiza et al., 2019). In these studies, various types of games have been observed, such as serious games (Gómez-Urquiza et al., 2019; Johnsen, Fossum, Vivekananda-Schmidt, Fruhling, & Slettebø, 2018) and card games (Milner & Cosme, 2017), among others. These types of games allow us to practice various essential aspects of nursing students' learning, such as communication skills, teamwork, and critical thinking (Friedrich, Teaford, Taubenheim, Boland, & Sick, 2018; Nicholson, 2018). In addition, learning through the use of games leads to an increase in attention and motivation (Dale, 2014). It also helps with knowledge retention (Bruder, 2015), academic performance (Roche et al., 2018), and decision-making skills (Huotari & Hamari, 2017).
This new learning technique allows students to improve their clinical skills in a safer and more realistic setting, as well as encourages students to work on their clinical decision-making skills within the context of their nursing education (Cant & Cooper, 2014; Johnsen, Fossum, Vivekananda-Schmidt, Fruhling, & Slettebø, 2016). As we have previously pointed out, emotions play an important role in gamification, as they influence cognition and decision making (Pessoa, 2013). In the past few decades, there has been an increased interest in exploring emotions, with the potential of creating a paradigm shift in decision theory. Research reveals that emotions are powerful, widespread, predictable, sometimes harmful, and sometimes beneficial precursors to decision making (Lerner, Li, Valdesolo, & Kassam, 2015). Due to the aforementioned data, the objective of this study was to determine the influence of gamification on decision making in nursing students.
A quasi-experimental study was performed with an experimental group (EG) and a control group (CG).
A convenience sample was used to select the participants. A total of 191 students participated. Inclusion criteria included the following: being enrolled in the subject of Basic and Advanced Life Support in the nursing major, age over 18 years, and attending every session related to the study. Exclusion criteria included being a foreign exchange student, as most did not have sufficient command of the language used in the original questionnaire. Each participant was randomly assigned to one of the groups.
The following questionnaires were used for data collection: The Melbourne Decision Making Questionnaire is used to evaluate decision making (Mann, Burnett, Radford, & Ford, 1998). The questionnaire is made up of 22 questions divided into four dimensions;
- Vigilance (six questions; maximum score 12).
- Hypervigilance (five questions; maximum score 10).
- Buck passing (six questions; maximum score 12), which involves leaving decisions to others and avoiding responsibility.
- Procrastination (five questions; maximum score 10).
The questions are answered using a Likert-type scale from 0 (never) to 2 (always). The higher the score on the questionnaire, the greater the confidence level in decision making. The reliability, which verifies the consistency of the scale, was good, with a Cronbach's alpha coefficient of .87.
To evaluate the experiences of students in the game, the Gameful Experience Scale (GAMEX) was used (Eppmann, Bekk, & Klein, 2018), which has also been validated for its use with nursing students by Márquez-Hernández et al. (2019). It is made up of 27 questions rated on a Likert-type scale, from 1 (never) to 5 (always). The questions are divided into six dimensions: enjoyment, absorption, creative thinking, activation, absence of negative effects, and dominance. In this case, the higher the score, the greater the game experience. Cronbach's alpha was .839 for this sample, which shows good reliability of the scale. Additionally, sociodemographic characteristics such as gender and age were also collected.
Once permission was granted by the institutional review board, the students were asked to attend a meeting in a classroom in the department of health sciences. The chief researcher informed the students of the objective of the study, the voluntary nature of their participation and the anonymous treatment of their data, as well as the possibility of withdrawing from the study at any time. The students who were interested in participating signed the informed consent form prior to starting. The study was conducted in accordance to the ethical considerations of the Declaration of Helsinki.
Next, the students were divided into two groups: EG and CG. In the EG session, gamification was used through an app to learn Basic and Advanced Life Support. The subjects participated in pairs, trying to guess terms related to Basic and Advanced Life Support in under 30 seconds for each turn, without their partner saying the word they were trying to define. The winner was the student who was able to guess more terms related to the topic (Figure 1).
Screenshot of gamification application.
On completing this segment, the students filled out the questionnaires to evaluate the influence of their gamified experience on learning and decision making.
In the CG, the students received a traditional review lesson on the content, soon followed by filling out the form. To guarantee confidentiality, the students placed the completed questionnaires in a box located in one corner of the classroom. Data collection took place between December 2018 and February 2019.
For the data analysis, the statistical software SPSS® version 25 was used. First, a descriptive analysis of the results was performed. For the qualitative variables, the frequency and percentages were calculated, whereas for the quantitative variables, the mean and typical deviation were calculated. To check the hypothesis between qualitative and quantitative variables, the nonparametric Mann Whitney U test was used; on the other hand, Spearman's correlation test was used to contrast quantitative variables. A value of p < .05 was considered significant.
Sociodemographic Characteristics of the Participants
A total of 191 students took part in the study, of which, 77% (n = 147) were female and 23% (n = 44) were male. Regarding age, the average age of the participants was 20.93 ± 53.11. The participants were divided randomly into two groups: a CG (50.8%, n = 97) and EG (49.2%, n = 94).
Gameful Experience Scale
The EG filled out the GAMEX questionnaire immediately following the intervention. All items or questions were scored higher than average (2.50), except those related to negative effects.
Regarding gender, statistically significant differences were found in items 24, 25, and 27. More specifically, in item 24, male students reported having a greater feeling of control (3.39 ± 1.23) than female participants (2.76 ± 1.21; U = 595.500; Z = −2.001; p = .045). On the other hand, in item 25 it was observed that male participants felt more influential (3.70 ± 1.10) than female participants (3 ± 1.30; U = 564.500; Z = −2.27; p = .023). Finally, in item 27, it was also shown that male participants felt more secure (4.30 ± 0.82) than female participants (3.59 ± 1.09; U = 505,000; Z = −2.856; p = .004).
The Melbourne Decision Making Questionnaire
Upon comparing the CG and the EG, statistically significant differences were found in the following behavior patterns: vigilance, buck passing, and procrastination. In the vigilance dimension, the EG had an average score of 9.76 ± 1.71, higher than that of the CG which was 8.20 ± 2.10 (U = 2651.000; Z = −5.058; p = .000). On the other hand, in the case of buck passing, the EG reached a lower average score (4.02 ± 1.81) compared with the CG, whose average score was 5.07 ± 1.82 (U = 3201.500; Z = −3.597; p = .000). Likewise, in procrastination, the EG ended up obtaining a lower average score (4.32 ± 1.82) than the CG (5.53 ± 1.95; U = 2982.500; Z = −4.182; p = .000).
The objective of this study was to determine the influence of gamification on decision making in nursing students. In their gameful experience, the students reported little to no negative effects. This data coincide with what has been reported in other research, stressing the importance of the absence of negative qualities on the emergence of the true gameful experience (Eppman et al., 2018; Mullins & Sabherwal, 2018). In addition, the nursing students showed high scores in enjoyment, absorption, creative thinking, activation, and dominance. In the same vein, several studies have found that nursing students enjoy themselves, develop creative thinking skills, better retain knowledge, and improve their clinical skills with the use of gamification (Brull, Finlayson, Kostelec, MacDonald, & Krenzischeck, 2017; Roche et al., 2018; Wingo et al., 2019).
Nevertheless, differences were found by gender regarding certain gameful experience elements. More specifically, males reported having a greater sense of control, and feeling more secure and influential than the females. However, the original authors of the scale did not find any statistically significant differences between gender and these specific elements (Eppmann et al., 2018). Therefore, additional studies that explore the influence of gender on these aspects are necessary. Gamification has been shown to stimulate interactivity, increase motivation, minimize evasion, and make the learning process more dynamic for nursing students (Brull et al., 2017; Castro & Gonçalves, 2018).
Concerning gamification and decision making, statistically significant differences were found. The EG obtained a higher average score in vigilance than the CG. The use of gamification in learning has been positively evaluated by students, considering that learning teamwork and time management has a positive effect on decision making (Brown, Darby, & Coronel, 2019; Johnsen et al., 2016). Along the same lines, several studies report that gamification increases motivation and readiness in students (Brull et al., 2017; Gallegos et al., 2017). These aspects have been related to vigilance in decision making (Yi & Park, 2003). Additionally, students who showed higher levels of vigilance have also shown greater self-confidence, which has a positive effect on decision making and problem solving (Luna Bernal & Laca Arocena, 2014; Senol, Can, & Pektas, 2012).
Nevertheless, the students who had a higher score in vigilance (EG) reported lower scores in buck passing. These data concur with other studies (Kornilova, Chumakova, & Krasavtseva, 2018; Luna Bernal & Laca Arocena, 2014). Also, students who make decisions in an optimistic way and with greater initiative tend to earn better academic results (Kornilova et al., 2018; Senol et al., 2012). In contrast, students who tend to avoid responsibility and delegate it to others (buck passing) receive worse academic results and have greater difficulty in problem solving (Senol et al., 2012).
Regarding procrastination, the EG received lower scores than the CG. This may be due to the short time frame the EG had to select their responses, while the CG could delay selecting an answer right away, avoiding making the decision. This is consistent with data found in other studies that indicate that students tend to wait until the last minute to perform certain academic activities (Contreras-Pulache et al., 2011; Rosário et al., 2009). Recently, a study conducted on nursing students found a link between academic procrastination and an increase in anxiety (Custer, 2018).
Gamification can serve as a complementary learning tool to traditional training in both academic and clinical settings (Johnsen et al., 2016), as it has been shown to be a more effective teaching strategy than traditional methods (Brull et al., 2017). Gamification can promote the use of critical thinking, improve time management skills, foster teamwork, improve delegation abilities, and provide training for clinical skills, all of which are essential aspects in any clinical practice (Bauman & Wolfenstein, 2013). Furthermore, using this method makes students feel that they play an active role in their learning process, which in turn increases their motivation. It also allows the professors to better understand how the students think (Schaffhauser, 2017).
The results of this study should be regarded while taking into account a series of limitations. First of all, the sample was selected by convenience and from only one institution, which does not allow for a generalization of the results. Second, it should be kept in mind that all results were self-reported. Also, it is the first time that a study has been conducted to determine the influence of gamification on decision making, which limits the possibility of a thorough discussion and the ability to contrast this study with others. Finally, additional research is needed, specifically on the influence of gamification on decision making in a clinical setting, an essential aspect of any future nursing professional's clinical practice.
Gamification in nursing increases satisfaction, creative thinking, and control, with little to no negative effects during the gameful experience, which has a positive influence on decision making. Aspects such as vigilance, buck passing, and procrastination have been shown to be influenced by gamification. Therefore, gamification constitutes an important element in the promotion and practice of decision making in the education of future nursing professionals.
- Bauman, E. & Wolfenstein, M. (2013). Game-based teaching and simulation in nursing and health care. New York, NY: Springer.
- Brady, C.L. (2013). Understanding learning styles: Providing the optimal learning experience. International Journal of Childbirth Education, 28(2), 16–19.
- Brigham, T.J. (2015). An introduction to gamification: Adding game elements for engagement. Medical Reference Services Quarterly, 34(4), 471–480 https://doi.org/10.1080/02763869.2015.1082385 doi:10.1080/02763869.2015.1082385 [CrossRef]26496401
- Brown, N., Darby, W. & Coronel, H. (2019). An escape room as a simulation teaching strategy. Clinical Simulation in Nursing, 30(5), 1–6 https://doi.org/10.1016/j.ecns.2019.02.002 doi:10.1016/j.ecns.2019.02.002 [CrossRef]
- Bruder, P. (2015). Game on: Gamification in the classroom. Education Digest, 80(7), 56–60.
- Brull, S. & Finlayson, S. (2016). Importance of gamification in increasing learning. The Journal of Continuing Education in Nursing, 47(8), 372–375 https://doi.org/10.3928/00220124-20160715-09 doi:10.3928/00220124-20160715-09 [CrossRef]27467313
- Brull, S., Finlayson, S., Kostelec, T., MacDonald, R. & Krenzischeck, D. (2017). Using gamification to improve productivity and increase knowledge retention during orientation. Journal of Nursing Administration, 47(9), 448–453 https://doi.org/10.1097/NNA.0000000000000512 doi:10.1097/NNA.0000000000000512 [CrossRef]28834805
- Cant, R.P. & Cooper, S.J. (2014). Simulation in the Internet age: The place of Web-based simulation in nursing education. An integrative review. Nurse Education Today, 34(12), 1435–1442 https://doi.org/10.1016/j.nedt.2014.08.001 doi:10.1016/j.nedt.2014.08.001 [CrossRef]25156144
- Castro, T.C. & Gonçalves, L.S. (2018). The use of gamification to teach in the nursing field. Revista Brasileira de Enfermagem, 71(3), 1038–1045 https://doi.org/10.1590/0034-7167-2017-0023 doi:10.1590/0034-7167-2017-0023 [CrossRef]29924165
- Chi, M.T. & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243 https://doi.org/10.1080/00461520.2014.965823 doi:10.1080/00461520.2014.965823 [CrossRef]
- Contreras-Pulache, H., Quispe, E.M., Figueroa, N.L., Guevara, E.G., Camposano, W.H., Bolívar, D.R. & Cross, C.C. (2011). Procrastinación en el estudio: Exploración del fenómeno en adolescentes escolarizados. Revista Peruana de Epidemiología, 15(3), 6–5.
- Custer, N. (2018). Test anxiety and academic procrastination among preli-censure nursing students. Nursing Education Perspectives, 39(3), 162–163 https://doi.org/10.1097/01.NEP.0000000000000291 doi:10.1097/01.NEP.0000000000000291 [CrossRef]29505500
- Dale, S. (2014). Gamification: Making work fun, or making fun of work?Business Information Review, 31(2), 82–90 https://doi.org/10.1177/0266382114538350 doi:10.1177/0266382114538350 [CrossRef]
- Eppmann, R., Bekk, M. & Klein, K. (2018). Gameful experience in gamification: Construction and validation of a gameful experience scale [GAMEX]. Journal of Interactive Marketing, 43, 98–115 https://doi.org/10.1016/j.intmar.2018.03.002 doi:10.1016/j.intmar.2018.03.002 [CrossRef]
- Friedrich, C., Teaford, H., Taubenheim, A., Boland, P. & Sick, B. (2018). Escaping the professional silo: An escape room implemented in an interprofessional education curriculum. Journal of Interprofessional Care, 33, 573–575 https://doi.org/10.1080/13561820.2018.1538941 doi:10.1080/13561820.2018.1538941 [CrossRef]30362849
- Gallegos, C., Tesar, A.J., Connor, K. & Martz, K. (2017). The use of a game-based learning platform to engage nursing students: A descriptive, qualitative study. Nurse Education in Practice, 27, 101–106 https://doi.org/10.1016/j.nepr.2017.08.019 doi:10.1016/j.nepr.2017.08.019 [CrossRef]28881322
- Gómez-Urquiza, J.L., Gómez-Salgado, J., Albendín-García, L., Correa-Rodríguez, M., González-Jiménez, E. & Cañadas-De la Fuente, G.A. (2019). The impact on nursing students' opinions and motivation of using a “Nursing Escape Room” as a teaching game: A descriptive study. Nurse Education Today, 72, 73–76 https://doi.org/10.1016/j.nedt.2018.10.018 doi:10.1016/j.nedt.2018.10.018 [CrossRef]
- Hammedi, W., Leclerq, T. & Van Riel, A.C. (2017). The use of gamification mechanics to increase employee and user engagement in participative healthcare services: A study of two cases. Journal of Service Management, 28(4), 640–661 https://doi.org/10.1108/JOSM-04-2016-0116 doi:10.1108/JOSM-04-2016-0116 [CrossRef]
- Harwood, T. & Garry, T. (2015). An investigation into gamification as a customer engagement experience environment. Journal of Services Marketing, 29(7), 533–546 https://doi.org/10.1108/JSM-01-2015-0045 doi:10.1108/JSM-01-2015-0045 [CrossRef]
- Huotari, K. & Hamari, J. (2017). A definition for gamification: Anchoring gamification in the service marketing literature. Electronic Markets, 27, 21–31 https://doi.org/10.1007/s12525-015-0212-z doi:10.1007/s12525-015-0212-z [CrossRef]
- Johnsen, H., Fossum, M., Vivekananda-Schmidt, P., Fruhling, A. & Slettebø, A. (2018). Developing a serious game for nurse education. Journal of Gerontological Nursing, 44(1), 15–19 https://doi.org/10.3928/00989134-20171213-05 doi:10.3928/00989134-20171213-05 [CrossRef]29355884
- Johnsen, H.M., Fossum, M., Vivekananda-Schmidt, P., Fruhling, A. & Slettebø, Å. (2016). A serious game for teaching nursing students clinical reasoning and decision-making skills. Studies in Health Technology and Informatics, 225, 905–906 https://doi.org/10.3233/978-1-61499-658-3-90527332402
- Kapp, K.M. (2012). The gamification of learning and instruction: Game-based methods and strategies for training and education. San Francisco, CA: Wiley.
- Kornilova, T.V., Chumakova, M.A. & Krasavtseva, Y.V. (2018). Emotional intelligence, patterns for coping with decisional conflict, and academic achievement in cross-cultural perspective (evidence from selective Russian and Azerbaijani student populations). Psychology in Russia, 11(2), 114–133 https://doi.org/10.11621/pir.2018.0209
- Levine, A. & Dean, D.R. (2012). Generation on a tightrope: A portrait of today's college student. San Francisco, CA: Jossey-Bass.
- Lerner, J.S., Li, Y., Valdesolo, P. & Kassam, K.S. (2015). Emotion and decision making. Annual Review of Psychology, 66(1), 799–823 https://doi.org/10.1146/annurev-psych-010213-115043 doi:10.1146/annurev-psych-010213-115043 [CrossRef]
- Lim, S. & Lee, J.E.R. (2009). When playing together feels different: Effects of task types and social contexts on physiological arousal in multiplayer online gaming contexts. CyberPsychology and Behavior, 12(1), 59–61 https://doi.org/10.1089/cpb.2008.0054 doi:10.1089/cpb.2008.0054 [CrossRef]
- Luna Bernal, A.C.A. & Laca Arocena, F.A.V. (2014). Patrones de toma de decisiones y autoconfianza en adolescentes bachilleres. Revista de Psicología, 32(1), 39–66.
- Mann, L., Burnett, P., Radford, M. & Ford, S. (1998). The Melbourne decision making questionnaire: An instrument for measuring patterns for coping with decisional conflict. Journal of Behavioral Decision Making, 10(1), 1–19 https://doi.org/10.1002/(SICI)1099-0771(199703)10:1<1::AID-BDM242>3.0.CO;2-X doi:10.1002/(SICI)1099-0771(199703)10:1<1::AID-BDM242>3.0.CO;2-X [CrossRef]
- Márquez-Hernández, V.V., Garrido-Molina, J.M., Gutiérrez-Puertas, L., García-Viola, A., Aguilera-Manrique, G. & Granados-Gámez, G. (2019). How to measure gamification experiences in nursing? Adaptation and validation of the Gameful Experience Scale [GAMEX]. Nurse Education Today, 81, 34–38. doi:10.1016/j.nedt.2019.07.005 [CrossRef]
- Milner, K.A. & Cosme, S. (2017). The PICO game: An innovative strategy for teaching step 1 in evidence-based practice: The PICO game. Worldviews on Evidence-Based Nursing, 14(6), 514–516 https://doi.org/10.1111/wvn.12255 doi:10.1111/wvn.12255 [CrossRef]28799703
- Mullins, J.K. & Sabherwal, R. (2018). Gamification: A cognitive-emotional view. Honolulu: University of Hawaii at Manoa.
- Nicholson, S. (2018). Creating engaging escape rooms for the classroom. Childhood Education, 94(1), 44–49 https://doi.org/10.1080/00094056.2018.1420363 doi:10.1080/00094056.2018.1420363 [CrossRef]
- Pessoa, L. (2013). The cognitive-emotional brain: From interactions to integration. Cambridge, MA: MITPress. doi:10.7551/mitpress/9780262019569.001.0001 [CrossRef]
- Ravaja, N., Saari, T., Salminen, M., Laarni, J. & Kallinen, K. (2009). Phasic emotional reactions to video game events: A psychophysiological investigation. Media Psychology, 8(4), 343–367 https://doi.org/10.1207/s1532785xmep0804_2 doi:10.1207/s1532785xmep0804_2 [CrossRef]
- Roche, C.C., Wingo, N.P., Westfall, A.O., Azuero, A., Dempsey, D.M. & Willig, J.H. (2018). Educational analytics: A new frontier for gamification?Computers, Informatics, Nursing, 36(9), 458–465 https://doi.org/10.1097/CIN.0000000000000455 doi:10.1097/CIN.0000000000000455 [CrossRef]29985195
- Rosário, P., Costa, M., Núñez, J.C., González-Pienda, J., Solano, P. & Valle, A. (2009). Academic procrastination: Associations with personal, school, and family variables. Spanish Journal of Psychology, 12(1), 118–127 https://doi.org/10.1017/S1138741600001530 doi:10.1017/S1138741600001530 [CrossRef]19476225
- Schaffhauser, D. (2017). Breakout! Gaming to learn. Technical Horizons in Education Journal, 44(4), 6–11.
- Senol, S., Can, H.O. & Pektas, I. (2012). Decision making and problem-solving skills of midwifery students in Turkey. HealthMed, 6(2), 560–567.
- Solís de Ovando, A., Rodríguez, A. & Hullin, C. (2018). Digital game-based learning: A didactic experience in the pre-degree nursing career. Studies in Health Technology and Informatics, 250, 88–89.
- Tanabe, L.P. & Kobayashi, R.M. (2013). Profile, competencies and digital fluency of nurses in the professional improvement program. Revista da Escola de Enfermagem da USP, 47(4), 943–949 https://doi.org/10.1590/S0080-623420130000400024 doi:10.1590/S0080-623420130000400024 [CrossRef]
- White, M. & Shellenbarger, T. (2018). Gamification of nursing education with digital badges. Nurse Educator, 43(2), 78–82 https://doi.org/10.1097/NNE.0000000000000434 doi:10.1097/NNE.0000000000000434 [CrossRef]
- Wingo, N.P., Roche, C.C., Baker, N., Dunn, D., Jennings, M., Pair, L. & Willig, J.H. (2019). “Playing for bragging rights”: A qualitative study of students' perceptions of gamification. Journal of Nursing Education, 58(2), 79–85 https://doi.org/10.3928/01484834-20190122-04 doi:10.3928/01484834-20190122-04 [CrossRef]30721307
- Yi, J.S. & Park, S. (2003). Cross-cultural differences in decision-making styles: A study of college students in five countries. Social Behavior and Personality, 31(1), 35–47 https://doi.org/10.2224/sbp.2003.31.1.35 doi:10.2224/sbp.2003.31.1.35 [CrossRef]