The Institute for Healthcare Improvement's Triple Aim of (a) improving the patient experience of care, including quality and satisfaction, (b) improving the health of populations, and (c) reducing the per capita cost of health care (Berwick, Nolan, & Whittington, 2008; Hinshaw, 2011) ultimately translates into forces that influence nursing education. Safety and quality improvement education has the potential to impact all three aims. The American Association of Colleges of Nursing (AACN, 2008) Essentials of Baccalaureate Education for Professional Practice (Essentials) highlight the importance of safety and quality improvement education. Often, nurse educators add components to courses and curricula that reflect national directives, such as the Triple Aim and the Essentials; however, they may struggle to provide motivators to support this learning. Digital badges (DBs) may represent a viable, pedagogical approach to student learning. The current study describes how DBs may be used as potential motivators to baccalaureate nursing students as they learn content related to safety and quality improvement.
Youth organizations use merit or proficiency badges to showcase their achievements on shirts and sashes, as do many virtual games. In addition to what has been achieved, badges demonstrate to the individual what can be earned, creating pathways to accomplishments (Gibson, Ostashewski, Flintoff, Grant & Knight, 2015). In an educational setting, DBs may achieve these same aims as they employ technology to create a living, secure mechanism for students to display their accomplishments. Digital badges are housed in online portals that may be accessed by future and current employers, peers, and colleges and universities, affording ready access that is not typical of some traditional education documents. Described as an emerging technology, DBs are also called open badges, or microcredentials. We define DBs as virtual, visual, and iconic representations of an achievement, knowledge, skill, and competency achieved by an individual learner that is available online and includes metadata with links to the content, process, and result of the activity (Casilli & Knight, 2012; Gibson et al., 2015). Perdue University, the site for this study, has the infrastructure to support student-earned DBs through software called Passport ( http://www.itap.purdue.edu/studio/passport/), a sophisticated platform that hosts individual accounts. These accounts allow the student to create an online portfolio to showcase DBs.
Despite the potential use for motivation in learning, limited badge-related research has been conducted in higher education; no studies have been conducted in nursing education to the authors' knowledge. Thus, our study will be the first to examine the influence of DBs in nursing education. Empirical evidence that describes the potential relationship between DBs and motivation is also scarce. Abramovich, Schunn, and Higashi (2013) examined the impact badges had on middle school student motivation. They concluded that although educational badges increased interest by students overall, variable factors, such as the type of learner, level of performance, and expectations, influenced motivation. In a study of 200 college students using badges as incentives in an online course, McDaniel, Lindgren, and Friskics (2012) reported that badges seemed to act as motivating factors. Women seemed to favor achievements awarded through badges slightly more than men. Student comfort level with technology had no effect on the overall system (i.e., badge) value.
The content, or the learning objective, related to the DB is important in a learning environment. This is specifically true in professions such as nursing, where competencies are tested prior to entering the professional world. In alignment with the Triple Aim (Berwick et al., 2008) and the AACN Essentials (2008), our study will combine the use of DBs with educational content related to patient safety and quality care.
The ARCS model (Keller, 1987, 2010) guided the study design and instruments used. The components of the ARCS model are: Attention (i.e., gaining the attention of the learner); Relevance (i.e., meeting the needs and goals of the learner); Confidence (i.e., giving the learner control and the belief in his or her success); and Satisfaction (i.e., strengthening achievements with rewards). In higher education, students strive to understand how new knowledge and skills will offer them value currently and in the future (i.e., Attention and Relevance; Skiba, 2005). As microcredentials, DBs function as platforms that expand on past learning activities, describe how learning was achieved, and outline the outcomes of the experiences (i.e., Confidence and Satisfaction; Bixler & Layng, 2013).
Using Keller's ARCS model, the aims of this exploratory study were to (a) describe nursing students' opinions of the DBs; (b) assess the influence of DBs on nursing student motivation toward a lesson on patient safety and quality; and (c) derive patterns of responses and themes related to earning DBs from open-ended questions.
After institutional review board approval for the study was received, approximately 100 students entering their second semester of their sophomore year were informed of the study. These students were enrolled in a large university baccalaureate program in the midwestern United States. As part of the school of nursing's curricular evaluation plan, the students were required to participate in the study but were also informed that they could skip an item if they felt uncomfortable in responding. The first step in the study was to provide text in the course syllabus that introduced safety and quality content and tied this to the DB. The second step that occurred at week 2 of the semester was for the instructional technology staff to present the specific lesson on how to navigate the Passport system. Faculty also informed students about the content they would be reviewing to earn the DB that consisted of four online tutorials provided by the Institute for Healthcare Improvement Open School (2014): (a) Introduction to Patient Safety, (b) Teamwork and Communication, (c) Introduction to the Culture of Safety, and (d) Fundamentals of Improvement. These tutorials required approximately 4 hours to complete (i.e., 1 hour per module). Step three occurred 2 weeks later when the online survey link was sent to the students. The students were instructed to independently finish the Open School safety modules by week 10 of the semester as homework assignments.
The online survey consisted of four sections. The first section consisted of demographic items: age, gender, race, or ethnic background and religious background. The second section was the 12-item Badge Opinion Survey (BOS) developed by Abramovich et al. (2013) and Abramovich, Higashi, Hunkele, Schunn, and Shoop (2011), using a Likert scale from 5 = strongly agree to 1 = strongly disagree. Minor wording changes that reflected content related to the online safety modules were made to this tool. For example, the original item in the BOS was “I think that the badge/achievements are a good addition to the cognitive tutor” (Abramovich et al., 2013). We revised the item to read “I think the badge was a good addition to the online module assignment.”
The third section used Keller's (2010) ARCS conceptual model. An 11-item instrument was generated, including one global item to assess overall motivation and 10 additional items related to attention (i.e., 3 items), relevance (i.e., 3 items), confidence (i.e., 2 items), and satisfaction (i.e., 2 items). These dimensions parallel the Instructional Materials Motivation Survey (IMMS), a 36-item tool that is well validated and widely used to assess students' motivation toward a course or lesson (Keller, 2010). An abbreviated, 12-item version of the IMMS demonstrated a preferred parsimonious version (i.e., the Reduced Instructional Materials Motivation Survey [RIMMS]; Loorbach, Peters, Karreman, & Steehouder, 2014). However, both the IMMS and RIMMS relate to lessons versus the earning of a DB. Therefore, the researchers in the current study decided to generate items specific to DBs that also modeled the theoretical ARCS constructs. These items were vetted for content validity and were independently coded by ARCS dimensions by each investigator for construct validity. Next, the three investigators met and reached consensus on which ARCS dimension was assessed by each item, calling the survey the ARCS-Badge Motivation Survey (BMS) using a Likert scale from 5 = strongly agree to 1 = strongly disagree). Interitem reliability coefficients (i.e., Cronbach's alpha coefficients) were calculated for each of the four ARCS constructs: relevance = .89, attention = .82, confidence = .71, and satisfaction = .87. An overall Cronbach's alpha coefficient was calculated for the ARCS-BMS as .92.
The fourth section of the online survey consisted of open-ended items. Questions were clustered along two themes: Past Exposure and Motivation Related to Badges, and Current Perceptions of Motivation and DBs.
Data Analysis and Results
Descriptive statistics were calculated to summarize the characteristics of the sample. Means and standard deviations were derived for the BOS (Abramovich et al., 2013) and the ARCS-BMS. For the open-ended responses, the three primary investigators used a qualitative content analysis and coded the data using a deductive, unrestrained matrix approach (Elo & Kyngäs, 2007). Themes, as defined by DeSantis and Ugarriza (2000), were the unit of analysis. The three investigators separately analyzed the raw data and generated overall themes. Reactions to the student texts were also noted. Subsequently, the research team discussed discrepancies in the thematic coding and ultimately reached consensus.
Participant Characteristics and Quantitative Findings
The study sample consisted of students who were between 19 and 22 years of age (n = 73 [81.1%] were 19 or 20 years), primarily women (n = 83 [92.2%]), and Caucasian (n = 78 [86.7%]). The majority were admitted into the nursing program from high school (n = 79 [87.8%]), and belong to the Roman Catholic, Protestant, or other Christian religions (n = 68 [75.6%]). Ninety-one students completed the BOS and ARCS-BMS surveys. Findings related to the BOS included the two highest rated items: most students understood why they earned the badge (M = 4.33; SD = 0.84), and felt the badge was a good addition to the safety module (M = 4.01; SD = 0.87). However, students tended to be neutral or disagree that they knew what a DB was before starting the online safety modules (M = 2.73; SD = 1.26) and that the patient safety and quality care badge made them want to keep working at earning badges (M = 3.49; SD = 0.94).
The results of the ARCS-BMS revealed that motivation to earn the IHI safety modules was positively influenced by earning the DB (Global Item: M = 3.71, Table). The mean scores for items related to the constructs of attention, relevance, confidence and satisfaction ranged from a low score of 3.43 (i.e., question related to Confidence) to a high score of 3.74 (i.e., question related to Attention and Confidence).
ARCS-Badge Motivation Survey
Previous Experience and Salience of Past Badges. Of the 87 student responses, previous activities in scouting was the most frequently cited exposure to badges (n = 37). Thirty students reported no previous experiences with badges. Forty-five students reported both internal and external motivators related to past badge work: “You work hard and you receive something for it,” (i.e., external motivation) versus “Earning a badge in Brownies made me much more proud of what I accomplished and wanted to show them off” (i.e., internal and external motivation).
Current Impact on Motivation. Fifty-five students of the 86 students reported an increase in motivation. Several students' answers reflected knowledge and mastery of safety content that had been gained from earning the badge. They reported that this knowledge could then be applied in their practice. Other student responses indicated confusion over the badge's purpose and implementation.
Theme: Increased Motivation Related to Earning DB. Students felt badges provided evidence to future employers, presented an innovative aspect to the technology, and visible proof of their achievements. The ARCS conceptual underpinnings were also evidenced in the responses. Several subthemes were identified.
Subtheme: Innovative use of technology. “Hearing about how cutting edge this idea of digital badges is made me want to log on and experience it for myself. Earning the badge made me feel a little ahead of the game, and knowing that, I'm even more motivated to complete badges.”
Subtheme: Visual or tangible evidence. “Yes, it was a visual acknowledgement of your completion of the module,” and “It did increase because there was something tangible to earn.”
Subtheme: Benefits for the future. “Receiving the digital badge increased my motivation to complete the mandatory badge assignment because I knew that I would be able to refer future employers to view this badge.”
Subtheme: ARCS. “Increased importance of assignment,” “Yes, knowing that I was going to earn a badge motivated me to pay more attention to the modules,” “I felt that receiving the digital badge motivated me to do the assignment because I wanted the badge to show that I had accomplished more nursing skills,” and “I felt like I was earning something important and official, and needed to take the assignment seriously.”
Theme: Somewhat Motivating. Other students felt less definitive about being motivated by earning a DB: “It somewhat increased my motivation. The material was important regardless, however the badge gave me a reason not to give up if I got questions wrong.”
Theme: Did Not Affect Motivation. Some individuals completed the module because it was assigned. Another student emphasized internal motivation as a factor: “Receiving a digital badge did not affect my motivation, wanting to gain knowledge was my motivation.”
This study explored the impact of earning DBs on motivation toward learning safety and quality of care content. Our quantitative findings include that most students understood why they earned the badge and felt the badge was a good addition to the safety modules. Overall, students' motivation to earn the safety modules was positively influenced by earning the DBs. Although the majority of students reported being motivated by earning DBs, qualitative data provided an inconclusive description of the extent to which badges motivated nursing students. Content analysis resulted in somewhat polarized perceptions of the badges' motivational worth, as well as a need to educate students about DBs and the contexts in which they are used. Learning opportunities of these contexts may help students in appreciating the current and future use of DBs. The ARCS model (Keller, 1987, 2010) appeared to be a useful framework as motivation and DBs were explored. The ARCS-BMS revealed good internal consistency parameters, but it should be tested in a more heterogeneous sample.
In addition to current learning, DBs appear to have implications for professional development, and many students reported the influence of securing jobs as a motivator to earning a badge. In a discussion related to teachers' educational development, DBs were found to offer customization and personalization of learning (Gamrat, Zimmerman, Dudek, & Peck, 2014). Badges may be considered as tools to motivate and to stimulate the acquisition of knowledge and skills within the learning community and environment (O'Byrne, Schenke, Willis, & Hickey, 2015). This principle was reflected in students who responded that the safety and quality of care knowledge was the motivating factor and in others who reported that further motivation would come from earning additional DBs based on their own personalized learning needs.
The limited sample size and homogeneous nature of the sample limit generalizability. Explaining DBs to the students in a relatively short amount of time may also have limited the motivational impact of badges. However, data situated within a conceptual framework and the triangulation of data strengthen the study findings.
Although most nursing students reported that DBs motivated them in the learning activity, their perceptions of DBs as a motivator was contingent on factors that require further exploration. These factors include previous exposure to badges, internal versus external motivation, an understanding of the uses and context of DBs, and, finally, an appreciation of future uses of DBs in their professional lives. Overall, this exploratory study provides preliminary evidence that DBs may be useful to nurse educators as a tool to enhance student motivation and as a mechanism to verify student achievement.
- Abramovich, S., Higashi, R., Hunkele, T., Schunn, C. & Shoop, R. ( 2011, July. ). An achievement system to increase achievement motivation. Paper presented at the Games Learning Society 7.0, Madison, WI. .
- Abramovich, S., Schunn, C. & Higashi, R. (2013). Are badges useful in education?: It depends upon the type of badge and expertise of learner. Educational Technology Research and Development, 61, 217–232. doi:10.1007/s11423-013-9289-2 [CrossRef]
- American Association of Colleges of Nursing. (2008). The essentials of baccalaureate education for professional nursing practice. Washington, DC: Author. Retrieved from http://www.aacn.nche.edu/education-resources/BaccEssentials08.pdf
- Berwick, D.M., Nolan, T.W. & Whittington, J. (2008). The triple aim: Care, health and cost. Health Affairs, 27, 759–769. doi:10.1377/hlthaff.27.3.759 [CrossRef]
- Bixler, B. & Layng, K. (2013). Digital badges in higher education: An overview. Retrieved from http://newcomb.tulane.edu/blogs/uccbadges/case-studies/digital-badges-in-higher-education/
- Casilli, C. & Knight, E. (2012). 7 things you should know about badges. EDUCAUSE Learning Initiative. Retrieved from http://net.educause.edu/ir/library/pdf/ELI7085.pdf
- DeSantis, L. & Ugarriza, D.N. (2000). The concept of theme as used in qualitative nursing research. Western Journal of Nursing Research, 22, 351–372. doi:10.1177/019394590002200308 [CrossRef]
- Elo, S. & Kyngäs, H., (2007). The qualitative content analysis process. Journal of Advanced Nursing, 62, 107–115. doi:10.1111/j.1365-2648.2007.04569.x [CrossRef]
- Gamrat, C., Zimmerman, H.T., Dudek, J. & Peck, K. (2014). Personalized workplace learning: An exploratory study on digital badging within a teacher professional development program. British Journal of Educational Technology, 45, 1136–1148. doi:10.1111/bjet.12200 [CrossRef]
- Gibson, D., Ostashewski, N., Flintoff, K., Grant, S. & Knight, E. (2015). Digital badges in education. Education and Information Technologies, 20, 403–410. doi:10.1007/s10639-013-9291-7 [CrossRef]
- Hinshaw, P.M. (2011). Understanding the triple aim. Nursing Management, 42, 18–19. doi:10.1097/01.NUMA.0000393008.26504.b4 [CrossRef]
- Institute for Healthcare Improvement. (2014). IHI open school online courses. Retrieved from http://www.ihi.org/education/webtraining/Pages/default.aspx
- Keller, J.M. (1987). Development and use of the ARCS model of motivational design. Journal of Instructional Development, 10, 2–10. doi:10.1007/BF02905780 [CrossRef]
- Keller, J.M. (2010). Motivational design for learning and performance. New York, NY: Springer. doi:10.1007/978-1-4419-1250-3 [CrossRef]
- Loorbach, N., Peters, O., Karreman, J. & Steehouder, M. (2014). Validation of the Instructional Materials Motivation Survey (IMMS) in a self-directed instructional setting aimed at working with technology. British Journal of Educational Technology, 46, 204–218. doi:10.1111/bjet.12138 [CrossRef].
- McDaniel, R., Lindgren, R. & Friskics, J. ( 2012, October. ). Using badges for shaping interactions in online learning environments. Presented at the IEEE International Professional Communication Conference. . 1–4. doi:10.1109/IPCC.2012.6408619 [CrossRef]
- O'Byrne, W.I., Schenke, K., Willis, J.E. III. & Hickey, D.T. (2015). Digital badges: Recognizing, assessing, and motivating learners in and out of school contexts. Journal of Adolescent & Adult Literacy, 58, 451–454. doi:10.1002/jaal.381 [CrossRef]
- Skiba, D.J. (2005). The millennials: Have they arrived at your school of nursing?Nursing Education Perspectives, 25, 370–371.
ARCS-Badge Motivation Survey
| I felt more motivated to complete the IHI modules because I was also earning a digital badge.||3.71||1.0|
| The badge helped to gain my attention related to the lesson on patient safety and quality care.||3.74||0.89|
| I was more aware of this assignment because I would also be earning a badge.||3.71||0.97|
| Because I earned a badge, I knew the assignment was important.||3.62||0.91|
| Earning the badge made a difference in how I viewed completing the assignment.||3.68||0.95|
| Earning the badge made the assignment more significant to me.||3.73||1.0|
| The badge increased how relevant the assignment was.||3.58||0.95|
| The badge increased my confidence in being able to demonstrate my knowledge of the content.||3.43||0.95|
| The badge was a symbol that I had mastered this content.||3.74||0.93|
| The badge increased my overall level of satisfaction with completing the assignment.||3.68||0.89|
| By earning the badge, I was more fulfilled as a student by the assignment.||3.63||0.95|