Journal of Nursing Education

Major Article 

Revision and Psychometric Testing of the Incivility in Nursing Education (INE) Survey: Introducing the INE-R

Cynthia M. Clark, PhD, RN, ANEF, FAAN; Celestina Barbosa-Leiker, PhD; Larecia Money Gill, MSN/ED, RN; Danh Nguyen, BSN, RN

Abstract

Background:

Academic incivility is a serious challenge for nursing education, which needs to be empirically measured and fully addressed.

Method:

A convenience sample of nursing faculty and students from 20 schools of nursing in the United States participated in a mixed-methods study to test the psychometric properties of the Incivility in Nursing Education-Revised (INE-R) Survey.

Results:

A factor analysis and other reliability analyses support the use of the INE-R as a valid and reliable measurement of student and faculty perceptions of incivility in nursing education.

Conclusion:

The INE-R is a psychometrically sound instrument to measure faculty and student perceptions of incivility; to examine differences regarding levels of nursing education, program type, gender, age, and ethnicity; to compare perceptions of incivility between and among adjunct, clinical, teaching, and research faculty; and to conduct pre- and postassessments of the perceived levels of faculty and student incivility in nursing programs to inform evidence-based interventions. [J Nurs Educ. 2015;54(6):306–315.]

Dr. Clark is Nurse Consultant, ATI Nursing Education, and Professor Emeritus, Boise State University; Mr. Nguyen is Staff Nurse, Saint Alphonsus Regional Medical Center, Boise, Idaho; Dr. Barbosa-Leiker is Assistant Professor, Washington State University, College of Nursing, Spokane, Washington; and Ms. Gill is Assistant Professor of Nursing, Darton State College, Albany, Georgia.

The statistical analysis was funded in part by the Jody DeMeyer Endowment, Boise State University.

Dr. Barbosa-Leiker received a consulting fee to perform the statistical analysis for this project. The remaining authors have disclosed no potential conflicts of interest, financial or otherwise.

Address correspondence to Cynthia M. Clark, PhD, RN, ANEF, FAAN, Nurse Consultant, ATI Nursing Education, 11161 Overbrook Road, Leawood, KS 66211; e-mail: cindy.clark@atitesting.com.

Received: November 02, 2014
Accepted: March 18, 2015

Abstract

Background:

Academic incivility is a serious challenge for nursing education, which needs to be empirically measured and fully addressed.

Method:

A convenience sample of nursing faculty and students from 20 schools of nursing in the United States participated in a mixed-methods study to test the psychometric properties of the Incivility in Nursing Education-Revised (INE-R) Survey.

Results:

A factor analysis and other reliability analyses support the use of the INE-R as a valid and reliable measurement of student and faculty perceptions of incivility in nursing education.

Conclusion:

The INE-R is a psychometrically sound instrument to measure faculty and student perceptions of incivility; to examine differences regarding levels of nursing education, program type, gender, age, and ethnicity; to compare perceptions of incivility between and among adjunct, clinical, teaching, and research faculty; and to conduct pre- and postassessments of the perceived levels of faculty and student incivility in nursing programs to inform evidence-based interventions. [J Nurs Educ. 2015;54(6):306–315.]

Dr. Clark is Nurse Consultant, ATI Nursing Education, and Professor Emeritus, Boise State University; Mr. Nguyen is Staff Nurse, Saint Alphonsus Regional Medical Center, Boise, Idaho; Dr. Barbosa-Leiker is Assistant Professor, Washington State University, College of Nursing, Spokane, Washington; and Ms. Gill is Assistant Professor of Nursing, Darton State College, Albany, Georgia.

The statistical analysis was funded in part by the Jody DeMeyer Endowment, Boise State University.

Dr. Barbosa-Leiker received a consulting fee to perform the statistical analysis for this project. The remaining authors have disclosed no potential conflicts of interest, financial or otherwise.

Address correspondence to Cynthia M. Clark, PhD, RN, ANEF, FAAN, Nurse Consultant, ATI Nursing Education, 11161 Overbrook Road, Leawood, KS 66211; e-mail: cindy.clark@atitesting.com.

Received: November 02, 2014
Accepted: March 18, 2015

Academic incivility, whether instigated or experienced by faculty or students, is a serious challenge for nursing education and an issue that needs to be adequately measured and fully addressed. The early work of Lashley and de Meneses (2001) and Thomas (2003) called attention to the problem, and, since then, several researchers have conducted studies to better understand the phenomena (Altmiller, 2012; Clark, 2008a, 2008b, 2011b, 2013a, 2013b; Clark, Olender, Kenski, & Cardoni, 2013; Clark & Springer 2007a, 2007b; Clarke, Kane, Rajacich, & Lafreniere, 2012; Cooper et al., 2009; Gallo, 2012; Lasiter, Marchiondo, & Marchiondo, 2012; Luparell, 2004, 2007, 2011; Marchiondo, Marchiondo, & Lasiter, 2010). Clark (2013a, 2013b) and Griffin and Clark (2014) defined incivility as rude or disruptive behavior that often results in psychological or physiological distress for the people involved (including targets, offenders, bystanders, peers, stakeholders, and organizations) and, if left unaddressed, may progress into threatening situations (or result in temporary or permanent illness or injury).

The dynamics of incivility are multidimensional and may occur between faculty–student, student–student, student–faculty, and faculty–faculty (including administrators). However, despite the dimension and direction of academic incivility, the effects of an uncivil encounter can be lasting and significant (Clark, 2013a). Not only do these encounters disrupt the learning environment (Clark, Farnsworth, & Landrum, 2009; Feldman, 2001; Luparell, 2007, 2011) but they are considered unprofessional and in violation of several foundational statements and positions. For example, Essential VIII: Professionalism and Professional Values described by the American Association of Colleges of Nursing (2008) underscores the importance of nurses being accountable and responsible for their individual actions and ensuring that civility is present so that professionalism can occur. Similarly, Provision 1.5 of the American Nurses Association Code of Ethics (2015) “requires nurses to create an ethical environment and culture of civility and kindness, treating colleagues, co-workers, employees, students, and others with dignity and respect…and that any form of bullying, harassment, intimidation, manipulation, threats or violence are always morally unacceptable behaviors” (p. 4). In addition, the Institute of Medicine report (2010) recommends empowering nurses to participate in collaborative efforts to improve work environments and health care systems. These foundational documents support the need for civil behaviors that must be continuously demonstrated by nurses in all areas of nursing education and practice.

It is important to measure the problem of incivility in nursing education so that strategies may be developed and interventions implemented to improve academic workplaces, the teaching–learning environments, student learning outcomes, student and faculty relationships, and the retention rates of faculty and students, as well as to change the culture in nursing practice. The purpose of this article is to detail the current instruments used to measure incivility in nursing education and to describe the revision and psychometric testing of the quantitative portion of the Incivility in Nursing Education-Revised (INE-R) Survey.

Measures of Incivility in Nursing Education

This article describes all known empirical instruments used to measure incivility in nursing education, including the psychometric properties and a brief critique of each instrument.

Incivility in Nursing Education Survey

The Incivility in Nursing Education (INE) Survey (Clark et al., 2009) is the first known empirical instrument developed to measure incivility in nursing education. When an extensive literature review conducted in 2004 revealed no known instruments designed to measure incivility in nursing education, Clark et al. (2009) developed the INE survey, which was based on a comprehensive literature review, numerous faculty and student interviews, and extensive pilot testing. The INE is a mixed-methodological tool designed to measure student and faculty perceptions of the types and frequencies of incivility in nursing education. Three previously developed instruments were used as the foundation for the INE—the Defining Classroom Incivility (DCI) Survey (Indiana University Center for Survey Research, 2000), the Student Classroom Incivility Measure (SCIM), and the Student Classroom Incivility Measure–Faculty (SCIM–F; Fryer Hanson, 2000). Cronbach’s alpha for the SCIM ranged from 0.84 to 0.86, whereas it was 0.67 for the SCIM-F. No reliability was reported for the DCI. The DCI, SCIM, and SCIM-F were designed for use in general higher education and were adapted by Clark et al. (2009) to address incivility in nursing education. A combination of these three instruments was used to develop the INE because separately the instruments did not sufficiently cover the full constructs of faculty and student incivility, nor did they specifically address incivility in nursing education.

The INE is divided into three sections, including a demographic component (section I), a list of uncivil student and faculty behaviors (section II), and four open-ended questions (section III). The uncivil behaviors listed in section II are divided into two groups—behaviors that may be considered uncivil (16 student behaviors and 20 faculty behaviors) and behaviors that are known to be threatening (13 student and 13 faculty behaviors). Participants are asked whether the behaviors listed are considered uncivil and how often they experienced each behavior in the past 12 months. Section II also includes two items that assess the degree to which students and faculty identify incivility as a problem within their academic program, as well as whether they view students or faculty as the most common perpetrator of incivility. Initial interitem reliability coefficients ranged from 0.81 to 0.89 (student incivility) and from 0.92 to 0.96 (faculty incivility; Clark et al., 2009). The INE also includes a qualitative portion that gathers respondents’ opinions about potential contributors to academic incivility, as well as ways to prevent and address incivility.

Bullying in Nursing Education Questionnaire

The Bullying in Nursing Education Questionnaire (BNEQ) was developed by Cooper et al. (2009) to assess student perceptions of bullying in nursing education. The BNEQ is composed of two previously established instruments—the Negative Acts Questionnaire (NAQ; Einarsen, Raknes, Matthiesen, & Hellesoy, 1994) and Celik’s and Bayraktar’s (2004) unnamed nursing student abuse questionnaire. The developers modified each instrument to facilitate data collection and analysis of one form of incivility—bullying. The NAQ measures frequency, intensity, and prevalence of workplace bullying and reports Cronbach’s alphas ranging from 0.87 to 0.93. The NAQ has been used worldwide (Einarsen et al., 1994). The unnamed nursing student abuse questionnaire collects data on verbal, physical, sexual, and academic abuse of nursing students from the students’ perspective (Celik & Bayraktar, 2004). Students’ reported sources of abuse ranged from classmates, faculty, nurses, physicians, patients, and patients’ family members. However, no psychometric properties were presented describing the validity and reliability of the BNEQ.

The BNEQ is a one-page survey that includes 15 items assessing the frequency and source of bullying (items 1 to 12), the behaviors used to cope with bullying (item 13), and resources provided by the nursing program to help students cope with bullying (items 14 and 15). Respondents answer the survey using a 5-point Likert scale, which ranges from 0 = never, 1 = seldom, 2 = intermittent, 3 = frequent, 4 = always. However, the survey fails to capture the qualitative data that could further the understanding of the phenomena of bullying. In addition, the study targeted nursing students in their final year of nursing school; previous studies have been inconclusive regarding which year nursing students are most at risk of experiencing bullying. Failure to include students of varying degrees of academic experience may have led the researchers to miss the students at greatest risk of experiencing bullying and thus miss the population of interest. The BNEQ focuses solely on nursing students’ perceptions of bullying, despite the fact that nursing faculty experience bullying at rates comparable to students (Beckmann, Cannella, & Wantland, 2013; Lashley & Menses, 2001). Therefore, the BNEQ provides only limited insight into the phenomena.

Nursing Education Environment Survey

The Nursing Education Environment Survey (NEES; Marchiondo et al., 2010) measures nursing students’ experiences of incivility from nursing faculty, including the most frequent types of uncivil behaviors encountered, frequencies of uncivil behaviors, and where the behaviors took place (e.g., classroom, clinical experience, laboratory). Students select from 14 uncivil (faculty) behaviors and 17 response actions used to cope with incivility. One open-ended question is included at the conclusion of the survey, asking students to describe their worst experience with faculty incivility. However, the survey does not assess how often students apply each of the listed responses to incivility nor does the open-ended question prompt respondents to include how they usually respond to the uncivil behaviors. Such questions would provide students with the opportunity to provide information on how they commonly respond, particularly if their response is not included in the 17 actions provided on the scale. Therefore, the NEES generates limited data to ascertain information regarding the frequency of student responses to incivility. Cronbach’s alpha for the NEES is 0.86; content validity was established by two nursing researchers and one content expert.

The NEES was developed from two established instruments—the Workplace Incivility Scale (WIS; Cortina, Magley, Williams, & Langhout, 2001) and the INE (Clark et al., 2009). The WIS, which measures the type and frequency of common uncivil behaviors exhibited by supervisors and coworkers, is composed of seven items and uses a 4-point Likert scale that ranges from 1 = never to 4 = most of the time. Cronbach’s alpha for the WIS ranges from 0.84 to 0.89. The INE is described earlier. Although the NEES is a relatively new instrument, the validity and reliability of the survey is strengthened by the fact that it consists of two established, valid, and reliable instruments. However, the NEES survey assesses student experiences with faculty incivility only, whereas previous research has shown that students frequently cite other students as common instigators of incivility (Cooper et al., 2009). Therefore, the NEES does not assess the experiences of students from the most common source of incivility. Further, it focuses solely on student experiences; hence, the NEES provides only limited insight into the phenomena.

Nurses’ Intervention for Civility Education Questionnaire

A second survey used to assess nursing students’ perceptions of incivility in nursing education is the Nurses’ Intervention for Civility Education Questionnaire (NICE-Q; Kerber, Jenkins, Woith, & Kim, 2012). The NICE-Q measures civility (and incivility) and academic integrity, using a 4-point Likert scale that ranges from 1 = never to 4 = more than six times per semester. The instrument is composed of six subscales that contain six to 13 items—five are associated with civility and one with academic integrity. Researchers used the scale in a pretest–posttest design and reported reliability scores that ranged from 0.68 to 0.86 in both pretest and posttest analyses of four of the civility subscales (the reliability for one subscale was not provided due to a low reliability score). Again, the instrument assessed only nursing students’ perceptions and experiences with incivility and civility (versus including nursing faculty). In addition, the sample size was small (n = 79), and researchers did not include randomization. Although the posttest survey included two qualitative questions, both were about how participation in the study affected participants’ behavior toward others, rather than expanding the understanding of incivility and seeking ways to foster academic civility.

Incivility in Online Learning Environments Survey

The Incivility in Online Learning Environments (IOLE) survey (Clark, Ahten, & Werth, 2012; Clark, Werth, & Ahten, 2012) measures faculty and student perceptions of uncivil behaviors, as well as the frequency of uncivil behaviors (from both faculty and students), suggestions for promoting civility in online learning environments (OLE), and the challenges and advantages of OLE. The IOLE survey was developed based on previous research findings and was reviewed for content validity by three OLE experts. The IOLE was then pilot tested by seven nursing faculty and 31 nursing students. Revisions were made based on feedback from the pilot study. The IOLE is composed of 18 uncivil student behaviors and 24 uncivil faculty behaviors. Using a Likert scale (always, usually, sometimes, and never), respondents are asked to describe their perceptions of whether a behavior is considered uncivil and the frequency (often, sometimes, rarely, never) of each behavior. In addition, the IOLE measures the perceived extent of incivility in OLE and whether students or faculty are more likely to perpetuate uncivil behaviors. The survey also includes three open-ended questions that ask what are the challenges and advantages of OLE and how to foster civility in OLE. The interitem Cronbach’s alpha was 0.99; internal consistency reliability was 0.98 for faculty behaviors and was 0.96 for student behaviors. Although the survey is reliable, the participants were enrolled in a baccalaureate completion program from one university. Further, the sample lacked gender and ethnic diversity; therefore, findings may not be generalizable to other populations.

Faculty-to-Faculty Incivility Survey

The Faculty-to-Faculty Incivility Survey (F-FI; Clark, 2013b; Clark et al., 2013; Grust, 2013) is used to measure nursing faculty perceptions of incivility with other faculty (including administrators), as well as the frequency of uncivil behaviors and ways to foster civility. The F-FI Survey is structured similarly to the INE—section I collects demographic information, section II lists 31 faculty behaviors that are commonly identified as uncivil, and section III includes two open-ended questions. section II uses a Likert scale ranging from always, sometimes, rarely, and never to assess faculty perceptions of the degree of incivility for the 31 behaviors listed. To assess the frequency of uncivil behaviors, faculty answer a second set of questions using a Likert scale that ranges from often, sometimes, rarely, and never. Section III asks faculty to describe an episode of incivility experienced with another faculty member, as well as ways to effectively to address faculty incivility. Cronbach’s alpha for the F-FI Survey is 0.97. However, the F-FI Survey focuses solely on faculty perceptions of faculty-to-faculty incivility and excludes student incivility, which is also a pressing issue for nursing education.

Summary of Instruments

Few instruments used to research incivility in nursing education have established both validity and reliability. Further, most focus on either nursing student or nursing faculty experiences and perceptions of incivility, despite evidence that both groups frequently encounter uncivil behaviors (Beckmann et al., 2013; Celik & Bayraktar, 2004; Clark, 2008a, 2008b, 2011; Clark, Ahten et al., 2012; Clark, Juan et al., 2012; Clark, Werth et al., 2012; Clarke et al., 2012; Clark & Springer 2007a, 2007b; Foster, Mackie, & Barnett, 2004; Lashley & de Meneses, 2001). The NEES (Marchiondo et al., 2010) focuses solely on student experiences with faculty incivility, whereas the NICE-Q (Kerber et al., 2012) assesses student experiences with both faculty and student incivility. The F-FI Survey focuses on nursing faculty perceptions of incivility experienced with other nursing faculty and administrators only. Previous research has shown that nursing students frequently cite other nursing students as the most common source of incivility (Cooper et al., 2009). Therefore, surveys that omit perceptions of nursing student uncivil behaviors may be missing the population of greatest impact. Further, instruments that exclude either nursing faculty or nursing student perceptions of uncivil behaviors do not acquire adequate information to provide a complete depiction of the phenomenon.

Only the INE Survey (and now the INE-R) includes quantitative and qualitative items to measure perspectives of uncivil student and faculty behaviors from the points of view of both students and faculty. The open-ended questions allow for rich, detailed descriptions of perceived acts of incivility and, more importantly, are designed to garner proposed strategies and solutions to foster civility in nursing education. A unique feature of the INE (and INE-R) is that the same instrument can be completed by both groups and may be used to measure both faculty and student perceptions of incivility in the same study (Clark et al., 2009). However, the INE and INE-R measure respondents’ perceptions, which may be open to individual interpretation.

Designing The INE-R

The INE-R was revised from the original INE and is built on an evidence-based continuum of uncivil behaviors, ranging from irritating behaviors (e.g., eye rolling and heavy sighing) to intimidating or violent behaviors, such as workplace mobbing or physical threats. Clark first developed the Continuum of Incivility (Figure) in 2009 (Clark, Ahten, & Stokowski, 2011) and has consistently revised it (Clark & Academic Consulting Group, 2013) based on several empirical studies (Altmiller, 2012; Clark, 2008a, 2008b, 2011, 2013a, 2013b; Clark, Ahten, & Werth, 2012; Clark, Nguyen, & Barbosa-Leiker, 2014; Clark, Juan et al., 2012; Clark et al., 2013; Clark et al., 2010; Clark & Springer 2007a, 2007b; Clark, Werth, et al., 2012; Clarke 2012; Cooper et al., 2009; Gallo, 2012; Hoffman, 2012; Lasiter et al., 2011; Luparell, 2004, 2007, 2011; Marchiondo et al., 2010). A sampling of the empirical evidence used to develop and revise the INE-R is included in this article to describe the basis for inclusion of specific student and faculty behaviors.

Continuum of Incivility. Copyright 2009 by Clark; revised 2013, 2014.

Figure.

Continuum of Incivility. Copyright 2009 by Clark; revised 2013, 2014.

Several studies revealed the most frequently reported uncivil student behaviors, including students arriving late to class, engaging in distracting side conversations, dominating class discussions, leaving early, being unprepared, and acting bored or apathetic (Clark, 2008a, 2008b, 2011; Clark & Springer 2007a, 2007b). More serious behaviors included students openly challenging faculty credibility, making taunts or disrespecting faculty and other students, sending inappropriate e-mails, and using vulgarity directed at faculty (Clark, 2008a, 2008b, 2011; Clark & Springer 2007a, 2007b). Despite obvious cultural differences, those findings were surprisingly similar to a study conducted in the People’s Republic of China using the INE, where the most frequently experienced student incivilities included being unprepared for class, sleeping in class, and engaging in distracting side conversations (Clark et al, 2010; Clark, Juan et al., 2012). Of note, the most frequently experienced threatening student behaviors were identical to those reported in the United States (Clark et al., 2010). These findings are also consistent across types of nursing programs (Hoffman, 2012).

Frequently reported faculty incivilities include faculty exerting superiority, threatening to fail students, not allowing open discussion, being inflexible and rigid, and being unavailable outside of class (Clark, 2011). Altmiller (2012) found similar results by exploring students’ perceptions of faculty incivility, including faculty demonstrating unprofessional behavior, using poor communication techniques, and exerting power over students. In a study by Lasiter et al. (2011) students described four major categories of faculty incivility and expressed the significant effects that faculty incivility had on academic and clinical performance and on decreasing program satisfaction and retention.

The Continuum of Incivility formed the basis for the revision, as it is constructed from numerous empirical studies and was found to be a reliable framework for a range of uncivil behaviors. The original INE had been used in several empirical studies (Beck, 2009; Clark, 2008a, 2008b, 2011; Clark, Juan et al., 2012; Clark et al., 2010; Clark & Springer, 2007a, 2007b; Hoffman, 2012), and results from those studies were used to reorganize and revise the INE to create the INE-R. Using the Continuum of Incivility as an organizing framework, three researchers (C.M.C., L.M.G. D.N.) rearranged the list of uncivil and threatening student and faculty behaviors into matched pairs and pilot tested the INE-R with 10 nursing faculty and 10 nursing students (who did not participate in the subsequent study). Results of the pilot test were favorable regarding the ease of survey administration and completion, content validity, readability, and logical flow. Slight modifications were made to the revised survey on the basis of faculty and student feedback derived from pilot testing. Ordinal variables mapping onto an underlying continuous construct is debated in measurement theory (Kampen & Swyngedouw, 2000). Although Likert-type scales are ordinal, it may be appropriate to consider them interval like, given the existence of the underlying, continuous latent variable. Given that the INE-R was based on the Continuum of Incivility, and pilot test results found that the response categories were able to adequately cover the continuum of responses, the authors chose to view this scale as interval like. However, it is worthy to note that because of the severity of the behaviors at the right end of the continuum, it was expected that more behaviors would fall onto the left- to mid-end of the continuum. In other words, it is more likely to see rude or uncivil nonverbal behaviors and utterances of demeaning remarks than threatening behaviors or violent acts.

Similar to the INE, the INE-R consists of three sections—demographic items (section I), which may be amended to fit the specific study; a list of uncivil student and faculty behaviors (section II); and four open-ended questions (section III). In the INE-R, the 16 uncivil student behaviors, 20 uncivil faculty behaviors, 13 threatening student behaviors, and 13 threatening faculty behaviors from the INE were collapsed into one categorical list, which reduced the list of total items to 24 student behaviors and 24 faculty behaviors. The respondent is asked to provide his or her perception regarding the level of incivility and how often each behavior occurred over the past 12 months for each of the 48 items. Section II in the INE-R also includes items that assess the degree to which students and faculty identify incivility as a problem within their academic program, as well as whether they view students or faculty as the most common perpetrator of incivility. An additional item was added to section II of the INE-R that asks respondents to select the top three strategies for improving the level of civility in nursing education from a drop-down menu of 10 items. The four qualitative fill-in-the-blank items were slightly revised to include a space for a narrative entry describing an example of an uncivil encounter, the primary reason or cause for incivility in nursing education, the most significant consequence of incivility in nursing education, and the most effective strategies to promote academic civility.

Instrument and Analysis of the INE-R

The INE-R is a 48-item, Likert-type survey consisting of 24 student behaviors and 24 matched faculty behaviors (e.g., student behavior: “skipping class or other scheduled activities,” matched faculty behavior: “cancelling class or other scheduled activities without warning”). Participants were asked to rate the level of incivility of each behavior with the following response categories: 1 = not uncivil, 2 = somewhat uncivil, 3 = moderately uncivil, and 4 = highly uncivil. Participants were also asked to indicate how often each behavior occurred in the past 12 months (1 = never, 2 = rarely, 3 = sometimes, and 4 = often). Demographic questions about status (student or faculty), gender, age, ethnic/racial background, years taught (faculty), level of program taught in (faculty), position (faculty), and current program level (student) were asked at the start of the survey.

Psychometric Analyses

Preliminary item analyses evaluated the skewness and kurtosis of the distribution of each item and examined the use of all response categories. The factor structure of the INE-R was analyzed using exploratory factor analysis, with robust maximum likelihood estimation. Missing data were assumed to be missing at random; therefore, full-information maximum likelihood estimation was used to estimate missing data. The exploratory factor analysis examined one- and two-factor models of the INE-R for student and faculty participants separately. Oblique rotation was used for >1 factors due to a hypothesized correlation among factors. The pattern of Geomin (oblique) rotated factor loadings and fit indices were examined. Standard criteria were used to assess model fit—the comparative fit index (CFI) ⩾ 0.90, root mean square error of approximation (RMSEA) ⩽ 0.08, and standardized root mean square residual (SRMR) ⩽0.08 (Brown, 2006). Cronbach’s alpha was estimated for each factor and for the total scale. P values (two-tailed) of ⩽0.05 were used to indicate significance of factor loadings and factor correlations. Mplus version 7.11 software (Muthén & Muthén, 1998–2012) was used for all factor analyses, and SPSS® software was used for reliability analyses.

Procedure

Institutional review board approval was obtained to conduct psychometric testing on the INE-R survey. A convenience sample of nursing faculty and students from 20 randomly selected schools of nursing (five from each quadrant of the United States) participated in the study. The principle investigator (C.M.C.) contacted the deans and directors from the 20 schools of nursing by e-mail, inviting them to ask their faculty and students to complete the INE-R. The link to the INE-R was provided in the body of an e-mail sent by the deans and directors to their nursing faculty and students; the link to the survey was administered using secure Web-based technology (Qualtrics). After obtaining consent, the respondents completed the INE-R. All responses were collected anonymously and were reported as aggregate data.

Results

The student sample (n = 310) was 86.1% female and 82.9% Caucasian, with a mean age of 32.15 years (SD = 9.74 years). The faculty sample (n = 182) was 90.7% female and 90.1% Caucasian, with a mean age of 52.24 years (SD = 9.99 years). Preliminary item analyses revealed that all response categories were used for each item. For the faculty behaviors, skewness ranged from −0.40 to −1.56 for the student participants and from −0.41 to 2.07 for the faculty participants; for the student behaviors, skewness ranged from 0.15 to −1.50 for the student participants and 1.80 to −2.20 for the faculty participants. For the faculty behaviors, kurtosis ranged from 0.57 to −1.24 for the student participants and −0.93 to 2.34 for the faculty participants; for the student behaviors, kurtosis ranged from 0.33 to −1.40 for the student participants and from 2.97 to −1.13 for the faculty participants. Robust maximum likelihood estimation was therefore used for the factor analyses.

Fit indices for the one- and two-factor models for student and faculty behavior by participant status are shown in Table 1. As can be seen, the one-factor model did not fit the data (CFIs as low as 0.614, RMSEAs as high as 0.162, and SRMRs as high as 0.172). The one-factor model fit the data for only the faculty behaviors rated by the faculty participants. The two-factor model provided better fit to the data, where only the faculty behaviors rated by the faculty participants did not meet all three of the study criteria fit values (CFI = 0.894 [study criterion ⩾0.90], RMSEA = 0.101 [study criterion ⩽0.08]).

Fit Indices for the One- and Two-Factor Models of the Incivility in Nursing Education-Revised Survey for Student and Faculty Behaviors, by Participant Status

Table 1:

Fit Indices for the One- and Two-Factor Models of the Incivility in Nursing Education-Revised Survey for Student and Faculty Behaviors, by Participant Status

The research team categorized the factors based on the factor analysis and after reviewing the pattern of factor loading. Tables 23 consist of the factor loadings for the two-factor model of the INE-R for student and faculty items, respectively, by participant status. As can be seen by the pattern of salient factor loadings, items reflect a lower level of incivility factor and a higher level of incivility factor. Note that the same factor loading pattern is found across student and faculty behaviors, as rated by both students and faculty, with the exception of two items. The student behavior, “being unresponsive to e-mails or other communications” loaded on the higher level of incivility factor for student participants, but it loaded on the lower level of incivility factor for faculty participants. The matched faculty behavior (“being unavailable outside of class [not returning calls or e-mails], not maintaining office hours”) loaded on the lower level of incivility factor for both student and faculty participants. Therefore, this item is scored on the lower level of incivility factor. The faculty behavior, “unfair grading” loaded on the lower level of incivility factor for faculty participants but loaded on the higher level of incivility factor for student participants. In addition, the matched student behavior (“cheating on exam[ination]s or quizzes”) loaded on the higher level of incivility factor for both student and faculty participants. Thus, this item was scored on the higher level of incivility factor. Therefore, the lower level of incivility consists of 15 items and the higher level of incivility consists of nine items.

Standardized Factor Loadings for the Two-Factor Model of the Incivility in Nursing Education-Revised Survey for Student Behaviors, by Participant Status

Table 2:

Standardized Factor Loadings for the Two-Factor Model of the Incivility in Nursing Education-Revised Survey for Student Behaviors, by Participant Status

Standardized Factor Loadings for the Two-Factor Model of the Incivility in Nursing Education-Revised Survey for Faculty Behaviors, by Participant Status

Table 3:

Standardized Factor Loadings for the Two-Factor Model of the Incivility in Nursing Education-Revised Survey for Faculty Behaviors, by Participant Status

The factor analyses revealed seven items that cross-loaded onto both factors. Although there is a strong primary loading, a secondary loading is also given for what may be nondiscriminant items. Even though most items cross-loaded in the factor models for either students or faculty behaviors, or student or faculty participants, the item “making rude gestures or nonverbal behaviors toward others (eye rolling, finger pointing)” cross-loaded as a student behavior for both student and faculty participants, and as a faculty behavior for student participants.

Reliability coefficients are provided in Tables 23. Cronbach’s alpha indicates that the higher level of incivility factor and the lower level of incivility factor are reliable for the student and faculty behaviors, as rated by the student and faculty participants (Cronbach’s alphas ⩾0.94). Although a one-factor higher order model using confirmatory factor analysis is needed in future research to validate a single INE-R score, the Cronbach’s alpha for the total score is ⩾0.96 for student behaviors and ⩾0.98 for faculty behaviors.

Scoring Information

Numerical values may be assigned to the response categories (1 = not uncivil, 2 = somewhat uncivil, 3 = moderately uncivil, 4 = highly uncivil) for student and faculty behaviors, and a similar scoring may be assigned for frequency of perceived uncivil behaviors (1 = never, 2 = rarely, 3 = sometimes, 4 = often). Creating these scores may be done to evaluate the sample as a whole or to conduct comparisons across individual items or subscale and total scores.

Single Items. Each item on the INE-R can be calculated separately by means and overall percentages for each response category. Further, individual items’ response categories may be collapsed into dichotomous categories. For example, regarding the item “Expressing disinterest, boredom, or apathy about course content or subject matter,” the results can be bifurcated and reported as not uncivil or somewhat uncivil and moderately uncivil or highly uncivil. For the frequency of the observed behaviors, the items can be collapsed into often or sometimes and rarely or never.

Subscales and Total Scale. As noted above, the lower level of incivility factor consists of 15 items, and the higher level of incivility factor consists of nine items. Subscales can be created for the lower level of incivility factor by summing items 1 to 12, 15, 16, and 18 and dividing by 15 ([1+2+3+4+5+6+7+8 +9+10+11+12+15+16+18]/15). For the higher level of incivility factor, sum items 13, 14, 17, and 19 to 24 and divide by 9 ([13+14+17+19+20+21+22+23+24]/9). Note that the numbering of items matches the order of items in Tables 23; if other researchers create a survey with the items in a different order, the numbering will need to be adjusted accordingly.

Discussion and Application

Similar to the INE, the INE-R is used to describe student and faculty perceptions of incivility in nursing education. It includes quantitative and qualitative items to measure uncivil student and faculty behaviors in nursing education from both the faculty’s and students’ points of view. New items include a perceived overall rating of organizational civility and a list of evidence-based strategies for improving civility in nursing education. Four open-ended items are designed to garner narrative descriptions about uncivil experiences, potential causes of incivility, the impact and consequences of uncivil behavior, and the most effective ways to promote academic civility. A unique feature of the INE-R is that the same instrument can be completed by both faculty and students, thus gathering the perceptions of incivility of both groups in the same study. Alternatively, the INE-R may be administered to only students or faculty and may be further separated into faculty and student sections and administered to one or both groups, depending on the purpose of the study. In addition, the demographic items may be modified to meet the study’s objectives.

Like the INE, the INE-R has the potential to assist researchers with studies designed to measure faculty and student perceptions of incivility in nursing education and to examine differences regarding levels of nursing education, program type, gender, age, and ethnicity. The INE-R may also be used to compare uncivil behaviors between and among adjunct, clinical, teaching, and research faculty. Findings from the INE-R may also be used as a pre- and postassessment tool to measure the perceived levels of faculty and student incivility in nursing programs. For example, findings may stimulate dialogue among nursing administrators, faculty, and students about prevention and intervention strategies. Targeted interventions may be implemented based on the findings of the INE-R. After the interventions are established and firmly in place, the INE-R may be used as a follow-up measure to determine the extent to which incidents of incivility have declined and other outcomes have improved. At this time, item and mean subscale incivility scores can only be interpreted continuously, as thresholds of incivility scores have not been tested. That is, comparisons across groups on mean subscales is appropriate, whereas indicating that a mean score greater than a certain value equates to incivility is not warranted at this time.

The original INE Survey has been translated into several non-English languages and has been used to measure incivility in nursing programs within and outside the United States. Because the INE-R is heavily based on and is similar to the INE, translation may be easily accomplished by using the translated versions of the INE that have already been developed. Further, the INE-R may be administered using a paper-and-pencil version or a secure Web-based platform, such as Qualtrics or Survey Monkey. The use of a Web-based platform allows for easy data collection, download, and analysis.

Limitations

Because the INE-R utilizes self-report items with voluntary participation, it carries the risk of common method biases. Also, because the INE-R is a newly revised instrument, additional studies are needed to examine concurrent and predictive validity to adequately define and test subscale thresholds. Method effects based on the ordering of the items from least to most uncivil behaviors have not yet been examined. Finally, it is possible that results may be skewed due to the proportion of students and faculty exhibiting uncivil behaviors. For example, it is possible for one student or one faculty member to commit numerous acts of incivility and, in another case, the same number of uncivil behaviors may be displayed by many students and faculty, resulting in the same frequency score.

Conclusion

This article provides documentation of the development and validation of the INE-R, which measures faculty and student perceptions of incivility in nursing education, contributors to uncivil behavior, and strategies to prevent, address, and intervene for uncivil encounters. The INE-R is the revised version of the INE Survey, which has been used in numerous empirical studies nationally and internationally.

References

  • Altmiller, G. (2012). Student perceptions of incivility in nursing education: Implications for educators. Nursing Education Perspectives, 33, 15–20. doi:10.5480/1536-5026-33.1.15 [CrossRef]
  • American Association of Colleges of Nursing. (2008). The essentials of baccalaureate education for professional nursing practice. Retrieved from http://www.aacn.nche.edu/education-resources/baccessentials08.pdf
  • American Nurses Association. (2015). Code of ethics for nurses with interpretive statements. Washington, DC: Author. Retrieved from http://www.nursingworld.org/codeofethics
  • Beck, J.W. (2009). Deconstructing student perceptions of incivility in the nursing education triad (Unpublished doctoral dissertation). Louisiana State University, Baton Rouge. Retrieved from http://etd.lsu.edu/docs/available/etd-10312009-095845/unrestricted/JWBeck_diss.pdf
  • Beckmann, C.A., Cannella, B.L. & Wantland, D. (2013). Faculty perception of bullying in schools of nursing. Journal of Professional Nursing, 29, 287–294. doi:10.1016/j.profnurs.2012.05.012 [CrossRef]
  • Brown, T.A. (2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.
  • Celik, S.S. & Bayraktar, N. (2004). A study of nursing student abuse in Turkey. Journal of Nursing Education, 43, 330–336.
  • Clark, C.M. (2008a). The dance of incivility in nursing education as described by nursing faculty and students. Advances in Nursing Science, 31(4), E37–E54. doi:10.1097/01.ANS.0000341419.96338.a3 [CrossRef]
  • Clark, C.M. (2008b). Faculty and student assessment and experience with incivility in nursing education: A national perspective. Journal of Nursing Education, 47, 458–465. doi:10.3928/01484834-20081001-03 [CrossRef]
  • Clark, C.M. (2011). Pursuing a culture of civility: An intervention study in one program of nursing. Nurse Educator, 36, 98–102. doi:10.1097/NNE.0b013e3182161238 [CrossRef]
  • Clark, C.M. (2013a). Creating and sustaining civility in nursing education. Indianapolis, IN: Sigma Theta Tau International.
  • Clark, C.M. (2013b). National study on faculty-to-faculty incivility: Strategies to foster collegiality and civility. Nurse Educator, 38, 98–102. doi:10.1097/NNE.0b013e31828dc1b2 [CrossRef]
  • Clark, C.M.Academic Consulting Group. (2013, September13). The pedagogy of civility: Innovative strategies to create an engaged learning environment. Retrieved from http://pages.elsevieradvantage.com/WhitePaperSeptember2013.html
  • Clark, C.M., Ahten, S.M. & Stokowski, L.A. (2011, March24). The downward spiral: Incivility in nursing. Medscape Nurses. Retrieved from http://www.medscape.com/viewarticle/739328
  • Clark, C.M., Ahten, S.M. & Werth, L. (2012). Cyber-bullying and incivility in the online learning environment, part 2: Promoting student success in the virtual classroom, Nurse Educator, 37, 192–197. doi:10.1097/NNE.0b013e318262eb2b [CrossRef]
  • Clark, C.M, Farnsworth, J. & Landrum, R.E. (2009). Development and description of the incivility in nursing education (INE) survey. The Journal of Theory Construction and Testing, 13, 7–15.
  • Clark, C.M., Juan, C.M., Allerton, B.W., Otterness, N.S., Jun, W.Y. & Wei, F. (2012). Faculty and student perceptions of academic incivility in the People’s Republic of China. Journal of Cultural Diversity, 19, 85–93.
  • Clark, C.M., Nguyen, D.T. & Barbosa-Leiker, C. (2014). Student perceptions of stress, coping, relationships, and academic civility: A longitudinal study. Nurse Educator, 39, 170–174. doi:10.1097/NNE.0000000000000049 [CrossRef]
  • Clark, C.M., Olender, L., Kenski, D. & Cardoni, C. (2013). Exploring and addressing faculty-to-faculty incivility in nursing education: A national perspective and literature review. Journal of Nursing Education, 52, 211–218. doi:10.3928/01484834-20130319-01 [CrossRef]
  • Clark, C.M., Otterness, N.S., Jun, W.Y., Allerton, B.W., Juan, C.M., Black, M. & Wei, F. (2010). Descriptive study of student incivility in the People’s Republic of China. Journal of Cultural Diversity, 17, 136–143.
  • Clark, C.M. & Springer, P.J. (2007a). Incivility in nursing education: A descriptive study of definitions and prevalence. Journal of Nursing Education, 46, 7–14.
  • Clark, C.M. & Springer, P.J. (2007b). Thoughts on incivility: Student and faculty perceptions of uncivil behavior in nursing education. Nursing Education Perspectives, 28, 93–97.
  • Clark, C.M., Werth, L. & Ahten, S. (2012). Cyber-bullying and incivility in the online learning environment, part 1: Addressing faculty and student perceptions. Nurse Educator, 37, 150–156. doi:10.1097/NNE.0b013e31825a87e5 [CrossRef]
  • Clarke, C.M., Kane, D.J., Rajacich, D.L. & Lafreniere, K.D. (2012). Bullying in undergraduate clinical nursing education. Journal of Nursing Education, 51, 269–276. doi:10.3928/01484834-20120409-01 [CrossRef]
  • Cooper, J.R.M., Walker, J.T., Winters, K., Williams, P.R., Askew, R. & Robinson, J.C. (2009). Nursing students’ perceptions of bullying behaviours by classmates. Issues in Educational Research, 19, 212–226.
  • Cortina, L.M., Magley, V.J., Williams, J.H. & Langhout, R.D. (2001). Incivility in the workplace: Incidence and impact. Journal of Occupational Health Psychology, 6, 64–80. doi:10.1037/1076-8998.6.1.64 [CrossRef]
  • Einarsen, S., Raknes, B., Matthiesen, S. & Hellesoy, O. (1994). Negative Acts Questionnaire, revised. Bergen, Norway: Bergen Bullying Research Group.
  • Feldman, L.J. (2001). Classroom civility is another of our instructor responsibilities. College Teaching, 49, 137–140. doi:10.1080/87567555.2001.10844595 [CrossRef]
  • Foster, B., Mackie, B. & Barnett, N. (2004). Bullying in the health sector: A study of bullying of nursing students. New Zealand Journal of Employment Relations, 29, 67–83.
  • Fryer Hanson, M.R. (2000). Classroom incivility: Management practices in large lecture courses. Available from ProQuest Dissertations and Theses database. (UMI No. 9978835)
  • Gallo, V.J. (2012). Incivility in nursing education: A review of literature. Teaching and Learning in Nursing, 7, 62–66. doi:10.1016/j.teln.2011.11.006 [CrossRef]
  • Griffin, M. & Clark, C.M. (2014). Revisiting cognitive rehearsal as an intervention against incivility and lateral violence in nursing: 10 years later. Journal of Continuing Education in Nursing, 45, 535–542. doi:10.3928/00220124-20141122-02 [CrossRef]
  • Grust, P.L. (2013). A study of the relationship of incivility to faculty satisfaction and persistence in nursing education. Available from ProQuest Dissertations and Theses database. (UMI No. 3597793)
  • Hoffman, R.L. (2012). Differences in student perceptions of student and faculty incivility among nursing program types: An application of attribution theory. Available from ProQuest Dissertations and Theses database. (UMI No. 3547431)
  • Indiana University Center for Survey Research. (2000). A survey on academic incivility at Indiana University. Retrieved from http://spea.iupui.edu/documents/Incivility%20at%20IU.pdf
  • Institute of Medicine. (2010). The future of nursing: Leading change, advancing health. Washington, DC: National Academies Press.
  • Kampen, J. & Swyngedouw, M. (2000). The ordinal controversy revisited. Quality & Quantity, 34, 87–102. doi:10.1023/A:1004785723554 [CrossRef]
  • Kerber, C., Jenkins, S., Woith, W. & Kim, M. (2012). Journal clubs: A strategy to teach civility to nursing students. Journal of Nursing Education, 51, 277–282. doi:10.3928/01484834-20120323-02 [CrossRef]
  • Lashley, F.R. & de Meneses, M. (2001). Student civility in nursing programs: A national survey. Journal of Professional Nursing, 17, 81–86. doi:10.1053/jpnu.2001.22271 [CrossRef]
  • Lasiter, S., Marchiondo, L. & Marchiondo, K. (2011). Student narratives of faculty incivility. Nursing Outlook, 60, 121–126. doi:10.1016/j.outlook.2011.06.001 [CrossRef]
  • Luparell, S. (2004). Faculty encounters with uncivil nursing students: An overview. Journal of Professional Nursing, 20, 59–67. doi:10.1016/j.profnurs.2003.12.007 [CrossRef]
  • Luparell, S. (2007). The effects of student incivility on nursing faculty. Journal of Nursing Education, 46, 15–19.
  • Luparell, S. (2011). Incivility in nursing: The connection between academia and clinical settings. Critical Care Nurse, 31, 92–95. doi:10.4037/ccn2011171 [CrossRef]
  • Marchiondo, K., Marchiondo, L.A. & Lasiter, S. (2010). Faculty incivility: Effects on program satisfaction of BSN students. Journal of Nursing Education, 49, 608–614. doi:10.3928/01484834-20100524-05 [CrossRef]
  • Muthén, L. & Muthén, B.O. (1998–2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén.
  • Thomas, S.P. (2003). Handling anger in the teacher-student relationship. Nursing Education Perspective, 24, 17–24.

Fit Indices for the One- and Two-Factor Models of the Incivility in Nursing Education-Revised Survey for Student and Faculty Behaviors, by Participant Status

Modeldfχ2CFIRMSEASRMR
One-factor model
  Student participants
    Student behaviors25215780.7690.1300.109
    Faculty behaviors25213210.8020.1210.055
  Faculty participants
    Student behaviors25214500.6140.1620.172
    Faculty behaviors971640.9160.0730.054
Two-factor model
  Student participants
    Student behaviors2295910.9370.0710.029
    Faculty behaviors2296660.9300.0810.020
  Faculty participants
    Student behaviors2295370.9010.0860.041
    Faculty behaviors2296180.8940.1010.030

Standardized Factor Loadings for the Two-Factor Model of the Incivility in Nursing Education-Revised Survey for Student Behaviors, by Participant Status

Student BehaviorStudent ParticipantsFaculty Participants


Lower Level of IncivilityHigher Level of IncivilityLower Level of IncivilityHigher Level of Incivility
Expressing disinterest, boredom, or apathy about course content or subject matter0.800.72
Making rude gestures or nonverbal behaviors toward others (e.g., eye rolling, finger pointing)0.45(0.39)0.45(0.40)
Sleeping or not paying attention in class (doing work for other classes, not taking notes)0.680.69
Refusing or reluctant to answer direct questions0.520.53
Using a computer, mobile telephone, or other media device in a class, meeting, or activity for unrelated purposes0.730.74
Arriving late for class or other scheduled activities0.760.82
Leaving class or other scheduled activities early0.810.81
Being unprepared for class or other scheduled activities0.770.78
Skipping class or other scheduled activities0.700.69
Being distant and cold toward others (unapproachable, rejecting faculty or other student’s opinions)0.560.44(0.41)
Creating tension by dominating class discussion0.550.49
Holding side conversations that distract you or others0.600.65
Cheating on examinations or quizzes0.800.59
Making condescending or rude remarks toward others0.650.65
Demanding make-up examinations, extensions, or other special favors0.570.60
Ignoring, failing to address, or encouraging disruptive behaviors by classmates0.48(0.45)0.54
Demanding a passing grade when a passing grade has not been earned0.69(0.41)0.54
Being unresponsive to e-mails or other communications0.540.59
Sending inappropriate or rude e-mails to others0.840.74
Making discriminating comments (racial, ethnic, gender) directed toward others0.930.89
Using profanity (swearing, cussing) directed toward others0.750.79
Threats of physical harm against others (implied or actual)1.020.98
Property damage1.030.99
Making threatening statements about weapons0.990.99
Cronbach’s alpha by factor0.950.990.940.97
Cronbach’s alpha total scale0.960.98

Standardized Factor Loadings for the Two-Factor Model of the Incivility in Nursing Education-Revised Survey for Faculty Behaviors, by Participant Status

Faculty BehaviorStudent ParticipantsFaculty Participants


Lower Level of IncivilityHigher Level of IncivilityLower Level of IncivilityHigher Level of Incivility
Expressing disinterest, boredom, or apathy about course content or subject matter0.880.77
Making rude gestures or nonverbal behaviors toward others (e.g., eye rolling, finger pointing)0.56(0.41)0.59
Ineffective or inefficient teaching method (deviating from course syllabus, changing assignment or test dates)1.050.80
Refusing or reluctant to answer direct questions0.860.73
Using a computer, mobile telephone, or another media device in faculty meetings, committee meetings, or other work activities for unrelated purposes0.860.88
Arriving late for class or other scheduled activities0.880.83
Leaving class or other scheduled activities early0.880.87
Being unprepared for class or other scheduled activities0.770.85
Canceling class or other scheduled activities without warning0.600.73
Being distant and cold toward others (unapproachable, rejecting student’s opinions)0.730.68
Punishing the entire class for one student’s misbehavior0.48(0.46)0.57
Allowing side conversations by students that disrupt class0.770.86
Unfair grading0.630.54(0.43)
Making condescending or rude remarks toward others0.600.69
Refusing to discuss make-up examinations, extensions, or grade changes0.690.71
Ignoring, failing to address, or encouraging disruptive student behaviors0.650.70
Exerting superiority, abusing position, or rank over others (e.g., arbitrarily threatening to fail students)0.650.65
Being unavailable outside of class (not returning calls or e-mails, not maintaining office hours)0.540.69
Sending inappropriate or rude e-mails to others0.730.64
Making discriminating comments (racial, ethnic, gender) directed toward others0.910.88
Using profanity (swearing, cussing) directed toward others0.860.78
Threats of physical harm against others (implied or actual)1.001.00
Property damage0.991.02
Making threatening statements about weapons1.020.99
Cronbach’s alpha by factor0.980.990.970.99
Cronbach’s alpha total scale0.990.98

10.3928/01484834-20150515-01

Sign up to receive

Journal E-contents