Journal of Nursing Education

The articles prior to January 2012 are part of the back file collection and are not available with a current paid subscription. To access the article, you may purchase it or purchase the complete back file collection here

The Development and Validation of a Scale Measuring Effective Clinical Teaching Behaviors

Lani Zimmerman, PhD, RN; Joan Westfall, MSN, RN

Abstract

ABSTRACT

The purpose of this methodological study was to describe the development and validation of an instrument that would measure effective clinical teaching behaviors of nursing faculty. Two hundred eighty-one nursing students evaluated their clinical instructors using a 43-item Likert scale entitled "Effective Teaching Clinical Behaviors" (ETCB), and data were used to test the validity and reliability of the instrument. Content validity of the scale is adequately documented, and factor analysis suggests that the tool is measuring one major factor, "effective clinical teaching behaviors." Internal consistency and test-retest reliability coefficients achieved satisfactory standards for reliability. Alternate scales reliability coefficient was also satisfactory. The ETCB fills a void in the instrumentation available for clinical instructors. Since students are the direct recipients of instruction, it seems imperative that a valid and reliable scale be available for them to evaluate their instruction.

Abstract

ABSTRACT

The purpose of this methodological study was to describe the development and validation of an instrument that would measure effective clinical teaching behaviors of nursing faculty. Two hundred eighty-one nursing students evaluated their clinical instructors using a 43-item Likert scale entitled "Effective Teaching Clinical Behaviors" (ETCB), and data were used to test the validity and reliability of the instrument. Content validity of the scale is adequately documented, and factor analysis suggests that the tool is measuring one major factor, "effective clinical teaching behaviors." Internal consistency and test-retest reliability coefficients achieved satisfactory standards for reliability. Alternate scales reliability coefficient was also satisfactory. The ETCB fills a void in the instrumentation available for clinical instructors. Since students are the direct recipients of instruction, it seems imperative that a valid and reliable scale be available for them to evaluate their instruction.

Introduction

Evaluation of clinical teaching plays an important role in the teaching-learning process both for the student who is evaluating the instructor, and for the teacher who is seeking improvement. Review of the evaluation data can identify effectiveness, as well as problem areas in teaching. The evaluation of clinical teaching is also a vital component of the comprehensive evaluation of nursing faculty. According to Bell, Miller, and Bell (1984), evaluation of faculty can be a constructive process to improve teaching performance and to provide data in reaching decisions about merit, promotion, and tenure.

In general, clinical teaching evaluations by students of their learning experiences are recognized as a valid source of data which facilitates changes in clinical teaching and improves the quality of teaching. As Miller (1982) cites, these evaluations provide important data on a multitude of teaching behaviors.

Nurse educators are expected to demonstrate accountability in their work environment. Since clinical teaching evaluation of nursing faculty is a dimension of professional accountability (Curry, 1981), nurse educators have a responsibility to their students and to their profession to develop valid and reliable instruments to measure effective clinical instruction (Sullivan, 1985; Zimmerman & Waltman, 1986).

Review of the Literature

Most articles written on evaluation of clinical instruction concentrate on identifying clinical teaching behaviors and overlook the development of valid and reliable instruments for measuring effective teaching behaviors. In a review of the literature, Zimmerman and Waltman (1986) found that some of the authors developed categories of behaviors for use in evaluating clinical instruction (Armington, 1972; Jacobson, 1966; Kiker, 1973; O'Shea & Parsons, 1979). Other authors developed a list of specific behaviors for evaluating clinical instruction (Barham, 1965; Brown, 1981; Craig & Page, 1981; Stuebbe, 1981).

Although the articles cited above were helpful in identifying major categories of behaviors and specific behaviors that are reflective of effective clinical instruction, no attempt was made to reveal the dimensions of clinical teaching through factor analysis of student evaluations. According to Romberg (1984), one way to determine the major elements of clinical teaching behavior is to attempt to identify, through factor analysis, the underlying teaching dimensions or factors on which clinical instructors vary. The number of studies using factor analyses of clinical teaching behaviors is limited, especially in nursing education.

However, a few factor analytic studies have been conducted on evaluation of clinical instruction in other educational settings. Through factor analysis, Romberg (1984) found four factors to describe the clinical teaching behaviors inherent in a set of dental students' evaluations of clinical instruction. In this study, Romberg found that the four factors (meeting teaching responsibilities, behaviors conducive to clinical learning, instructor competence, and job satisfaction) accounted for 92.1% of the total variance. The factor analytic study (Miller, 1982) of responses of third-year medical students' evaluations of clinical instruction indicated that the dimensionality of clinical teaching behaviors could be reduced. Although no indication was given of the contribution of each factor to the total variance in the data, it did demonstrate the factorial validity of an evaluation instrument used to measure clinical teaching behaviors.

In the assessment of laboratory and seminar/discussion group experiences among college students, Meredith (1983) found that the distribution of eigenvalues indicated a single "large" factor accounting for the majority of variance. No studies were found in the literature to reveal the dimensions of clinical instruction through factor analytic techniques of student evaluations in nursing education settings. Furthermore, there is a paucity of instruments that measure the validity and reliability of clinical instruction in nursing education. Therefore, the purpose of this methodological study was to describe the development and validation of an instrument entitled "Effective Clinical Teaching Behaviors" (ECTB).

Specific objectives of this study were to (1) assess the content validity of the ECTB, (2) evaluate the construct validity of the ECTB through factor analysis, and (3) evaluate the internal consistency and test-retest reliability of the scale.

Methodology

The ECTB was developed to provide a means for students to evaluate effective teaching behaviors of their clinical instructor. There were originally 53 items on the scale which were selected from the literature reviewed.

Ten senior faculty who were perceived by their peers to be excellent clinical instructors were given 53 items reported in the literature and asked to rate the importance on a three-point scale (with "1" being Not Important to "3" being Very Important) of the items that were appropriate for inclusion on a tool which would be used by students. The evaluation of the 53 items in terms of content validity by the 10 faculty resulted in the deletion of 10 items. The following criteria were used for maintaining or deleting an item:

* If a specific item was reported to be Not Important by greater than or equal to 50% of the faculty it was deleted, or if a specific item was reported to be Very Important by greater than or equal to 50% of the faculty it was maintained on the tool.

* If items did not fall within the range described in criterion I, they had to have a mean greater than or equal to 2.0 to be maintained on the tool.

The convenience sample was comprised of 281 students who evaluated 29 clinical faculty teaching at three different nursing programs in a midwestern state. Informed consent was obtained from both faculty and students. One of the nursing programs was located in a large university (a fouryear program), one in a state college (a four-year program), and one was a three-year diploma nursing program. Equal numbers of students were selected to represent different years and types of program. First year students were not used in the sample because they were not in the clinical settings. All students were asked to evaluate their clinical instructor at the end of their clinical rotation. The 43-item Likert scale took approximately 10 to 15 minutes to complete. Students were asked to rate the frequency of an observed behavior on a scale of 1 to 5 with "1" indicating almost never, "5" indicating almost always, and N/O indicating "not observed." Evaluations were placed in a sealed envelope by the students, then given to the instructor who returned them to the researchers. Data analysis included the following statistical procedures:

* Factorial validity of the ECTB was investigated for the intercorrelations of the 43 items from the scale data of the 281 nursing students using a principal factor solution followed by a varimax rotation.

* Reliability of the ECTB was evaluated using coefficient alpha for the data from the 281 nursing students. A splithalf reliability coefficient alpha was employed on alternate tests for the data from the 281 nursing students.

* A test-retest reliability coefficient for the alternate forms was used with an additional 44 nursing students who were enrolled in summer nursing courses.

Findings

VALIDITY ASSESSMENT

Content validity. Face validity was demonstrated by a thorough review of the literature which covered the domain "effective teaching behaviors." Logical validity was established by having a panel of 10 senior clinical faculty evaluate the ETCB.

Construct validity. The alpha reliability coefficient is one source of indirect support for the construct validity of the ETCB. Factor analysis also provided a method for construct validation by identifying clusters of related variables which share a common variance and, thus, are measuring the same attribute.

Principal factor extraction was used to determine the minimum number of independent factors. Kaiser's criterion (an eigenvalue of one or greater), the scree test, and conceptual interpretability were used to determine which factors were meaningful enough to extract and rotate. Three factors had eigenvalues greater than one, thus, a principal factor analysis extracting three factors with rotation was conducted. Factor 1 had an eigenvalue of 20.61 which accounted for 48% of the variance; factor 2 had an eigenvalue of 1.62 which accounted for an additional 4% of the variance; and factor 3 had an eigenvalue of 1.07 which accounted for an additional 21/2% of the total variance. The data strongly suggest that there is clearly one overall factor that is being measured by this tool. The Table depicts the composition for the three-factor solutions of the ECTB.

Table

TABLECOMPOSITION OF THREE-FACTOR SOLUTION OF ECTB AND ALTERNATE FORMS

TABLE

COMPOSITION OF THREE-FACTOR SOLUTION OF ECTB AND ALTERNATE FORMS

RELIABILITY

Homogeneity. Prior to assessing construct validity, Cronbach's coefficient alpha was computed on the ETCB scale data to estimate the internal consistency reliability of the instrument (Nunnally, 1978). Since internal consistency is concerned with whether something in common is being measured, and construct validity is concerned with what that common element is, homogeneity is related to both validity and reliability. A coefficient alpha of .97 was obtained which indicates overall homogeneity of the content on the scale.

Split-half reliability matching items of similar behaviors resulted in a correlation of .96. The coefficient alpha with Form A was .95 and with Form B the coefficient was .95. (See the Table for the alternate forms of the ECTB.)

Stability. Test-retest reliability was evaluated using 44 nursing students taking a summer nursing course. The two administrations of the ETCB were administered at twoweek intervals. A Pearson correlation coefficient was used to assess the relationship between the test-retest data. The correlation for Form A (n = 22) was r = .94 and Form B (n = 22) was r = .93 with significance at the p < .001 level.

Discussion

Both forms of the ECTB scales were found valid and reliable. Content validity of the tool is adequately documented. Factor analysis suggests that the tool is measuring one major factor - that is effective clinical teaching behaviors. This was somewhat surprising since the literature suggested other major categories that have been identified; however, these categories did not evolve from the factor analysis. In reviewing these reports, the major categories identified in these scales were not assessed for their psychometric properties.

Internal consistency, alternate scales reliability, and test-retest reliability were highly satisfactory. Having two alternate forms with such high reliability should prove very useful in clinical practice. Students generally have two or three rotations per semester and are asked to evaluate their clinical instructor at least once every rotation. Having valid and reliable scales that can be administered in 5 to 10 minutes should prove effective in obtaining accurate data.

The ETCB fills a void in the instrumentation available for evaluation of clinical instructors. Students are the direct recipients of instruction, and, therefore, it seems imperative that a valid and reliable scale should be available for them to evaluate their instruction. In addition, the ETCB is invaluable for a faculty member to document his or her effectiveness as a clinical instructor, especially in relation to promotion and tenure endeavors.

Further study needs to be conducted on this instrument to determine the soundness and replicability of this scale. Other measures of construct validity should also be considered: for example, testing this scale on beginning nurse educators as compared with expert senior clinical faculty to see if there is any difference in scores obtained. Criterion validity (concurrent) could possibly be assessed by comparing peer and student evaluations of individual faculty members.

References

  • Armington, C. (1972). Student evaluation: Threat or incentive. Nursing Outlook, 20(12), 789-792.
  • Barham, V. Z. (1965). Identifying effective behaviors of the nursing instructor through critical incidents. Nursing Research, 14(1), 65-69.
  • Bell, D. F., Miller, R. L, & Bell, D. L. (1984). Faculty evaluationteaching, scholarship, and services. Nurse Educator, 9(2), 18-27.
  • Brown, S. T. (1981). Faculty and student perceptions of effective clinical instruction. Journal of Nursing Education, 20(9), 4-15.
  • Craig, J., & Page, G. (1981). The questioning skills of nursing instructors. Journal of Nursing Education, 20(5), 18-23.
  • Curry, M. A. (1981). Clinical evaluation of the nursing instructor. Nursing Forum, 22(1), 62-71.
  • Jacobson, M. D. (1966). Effective and ineffective behavior of teachers of nursing as determined by their students. Nursing Research, 15(4), 218-224.
  • Kiker, P. J. (1973). Characteristics of the effective teacher. Nursing Outlook, 21(1), 721-723.
  • Meredith, G.M. (1983). Factor-specific items for appraisal of laboratory and seminar/discussion group experiences among college students. Perceptual and Motor Skills, 56, 133-134.
  • Miller, M. D. (1982). Factorial validity of a clinical teaching scale. Educational and Psychological Measurement, 42 1141-1147.
  • Nunnally, J.C. (1978). Psychometric Theory (2nd ed.). New York: McGraw-Hill.
  • O'Shea H., & Parsons, M. (1979). Clinical instruction: Effective and ineffective teacher behavior. Nursing Outlook, 27(6), 411-419.
  • Romberg, E. (1984). A factor analysis of students' ratings of clinical teaching. Journal of Dental Education, 48(5), 258-262.
  • Stuebbe, B. (1981). Student and faculty perspectives on the role of nursing instructors. Journal of Nursing Education, 7(9), 4-9.
  • Sullivan, M.J. (1985). Faculty evaluation: Our shackle or parachute? Nursing and Health Care, 6(8), 447-448.
  • Zimmerman, L., & Waltman, N. (1986). Effective clinical behaviors of faculty: A review of the literature. Nurse Educator, 11(1), 31-34.

TABLE

COMPOSITION OF THREE-FACTOR SOLUTION OF ECTB AND ALTERNATE FORMS

10.3928/0148-4834-19880601-08

Sign up to receive

Journal E-contents