Journal of Nursing Education

RESEARCH BRIEFS 

A Pilot Study to Investigate the Impact of Interactional Television on Student Evaluation of Faculty Effectiveness

Susan J Fetzer, PhD, RN, BA, BSN, MSN, MBA

Abstract

The desire of the growing number of associate degree nurses based in rural areas to return for a baccalaureate degree and the needs of the profession to increase the number of baccalaureate prepared nurses requires the development and refinement of distance education delivery systems. Distance education takes the form of print, audio, computer, or video technology to create the virtual classroom. Of all distance education systems, interactive television (ITV) bears the closest resemblance to the traditional classroom. Through two-way video and two-way audio technology the teacher and learner see and hear each other in real time. The characteristic that distinguishes ITV education from other types of off-campus programs is the physical separation of the learner and the teacher. While numerous strategies have been promoted to mediate the difficulties created by this teacher-learner separation, the limited research available on student evaluation of faculty effectiveness suggests that the ITV distance education format may affect student perceptions. No empirical literature could be identified that described the effect on student evaluation of nursing faculty using ITV A pilot study was designed to answer the research question: Does the ITV delivery system impact nursing students' evaluation of teacher effectiveness?

LITERATURE REVIEW

Experts agree that distance education is a cost-effective method of providing instruction with comparable learning outcomes. Whittington's (1987) meta-analysis on over 100 studies identified that, regardless of the distance education delivery system, students received a comparable education. Billings and Bachmeier (1994) reached similar conclusions about costeffectiveness, learning outcomes, and teaching strategies in an integrative review of the nursing literature related to distance education.

There is a general consensus that traditional classroom teaching strategies must be adapted for ITV distance learning and an abundance of how-to literature is available (Boyd & Baker, 1987; Hegge, 1993; Parks & O'Shea, 1995). One of the key variables noted in nearly all of the distance education literature is the importance of faculty; however, research related to ITV and faculty issues is limited. Dillon and Walsh (1992) identified 225 articles related to distance education yet only 9 articles were specific to ITV, the majority of these describing faculty characteristics.

There is nearly unanimous opinion that teaching the distance learner is different from teaching the learner in the traditional classroom (Shaeffer & Farr, 1993; Shomaker, 1993). Experienced distance educators have offered valuable suggestions, methods, and strategies to bridge the physical separation between teacher and learner (Fulmer, Hazzard, Jones, & Keene, 1992; Parks & O'Shea, 1995; Shomaker, 1993). Teacher compensation and workload adjustment are frequently mentioned faculty concerns. Billings and colleagues (1994) reported that 11 of 15 nursing faculty queried indicated that their workload was greater for distance educators. Narrative comments of faculty in Scriven's (1986) study felt the ITV system of teaching was less rewarding, less scholarly, offered fewer career advantages, and lacked prestige among colleagues. The lack of external incentives supports the notion that faculty motivation to use ITV is intrinsic (Dillon & Walsh, 1992).

Three studies have described the effect of ITV on student perception of teaching effectiveness. Parkinson and Parkinson (1989) compared two groups of LPN students enrolled in an associate degree pathophysiology course. Though the same instructor taught both groups, the ITV students rated the instructor's effectiveness significantly lower on 6 of 7 criteria. The researchers noted that the ITV students considered the learning process impersonal and suggested that teacher friendliness and student support were compromised by the ITV format. However, methodological issues make these results difficult to interpret because teacher contact time was not equivalent between the groups (33.3 hours for traditional versus 10 hours for ITV).

Fulmer et al. (1992) queried students' opinion of overall…

The desire of the growing number of associate degree nurses based in rural areas to return for a baccalaureate degree and the needs of the profession to increase the number of baccalaureate prepared nurses requires the development and refinement of distance education delivery systems. Distance education takes the form of print, audio, computer, or video technology to create the virtual classroom. Of all distance education systems, interactive television (ITV) bears the closest resemblance to the traditional classroom. Through two-way video and two-way audio technology the teacher and learner see and hear each other in real time. The characteristic that distinguishes ITV education from other types of off-campus programs is the physical separation of the learner and the teacher. While numerous strategies have been promoted to mediate the difficulties created by this teacher-learner separation, the limited research available on student evaluation of faculty effectiveness suggests that the ITV distance education format may affect student perceptions. No empirical literature could be identified that described the effect on student evaluation of nursing faculty using ITV A pilot study was designed to answer the research question: Does the ITV delivery system impact nursing students' evaluation of teacher effectiveness?

LITERATURE REVIEW

Experts agree that distance education is a cost-effective method of providing instruction with comparable learning outcomes. Whittington's (1987) meta-analysis on over 100 studies identified that, regardless of the distance education delivery system, students received a comparable education. Billings and Bachmeier (1994) reached similar conclusions about costeffectiveness, learning outcomes, and teaching strategies in an integrative review of the nursing literature related to distance education.

There is a general consensus that traditional classroom teaching strategies must be adapted for ITV distance learning and an abundance of how-to literature is available (Boyd & Baker, 1987; Hegge, 1993; Parks & O'Shea, 1995). One of the key variables noted in nearly all of the distance education literature is the importance of faculty; however, research related to ITV and faculty issues is limited. Dillon and Walsh (1992) identified 225 articles related to distance education yet only 9 articles were specific to ITV, the majority of these describing faculty characteristics.

There is nearly unanimous opinion that teaching the distance learner is different from teaching the learner in the traditional classroom (Shaeffer & Farr, 1993; Shomaker, 1993). Experienced distance educators have offered valuable suggestions, methods, and strategies to bridge the physical separation between teacher and learner (Fulmer, Hazzard, Jones, & Keene, 1992; Parks & O'Shea, 1995; Shomaker, 1993). Teacher compensation and workload adjustment are frequently mentioned faculty concerns. Billings and colleagues (1994) reported that 11 of 15 nursing faculty queried indicated that their workload was greater for distance educators. Narrative comments of faculty in Scriven's (1986) study felt the ITV system of teaching was less rewarding, less scholarly, offered fewer career advantages, and lacked prestige among colleagues. The lack of external incentives supports the notion that faculty motivation to use ITV is intrinsic (Dillon & Walsh, 1992).

Three studies have described the effect of ITV on student perception of teaching effectiveness. Parkinson and Parkinson (1989) compared two groups of LPN students enrolled in an associate degree pathophysiology course. Though the same instructor taught both groups, the ITV students rated the instructor's effectiveness significantly lower on 6 of 7 criteria. The researchers noted that the ITV students considered the learning process impersonal and suggested that teacher friendliness and student support were compromised by the ITV format. However, methodological issues make these results difficult to interpret because teacher contact time was not equivalent between the groups (33.3 hours for traditional versus 10 hours for ITV).

Fulmer et al. (1992) queried students' opinion of overall instructor effectiveness dining a pharmacology class. While all students (24 traditional and 26 ITV) believed the teacher was effective, the researcher reported that ITV students were not as "strong in their opinion" (p. 292). A small sample size and a single evaluation criterion limit the application of these study findings.

Egan, Welch, Page, and Sebastian (1992) used an institutional teacher evaluation instrument for 154 traditional and 93 ITV learners in a teacher preparation course for baccalaureate students. Experienced, tenure track faculty taught both groups. Evaluation scores on content organization, clarity of content, and content relevant objectives were ranked higher by the traditional students than by the ITV students. However, a different instructor was responsible for each delivery system and variability of teaching style may have contributed to the differences between groups.

Table

TABLEStudent Evaluation of Teaching Scores [M (SD)] Using Traditional and ITV Systems

TABLE

Student Evaluation of Teaching Scores [M (SD)] Using Traditional and ITV Systems

The limited research available on student evaluation of faculty effectiveness suggests that the ITV distance education format may affect student perceptions. Previous research designs have not controlled for faculty, course content, or student-teacher contact time. There has not been any published work on student evaluation of faculty teaching nursing courses using ITV.

METHOD

The convenience sample (N = 114) consisted of diploma and associate degree prepared registered nurse students enrolled in a 2-credit nursing research course offered at an NLN accredited baccalaureate (BSN) program in the northeast. For the majority of student's this was the first course in the BSN curriculum and no student had prior course experience with the instructor. The traditional (TRAD) group (n = 61) participated in an on-campus research course during the fall semester and the ITV group (n = 53) participated in the same research course the following semester. The ITV group was enrolled at one of three sites; each supported by full-time support technicians and described as two-way audio and twoway full motion video. The same instructor taught both groups with identical studentto-faculty class contact hours. During the semester the faculty member rotated sites to be physically present for each ITV section. The needs and requests of each student determined faculty-student contact outside the classroom. The objectives, required text, syllabus, and outcome criteria were the same for both courses. During the final class each student anonymously completed the required institutional evaluation instrument, which was collated and reported as group data by the Office of Institutional Research.

INSTRUMENT

Traditionally, evaluation of faculty performance includes self-evaluation, expert review of teaching materials, peer review, and student evaluations. Self-evaluation and expert review have been criticized as subjective assessments (Bell, Miller, & Bell, 1984). Peer-review is episodic and subject to the Hawthorne effect. Student evaluations of faculty are the most common form of systematic evaluation and ara typically required by universities across all types of delivery systems (Bell et al., 1984). Student's ratings, according to Marsh (1984) are reliable, valid, and stable over time.

The instrument for this study was the Student Evaluation of Teaching (SET) questionnaire required by the investigator's institution and consists of 14 items using a 5-point Likert scale. Specific teaching qualities are reflected in 13 items and 1 item asks for an overall rating of the instructor. Anchors are designated as strongly agree (5) and strongly disagree (1) for each specific item. The anchors for the overall rating are excellent (5) and poor (1). While the SET has been used for over 15 years, reliability and validity have not been reported.

RESULTS

A retrospective analysis of students' evaluation of teaching compared the perceptions of the TRAD and ITV groups. Nursing students in the TRAD group rated teaching effectiveness significantly higher (P<.05) than RN students in the ITV group on 12 of the 13 SET items (Table). The TRAD group rated the instructor significantly higher on overall teaching effectiveness than did the ITV group. The difference between the TRAD and ITV groups on the criterion "available outside of class" was not significant.

DISCUSSION

The student evaluations of teaching using the traditional format in this pilot study were higher than average, indicating the faculty was proficient in the teaching role and course content. The findings suggest that when the same faculty teaches the same course to a similar population of students using ITV systems, there is a negative impact on student evaluation of teaching. The data support and extend the findings of previous researchers (Egan et al., 1992; Fulmer et al., 1992; Parkinson & Parkinson, 1989) who identified the difficulties posed by ITV format and suggested possible consequences on student evaluation of faculty effectiveness.

It is not surprising that there was no between the two groups on the criterion "available outside of class." Unlike the other criteria, availability outside of class is independent of the type of classroom or system used for teaching.

A limitation of this pilot study is the instrument used to evaluate teacher effectiveness. While most university evaluation methods are designed to promote selfimprovement and course modifications of classroom teaching, their validity in evaluating student perception of ITV teaching has not been established. However, in most settings, the required evaluation tool is applied and the results, regardless of teaching system, are weighed heavily in promotion and tenure decisions (Morton, 1987).

The results of this study can not be generalized beyond this group of RN baccalaureate students and the teaching performance of one nursing instructor. Further investigation is needed on student evaluation of faculty teaching other types of nursing courses at different educational levels. However, the findings of the study should be of concern to administrators assigning ITV courses to junior faculty. New, inexperienced traditional classroom faculty may be overwhelmed with the verbal and nonverbal communication obstacles faced in ITV systems. For less adept faculty, a possible decline in student evaluation scores from the traditional to ITV format can be devastating. Scriven (1986) suggested that the reason full professors were more satisfied with their distance education experience than lower ranking faculty was the positive effect of immunity granted by tenure.

Greater workload requirements and lack of financial and psychological recognition make ITV teaching less than appealing. If faculty are worried about student evaluations, the attractiveness of experimenting with the ITV format may be lost. Perhaps only tenured faculty with strong student evaluations should be assigned ITV courses. Nontenured or tenure-track faculty should have strong administrative support and understanding of the possible negative effects of ITV on student perceptions of teacher effectiveness before the course begins.

SUMMARY

While the dominant theme of distance education research has been the learner, learner achievement is not necessarily the only important consideration in assessing the effectiveness of ITV programs. With nearly half of the nursing programs recently surveyed by Reinert and Fryback (1997) offering or planning to offer distance learning in the near future, researchers and developers must seek to understand the impact of ITV on the teacher. These programs require faculty who are committed and prepared to teach outside of the traditional classroom. Selection of faculty requires attention to appropriate qualifications and knowledge of strategies to bridge teacher-learner separation created by distance education. Faculty, peers, and administrators should consider the teaching environment when interpreting data designed to evaluate teacher effectiveness of ITV teaching.

REFERENCES

  • Bell, D.F., Miller, R.I., & Bell, D.L. (1984). Faculty evaluation. Nurse Educator, 2, 1827.
  • Billings, D., Durham, J., Finke, L., Boland, D., Smith, S., & Manz, B. (1994). Faculty perceptions of teaching on television: One school's experience. Journal of Professional Nursing, 10, 307-312.
  • Billings, D., & Bachmeier, B. (1994). Teaching and learning at a distance: A review of the nursing literature. In L. Allen, (Ed.), Review of research in nursing education (pp. 1-32). New York: National League for Nursing.
  • Boyd, N.J., & Baker, CM. (1987). Using television to teach. Nursing and Health Care, 8, 523-527.
  • Dillon, CL., & Walsh, S.M. (1992). Faculty: The neglected resource in distance education. American Journal of Distance Education, 6(3), 5-21.
  • Egan, M.W., Welch, M., Page, B., & Sebastian, S. (1992). Learner's perceptions of instructional delivery systems: Conventional and television. American Journal of Distance Education, 6(2), 47-55.
  • Fulmer, J., Hazzard, M., Jones, S., & Keene, K. (1992). Distance learning: An innovative approach to nursing education. Journal of Professional Nursing, 8, 289-294.
  • Hegge, M. (1993). Interactive television presentation style and teaching materials. Journal of Continuing Education, 24, 39-42.
  • Marsh, H. W (1984). Students' evaluation of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11, 253-388.
  • Morton, RG. (1987). Student evaluation of teaching: Potential and limitations. Nursing Outlook, 35(2), 86-88.
  • Parkinson, CF, & Parkinson, S.B. (1989). A comparative study between interactive television and traditional lecture course offerings for nursing students. Nursing and Health Care, 10, 499-502.
  • Parks, RL., & O'Shea, K. (1995). Teaching nursing research using two-way video technology. Western Journal of Nursing Research, 17, 339-342.
  • Reinert, B.R., & Fryback RB. (1997). Distance learning and nursing education. Journal of Nursing Education, 36, 421-427.
  • Scriven, B. (1986). Staff attitudes to external studies. Media in Education Development, 19, 176-183.
  • Shaeffer, J.M., & Farr, CW (1993). Evaluation: A key piece in the distance education puzzle. Technological Horizons in Education Journal, 20, 79-82.
  • Shomaker, D. (1993). A statewide instructional television program via satellite for RN to BSN students. Journal of Professional Nursing, 9, 153-158.
  • Whittington, N. (1987). Is instructional television educationally effective? A research review. American Journal of Distance Learning, 1, 47-57.

TABLE

Student Evaluation of Teaching Scores [M (SD)] Using Traditional and ITV Systems

10.3928/0148-4834-20000201-12

Sign up to receive

Journal E-contents