Journal of Nursing Education

The articles prior to January 2012 are part of the back file collection and are not available with a current paid subscription. To access the article, you may purchase it or purchase the complete back file collection here

Relationship Between College Success and Employer Competency Ratings for Graduates of a Baccalaureate Nursing Program

Sharon E Bolin, RN, MS; Edith L Hogle, RN, MS

Abstract

ABSTRACT

This expost facto correlational study sought to determine which measures of academic success in one class of BSN graduates predicted their competence as employees one year after graduation, as judged by their employers. The relationship between pre-entrance test scores, clinical experience grades, GPA, State Board Test Pool examination scores, and employer competency ratings were also determined. In keeping with the literature in fields other than nursing, the findings suggest that there may be little relationship between academic performance in a nursing program and subsequent job performance as a nurse, even though verbal ability may be predictive of success in school. While significant positive correlations were found between pre-entrance test data and final grade point averages, as well as pre-entrance test scores and State Board Test Pool examination scores, there was little evidence that preentrance test scores were predictive of nursing abilities. Isolated correlations were found between the clinical components of some nursing courses and specific nursing abilities. Using multiple regression analysis, no clinical course grade was found to be a significant predictor of the mean employer competency rating. Significant predictors were found for only four of the individual nursing abilities, with the clinical component of Leadership in Nursing being the most frequent and best predictor.

Abstract

ABSTRACT

This expost facto correlational study sought to determine which measures of academic success in one class of BSN graduates predicted their competence as employees one year after graduation, as judged by their employers. The relationship between pre-entrance test scores, clinical experience grades, GPA, State Board Test Pool examination scores, and employer competency ratings were also determined. In keeping with the literature in fields other than nursing, the findings suggest that there may be little relationship between academic performance in a nursing program and subsequent job performance as a nurse, even though verbal ability may be predictive of success in school. While significant positive correlations were found between pre-entrance test data and final grade point averages, as well as pre-entrance test scores and State Board Test Pool examination scores, there was little evidence that preentrance test scores were predictive of nursing abilities. Isolated correlations were found between the clinical components of some nursing courses and specific nursing abilities. Using multiple regression analysis, no clinical course grade was found to be a significant predictor of the mean employer competency rating. Significant predictors were found for only four of the individual nursing abilities, with the clinical component of Leadership in Nursing being the most frequent and best predictor.

In an attempt to determine how graduates of nursing schools perform in the work situation in the real world, many schools of nursing conduct graduate follow-up studies at some designated point in time following graduation. In addition to collecting data from the nurse herself, job evaluations, based on the school's terminal objectives, are elicited from employers.

The literature is not encouraging when one looks at studies in many fields which concern themselves with academic performance, and subsequent job performance. Olson (1977) argues that teaching the verbal representations of a practical activity has as its function the teaching of information, but "the disadvantage of frequently not transferring back to the practical activity" (p. 78). One may know the theory but not be able to apply it. Olson further states that:

. . . translating complex practical activities into verbal formulas has the effect of removing the correlation between succeeding in learning in the instructional program and success in learning the practical performance. That is, people who learn the lists well are not necessarily the ones who perform the activity well. There is a gap between theory and practice (p. 79).

Olson's position is corroborated by Hoyt (1965) in a metaanalysis of 46 studies which reviews research on the relationship between college grades and adult achievement in the areas of business, teaching, engineering, scientific research, miscellaneous occupations, studies of eminence, and non-vocational accomplishments. His conclusion is that present evidence strongly suggests that college grades bear little or no relationship to measures of later accomplishment.

Hoyt (1965) notes that there is not necessarily a relationship between what a person knows and what he does with his knowledge, and that "there is good reason for believing that academic achievement (knowledge) and other types of student growth are relatively independent of each other" (p. 48). This is perhaps the "knowing with" that Broudy (1977) discusses, and about which little is understood.

In terms of selection of students for professional education, Hoyt (1965) suggests that there is little to support the practice of using a high grade point average cut-off before considering other characteristics which might indicate a student's potential for making a professional contribution. It is these characteristics which need to be determined. In summarizing Hoyt's work, Olson (1977) states that "verbal IQ predicts performance in the training program, but performance in the training program does not predict performance on the job" (p. 78).

In directing attention toward studies in nursing which have to do with successful nursing performance, a study conducted by Schwirian (1977), and funded by the Department of Health, Education, and Welfare, reviewed the research related to the prediction of successful nursing performance between 1965 and 1975. The findings suggested that on-the-job performance "seemed to be associated with what might be called a 'mature personality' structure, i.e., lower susceptibility to stress, openness to input from others, flexibility, relatively self-actualized, selfassured, etc." (p. 12).

In addressing illusions of learner accomplishment, Caulley and Grotelueschen (1978) suggest that we "take that news too seriously if we assume that indices of accomplishment in school tasks relate directly to life tasks" because "little is directly replicated from a school situation to a subsequent adult life context" (p. 282).

The foregoing data led the authors to question what, if any, relationships might be found between measures of academic success in college, and graduate nurse competence on the job as described by employers. In order to test the assumption that verbal ability predicts success in school, but that success in school does not predict performance on the job, four issues were explored. These were the relationships between:

1. pre-entrance test scores and final grade point average;

2. pre-entrance test scores and State Board Test Pool examination scores;

3. pre-entrance test scors and employer competency ratings; and

4. clinical experience grades in nursing courses and employer competency ratings.

The subjects in this study were graduates of a baccalaureate nursing program located in a small liberal arts college, who had all passed the State Board Test Pool examination and were licensed to practice nursing. All were women in their twenties and had been out of school for one year when this study was conducted. They were those graduates whose employers had returned the questionnaire given to them by the graduates as a part of a one-year follow-up of graduates conducted by their educational institution. Of the 47 questionnaires sent to the graduates who were responsible for giving them to their employer, 18 were returned.

The data for this study were documents in the subject's academic file and the questionnaire returned by the employer. The questionnaire completed by the employer was constructed by the nursing faculty of the school in which the study was conducted. It included the following twelve nursing abilities and the employer was asked to measure each ability on a five-point scale:

1. nursing knowledge;

2. independent decision making;

3. organizational skills in nursing practice;

4. patient and family teaching;

5. evidence of critical thinking;

6. interpersonal skills with clients;

7. psychomotor skills;

8. ability as a change agent;

9. leadership ability;

10. ability to get along with co-workers;

11. ability to accept criticism; and

12. ability to function well with an average amount of orientation.

For those subjects whose employers returned the evaluation questionnaires, the following information was obtained from their school records: ACT scores including English, Math, Social Science, Natural Science, and Composite; SAT Verbal and Math scores; clinical experience grades for the junior level courses Maternal Health Nursing, Child Health Nursing, Mental Health Nursing, Adult Health Nursing I and the senior level courses Adult Health Nursing II (separate grades for acute and nonacute components), Community Health Nursing, and Leadership in Nursing; overall grade point average, including general education and nursing courses; grade point average for nursing courses only; scores on the State Board Test Pool examination, including Medical Nursing, Surgical Nursing, Obstetric Nursing, Pediatric Nursing, and Psychiatric Nursing. For missing data for any of the above variables, the subject was given the mean score computed from the complete data on that variable (Borg and Gall, 1979). From this data a mean State Board Test Pool examination score and a mean clinical experience grade were constructed as additional variables.

The rating given by the employer to each of the subjects on each of the 12 nursing abilities was obtained from the questionnaire returned by the employer. For any missing data, the subject was given the mean score on the particular nursing ability computed from the complete data on that nursing ability. From this data a new variable, mean employer competency score, was constructed.

In addressing the question of the relationship between pre-entrance test scores and final grade point averages, Spearman rank-order correlations were performed. All statistical operations were computer-performed, using the Statistical Package for the Social Sciences (SPSS) (Nie, Hull, Jenkins, Steinbrenner and Bent, 1975). Correlations were obtained between students' final grade point averages and ACT composite and subscores (English, Math, Social Science and Natural Science), and between their final grade point averages and SAT Verbal and SAT Math scores. Significant positive correlations at the .01 level were found between the final grade point average and the composite ACT; final grade point average and English and Math ACT subscores; and final grade point average and SAT Math score.

The relationship between pre-entrance test scores and State Board Test Pool examination scores was tested by finding Pearson correlation coefficients. Scores on each of the State Board Test Pool examinations in Medical Nursing, Surgical Nursing, Obstetric Nursing, Pediatric Nursing, and Psychiatric Nursing, as well as the constructed mean score were correlated with the ACT composite score, each of the ACT subscores, the SAT Verbal score and the SAT Math score. Of a possible 42 corrélations, 21 were positive and significant. The mean State Board Test Pool examination score was found to be significantly correlated with the ACT composite score, three of the ACT subscores (English, Social Science, and Natural Science), the SAT Verbal score, and the SAT Math score. No significant relationship was found between Obstetric Nursing or Psychiatric Nursing and any of the pre-entrance test scores. Other findings revealed that the ACT composite score correlated significantly and positively with Medical Nursing, Surgical Nursing, and Pediatric Nursing; the ACT English subscore with Medical Nursing, Surgical Nursing and Pediatric Nursing; the ACT Math subscore with Surgical Nursing and Pediatric Nursing; the ACT Social Science subscore with Medical Nursing; the ACT Natural Science subscore with Medical Nursing and Pediatric Nursing; the SAT Verbal score with Pediatric Nursing; and the SAT Math score with Medical Nursing, Surgical Nursing, and Pediatric Nursing.

Table

TABLE 1KENDALL RANK ORDER CORRELATIONS BETWEEN PRE-ENTRANCE TEST SCORES AND EMPLOYER COMPETENCY RATINGS*

TABLE 1

KENDALL RANK ORDER CORRELATIONS BETWEEN PRE-ENTRANCE TEST SCORES AND EMPLOYER COMPETENCY RATINGS*

In looking at the relationship between pre-entrance test scores and employer competency ratings, Kendall rankorder correlations were performed between the subjects' pre-entrance test scores and each of the twelve nursing abilities on the employer competency rating, as well as the mean employer competency constructed variable. Of 91 correlations, thirteen (14%) were found to be significant (Table 1). The mean employer competency rating was found to be significantly positively correlated with the composite ACT score and the English ACT subscores.

When each nursing ability on the employer questionnaire was correlated with the pre-entrance test scores, no significant relationship was found between nursing abilities 1, 2, 4, 5, 6, 7, or 11 and any of the pre-entrance test scores (Table 1 ). A significant positive correlation was found between the ACT English subscore and nursing abilities 3 (organization), 9 (leadership), 10 (interpersonal relationship with co-workers), and 12 (average orientation). The composite ACT score was found to be significantly positively correlated with nursing abilities 3, 10, and 12. The remaining significant positive correlations were the ACT Social Science subscore with nursing abilities 3 and 12, and SAT Verbal subscore with nursing ability 3. A significant negative correlation was found between ACT Math subscore and nursing ability 8 (change agent). No significant relationship was found between the SAT Math subscore and the mean employer competency rating or any of the twelve nursing abilities on the employer questionnaire. Stepwise multiple regression was performed to determine which, if any, of the pre-entrance tests would predict the mean employer competency rating. No significant predictors were found.

The Kendall rank-order correlation procedure was used to locate any relationships between the employer competency ratings and each nursing course clinical grade (Maternal Health Nursing, Child Health Nursing, Mental Health Nursing, Adult Health Nursing I, Adult Health Nursing II Acute, Adult Helth Nursing II Nonacute, Leadership in Nursing, and Community Health Nursing) as well as the mean nursing course clinical grade. Of a possible 117 significant correlations, only eighteen (15%) were found (Table 2). The mean employer competency rating was found to be significantly positively correlated with two courses, Adult Health Nursing I and Leadership in Nursing. The mean nursing course clinical grade was found to be significantly positively correlated only with nursing ability 4 (teaching). There was no significant correlation between the mean employer competency rating and the mean nursing course clinical grade.

In correlating each of the nursing abilities on the employer questionnaires with the nursing course clinical grades, no significant relationships were found between nursing abilities 3, 6, 8, 9, 10, or 11 and any of the nursing course clinical grades (Table 2). No significant relationship was found between Mental Health Nursing, Adult Health Nursing II Acute, and Community Health Nursing clinical course grades and any of the employer competency ratings. Leadership in Nursing clinical course grade was found to be significantly positively correlated with nursing abilities 1 (nursing knowledge), 2 (independent nursing decisions), 4 (teaching), 5 (critical thinking), 7 (psychomotor skills), and 12 (average orientation). Adult Health Nursing I clinical course grade was found to be significantly positively correlated with these same nursing abilities with the exception of nursing ability 4. Adult Health Nursing II Nonacute clinical course grade was significantly positively correlated with nursing ability 12 (average orientation) and 2 (independent nursing decisions). The only remaining significant positive correlations were between Maternal Halth Nursing and nursing ability 4 (teaching) and between Child Health Nursing and nursing ability 4.

Stepwise multiple regression was performed to determine which, if any, of the nursing clinical course grades might be significant predictors of the mean employer competency rating, as well as of each nursing ability on the employer questionnaire. Table 3 reveals that only four of the nursing abilities had significant clinical experience course grade predictors. The mean employer competency rating had no significant predictors. The best predictor for nursing ability 2 (independent nursing decisions) was Leadership in Nursing. Maternal Health Nursing was the best predictor for nursing ability 4 (teaching). The best predictors (in descending order) for nursing ability 5 (critical thinking) were Leadership in Nursing, Adult Health Nursing II Acute, and Adult Health Nursing I. Leadership in Nursing and Adult Health Nursing II Nonacute were the best predictors for nursing activity 7 (psychomotor skills). Leadership in Nursing was the best predictor for three of the four nursing abilities with significant predictors. Maternal Health Nursing was the best predictor for one nursing ability and second best for another of the four nursing abilities with significant predictors.

The results of this study tend to corroborate previous research in both nursing and other fields which indicates that college grades bear little or no relationship to measures of later accomplishment, even though verbal ability may predict success in school. While significant positive correlations were found between pre-entrance test data and final grade point average, and pre-entrance test scores and State Board Test Pool examination scores, there was little evidence that pre-entrance test scores were predictive of nursing abilities, as judged by employers who rated the subjects one year following graduation from their school of nursing.

Olson (1977) refers to the gap between theory and practice, noting that teaching verbal representations of a practical activity does not assure transfer to the practical activity itself. The clinical experience of the student was most likely not a match for the real situation in the work world. This concept is supported by Gagne and Beard (1978) in their distinction between proximate and ultimate goals. In the case of this study, the proximate goals were represented by course grades and the ultimate goals were represented by employer ratings. Gagne and Beard (1978) point out that ". . . the proximate objectives of a course of study are those that can reasonably and feasibly be measured at the end of some period of instruction. They are related to ultimate goals by some process of rational analysis or, more rarely, on the basis of empirical evidence" (p. 273).

Gagne and Beard (1978) further indicate that the cues in the test situation are usually considerably different from the real situation, and that "the responses required for assessment may also differ from those of the real situation" (p. 275). In the case of this study, ". . . the location, the people present, the extraneous noise, the mental set of the learner . . ." (p. 275) were different between the clinical learning situation of the nursing student and the work world of the employed nurse.

Table

TABLE 2KENDALL RANK ORDER CORRELATIONS BETWEEN EMPLOYER COMPETENCY RATINGS AND NURSING COURSE CLINICAL GRADES

TABLE 2

KENDALL RANK ORDER CORRELATIONS BETWEEN EMPLOYER COMPETENCY RATINGS AND NURSING COURSE CLINICAL GRADES

Replication of the study with a larger sample might result in a greater range of the dependent variable, the employer competency ratings, and might result in revealing significant predictor variables. Use of subjects with less clinical experience hours per week might also yield a greater range in both the nursing course clinical grades and the employer competency ratings. Use of subjects with failures on the State Board Test Pool examinations would also provide for greater range in this variable and might result or be reflected in greater range for other variables used in the study.

Because the year of experience in the work situation may be acting as a treatment variable, it would be instructive to have a representative of the staff who worked with the subject during the Leadership in Nursing course evaluate the subject on the twelve nursing activities at the completion of this experience. This rating on the nursing activities which comprise the employer competency ratings could then be correlated with the employer competency ratings at the end of one year of experience for the subject. A lack of correlation would tend to validate the treatment effect of the year of experience.

Table

TABLE 3NURSING ABILITIES ON EMPLOYER COMPETENCY QUESTIONNAIRE PREDICTED BY NURSING CLINICAL COURSE GRADES. USING STEPWISE MULTIPLE REGRESSION*

TABLE 3

NURSING ABILITIES ON EMPLOYER COMPETENCY QUESTIONNAIRE PREDICTED BY NURSING CLINICAL COURSE GRADES. USING STEPWISE MULTIPLE REGRESSION*

References

  • Borg, W.R., & Gall, M.D. (1979). Educational research (3rd ed.). New York: Longman.
  • Broudy, H. S. (1977). Types of knowledge and purposes of education. In R.C. Anderson (Ed). Schooling and the acquisition of knowledge. New York: John Wiley & Sons.
  • Caulley, D.N. & Grotelueschen, A.D. (1978). The illusions of learner accomplishment. Educational Leadership, 35(4), 280-283.
  • Gagne, R.M., & Beard, J.G. (1978). Assessment of earning outcomes. In R. Glaser (Ed.) Advances in instructional psychology (Vol. 1). Hillside, NJ: Lawrence Erlbaum.
  • Hoyt, D. R (1965). The relationship between college grades and adult achievement: a review of the literature. American College Testing Program Research Reports, pp. 1-58.
  • Nie, N.H., Hull, C.H., Jenkins, JG., Steinbrenner, K., & Bent, D.H. (1975). Statistical package for the social sciences (2nd ed. ) New York: McGraw Hill.
  • Olson, D. R. (1977). The languages of instruction: the literate bias of schooling. In R.C. Anderson (Ed. ) Schooling and the acquisition of knowledge. New York: John Wiley & Sons.
  • Schwirian, P.M. (1977). Prediction of successful nursing performance, Parts I & II. Washington, DC: U.S. Department of Health, Education and Welfare.

TABLE 1

KENDALL RANK ORDER CORRELATIONS BETWEEN PRE-ENTRANCE TEST SCORES AND EMPLOYER COMPETENCY RATINGS*

TABLE 2

KENDALL RANK ORDER CORRELATIONS BETWEEN EMPLOYER COMPETENCY RATINGS AND NURSING COURSE CLINICAL GRADES

TABLE 3

NURSING ABILITIES ON EMPLOYER COMPETENCY QUESTIONNAIRE PREDICTED BY NURSING CLINICAL COURSE GRADES. USING STEPWISE MULTIPLE REGRESSION*

10.3928/0148-4834-19840101-05

Sign up to receive

Journal E-contents