By: Carla Guevara y Scott Stewart
Student end-of-course evaluations are widely used by colleges and universities to determine the success of courses and the effectiveness of their instructors. Schools often use such student feedback even to determine instructors’ pay and promotions, to identify instructors who need supplemental training, and to restructure coursework.
There is an extensive academic literature that looks at student evaluations. Many studies explore the practice of rating instructors according to their classroom behavior and to students’ performance on tests. Others delve into the influence of program design on the success of online classes. Yet studies of student evaluations rarely address what matters most–the long-term benefits to students once their educations are complete and they have moved into the working world.
That is despite the fact that the primary objective of education is for students to learn what will be useful to them after they graduate. That does not necessarily mean specific vocational skills; it can mean understanding of the economy and government, public speaking skills, or increased reasoning abilities.
It intuitively seems to be so.
Therefore, as teachers, administrators and researchers, we (the authors) set out to discover the things that graduating students and alumni considered to be the most valuable about their university courses. Our goal was twofold: first, to identify what students view as truly important for measuring courses once they have had time to reflect and put their knowledge into practice, and second, to determine which student survey questions best can forecast these metrics.
Our research was conducted over a nine-year period on a specialized master’s program in business offered at a large northeastern university and included questions regarding instructor performance and career relevance. We used multiple regression techniques to allocate statistical importance to students’ perceptions of learning, the quality of the instructor, and career relevance.
Traditional end-of-course surveys served as a control for the study. Typically, student surveys measure students’ perceptions regarding the classroom environment as well as the instructor’s ability to encourage learning. According to the available research literature, an instructor’s ranking is influenced by a number of factors–including the instructor’s apparent sense of humor. Interestingly, there is a positive correlation between instructor rankings and student perceptions of learning, but not actual learning.
Since we wanted to examine the differences in student opinions at different periods, we also used program exit surveys at graduation time. For one thing, this helped us identify which courses taken early in a college career are more helpful with subsequent coursework. We also used alumni surveys, as they permit former students to evaluate courses more objectively. With some passage of time after completing a program, alumni should be able to better evaluate the relationship between knowledge and skills acquired through an educational program and the ones required in the workplace.
After all three surveys were completed, we compared the results of our end-of-program and alumni surveys with the end-of-course evaluations.
The statistical results suggested that the most important factor for determining satisfaction at the end of a course was the perceived performance of the instructor. The perceived career relevance of a course was not a significant factor using student evaluation data. Statistically, there was a 56 percent chance that students felt that the course material’s relevance to their future careers wasn’t important at all.
However, this changed once students had time to reflect and learn what worked for them on the job. At the end of their program and as alumni, the importance of the instructor for determining course satisfaction declined and the value of career relevance grew to become the single most important factor for determining course satisfaction. In fact, in the regression on alumni course satisfaction, career relevance was 58 percent more influential than the performance of the instructor. A further test determined that students seemed to perceive what courses were relevant for their future careers, but that they did not value that relevance until they completed their education. The results are detailed in the June 2011 edition of the journal Managerial Finance.
While our study is unique in its analysis of data linking students’ perceptions over time, we only had a small data set at our disposal. Perhaps additional studies could be conducted to confirm our results.
Still, the implications are broad. One is that career relevance clearly grows with time in importance for determining course satisfaction. It is not a statistically significant factor for course satisfaction using end-of-course student survey responses, but grows in importance in the later surveys, surpassing both the extent of learning and instructor performance; moreover, instructor performance becomes less important with time.
These findings suggest that colleges and universities should not weigh too heavily their traditional measures of student satisfaction based on end-of-term course evaluations for measuring teaching effectiveness and for policy-making. However, this conclusion places instructors in an interesting position. If they want their students, once they become alumni, to feel satisfied with their course experience, the instructor needs to teach material which will be useful in the real world, even if students do not fully appreciate it during class and it therefore does not substantially help the instructor’s student evaluations.
The results indicate that both teacher performance and course content are important to graduating students and alumni. In order for instructors to be successful in the short and long term, they need to keep their students engaged in the classroom, teach material that is relevant post-graduation, and explain in class the relevance of the course material.
University presidents may also be interested in the results of this study. If presidents want satisfied alumni, they need to ensure their school’s curriculum includes material useful for the working world. Perhaps, one approach to ensuring a program’s success is to initiate a council of alumni, who can consult and provide advice on curriculum and how it is taught in the classroom.?
This article originally appeared at the Pope Center for Higher Education.
Evaluating Student-Faculty Evaluations
Student end-of-course evaluations are widely used by colleges and universities to determine the success of courses and the effectiveness of their instructors. Schools often use such student feedback even to determine instructors’ pay and promotions, to identify instructors who need supplemental training, and to restructure coursework.
There is an extensive academic literature that looks at student evaluations. Many studies explore the practice of rating instructors according to their classroom behavior and to students’ performance on tests. Others delve into the influence of program design on the success of online classes. Yet studies of student evaluations rarely address what matters most–the long-term benefits to students once their educations are complete and they have moved into the working world.
That is despite the fact that the primary objective of education is for students to learn what will be useful to them after they graduate. That does not necessarily mean specific vocational skills; it can mean understanding of the economy and government, public speaking skills, or increased reasoning abilities.
It intuitively seems to be so.
Therefore, as teachers, administrators and researchers, we (the authors) set out to discover the things that graduating students and alumni considered to be the most valuable about their university courses. Our goal was twofold: first, to identify what students view as truly important for measuring courses once they have had time to reflect and put their knowledge into practice, and second, to determine which student survey questions best can forecast these metrics.
Our research was conducted over a nine-year period on a specialized master’s program in business offered at a large northeastern university and included questions regarding instructor performance and career relevance. We used multiple regression techniques to allocate statistical importance to students’ perceptions of learning, the quality of the instructor, and career relevance.
Traditional end-of-course surveys served as a control for the study. Typically, student surveys measure students’ perceptions regarding the classroom environment as well as the instructor’s ability to encourage learning. According to the available research literature, an instructor’s ranking is influenced by a number of factors–including the instructor’s apparent sense of humor. Interestingly, there is a positive correlation between instructor rankings and student perceptions of learning, but not actual learning.
Since we wanted to examine the differences in student opinions at different periods, we also used program exit surveys at graduation time. For one thing, this helped us identify which courses taken early in a college career are more helpful with subsequent coursework. We also used alumni surveys, as they permit former students to evaluate courses more objectively. With some passage of time after completing a program, alumni should be able to better evaluate the relationship between knowledge and skills acquired through an educational program and the ones required in the workplace.
After all three surveys were completed, we compared the results of our end-of-program and alumni surveys with the end-of-course evaluations.
The statistical results suggested that the most important factor for determining satisfaction at the end of a course was the perceived performance of the instructor. The perceived career relevance of a course was not a significant factor using student evaluation data. Statistically, there was a 56 percent chance that students felt that the course material’s relevance to their future careers wasn’t important at all.
However, this changed once students had time to reflect and learn what worked for them on the job. At the end of their program and as alumni, the importance of the instructor for determining course satisfaction declined and the value of career relevance grew to become the single most important factor for determining course satisfaction. In fact, in the regression on alumni course satisfaction, career relevance was 58 percent more influential than the performance of the instructor. A further test determined that students seemed to perceive what courses were relevant for their future careers, but that they did not value that relevance until they completed their education. The results are detailed in the June 2011 edition of the journal Managerial Finance.
While our study is unique in its analysis of data linking students’ perceptions over time, we only had a small data set at our disposal. Perhaps additional studies could be conducted to confirm our results.
Still, the implications are broad. One is that career relevance clearly grows with time in importance for determining course satisfaction. It is not a statistically significant factor for course satisfaction using end-of-course student survey responses, but grows in importance in the later surveys, surpassing both the extent of learning and instructor performance; moreover, instructor performance becomes less important with time.
These findings suggest that colleges and universities should not weigh too heavily their traditional measures of student satisfaction based on end-of-term course evaluations for measuring teaching effectiveness and for policy-making. However, this conclusion places instructors in an interesting position. If they want their students, once they become alumni, to feel satisfied with their course experience, the instructor needs to teach material which will be useful in the real world, even if students do not fully appreciate it during class and it therefore does not substantially help the instructor’s student evaluations.
The results indicate that both teacher performance and course content are important to graduating students and alumni. In order for instructors to be successful in the short and long term, they need to keep their students engaged in the classroom, teach material that is relevant post-graduation, and explain in class the relevance of the course material.
University presidents may also be interested in the results of this study. If presidents want satisfied alumni, they need to ensure their school’s curriculum includes material useful for the working world. Perhaps, one approach to ensuring a program’s success is to initiate a council of alumni, who can consult and provide advice on curriculum and how it is taught in the classroom.?
This article originally appeared at the Pope Center for Higher Education.