Student ratings of satisfaction have very little relationship to
their performance in the final exam.
Boring et al 2016 analyze an interesting set of data on student
evaluations of teaching.
The data have some nice features for relating teaching effectiveness to
student ratings. There is one main professor for the course, who gives
the lectures and sets the final exam. Section instructors teach the
students in small groups of 10-24 students. In effect, the university
assigns the students randomly to section instructors. The university
uses various means to force the students to fill out evaluation
questionnaires, giving a response rate of "nearly 100%".
The group-average (see paper) correlation of the score for "What is your
overall level of satisfaction?" and the final year exam mark was r=0.04.
This was not significant using a carefully designed permutation test. On
the other hand, the correlation between the satisfaction score and the
instructor's gender was r=0.09, and highly significant, in favor of male
instructors.
Conclusion
In two very different universities and in a broad range of course topics, SET measure students’ gender biases better than they measure the instructor’s teaching effectiveness. Overall, SET disadvantage female instructors. There is no evidence that this is the exception rather than the rule. Hence, the onus should be on universities that rely on SET for employment decisions to provide convincing affirmative evidence that such reliance does not have disparate impact on women, underrepresented minorities, or other protected groups. Because the bias varies by course and institution, affirmative evidence needs to be specific to a given course in a given department in a given university. Absent such specific evidence, SET should not be used for personnel decisions.
https://www.scienceopen.com/document?vid=818d8ec0-5908-47d8-86b4-5dc38f04b23e
The Google+ URL for this post was
https://plus.google.com/+MatthewBrett/posts/TGKbxXruhqt