Those in academics should be familiar with RateMyProfessor.com (RMP) by now -- the website that lets any student at a particular school rate any professor and the courses s/he teaches on that campus. As I’ve written about before, unlike many professors who vehemently denounce RMP, I don’t necessarily object to its existence, but neither do I care to log onto it and see what students say about me because I realized long ago that, in the words of Bill Cosby, “I don’t know what’s the key to success, but the key to failure is trying to please everybody.”

At any rate, one of the criticisms about RMP is that it since the students’ ratings for a particular professor are not a statistically representative sample, those ratings ultimately bear little similarity with the professor’s actual formal course evaluations. However, as Inside Higher Education reports, a new study argues that at least for good professors, there does seem to be a strong correlation between their ratings on RMP and their formal course evaluations:

The key findings are that RateMyProfessors.com ratings have a significant correlation with the formal student evaluations on the questions about the overall quality of the course and the relative difficulty or ease of the course. . . . At the same time, Coladarci [one of the study’s authors] cautioned that the correlation isn’t universally high.

The overlap is highest among those professors who are popular on RateMyProfessors.com — they also do extremely well with traditional student evaluations. “The pattern of this association suggests that when an instructor’s RMP overall quality is particularly high, one can infer that the instructor ‘truly’ is regarded as a laudatory teacher,” the study says. However, the correlations are much weaker for those who don’t score well, so Coladarci is much more hesitant to assume that poor RateMyProfessors.com ratings are equally meaningful.

In other words, professors who rate high on RMP also tend to rate high on their formal course evaluations. However, the correlation is much weaker for those professors who don’t rate so high on RMP -- in many cases, their formal course evaluations may also be low and in other cases they might be average or higher.

The article also notes that previous research has shown that on RMP, those professors who students judge as “hot” (i.e., physically attractive) or an “easy grader” are also much more likely to have a high overall course rating. However, the new study finds that there is no correlation between being judged as “hot” on RMP and that professor’s overall performance on formal course evaluations.

So I suppose there’s something for everyone here. RMP critics can claim victory that at least for professors who tend to score low on RMP, there is only weak evidence that their formal course evaluations may also be low. On the other hand, RMP supporters can crow about the finding that professors who score well on RMP also tend to score well on their formal course evaluations, lending more credibility to that particular aspect of the website.

I would now like to see a similar study on the correlations between students who gripe about professors on RMP and their actual grades. Now that would be quite interesting.