RateMyProfessor.com is an Internet site “where students do the grading.” College students are comfortable sharing all kinds of information online, and this site has attracted millions of them. The site is designed to be inviting and easy to use, but its ease of use also raises questions about the results. Anyone with access to the Internet can rate any professor. With a little extra effort, someone could rate the same professor more than once, and a student's assessment may be influenced by ratings and comments already on the site. Each professor's summary rating aggregates ratings across courses and across years. Some professors have not yet been rated, and some have few ratings thus far. Thus, if you wanted to dismiss this entire enterprise, you wouldn't have to look far to find fault.

Still, we should not ignore what this site has to offer simply because these evaluations have not been supervised by someone in authority and delivered to us in sealed manila envelopes. Where else can you read what six million students think about hundreds of thousands of professors? Where else can you find answers to some questions that traditional evaluations ignore, such as how easy or how attractive a professor is? And where else can you find teacher ratings searchable in a variety of useful ways? Advertisers on the site, such as Citibank and Discover Card, apparently find the site worth their investment. Taken with a grain of salt, RateMyProfessor.com may hold lessons for us all. I have taken a look at the site and offer some summary statistics, mostly about economists. Later I quote student comments indicating what they liked and disliked about their economics professors.

First some background. Ratings on the site are based on a five-point scale from 1, the worst, to 5, the best. To generate the key measure reported on the site, students are asked to rate professors in two separate categories: “helpfulness” and “clarity.” These two ratings are then averaged to get a student's assessment of the professor's “overall quality.” This measure is averaged for all raters over time and across courses to yield the overall quality for that professor. Professors in the 3.5-to-5.0 range are identified by the site as “good,” those in the 2.5-to-3.4 range are “average,” and those in the 1.0-to-2.4 range are “poor.”

As a crude check on the validity of this overall quality measure, I focused on professors who won the most prestigious teaching award at the University of Connecticut . Of the 14 winners who were also evaluated on RateMyProfessor.com, 12, or 86%, were rated as good. One award winner, or 7%, was rated average, and one winner, or 7%, was rated poor. How does this distribution compare with all rated faculty at UConn? Among the 680 UConn faculty rated on the site, 58% were rated good, 22% were average and 20% were poor. Compared to award-winning teachers at UConn, the typical faculty member was about three times more likely to be rated average or poor. Thus, there is at least some rough association between the online ratings and one independent measure of teacher quality.

To examine overall quality ratings for a cross-section of higher-education institutions, I selected a sample consisting of all rated teachers at 32 schools, sorted into four groups: the eight Ivies (Brown, Columbia, Cornell, Dartmouth, Harvard, Penn, Princeton, and Yale); eight elite liberal arts colleges (Amherst, Holy Cross, Pomona, Oberlin, Swarthmore, Wellesley, Wesleyan, and Williams); eight large public universities (UCLA, Berkeley, UConn, Florida, Michigan, Michigan State, Texas, and Virginia); and eight large community colleges (Glendale, Houston, Macomb, Mesa, Miami Dade, Monroe, Northern Virginia, and Portland).

The entire 32-school sample includes 22,102 teachers evaluated on RateMyProfessor.com. The average school in the sample had 691 rated teachers. The ratings are cumulative through the first week of September 2006, with the first student entries in a few cases dating back to 2001. I computed the distribution of good, average, and poor ratings by school, then averaged these to get the average distribution for the entire sample. The 32-school profile shows 66% were rated good, 18% average, and 16% poor. With two thirds of faculty rated good, students do not seem to be particularly tough graders. That variation of the Lake Wobegon effect, where most are above average, also holds for many college-sponsored ratings.

Faculty members at the eight elite colleges rated the highest among the four types of institutions. At elite colleges, 72% were rated as good, compared to 65% at community colleges, 64% at the Ivies, and 63% at public universities. And only 9% at elite colleges were rated poor, compared to 16% at community colleges, 18% at public universities, and 19% at the Ivies.

The sample includes a total of 621 economics teachers evaluated on the site, or an average of 19.4 economists per institution. How do the overall quality ratings of economists compare with those of all faculty? Among the 32 schools surveyed, 56% of economists were rated good, 26% average, and 18% poor. Thus, when compared to faculty more generally, a smaller share of economists was rated good (56% vs. 66%) and a larger share was rated average (26% vs. 18%) or poor (18% vs. 16%).

Economists at elite colleges had higher average ratings than economists at other institutions in the sample. For example, 70% of elite-college economists were rated good, versus 56% at community colleges, 51% at public universities, and 48% at the Ivies. And only 8% of economists at elite colleges were rated poor, versus 20% at the Ivies, 21% at community colleges, and 23% at public universities.

Drilling down to the college level, I found that at only 6 of the 32 institutions did the share of economists rated good exceed the share of all faculty at that institution rated good. This list included 1 of 8 Ivies (Yale), 0 of 8 public universities, 2 of 8 elite colleges (Holy Cross and Wellesley), and 3 of 8 community colleges ( Glendale , Monroe , and Northern Virginia ).

As points of reference, I identified the highest rated and lowest rated economists at each of the 32 institutions. The highest rated economists had an overall quality average of 4.6 out of 5.0. The lowest rated economists averaged 1.9 out of 5.0. Among the highest rated economists at each institution, those at elite colleges had the highest average at 4.8. Public universities and community colleges averaged 4.6, and the Ivies, 4.2. Among the lowest rated economists at each institution, elite colleges again had the highest average at 2.3. Averages at other institutions were tightly bunched between 1.6 and 1.7.

Students at RateMyProfessor.com are also asked to rate the "easiness" of the professor based on a 5 point scale, with 1 being ‘hard” and 5 “easy.” Is there any relation between overall quality and easiness? Economists rated highest in overall quality at each of the 32 institutions had an average easiness rating of 3.4, with a range of 2.2 to 4.3. Economists rated lowest in overall quality had an average easiness rating of 2.2, with a range from 1.2 to 3.3. Thus, professors rated highest in overall quality were judged to be easier than those rated lowest in overall quality. Or perhaps the causality runs the other way: Easier professors were judged to be of higher overall quality. As one student commented, the professor “ Gave lots of examples, made econ easier than I thought it would be.”

In an optional category that RateMyProfessor.com says is “just for fun,” students are asked to rate the professor's “appearance.” Students who respond have only two choices: “hot” or “not,” with no gradations in between. A professor's “hotness” rating is based on the sum of the "hot" votes and the "not" votes, where “hot” is +1 and “not” is -1. If the sum is positive, a red chili pepper identifies the professor's appearance as “hot.” If the sum is zero or negative, no chili pepper or any other identifier appears.

This hotness rating may have some relevance, given the possible role that an instructor's appearance may play in teaching evaluations (as discussed in “Beauty Contest,” The Teaching Economist , Issue 25, Fall 2003). First, I determined the average degree of “hotness” at the sample institutions. On average, 31% of all rated faculty were judged to be hot. Among economists, 23% were hot. So faculty on average were hotter than economists—about one third hotter in this sample.

Although faculty averaged hotter than economists at each type of institution, the faculty advantage was smallest at elite colleges, where 36% of the faculty were hot versus 33% of economists. Elite-college economists were also hotter than economists at other types of institutions (33% vs. 20%). The least hot economists were at community colleges, where only 14% were judged hot, versus 27% for all rated faculty at community colleges and 27% for economists at other types of institutions.

At the school level, economists were hotter than the respective faculty at 9 of the 32 institutions and cooler at 23. Economists were hotter at 5 of 8 Ivy League institutions ( Columbia , Cornell, Dartmouth , Penn, and Yale), 3 of 8 elite colleges (Amherst, Wellesley, and Wesleyan), 1 of 8 public universities (Berkeley), and 0 of 8 community colleges.

Is there a link between overall quality and appearance? Specifically, were the economists rated highest in overall quality at each institution more likely to be judged hot than economists rated lowest in overall quality? Of the 32 economists rated highest in overall quality, 16 were also judged to be hot. Among the 32 lowest rated economists in overall quality, only two were judged to be hot. Thus, 50% of the highest rated economists were rated hot versus only 6% of the lowest rated economists.

Other evidence suggests a link between hotness and overall quality. For the sample institutions, the correlation between the share of economists judged to be hot and the share rated good is 0.594. For all rated faculty at the sample institutions, it's 0.704. Both correlations are significant at the 0.001 level. One final linkage: RateMyProfessor.com lists the “Top 50 Hottest Professors” in the United States and Canada . Only one of these 50 was rated less than good—the 46 th ranked Hottie missed the good range by only 0.1.

In summary, faculty in the sample tended to be rated higher in “overall quality” and ‘hotter” than economists. Among different types of institutions, elite colleges had a highest share of economists rated good and a smallest share rated poor. The highest rated economists at each institution were judged on average to be easier than the lowest rated economists. The highest rated economists were also more likely to be judged as hot. Perhaps the most valuable lesson RateMyProfossor.com has to offer comes from student comments. These are next.

THE GRAPEVINE

For fifteen years The Grapevine has passed along teaching ideas from across the country and around the world—more than 150 items to date. For a change of perspective, let's give students their say. RateMyProfessor.com allows students to supplement their ratings with comments on the professor and the course. Here, I draw from hundreds of student comments to quote those that I found most instructive. I was less interested in relative remarks such as “best prof ever” or “worst prof ever,” and more focused on those teaching qualities that students found good or bad. All quotes are from students commenting on economists teaching at the sample schools. Although I offer an equal number of quotes identifying good and bad qualities, keep in mind that most comments on the site are positive.

The Good….

“Lectures are well organized and thorough. Sense of humor, good storyteller. Expects students to know a lot, but he teaches it all, and very well.”

“Very clear and concise. Dry humor is nice addition. Great lecturer—weeds through the BS to the heart of the material. Straightforward questions on problem sets and midterm.”

“Genuine, curious, funny, interestingly odd, intelligent conversationalist. Really thoughtful and prepared.”

“Amazing, clear, punctual, HOT.”

“If you look up the term “open-door policy,” you would see [her] face there. One-on-one attention is an absolute skill she has mastered. Extremely lucid lectures.”

“Such a wonderful professor. He's incredibly approachable and always available for office hours. He has definitely challenged me to look at econ in a different way.”

“He really does want his students to succeed, and if you have a problem with homework, he'll email you right back if you ask him.”

“He gets to know every student, and his explanations are really helpful. He cares that every student understands the difficult material.”

“She's clear, concise, and incredibly intelligent.”

“He cares so much about his students that he gives you his cell phone number to call if you need any help.”

“Explained everything thoroughly. Lots of great examples. Simplified situations but still showed the concept very well.”

“He actually makes me look forward to an economics class. I don't think I have ever laughed so hard at times. Engaging, intelligent, and focused, he is sure to please with his sure fire way of teaching. If you don't pay attention though, you'll be screwed.”

“Extremely organized, very helpful, and absolutely clear.”

“You miss a day, you miss a ton.”

“He will make you love macro, and you will want him to teach you again.”

And the Bad…

“Destroyed my interest in economics. Proves that maybe the policy isn't right, not all professors should have to teach undergrads.”

“The homework was nothing like the material covered in lectures, and the midterm/final wasn't anything like the lecture or the homework.”

“The first day of lecture, [he] tells you that he doesn't want you to come to his office hours. It's all downhill from there as he speaks his mumbo-jumbo.”

“Totally confusing. Only 10% of the class would turn up, and half would fall asleep. Doesn't like undergrads.”

“Constantly approval-seeking, generally disorganized, useless overall. Gives very weak and disorganized explanations littered with inappropriate political rhetoric.”

“Spends all lecture beating around the bush trying to explain one simple topic, and ultimately fails miserably.”

“Why is this woman still teaching? She doesn't like it, she doesn't like you, and she might not even like economics. One thing is for sure and that is the fact that she doesn't want to make this experience enjoyable for anyone.”

“He lectures about everything but micro; you'll know more about his life story than you will about the micro.”

“He lectured on the easy topics in the chapter that are pretty much common sense and left the rest for us to figure out ourselves.”

“He went so far as to make fun of my name because I drew something wrong. That was low. Tells people to leave egos out of the room yet can't leave his own.”

“Where to start? She can't explain concepts, scribbles illegible gibberish on the blackboard, gets mad at the students when they don't understand concepts (and blames them for not understanding), is late to classes (even on testing days), her tests are misnumbered and unclear and in fragmented English.”

“Arrogant, boring, impossible to know what he wants on tests and papers.”

“ Doesn't teach and won't answer questions.”

“His answer to everything: ‘Google it!'”

“Disorganized, unclear. Stood at the board with chalk in one hand, eraser in the other, waiting to wipe off his long-winded way of getting to an incorrect solution to a simple problem.”

Odds and Ends

In the hundreds of student comments that I read at RateMyProfessor.com, I found no good words about the use of overheads or PowerPoint ® slides in class. I wasn't even thinking about this topic until I noticed an accumulation of negative comments. Here's what I mean:

“He is amazed by PowerPoint and will prove this to you every single class.”

“He puts too much information on slides and goes so fast you can't copy anything down, and gets mad if you ask him to slow down.”

“All she does is read the damn overhead, if there is no overhead, she probably couldn't even teach.”

“Very hard to understand, and lectures are nothing more than him reading the textbook's PowerPoint slides and generally repeating them in slightly different language for about 2 hours.”

“If anyone has questions, he tells you to go to WebCT and download the PowerPoint slides! Absolutely no help to students at all!”

“PowerPoint slides are ineffective teaching tools.”

“She just reads off her PowerPoints WORD FOR WORD.”

“Reads long PowerPoint slides from the book and rushes through them giving you no time to copy.”

“Not only is this guy pretending to teach by reading slides, he'll even stray you away from the whole major. In those rare occasions in which he uses the chalkboard to explain a problem, you can see a glimpse of his teaching potential. Too bad, as he always returns to his notorious slides.”

The Spring 2006 issue of The Teaching Economist offered a fairly positive assessment of economic coverage at Wikipedia ( www.wikipedia.org ). A study in the journal Nature has found few differences in the accuracy of science entries between Wikipedia and Encyclopaedia Britannica ( www.nature.com/nature/journal/v438/n7070/full/438900a.html ). Britannica disagrees at corporate.britannica.com/britannica_nature_response.pdf . Wikipedia is free online. Britannica offers abridged versions of entries for free online but charges $70 a year for full access.

Early next year, Nielsen Media Research will begin including in its television ratings college students living away from home, a group not now included. According to Nielsen, college students watch an average of 24.3 hours of television a week.

Gretchen Morgenstern, business writer of The New York Times , argues that politics is more brutal than business because politics is more of a zero sum game. In business there is usually enough of a surplus generated by a deal or an exchange to make both parties better off.

I was at a Starbucks in Tokyo when an American teacher entered with his class of Japanese business students on what he told me was a “field trip” to show them how to network with others outside the office. He was using his conversation with me, a stranger, as a learning experience for them.

“Whereas, most people will admit their ignorance of physics or biology, the armchair economist is convinced that he knows exactly what he is talking about.”— The Economist

“Highlighted by honors student” –Selling point for a used textbook for sale online.

If you have developed any attention-getting examples, ways to "sensationalize" economic ideas, useful resources on the Internet, or more generally, ways to teach just for the fun of it, please share these with colleagues in “The Grapevine” by sending them to:

William McEachern, Editor
The Teaching Economist
Department of Economics
University of Connecticut
341 Mansfield Road , Unit 1063
Storrs , CT 06269-1063

e-mail: william.mceachern@uconn.edu

Electronic versions of all issues of The Teaching Economist are available at www.thomsonedu.com/economics . If you have not yet asked to receive a hard copy of this semiannual newsletter, compliments of Cengage South-Western, or if you need to change your address, please write to:

Cengage South-Western
The Teaching Economist
Attn: John Carey
5191 Natorp Blvd.
Mason , OH 45040-7945

Email: j.carey@cengage.com

 Return to Contents of Issue 31, Fall 2006