Welcome to Leave Luck to Us. Here we rate video games. But we don't rate them from 1-10. We rate them a different way but more on that later.
Whether it's women, restaurants, movies, one-liners, or video games; I've never liked the idea of rating things on a scale of 1-10. It seems to me that asking someone to rate something on a gamut more often than not turns into "Pick a number from 1-10 that's higher than 5."
It's not natural. When riffing with your friends on the latest RPG seldom does your friend comment on the 1-10ness of it. Rather, he'll comment on whether it's good or bad. Buy or sell. See or skip. Yes or no.
So why do most sites rate games numerically and how useful is the scale? How honest are the reviews?
I guess sites rate games with a number because they think that's what their readers want. To be fair, it is a convenient way to quickly encapsulate a review. But it's my opinion that therein lies a problem: It's too easy to wrap a review—whether it's positive or negative—in a 6 or 7 rating. Putting it into a box with a number on it can disguise a lot of what the author may really mean. For example, the writer may complain about control issues, audio problems, and inferior graphics. However, at the end of the review you may be surprised that the game has been issued a 7/10.
This way the writer can have it both ways: he isn't a fan of the game but a 7 means it's still playable for most of his audience. He's neither right nor wrong. If the author manned up and rated it an honest 3 he'll be bombarded with complaints from fans of the game. Now if a fan approaches him, he can say “Look. I gave it a 7. That means it's a decent title.” Conversely, if a critic of the game approaches him he can respond. “Yeah. That's why I only gave it a 7.” So my theory is that a rating of 7 is meaningless and has replaced 5 as the new average.
I wondered if I could somehow prove that 7 has become the new average and settled upon the following methodology:
I went to IGN.com for my dataset. I chose to look at PS2 games because I felt that the number of reviews would provide a fair test. Also, these reviews would span from system launch to system retirement and that should serve to flatten out any personal feelings people may have held about the system.
IGN allows users to select games with a given rating rounded down to the nearest whole number. I simply counted each title found under each rating. (IGN actually scores games to within 0.1 accuracy. I've discarded the decimals and scored the games as whole numbers. For example, 4.2 becomes a 4. It might be interesting to further break down the scores to include the decimals. I wonder if they would follow a normal distribution or IGN's curve. We'll leave that one to the chaos mathematicians.)
After counting all of the scores I was able to create the following graph:
The red bars indicate scores given to games while the black curve approximates a "normal distribution"; a bell curve.
In any healthy population you'd expect the black curve but IGN's scores appear quite skewed to the right side of the graph. In fact, it seems that "7" is the actual peak of the graph. Much as what I expected to find.
Basically, this graph shows that 7 is indeed the new average. I believe it's because, as mentioned earlier, a 7 is a nice “safe” score to give a game if a writer doesn't want to be too committal.
I started examining this issue in response to the recent controversy regarding the recent firing of Jeff Gerstmann from Gamespot allegedly for scoring Kane & Lynch a 6. Had he scored it a 7 he probably would have protected himself from much of the fallout.
At any rate, I have decided on a new system for rating games: Buy, Hold, and Sell. It's the same rating system investors use when evaluating stocks. It's honest, natural, compact, and tells you everything you need to know in a easily digested nugget. It also eliminates much of the infighting between fans because a game they like “only” scored an 8.8. It's also harder to gloss over bad reviews with a middling score. With only three available ratings—not including the qualifier “strong” as in a “strong buy”—an editor is forced to be more honest. No longer can he hide behind a meaningless 6 or 7.