Analysis
Why only the top 20?
Why not get the scores and averages for all the downloadable games? Because it’s useless. You’re not going to buy 100 games, and neither am I. Statistically, a system with 50 awesome games and 50 terrible games will look worse off than a system with 10 awesome games and 2 terrible games, but I know I’d just rather have the system with 50 awesome games.
Scores are highly compressed
Game reviews tend not to use their entire scale. If a game site or magazine rates games on a 1-10 scale, they’ll give an average game a 7-8 score, and a truly awful game a 5. In the games journalism business we make fun of this as the “7-9 scale”, and it’s dumb. But it’s there. Now take a site like Metacritic that averages game scores, and a few small points can mean a rather large difference in relative quality. Case in point: If you look at all games for all platforms released in the last year (since Aug 14, 2008), the 51st best game (Little King’s Story) has an average score of 87, the 101st best game (NBA2K9) an 84. Small differences in numbers represent dramatic differences in ranking, once you get away from the very top and very bottom scores. It’s a steep bell curve.
The averages are compressed even more
When you look at the average for the top 10 or top 20 games above, differences in scores are compressed even further! You’re talking about averages of averages of that infamous 7-9 review scale.
What’s with the “number of games over 85″ thing?
It’s a totally arbitrary number, I’ll admit. Looking at each platform, it became clear that if you score at least 85 on Metacritic, you’re a pretty damn good game (provided there are more than a small handful of reviews). I picked that as a “must have” threshold. You’ll be well off spending your money on any game that scores over 85, unless you just hate that genre.
XBLA wins, hands down
Microsoft’s box has an average score for it’s top 20 downloadable games that is higher (slightly) than the other two platform’s top 10! XBLA has 55% more “must haves” than PSN, and over 3x more than WiiWare. If you made a single top-20 list for all three platforms (the cutoff would be scores 87 or higher, by the way), XBLA would have 12 games on the list, PSN 7, and WiiWare 1, and 7 of the top 10 would be XBLA games.
Where’s [your favorite game here]?
I know, right? Where’s Castle Crashers? Where’s Fat Princess? I’m as shocked as you are that these games didn’t even make the top 20 for their respective platforms. Blame it on the rocky online play each game experienced at launch, which hurt reviews. If it makes you feel any better, each game just barely missed the top 20. There are other great games on these platforms that just didn’t make the cut, like Settlers of Catan.
WiiWare gets a pass in the press
The average scores for some of those WiiWare titles are, if you’ll excuse the technical terminology here, bullshit. The competition on XBLA is much stiffer, and PSN as well (maybe less so), due to the sheer number of downloadable titles on those platforms and the prominence of the online stores. Tetris Party on WiiWare is a fine game and all, but no way is it better than Castle Crashers or PixelJunk Eden. Simply put, there haven’t been enough high-quality WiiWare titles yet to raise the bar in the collective subconscious of reviewers. Conversely, the bar has been set pretty high on XBLA games.
Pages: 1 2
#1 by booya on August 14, 2009 - 10:22 am
Nice analysis! Are these statistics consistent with the total # of games for each online platform? Would be interesting to see the number of games for comparison.
#2 by jasoncross on August 14, 2009 - 8:25 pm
Hard to say. Certainly there are more games on XBLA than PSN or WiiWare. I think there may be more games on PSN than WiiWare, but I haven’t counted.
#3 by zoom on August 14, 2009 - 9:26 pm
This is kind of pathetic, Jason. Tetris party is one of the better implementations of a classic game – to single it out for dismissal like while giving SotN/Rez/Ikaruga/SSF2T and frigging Puzzle Quest a pass is … highly questionable. Then going further and claiming your cherrypicked example just shows Wiiware games in general are not only worse on average, they’re also *overrated* … that is insane. Your fanboy is showing. Also, there are better games than Tetris Party on ALL of those lists that don’t rate as high. Don’t cry because your favourite XBox games got rated lower – it just shows how arbitrary Metacritic score are and how they shouldn’t be taken seriously at all let alone ‘analysed’.
Also, if, as you said on Qt3, the goal was to compare all games on the download services without considering whether they were new or not – it seems exceedingly strange to ignore older games like XBox orginals and Virtual Console releases, which would change the ranking considerably.
#4 by jasoncross on August 14, 2009 - 10:12 pm
There’s no need for name calling. My fanboy is showing? Methinks thou dost protest too much, sir!
And yes, Tetris Party is a fine game. It’s good. But an 86 puts it in pretty rarefied air on the other lists, ahead of plenty of other games.
Take Tetris Splash. Yes, it’s a worse downloadable Tetris game than Tetris Party is, but not terribly worse. It gets a 53 on metacritic. Yes, Tetris Party has more stuff to do and is definitely a better game, but 86 to 53 better?
And no, my goal was not to compare every downloadable game on all platforms. It is to compare games available in XBLA, PSN, and WiiWare, not downloadable versions of retail games. Sorry if that was unclear. It’s almost impossible to really compare things like Xbox Originals because you can’t separate out reviews of the game today as a download title form reviews when it was a new Xbox game (where it was judged quite differently).
#5 by zoom on August 15, 2009 - 12:17 am
“Methinks thou doth protest too much, sir!” Well you would wouldn’t you!
But really, to pick out that one rating as if it were a pattern and use it to ‘prove’ how Wiiware is getting an easy ride when pretty much every other rating on that lists seems ~ right (ie. Wiiware is mostly composed of junkware and the top-rated games reflect that) – come *on*, Jason. It was just a lazy argument and an absurd reach in an otherwise fastidious-to-the-point-of-not-even-being-interesting article. It stuck out. Of ALL the things you suddenly decide to lose your faith in Metacritic over … that’s it? It’s just peculiar that we’re asked to accept without question that Puzzle Quest and Peggle and Rez are all better games than a fully fleshed-out version of Tetris with some pretty creative and even demented game modes. So what else is overrated there? Presumably you take issue with *many* of those scores. So let’s go. What is Jason Cross’ World of Goo score? 89? Orbient gets what? 75? I just want to know!
#6 by jasoncross on August 15, 2009 - 1:50 pm
Oh, nothing here is “proof.” You can’t “prove” an opinion. You’re confusing “proof” with “example.” No, I’m not asking you to “accept without question” Puzzle Quest or Peggle or Rez are better games than Tetris Party. It’s an opinion about the relative merits of WiiWare titles compared to PSN and XBLA. You are encouraged to have your own opinion. Neither of us will be correct or incorrect.
I’m merely expressing my opinion that many WiiWare games get scored more highly by critics (not me! not you! Critics reviewing the games!) than games of similar quality on PSN or XBLA. I think they get reviewed relative to other WiiWare games, and there haven’t been enough stellar WiiWare titles yet to “raise the bar” for that platform.
#7 by zoom on August 16, 2009 - 3:30 am
Opinion shmopinion – you were the one making authoratative statements about quality, falling back on ‘opinion’ now is a ridiculous copout – especially given that the aim of the article is to *quantify* opinions. Sheesh. There isn’t anything that makes Tetris Party somehow indictative of reviewer kid gloves when it comes to Wiiware. To act as if it’s just obviously a worse game than Puzzle Fighter or Puzzle Quest (or any of the other games on that list) is laughable, and generating some general rule a single game – you refuse to offer more examples – is psychotic. Tetris Splash was a barebones entry – it’s possible it was *underrated*, but it says nothing about Tetris Party, which is a much better version, no matter how you try to underplay it.
Finally, over half of the games on the 360 list are do-overs of ex-retail games. So you’re not sticking to your rules. PS3 and Wiiware have more original content in the top 20. I can understand not counting Virtual Console games/XB Originals/Playstation Classics because they have no new content at all, but it’s also worth pointing out that the distinction is narrow and technical.
#8 by work at home on September 9, 2009 - 11:27 pm
Very good work, interesting post, bookmarked !
#9 by Alex on November 27, 2009 - 6:31 pm
All my friends tell me I need to get BF 1943… now only if they’d buy it for me.
#10 by Riley Cooper on May 12, 2010 - 4:44 am
I easily get addicted to PSP games and i am always updated with the newest PSP games in town.~’~