Meta Madness

I saw this linked off of the Penny Arcade Report today and given my already public feelings on Metacritic and how we use it in game development, I couldn't simply remain silent.

Every game developer should be appalled at the notion of using Metacritic scores as the basis of a hiring decision. Maybe Irrational is just looking within that select (and lucky) caste for this position, a Design Manager is kind of a big deal gig after all, but regardless we need to take notice lest such requirements become common practice. I've worked on games that should have gotten higher Metacritic scores, but didn't and I've worked on ones that I knew were going to under perform. In both cases, there were maybe 5 people at the studio who could have done anything to change that fate and even then, it was a crap shoot.

Glorifying Metacritic scores within the game industry is only going to end up burning us in the end. We need to understand that Metacritic scores represent correlation and not causation. The scores don't drive sales, it's the factors that should exist to garner those scores that do it. What we need to understand however is that those factors don't always directly translate to the scores. I realize that an argument can be made that judging the relative quality of a game by it's Metacritic score is at least somewhat fair, it is not at all a fair however to use that same criteria to judge a developer; there are too many other factors that go into what makes a good developer and most of those are not and cannot be reflected by how the product performs on Metacritic. Let's look at this another way. Let's say you're applying for a public relations position somewhere and on the job listing it says "must have at least 500 Facebook friends" ... because someone who is in PR has to deal with people and therefore should have a lot of friends. Sure there's some correlation, but by and large the one thing has little to do with the other.

I've been in game development long enough to know that job listings usually represent a company's most pie-in-the-sky ambitions for the position and that when push comes to shove a guy with excellent qualifications and shipped titles with only an 80 on Metacritic could totally land that job. The point is that this is a stupid road to begin going down. We're already judged by our number of shipped titles (a value, that changes drastically depending on the types of games you work on) so why add even more arbitrary nonsense values into the hiring process? If you ask me, anyone using Metacritic as an exacting measure of a developer's quality simply isn't doing their homework. This industry is stressful and tough enough without having to worry that you're going to be judged for employment based on a Metacritic score that is largely beyond your ability to influence.

DISCLAIMER

The views expressed on Plenty For All are purely the opinions of Brian J. Audette and are not at all affiliated with, representative of, endorsed or supported by BioWare, EA, it's shareholders, partners, or subsidiaries.