In Defense of Subjectivity

“The idea of rating ballplayers is an arrogant bit of nonsense, incurring inherent intellectual costs which can lead, if unchecked, to intellectual bankruptcy.”

—Bill James, 1984 Baseball Abstract, in an essay prior to his player rankings

If this is true about baseball players, it is no less true about high school hockey teams. As someone who does this on a weekly basis, it’s something I remind myself of, every single week.

High school hockey rankings are a dime a dozen; everyone has their opinions, and it’s not too hard to broadcast them these days. The most notable is the coaches’ poll organized by Let’s Play Hockey, which, at some point in the mists of history remembered only by Lou Nanne and that State Tourney studio guy who looks like a character from Guess Who?, got “official” recognition in the media. To its credit, the LPH poll’s conservatism keeps it from having the wild swings one sees in other places, and I’d say it’s improved drastically even in the past five years.

Even so, LPH’s method is unexplained and seemingly arbitrary at times, leaving readers trying to figure out the logic behind their methods. In response, a whole bunch of people have created math-based computerized rating systems that perform much the same function. I grew up checking those of my fellow forum admins, Lee and Mitch; MyHockeyRankings uses similar principles for hockey nationwide. Some sports use QRF’s system for section seeding (though I find that one flawed beyond use in hockey), and this year, Doug over at FollowThePuck, who had previously done his own subjective rankings, has introduced an algorithm to do his work. I have a lot of respect for these dispassionate rankings, and check my preferred ones regularly. They’re a welcome antidote to the self-proclaimed hockey “experts” who spew out opinions left and right and invent rankings through narrow logic or facile knowledge of the teams.

At the same time, though, I’ve carved out a little niche for myself over on the forum over the past seven years, where I subject myself to weekly flagellation from the masses while trying to carefully explain my subjective rankings. In doing so, I have at times found myself in the unexpected position of being the great defender of subjectivity over the computer rankings. I’m not saying I’m better than the computers, but I do think I can offer things that they cannot.

For starters, let’s stop trying to pretend the computers are “objective.” They’re not. Sure, they don’t play favorites, care nothing for tradition or coaching, or for some of the excuses one often hears out of a losing team. They can see everything, which no human can do. But somewhere behind it all, a human has to decide how much weight to give to each of the results, and at what point we stop caring whether a team wins by 6 or 8 or 10 goals, and how much to value recent games versus old games, and so on. This replaces one form of subjectivity for another, and while people can study and adjust the formulas to give them even more predictive power, it is all backward-looking, and achieves success by narrowing the scope of study, and ignoring large parts of what goes on in a hockey game to fixate on goal difference and strength of schedule.

To illustrate this point, Doug and I had an amiable Twitter tit-for-tat earlier this week on the merits of weighting A vs. AA teams differently in his mathematical rankings. Over the course of the discussion, we found that Hermantown, who most people would consider the top Class A team, was 20 places apart between Mitch’s system and his when ranked against the AA field. Two “objective” systems spat out ridiculously different numbers. I say this not to slight either one, but to point out the poverty of the belief that these methods, which simplify our understanding by reducing everything that goes into a hockey game into a rating, can definitively answer these questions.

James again:

“My work has been described in a lot of ways, and I don’t like most of them, but one that I particularly don’t like is being called a baseball expert. I am not an expert; I am a student…I am not trying to lecture you—I am not trying to lecture anyone—about who is good and who is bad. I have my ideas on the subject, that’s all. I offer those ideas because people expect me to do that, and want me to do it…The ratings provide for an organization and framework for comments, and I do have things I want to say about the players.”

He goes on to explain how good scientific analysis tries to contribute to debates, not settle them (an insight that people would be wise to remember in discussions of topics far more weighty than hockey). This is what I try to do, and why I actually enjoy the give-and-take on the forum. Just this past week, I heard from someone who was upset his team didn’t get a mention—and he had a very good point, at least until his team suffered an unexpected loss on Tuesday night. The rankings are part of an ongoing dialogue as we try to make sense of the statewide hockey scene. The process does need one person to take control and shepherd it along, but because I’m just one person, I can’t possibly see everything. But with some help from a few friends, I can see enough of the complexity that goes into winning hockey games (ignored by the algorithms) to say something valuable that they cannot, even though I’m just some lowly, flawed, biased, Hound-loving human being.

That frees me to say something like this: “Hermantown lost to Hopkins and barely beat Roseville, and plays an offensive style that frees them to run up big scores on middling opponents, so they’re probably not quite a top-5 team. However, they have dominated everyone else in Class A, played powerful Wayzata tight, and the top Class A teams would usually crack a two-class top-ten, so they’re probably noticeably better than #25, too. After their annihilation of Grand Rapids on Tuesday, I’d have them around #7 or so—behind Wayzata, since they lost to them and haven’t beaten anybody better, but they have a few quality wins, and that one mediocre loss isn’t too big of a drag when we have teams like White Bear Lake (who lost to East Ridge) in the top ten.”

That may be right, and it may be wrong; I’d listen to arguments in either direction—preferably on the forum, since it’s hard to make coherent arguments in 140 Twitter characters. I’ve made mistakes, and doubtless I’ll make more, but I find the result far more enlightening than an unexplained list of twenty teams that appears in the paper every week. If you want a ranking to give you a definitive answer, you’ve missed the point.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s