social science | A Patient Cycle

“The idea of rating ballplayers is an arrogant bit of nonsense, incurring inherent intellectual costs which can lead, if unchecked, to intellectual bankruptcy.”

—Bill James, 1984 Baseball Abstract, in an essay prior to his player rankings

If this is true about baseball players, it is no less true about high school hockey teams. As someone who does this on a weekly basis, it’s something I remind myself of, every single week.

High school hockey rankings are a dime a dozen; everyone has their opinions, and it’s not too hard to broadcast them these days. The most notable is the coaches’ poll organized by Let’s Play Hockey, which, at some point in the mists of history remembered only by Lou Nanne and that State Tourney studio guy who looks like a character from Guess Who?, got “official” recognition in the media. To its credit, the LPH poll’s conservatism keeps it from having the wild swings one sees in other places, and I’d say it’s improved drastically even in the past five years.

Even so, LPH’s method is unexplained and seemingly arbitrary at times, leaving readers trying to figure out the logic behind their methods. In response, a whole bunch of people have created math-based computerized rating systems that perform much the same function. I grew up checking those of my fellow forum admins, Lee and Mitch; MyHockeyRankings uses similar principles for hockey nationwide. Some sports use QRF’s system for section seeding (though I find that one flawed beyond use in hockey), and this year, Doug over at FollowThePuck, who had previously done his own subjective rankings, has introduced an algorithm to do his work. I have a lot of respect for these dispassionate rankings, and check my preferred ones regularly. They’re a welcome antidote to the self-proclaimed hockey “experts” who spew out opinions left and right and invent rankings through narrow logic or facile knowledge of the teams.

At the same time, though, I’ve carved out a little niche for myself over on the forum over the past seven years, where I subject myself to weekly flagellation from the masses while trying to carefully explain my subjective rankings. In doing so, I have at times found myself in the unexpected position of being the great defender of subjectivity over the computer rankings. I’m not saying I’m better than the computers, but I do think I can offer things that they cannot.

For starters, let’s stop trying to pretend the computers are “objective.” They’re not. Sure, they don’t play favorites, care nothing for tradition or coaching, or for some of the excuses one often hears out of a losing team. They can see everything, which no human can do. But somewhere behind it all, a human has to decide how much weight to give to each of the results, and at what point we stop caring whether a team wins by 6 or 8 or 10 goals, and how much to value recent games versus old games, and so on. This replaces one form of subjectivity for another, and while people can study and adjust the formulas to give them even more predictive power, it is all backward-looking, and achieves success by narrowing the scope of study, and ignoring large parts of what goes on in a hockey game to fixate on goal difference and strength of schedule.

To illustrate this point, Doug and I had an amiable Twitter tit-for-tat earlier this week on the merits of weighting A vs. AA teams differently in his mathematical rankings. Over the course of the discussion, we found that Hermantown, who most people would consider the top Class A team, was 20 places apart between Mitch’s system and his when ranked against the AA field. Two “objective” systems spat out ridiculously different numbers. I say this not to slight either one, but to point out the poverty of the belief that these methods, which simplify our understanding by reducing everything that goes into a hockey game into a rating, can definitively answer these questions.

James again:

“My work has been described in a lot of ways, and I don’t like most of them, but one that I particularly don’t like is being called a baseball expert. I am not an expert; I am a student…I am not trying to lecture you—I am not trying to lecture anyone—about who is good and who is bad. I have my ideas on the subject, that’s all. I offer those ideas because people expect me to do that, and want me to do it…The ratings provide for an organization and framework for comments, and I do have things I want to say about the players.”

He goes on to explain how good scientific analysis tries to contribute to debates, not settle them (an insight that people would be wise to remember in discussions of topics far more weighty than hockey). This is what I try to do, and why I actually enjoy the give-and-take on the forum. Just this past week, I heard from someone who was upset his team didn’t get a mention—and he had a very good point, at least until his team suffered an unexpected loss on Tuesday night. The rankings are part of an ongoing dialogue as we try to make sense of the statewide hockey scene. The process does need one person to take control and shepherd it along, but because I’m just one person, I can’t possibly see everything. But with some help from a few friends, I can see enough of the complexity that goes into winning hockey games (ignored by the algorithms) to say something valuable that they cannot, even though I’m just some lowly, flawed, biased, Hound-loving human being.

That frees me to say something like this: “Hermantown lost to Hopkins and barely beat Roseville, and plays an offensive style that frees them to run up big scores on middling opponents, so they’re probably not quite a top-5 team. However, they have dominated everyone else in Class A, played powerful Wayzata tight, and the top Class A teams would usually crack a two-class top-ten, so they’re probably noticeably better than #25, too. After their annihilation of Grand Rapids on Tuesday, I’d have them around #7 or so—behind Wayzata, since they lost to them and haven’t beaten anybody better, but they have a few quality wins, and that one mediocre loss isn’t too big of a drag when we have teams like White Bear Lake (who lost to East Ridge) in the top ten.”

That may be right, and it may be wrong; I’d listen to arguments in either direction—preferably on the forum, since it’s hard to make coherent arguments in 140 Twitter characters. I’ve made mistakes, and doubtless I’ll make more, but I find the result far more enlightening than an unexplained list of twenty teams that appears in the paper every week. If you want a ranking to give you a definitive answer, you’ve missed the point.

Time to use some hockey to make a point about complexity and causes.

Here is a replay of the triple-overtime championship-winning goal in the 2011 Minnesota State High School Hockey Tournament. In that game, Eden Prairie defeated Duluth East, 3-2. (Sorry for the grainy video, but it’s the best I can do.)

This is about as straightforward as it comes in a sport: the puck goes in the net, one team wins, the other one loses. So, what caused that goal? What decided the 2011 championship?

At the most basic level, you have the excellent effort by the goal-scorer, Kyle Rau, diving to swat the puck into the net. There are also two Duluth East miscues: the goaltender letting the puck squirt through him, and the defenseman, Andrew Kerr, fanning on his attempt to clear it.

Others might point to luck or fate, too: as later replays showed, after Rau made contact with the puck on his dive, it hit the goalpost, bounced out, and deflected off Kerr’s skate before sliding into the back of the net. It was a great play by Rau, who was the state’s best player that year, but not even he could have planned something like that.

But why stop there? Let’s rewind this play a bit: you have the initial shot from the Eden Prairie defenseman, and the sequence of events from both teams that led up to that shot, of which you see only a fraction in this clip. And for Rau to even be in this position in the first place, countless other events had to fall in line. Both teams had scoring chances throughout the three overtimes, and in regulation as well. With East up 1-0 after two periods, one of their best defensemen went off with an injury. The second East goal was fairly soft—one the Eden Prairie goalie would have normally saved. The referees also played a role; they called only one penalty in the entire game, much to the chagrin of Eden Prairie fans—Kerr put together a complete highlight reel of vicious checks on Rau before the fateful play at the end. Any little change in a play, and this moment doesn’t happen. And that doesn’t even touch the months and years of preparation that led up to this game.

Keep in mind that hockey is, conceptually, fairly straightforward. It follows set rules, has a limited number of actors involved, and the goal is obvious: put the puck into your opponent’s net more often than they put it in yours. It has been studied by enough people over the years that we now have a reasonably good idea of what it takes to win a championship. We can know what to look for in individual players, and how they fit within the coach’s scheme; computer models can weed through the flood of information and statistics and predict outcomes with commendable accuracy. We can correlate many things with success.

But nailing down a precise cause; the exact catalyst that left Eden Prairie dancing in delight, and Duluth East flat on the ice in dejection? That’s an entirely different story.

And if it’s so difficult to do in hockey, how can it be any easier in any other realm of human affairs; ones with more actors, less clear goals, and fewer sets of rules we can all agree on? From politics to warfare to those mundane events that pop up in our daily lives, how can we pin down a sequence of events with any degree of accuracy?

Now, this train of thought could easily lead to a sort of causal nihilism. I don’t want to go there. The point here isn’t that it’s impossible to label a single cause. It is that finding these causes is a lot harder than it may at first seem, and that anyone who looks to study this sort of thing needs to go at it with a proper dose of humility. Any sort of analysis or study that tries to end the conversation, whatever the merits of its arguments, suffers from a conceit that does its audience a disservice. At some point, of course, we need to make a decision and move on. But social science, for all its explanatory power, is not a hockey game. Anyone who approaches it with the intent to win or lose has missed the point, and that can be a serious problem.

So, what does this Duluth East alumnus think caused that goal? My philosophy is that one has to boil it down to what one can control, which in this case means pointing out the two plays the East players could have made, but didn’t. There is no shame in taking that responsibility, especially for two otherwise rock-solid players who had fantastic high school careers. They were minor mistakes, but in a game that was so dead-even that it almost had to end on a fluky play, those two in tandem made the difference. Hockey can be a cruel sport, but, well, so can life. That’s my opinion, and while I doubt I’ll change it, it doesn’t invalidate the many other accounts of this game.

At any rate, this is the mindset I hope to use on this blog. Tomorrow, we’ll add some politics to the discussion.

Menu