Ahead of the third annual Ottawa Hockey Analytics Conference, The Hockey Writers sat down with one of the guest speakers at the conference, hockey analytics pioneer Rob Vollman. An author of five books and a consultant with multiple NHL teams, we asked Vollman about how he got into hockey analytics and the advanced analytics movement in hockey, as well as his thoughts on the Ottawa Senators’ success in the first round of the playoffs and the upcoming expansion draft.
(Some questions and answers have been edited for clarity, and some answers were provided by email.)
The Beginning
The Hockey Writers: What first prompted you to bring advanced analytics into the mainstream? What goals did you have in mind when you started this work?
Rob Vollman: My goal has always been to build this field. Whether it’s inventing stats, publishing books, going on TV and radio, writing for NHL.com, creating visualizations, organizing conferences, or consulting with teams, everything that I do is to prove that it can be done, how it can be done, and that it has value. It has always been my practice to stop working in an area once others have picked it up, and to invest that free time finding new areas to expand the frontier. One day, I hope that I’m incapable of advancing the field any further, and can simply sit back and enjoy the contributions of others.
THW: How was your work first received by the sport?
Vollman: The reaction can really be broken down into three groups. The mainstream media, who were largely resistant. The front offices, who were quick to accept the merits of advanced analytics but had no incentive to publish or broadcast that information to the public. And young people and new fans, who latched on and were immediately enthusiastic.
The Fancy Stats Movement
THW: Why do you think there was, and still is, a reluctance from mainstream media and fans to accept advanced stats as a legitimate way of understanding and evaluating the game?
Vollman: The mainstream media has to make difficult choices about the information that will best entertain and inform their audience, and it’s hard to accurately judge the merits of various new statistical perspectives and figure out how best to present them. Ultimately, it’s about presenting (the stats) in a fashion that people understand and that is entertaining.
THW: What kind of flaws exist in the use of advanced analytics in the media?
Vollman: Advanced stats are often perceived as the be-all and end-all, when in reality they lack perspective. That’s because writers have restrictions imposed by word counts, so they don’t have the space to explain the limitations of those stats in, for example, a 500-word article, and thus are forced to present them out of context.
THW: Many advanced statisticians are very critical of the old school and its way of looking at the game. Do you see room for a healthy medium to exist between the old school and new school?
Vollman: The rivalry between stats and the eye test is ridiculous because they complement each other so perfectly. Analyzing a game without double-checking an objective account of what happened is like covering one eye, but analyzing the game without watching it is like covering the other. We should be using both eyes.
THW: Based on your involvement in the game and your own observations, how often are advanced analytics used when making roster decisions?
Vollman: Almost every team has at least one person on staff looking at the numbers, in addition to arrangements with third-party data providers and analysts. Personally, I have had brief engagements and/or consultations with about a third of the league. However, I estimate that only handful of teams are including this perspective to any significant extent when making roster decisions. The main area in which advanced analytics are used most often is at the draft, using a method called Prospect Cohort Success. Essentially, all draft-eligible prospects are measured against all comparable prospects from previous drafts to determine how likely they are to play in the NHL and the kind of success they are likely to have as a pro.
THW: In what direction do you see the advanced analytics movement taking the game? Do you think the PHWA will follow suit with the MLB and make advanced stats an integral component in the award voting process?
Vollman: The next step is to use technology to get a more accurate account of what happened in a game, puck battle by puck battle, in much the same way baseball games are recorded pitch by pitch, and football play-by-play. As a member of the PHWA prior to my time with NHL.com, I used stats in the voting process, and helped others do so as well. Unless someone can actually watch all 82 games for all 31 teams, and in a completely unbiased fashion, then it makes sense to double-check what we have seen with an objective account of every game before we cast our votes.
“Analytics Aren’t About Destinies, They’re About Probabilities”
THW: Despite being the statistical underdog in their first-round series against the Boston Bruins, the Senators were the eventual victors. How much stock can be put into advanced stats in the smallest of sample sizes that is the postseason?
Vollman: Even a heavy favourite loses a seven-game series about a third of the time, and Boston was only a slight favourite. Take out all the overtimes they played and just look at regulation. What percentage of those minutes would you say the Sens outplayed the Bruins? (Editor’s note: we estimated it at roughly 50 or 55 percent.) This season in particular we have seen the impact random variation can have, because the “better” team arguably won the minority of the first-round matchups. Analytics aren’t about destinies, they’re about probabilities.
THW: What kind of impact does Erik Karlsson have on his team and do players of similar ilk have on their respective teams and teammates?
Vollman: The Senators are one team with Karlsson on the ice, and another without. He personally swings the team from the legitimate contender that they are, to a team that could potentially be in the draft lottery without him. Consider what happened to Tampa Bay without Steven Stamkos, or to Montreal without Carey Price last year. The same thing would happen to Ottawa without Karlsson – or worse. And he has a profound impact on his partners like Filip Kuba and Marc Methot, the former of whom was a borderline NHLer, and the latter of whom is probably a capable No. 4 defenseman without Karlsson, at best.
Expansion Draft Strategy
THW: If you were in George McPhee’s shoes and responsible for building a new team at the expansion draft, what would your approach be?
Vollman: McPhee should not view the expansion draft as a roster-building exercise, but as a method of acquiring assets without giving up anything in exchange. In this case, the assets are the contracts themselves, not the players to whom they belong. He should select the 30 most valuable contracts, no matter whose they are. Any contracts that belong to players that don’t fit his plans can be exchanged to other organizations for assets of equal value who do (fit his plan).