The Future of Hockey Analytics

Since the peak of NHL scoring in the 1980s, the average number of shots has remained comparatively stable. Scoring, however, has dropped precipitously. This trend suggests the declining importance of shots themselves and a corresponding increase in the importance of goals. Despite this reality, there remains an unwarranted emphasis on shot quantity to evaluate teams and individuals. This has the effect of encouraging a simplistic approach to hockey analytics. Hockey is a game that requires scoring more goals than your opponent. It seems that analysts should be promoting methods that consider goals, not simply possession.

This is not news to NHL players. In a recent piece for TSN, Frank Seravalli considered the views of NHLers on the advanced statistics that have infiltrated the thinking of pundits and teams. In a decidedly unscientific poll of 18 players in September 2017, TSN asked whether any of them knew their Corsi percentage last season. Not one player did.

While some did better recalling the seemingly more antiquated +/- statistic, the TSN questions prompted some thoughtful and unsolicited reflections from the players. Their comments are in contrast to the idea that hockey players are likely to speak in meaningless platitudes.

Joking aside, these players understand something that many commentators do not. In short: Corsi is overvalued as a statistic. They may be onto something. Adherents to advanced shot-based analytics believe it offers some sort of predictive power that as yet has not been demonstrated.

If Corsi is a statistic worth anything, it should be correlated over time with team success. Until teams with high Corsi numbers score more goals, win more games, and make it to the Stanley Cup finals year after year, what value is it?

Bill James and Moneyball

The hockey analytics awakening is still in its infancy. The infatuation with analytics in hockey has been imported from other sports, most notably baseball. The goal is to take lessons from the work of Bill James who developed a different way to consider statistics in baseball.

Sabermetrics is defined as “the search for objective knowledge about baseball.” James eschewed traditional statistics such as walks, strikeouts, home runs, runs batted in, and batting average. Instead, sabermetrics focuses on advanced metrics include wOBA (weighted on-base average), FIP (fielding-independent pitching) and WAR (wins above replacement). James was relatively unknown until Billy Beane began using his methods in Oakland and began hiring people who could manage the math. James is also credited with assisting the Boston Red Sox in building a team that won the World Series in 2004.

Casual baseball fans will know of James through the film adaptation of Michael Lewis’s best-selling book, “Moneyball: The Art of Winning an Unfair Game.” The essential idea is that organizations, especially those at a financial disadvantage, need to find market inefficiencies to succeed.

Part of the fun is that the story is about both the underdog Athletics making it to the postseason and the decisions behind the scenes to build a team based on relatively obscure stats. Less than three years after Beane adopted sabermetrics, the 2002 A’s won 20 games in a row, an American League record. Despite having the lowest payroll in baseball, Oakland tied for the best record in the majors.

Science vs Scouts

My favorite segments in the film explore how more traditional scouts and managers refused to adapt to this sort of statistical analysis. Viewers get to explore whether and how an unscientific culture like baseball adapts to the growing recognition of the value of the scientific revolution as applied to baseball. Essential is how to evaluate talent.

It is nothing like the subjective views of talent scouts, we are told. Many scouts tend to overvalue athleticism or the “look” of players. This is at the expense of actual performance – such as on-base percentage. Scouts are susceptible to the usual foibles of bias, drink, careerism, and plain old stubbornness. The numbers don’t lie, the movie quietly whispers over and over. Trust the math.

The problems with the book have often been portrayed as one of selectivity bias or using only the facts that support the premise. Both Lewis and the filmmakers essentially ignore the contributions of players – especially the A’s pitchers – who don’t support the underdog narrative. Related to this critique are the post-season failings of the A’s following the Cinderella season recounted in the film.

Perhaps the biggest problem is the false dichotomy the film portrays. The best analysis involves both talent scouts relying on expertise, and number monkeys who turn raw data into predictive metrics. For many, this remains an either/or proposition.

Debating On-ice Performance: Nerds vs Dinosaurs

Hockey is still without a Billy Beane of its own. Matt Larkin offers a useful history in hockey of the debate recounted in Moneyball. On one the hand, the advanced analytics adherents suggest relying on possession statistics that demonstrate who has the puck and which teams or players drive the play.

On the other hand, the old guard holds that quantifying events on the ice ignores the intangibles associated with the speed of the game and its inherent unpredictability. This debate became more animated as younger fans began to analyze team and player performance and share their work online. In some cases, bloggers combined novel analysis with a biting tone that mocked the mainstream dinosaurs. The rest, as they say, is history. Larkin observes:

Once the advanced stat proponents developed a strong social media presence, with tens of thousands of Twitter followers, their influence spiked. The sudden prevalence of analytics bloggers turned NHL team employees reflects that. Simmons insists many GMs in the league are forced to feign interest in possession numbers now to avoid being publicly flogged by the bloggers, and that “guys I know personally who really don’t care much for the stats will say when you ask them on the record, ‘Oh yeah, we do a lot of work in that area.’

Today, the advanced statistical movement is firmly established in many front offices around the NHL. A key metric is Corsi. Corsi serves as a proxy measure for puck possession during five-on-five play. To calculate a Corsi number, you add shots on target for, missed shots for and blocked shots against. Next, you subtract shots on target against, missed shots against, and blocked shots for.

Shane O’Donnell suggests Corsi as an effective measure because it is immune to the random fluctuations associated with other metrics. While some analysis suggests Corsi is a slightly better predictor than other metrics, the bar seems pretty low. Is a high Corsi number an effective indicator of good hockey teams?

The Case Against Corsi

THW’s own Andrew Wilson summarizes the problems with Corsi. One issue is the focus on quantity of shots, rather than quality. Evaluating a team or player using Corsi assumes that all shots are created equal. A shot from the slot is worth the same as a shot from outside the blue line. A shot from a 4th line call-up is valued the same as one from the NHL’s leading scorer. ‘

Another issue is that unsustainable levels of shot conversion rates or save-percentages can lead to major disparities and outliers. In other words, teams with strong goaltending or high shooting percentages can distort the metric.

A perceptive critic of Corsi is David Staples. One observation he makes is the semantic problem with reporting Corsi as an individual statistic as opposed to one that is contingent upon a player’s teammates. He states:

If folks regularly start to say, “Hall and his teammates had a Corsi of 44%,” then there’s no issue. But if you shorten it to just “Hall’s Corsi was 44%” it becomes semantic nonsense. It obscures what the number is really describing, and the danger is you’re going to start actually thinking of it that way, which will lead to confusion and problematic judgments.

Even if we accept Corsi as principally a team statistic, there are other more foundational problems. Good hockey teams generally have good Corsi ratings, but teams with good Corsi ratings are not guaranteed to be good hockey teams. This is a serious problem for any stat purporting to measure quality.

Proxy Measure

The challenge is that Corsi is a double proxy measure. First, shot-based metrics are being used as a proxy for offensive zone possession. Second, possession itself is being used as a proxy for goals. Do teams with better possession score more goals?

Short answer: they don’t. The top ten teams in 2016-2017 for Corsi include the LA Kings, Boston Bruins, Montreal Canadiens, Washington Capitals, Tampa Bay Lightning, San Jose Sharks, Philadelphia Flyers, St. Louis Blues, Anaheim Ducks, and Columbus Blue Jackets. Only Washington and Columbus were in the top ten of scoring.

What about the playoffs? Of the top ten teams in 2016-2017, seven made the playoffs. However, only three of the ten made it past the first round. The Kings had the highest Corsi For during the 2016-2017 season in all situations (5008). New Jersey had the lowest (4026).

If possession mattered for team success, we should expect the team with the highest Corsi to make the playoffs and the team with the lowest not to. In this case, neither team made the playoffs. Consider the Stanley Cup finalists. Nashville had a Corsi of 4776 and Pittsburgh ended the regular season with 4814. This is higher than the league average but lower than the Corsi for the Toronto Maple Leafs, New York Islanders, Dallas Stars, Bruins, and of course the beleaguered Kings.

It is silly to try to draw conclusions from a single year. As Nate Silver notes, this tendency is part of what he calls society’s probability problem. The desire for certainty leads to simplistic approaches to what are probabilistic uncertainties. The best way to address this uncertainty is to examine the potential value of a metric and assess its performance over time.

In 2015, Rob Found analyzed the value of Corsi. He demonstrated that over a 10-year span both winning and losing teams took a similar amount of shots over time. Based on this study, there is no significant relationship between winning and the number of shots a team takes.

Better Data Means Better Statistics

Today, teams are investing in approaches that will soon yield an unprecedented picture of player performance. This will involve a combination of the “eye test” and more sophisticated approaches to measurement, calculation, and prediction. It is clear goal-based metrics are better than possession-based metrics.

Based on Found’s ten-year review of NHL data, goal-based metrics that examined relative plus-minus divided by minutes of ice were more predictive of team success than shot-based metrics, especially for forwards. The irony is that the much-maligned +/- metric, still has value. This is something those NHL players that TSN asked already know.

The future of hockey analytics requires embracing the quantum nature of hockey in which no single metric exists in isolation. Goals can only come from shots, shots can only come from the possession of the puck, and possession is an inevitable result of winning faceoffs, getting power plays, forcing turnovers, and so on.

It means asking better questions about individual players. Do they help get the puck out of their own zone? Do they help get the puck into the opposition zone? How? Do they contribute to scoring chances? Do they make mistakes on scoring chances against? How do they do that? How much do they handle the puck? Who successfully makes and takes the most passes? Who makes the most mistakes?

Answering these questions, as Alison Mah points out, will be easier with the advent of new technology. Teams are exploring placing tracking chips into pucks and players’ jerseys to measure aspects of the game that were previously not measurable. Infrared cameras installed around the arena will track everything from skating speed to puck movement.

The good news is that actual data on individual players in numerous game situations will replace proxy measures like Corsi. These will build on interesting recent examples that are built on decades-old observations. The ultimate value of these newer approaches will require credible evidence that they are a good indicator of success over time. The good news is that they don’t have to be very good to supplant the over-used and poorly understood Corsi.