Panning For Gold: The NHL Draft Analyzed (Part I)

If you have ever traveled to Alaska, the Sierra Nevadas in California, or certain parts of Canada, you may have had the opportunity to pan for gold.  While the prospect seems a bit corny at first, you soon find yourself engrossed in the process, as you search pan after pan of dirt and water for one shiny nugget.   Sometimes you succeed, but far more often you come up short.

This is an apt metaphor for the process confronting NHL General Managers as they approach that annual rite of passage known as the NHL Entry Draft.  After months — and sometimes years — of research, scouting, analysis, interviews, arguing, cajoling, calculating and, ultimately, guessing — it all boils down to seven rounds of selections upon which millions of dollars, the success of an NHL franchise, and the fortunes of individual players rely.  No pressure, right?

The hockey journals, sports pages, blogs and forums are littered with “breakdowns” of the potential draftees, the strengths and weaknesses of each franchise, past draft failings by individual clubs and the identities of players who should have been taken in lieu of the selections actually made.  However, there are relatively few attempts to analyze the draft as a whole, over time, to see how it really works as a talent acquisition mechanism, how long players really take to develop, and how success on the ice relates to success or failure at the draft table.  To be sure, the draft is the only game in town — with few exceptions, every player has gone through the process and ended up in the league, so to that very basic extent, the process works.  However, as I read article after article about non-playoff teams pinning their hopes on this prospect or that prospect for immediate impact, I wondered if those dreams were realistic.  Hence, the germ of an idea for this series.

The analysis began with a review of the Entry Drafts from 2000 through 2009.  This represents ten years of data, and coincides with the entry of Columbus and Minnesota into the league, providing a consistent pool of 30 teams throughout the period.  I also started by limiting my review to the first three rounds, as this included 945 players — sufficient for statistical analysis, but not overwhelming to work with.  I then looked at the NHL stats for those players over that 10 year period ( nine years of actual play, due to the lockout), using only regular season numbers, to keep the comparison even.

For the first phase of the analysis, covered in this installment, I looked at two key metrics.  First — where are each of these players today?  There are really five possibilities —

• In the NHL, with the same team
• In the NHL, with a different team
• In the minors, college or juniors (collectivel referred to as “minors”)
• In Europe, with the KHL, SEL or another European league
• No longer an active hockey player

Using this breakdown, you can get some idea of the time it takes to bring a player to the league, identify how many players ever make it to the NHL level, and discover some intriguing trends in terms of mobility, both within the NHL and between the NHL and Europe.  To be sure, there is some subjectivity in these determinations.  All of the players were evaluated as of the end of the 20090-2010 NHL regular season.  Some had been called up to the big clubs to be on the roster for the playoffs, even though they would not play.  Should they really be regarded as “in the NHL”?  Sometimes yes, sometimes no.  Also, while this metric indicates the fact that a given player was at the NHL level, it makes no distinction between somebody who has played one game or 1oo games in the big league.    What was needed was a metric to define the quality of the draft pick — i.e.  how successful did the team ultimately prove to be in making that selection?

I ultimately came up with what I refer to as the “Player Participation Index” (PPI) , which, simply stated, represents the percentage of possible NHL regular season games  in which the player has participated since being drafted. In the final analysis, the goal of the NHL Entry Draft is to identify players capable of playing at the NHL level on a regular basis.  Points, goals, assists and other similar standards are too limiting, as players have different roles.  A checking forward who has played in 80% of his team’s games is as valuable a draft pick, for that role, as a scoring center is for that role.  For the same reason, TOI is also too confining, as it denigrates the contributions of third liners or those who are not special teams participants.   The PPI, however, does a very good job in accounting for players who are slow to develop or are injury prone  — both factors in determining whether a given pick is successful or not.

Note that I did not limit the PPI to games played for the team that drafted the player.  While this may seem strange at first, consider this — there are a wide variety of factors that go into the calculation over whether a player stays with a given club or goes elsewhere, none of which impact the merits of the original draft choice.  A player who attracts needed assets in a trade is just as valuable, in a different way, as a player who remains with his drafting club.  Team dynamics change over time, and weaker teams offer different opportunities than stronger teams.  Even if a player leaves via free agency, it does not impact the validity of the original draft choice.  Rather, that reflects the differing perceptions of value that the team and the player have at the time of his departure.

So, with the ground rules firmly in place, what do the results look like?

Now, when reviewing this data, keep in mind that we are talking about only the top three rounds here.  In other words, the cream of the crop.  Still, over the ten year period, only 30.58% of the players drafted are currently in the NHL, and only about 17% are with the same team that drafted them.  Sure, the numbers are a bit skewed by the fact that the players drafted in recent years haven’t percolated up to the NHL level, but that in itself tells a story all its own.  Only 6 of the 2009 draftees were at the NHL level, 12 from 2008 and 11 from 2007.   In 2006, the number jumps up to 26, giving some indication as to the minor league gestation period for an NHL player.    However, keep in mind that the number maxes out with the 2003 draft year, with just under 50% in the NHL at the end of the 2009-2010 season.  That’s a lot of time, effort and investment for what amounts to a coin flip, or worse, in terms of the odds of reaching the big time.

Also take a look at what happens over time.  Going back to 2000, only 33% remain in the league, and only 9% with the same club.  Twice that many (19%) are out of hockey altogether, and more are in Europe (40%) than are in the NHL right now.  Comparing the totals for Europe and the minors, there is a clear indication that players are willing to stay in the lower leagues here only so long, then venture off for more money in Europe.

Let’s look at this a different way –by Draft Round:
[table id=9 /]This is a pretty stark representation of the Law of Diminishing Returns. By the time you reach Round 3, only 45 of the 313 third round selections were in the NHL at season’s end.  In contrast, 72 were in Europe.  Even the first round numbers are not captivating — only 167 of 300 drafted (55.67%) in the first round over the past 10 years are in the NHL right now, and fewer than 1 in 3 are with the club that drafted them.  Clearly, the level of mobility shown over a relatively short period of time shows the “best available athlete” strategy in drafting.  Even if a club may not have the need for a puck-moving defenseman at draft time, if that is the best player on the board, he is likely to get picked, then used as trade material down the line.  (By the way, the differing draft numbers in the second and third rounds are attributable to compensatory picks awarded for a variety of reasons, making the total greater than 30 per year)

These numbers are eye-openers, to be sure, but they really only reflect a moment in time.  It tells us little about the quality of the players, nor about the depth or value in individual drafts.  For that, we need to look at the Player Participation Index for the past ten years:
[table id=10 /]
Here is where the details emerge — some of them quite surprising.  Consider that of 945 players drafted in the top three rounds over the decade reviewed, only 149 (15.76%) played more than half of the eligible games.  49% have never played an NHL minute, and fully 2/3 of those drafted in the top three rounds have played in fewer than 10% of the possible games.  If you are a General Manager, those are some pretty daunting numbers.  Obviously, those percentages go down precipitously if you were to consider all seven (formerly nine) rounds of the draft.

The numbers do support the notion that there are “good” drafts and “bad” drafts.   2003 was an unusually deep draft, with almost 30% of the players selected having a PPI of over .500.  Of the 2005 class, however, only 11% have exceeded that level.

Clearly, those teams seeking immediate help from the draft had better be both lucky and skilled at evaluating talent.  Only 6 players from the 2009 class played in more than 50% of the games last season, and only 6 from the 2008 class have a PPI over .500 for the two year period.  That number is only 5 for the 2007 class, and 8 for the 2006 crop of draftees.

These numbers show that the NHL GMs need to sift through lots and lots of sand, silt and dirt before finding some nuggets that are worth keeping.  Why is the success rate so low?  There are lots of reasons, including one-way contracts, limited opportunities, differing demands and the inherent difficulty in predicting future performance in the NHL.  We will be examining those topics in more detail in the final two installments of the series.

In Part II, we will dig a little more deeply into the PPI, look at differences by position, and examine some of the players falling at each end of the spectrum in an effort to show the difficulties posed by the drafting process.  In Part III, we will look at team draft performance over this ten year cycle, and see how (if at all) the draft performance has translated into success or failure on the ice.