Introducing DEV

by Zac Urback

TL; DR:

Introduction to Draft Analytics

Projecting the likelihood that an 18-year old succeeds in the NHL is extraordinarily difficult, as manifested by the sheer number of “busts” that were selected in previous NHL entry drafts. When amateur scouts evaluate draft eligible prospects, they attempt to project the likelihood a player makes the NHL, and the calibre of player that prospect will become. Scouts analyze a player’s skillset and character, along with a wide-range of other variables, which they believe predict future NHL success. One common tactic employed by scouts to help with their projections – whether it is conscious or unconscious – is to compare particular prospects to a player that they have previously scouted. Hayden and I have borrowed this tactic to create a tool that quantifies the expected value of a prospect, based off of his peers’ success-rate, and then compares those results with the expected value of a draft pick. The tool is not intended to replace traditional scouting, but it can be a useful aid to any team and/or scout looking to improve their draft selection efficiency.

Introduction to the Model

Hayden and I began discussing the viability of creating a tool that projects the success expectancy of individual NHL prospects in early 2016. After Hayden scraped the historic CHL data required for this project, we started analyzing which statistics were most predictive of future NHL point generation. Initially, we tested several dozen variables: advanced CHL statistics, such as estimated points per 60 minutes of ice-time; GF% relative to team; individual points percentage (IPP); and many other metrics, which can be found on Hayden’s website1. We eventually found out that the best predictor of future NHL production is draft year primary points per game played. We then implemented a modifier for the player’s age based off the work by Rhys Jessop with some tweaks2, and included an era and league adjustment by calculating marginal goals by league per year.3 Applying the aforementioned modifications to individual players, primary points per game played increased the predictive ability of the metric, and best allowed us to identify cohorts. With this, DEV (Draft Expected Value) was born.
DEV produces simple outputs: it determines an “expected NHL point per game” number for each prospect based off of the actual NHL success rate of the prospect’s most similar cohorts. To determine the cohorts for each player, the model searches for seasons in which other eligible, first-year draft players were of similar size and scored at a similar rate (by looking at age, era, and league adjusted primary points per game). Once we find a list of comparable players, we test similarity scores for the comparable players, similar to Emmanuel Perry’s similarity score calculator4. We use these scores as weights for each cohort. Using this information, we then find the weighted arithmetic mean of the successful comparable player’s NHL P/GP to find the prospect’s expected NHL P/GP — assuming that they make the NHL full time — and multiply this by their cohorts NHL success rate to determine an expected value. The result allows us to  compare each prospects expected NHL point per game to the expected NHL point per game of each draft selection, using Steve Burtch’s analysis.5 For example, there are 43 cohorts for Timo Meier’s draft-year. Of those 43, 24 became successful NHL players (NHL success is determined by them playing 200 NHL games or more). The weighted average NHL points per game of the successful NHL players compared to Meier is 0.57. By multiplying the success-rate of his cohorts (55.8%) and the weighted average of the successful player’s NHL points per game (0.57), we find that Meier’s draft-year expected value is 0.32 NHL P/GP, which is practically equal to picks 8-10 overall in the draft.

DEV is not meant to replace traditional scouting, but rather to help add perspective to particular prospects, identify potential “steals” or “busts”, and game the draft through the  identification of market efficiencies and inefficiencies. In the 2011 NHL draft, for example, Tyler Toffoli, Brendan Gallagher, and Mark Stone are likely to be the 3 biggest CHL forward steals selected outside of the first round. They went 47th, 147th, and 178th overall respectively, whereas our model ranked them 16-19th, 60-70th, and 97-110th. That being said, DEV is not perfect. The model ranked Greg McKegg 11-15th, which in hindsight, would have been a mistake. We acknowledge that there are many flaws inherent to the model; for instance, it cannot currently factor quality of line mates or style of play (outside of point generation). It can also be unreliable when a prospect has very few cohorts, such as Mitch Marner who only had 2 cohorts, or Connor McDavid who was unranked by the model because he had no comparable players. Although DEV serves as a useful aid, it is important not to solely rely on it. If used properly, DEV will improve upon organizations’ draft selection efficiency.

Conclusion

DEV was inspired by the "Prospect Cohort Success" (PCS)6 tool, created by Josh Weissbock and MoneyPuck, who have since been hired by the Florida Panthers for their tremendous work. Currently, DEV only analyzes prospects from the CHL, but future plans will add more leagues. We plan to make the tool available for public use prior to the 2016 NHL Entry Draft. Hayden will soon release a more technical write-up about DEV, but in the meantime, please contact us via Twitter (@Zac_Urback and @3Hayden2) or e-mail (ZacUrback@Rogers.com and 3Hayden2@gmail.com) with any questions or comments. Special thanks to Steve Burtch for providing invaluable help throughout the process of developing this model.

1 Prospect-Stats

2 Rhys Jessop’s CHL Age Adjustment Article

3 Alan Ryder’s Marginal Goals Article

4 Emmanuel Perry’s Similarity Scores Article

5 Stephen Burtch’s Draft Pick Expected Value Article

6 PCS (Prospect Cohort Success) Model