Statistical Analysis Versus Scouting

There are times when I can imagine how Christopher Columbus must have felt when ridiculed by the masses while expressing his idea that the world was indeed round. Or when Charles Darwin first published his “Origin of Species,” in which he first introduced the theory of evolution.

Of course nothing I write about or believe is quite as profound or meaningful as either of the aforementioned gentlemen and their beliefs but I still find myself ostracized in the same way nonetheless.

What I’m getting at is my belief that numbers can prove useful to evaluators and managers in the NHL and the blatant disregard that I and people that agree with my beliefs encounter from professional hockey managers and scouts.

I recently exchanged emails with a front office employee of an NHL team and was told that statistical analysis was a “tool in the toolbox but by no means could you complete a job just by analyzing numbers.” While I respect everyone’s opinion, especially that of a front office employee of an NHL team (a job I would love to have myself), I find it hard to understand the stance exhibited by most front office people and scouts in regards to analysis. Sometimes, it makes me want to ram my head into a wall or something.

I suppose this position comes from the misunderstanding of what we as analysts are trying to do. We aren’t trying to replace the art of scouting with the science of analysis. We are trying to augment what the scouts do with analysis. Numbers and facts are what we are trying to attach to the opinions of scouts and front office guys.

Let’s face it; they aren’t always right when evaluating talent so if information can be introduced that reduces the odds of missing on a draft pick or free agent signing then why wouldn’t they want that information at their disposal?

Allow me to articulate just how often and how badly teams can miss in the draft. I’ve already published a study on this site showing that during a 10 year period (1993–2002) just more than 25% of players drafted during that time appeared in even 50 NHL games. So if a team made 8 picks in a draft in that period the odds say that only 2 of them would see anything more than a “cup of coffee” at the NHL level. That’s not good.

Now I’ll look into the results of 3 drafts (1980, 1982, and 1983) to show you a little more. What I’ll do is give you the first overall pick in each of those drafts along with several players from each season that outperformed their respective #1 overall picks.

1980

#1 Doug Wickenheiser (556 GP, 276 Pts)
#8 (Just for good measure) Fred Arthur (80 GP, 9 Pts)
#69 Jari Kurri (1,251 GP, 1,398 Pts)
#73 Bernie Nicholls (1,127 GP, 1,209 Pts)
#120 Steve Larmer (1,006 GP, 1,012 Pts)

1982

#1 Gord Kluzak (299 GP, 123 Pts)
#43 Pat Verbeek (1,063 Pts, 522 G)
#56 Kevin Dineen (1,188 GP, 355 G)
#88 Ray Ferraro (1,258 GP, 408 G)
#134 Doug Gilmour (1,474 GP, 1,414 Pts)

1983

#1 Brian Lawton (483 GP, 112 G)
#125 Rick Tocchet (1,144 GP, 440 G)
#207 Dominik Hasek (389 Wins)

So four high draft choices that were relative busts (Wickenheiser, Arthur, Kluzak and Lawton) three sure-fire, hall-of-famers (Hasek, Kurri, and Gilmour) and 6 all-star/borderline hall-of-fame players that were drafted much later.

Well, it was 25 and more years ago but evidently the scouting back then wasn’t so good. It has to have gotten better, right?

Maybe not.

Here’s three players in this year’s top 25 in scoring that were discovered late in the draft or weren’t drafted at all.

#6 (tied) in scoring – Brad Richards (1998, Rd. 3, #64 overall)

#12 in scoring – Martin St. Louis (Undrafted)

#23(tied) in scoring – Dustin Penner (Undrafted)

So what exactly happened to the scouts there? Richards is an all-star and former Conn Smythe winner. St. Louis is also an all-star and won the Art Ross, Lester B. Pearson and Hart Trophy (as league MVP) after the 2003-04 seasons; and he wasn’t even drafted! Same with Penner who struggled in his first two years after being awarded a fat contract offer by the Oilers but has since regained his goal scoring touch this season and has a reasonable shot to make the all-star team this year. Looks like those scouts aren’t as perfect as they may think after all.

How often would you guess that two or more scouts from the same team have different opinions on a player? I would guess that happens fairly often.

Let’s say that one scout feels “Player A” isn’t big enough to play at the NHL level while another scout feels the same player has the instincts and elusiveness to make up for his lack of size, ala Marty St. Louis. What’s a GM to do? Well if he had some numbers, some facts that reinforce one opinion over the other then that GM would be in a position to make a more intelligent and informed decision. His odds of not missing on this player are reduced simply by the inclusion of some inarguable facts.

Does anyone think that the excessive contracts that legendary Rangers’ GM Glen Sather lavished on the likes of Chris Drury, Scott Gomez and Wade Redden might have been avoided had any sophisticated analysis been offered to the Rangers’ GM? Obviously scouting alone didn’t convince Sather to make the right decisions on those deals.

How about Bob Gainey and his severe overpayments in dealing for the aforementioned Gomez and signing Brian Gionta and Hal Gill to multiyear deals? Is that evidence of superior scouting?

None of the analysts that I have read or spoken with are of the opinion statistical analysis is better than scouting. However, we do believe that analysis makes scouting better and smarter. Analysis can help prove true what the scouts’ eyes see.

Another aspect of analysis that makes life easier for a manager is identifying market inefficiencies.

That was the real secret to the success of the Oakland A’s in the early 2000’s when “Moneyball,” was written. They discovered through research that On Base Percentage (OBP) was vital to the offensive success of a baseball team. It just so happens at that particular time, OBP was also an undervalued singular skill. This combination of factors enabled the A’s to stock up on players that excelled at getting on base (while not necessarily excelling at other offensive categories) and thus allowed the A’s to reside at the top of their league’s standings while fielding one of the lowest payrolls in the sport.

That is another goal of all analysts in hockey. We are working to identify skill set(s) that are currently undervalued by the current market. It isn’t easy but I believe that it can and will be done.

So my question to my faithful students is this; why can’t the analysts and old-school scouts and management co-exist together in NHL front offices? The current attitude among NHL management types today doesn’t bode well for the likely addition of statistical analysis to NHL front offices any time soon.

Maybe I’m wrong; maybe teams already have included analysis in their decision-making but don’t want to let on that they have. I would like to think that even if they haven’t yet, they soon will.

Thanks for joining me and I encourage you to hit me up with your thoughts on my topics and opinions at gmiller@insidehockey.com. Also, you can follow me on Twitter: gkmkiller.

facebooktwittergoogle_plusredditpinterestlinkedinmail

Comments are closed.