Preamble: The following is a paper I wrote while in college about 6 years ago. It is very theoretical, without understanding the realities of data quality in the real world. However, it still reflects my general attitude toward how goaltending performance should be measured, manifesting itself in my current Expected Goals model.
How new metrics concerning hockey’s most important position can offer critical insights into goaltender performance, development, and value.
During the last 20 years, the goaltending position has changed more than any other position in hockey. Advances in equipment and training have raised the benchmark for expected goaltender performance. Teams promptly began investing in the position in the mid-90’s as a new breed of goaltender found success in the NHL. From 1994-2006 an average of almost 3 goaltenders were selected in the 1st round. Of these 37 highly touted goaltenders, none had won a Vezina trophy as of 2011. With this surprising lack of success, teams began to avoid using high draft picks on goaltenders—from 2007-2011 less than 1 goaltender was drafted in the 1st round annually.
Teams will continue to invest less in the goaltending position for a number of reasons. First, it is a matter of economics—the supply of good goaltenders has increased, decreasing their value. Initially, the demand for goaltenders drove their stock up, but teams eventually realized that they struggle to correctly value goaltending prospects. Subsequently, many of the leagues most successful goaltenders during this period were late round picks. Outside of the legendary Martin Brodeur, the last 3 Vezina trophy winners were drafted in the 5th, 5th, and 9th rounds. In fact, in the last decade the only goaltenders to make the NHL 1st or 2nd All-Star teams that were drafted in the 1st round were Roberto Luongo and, of course, Martin Brodeur. Lastly, goaltenders appear to mature later, which means teams want to invest less in them, especially considering the new Collective Bargaining Agreement allows players to become free agents earlier. In summary, there are more good goaltenders, they are generally incorrectly valued, and teams are hesitant to develop goaltenders through the draft, preferring high-priced, experienced goaltenders.
These factors create a unique opportunity for teams that can properly value goaltenders. Goaltending is still a critical part of any team, but it can be acquired without giving up valuable assets. Goaltenders are generally selected later in the draft, exchanged for less than their intrinsic value via trade, or require no assets to acquire through free agency and from waivers. Solid NHL goaltending should ideally come at a friendly cap hit, since the premium for the highest paid goaltenders is diminishing. Another trend is evident: some of the most successful teams are using strong backups throughout the regular season to compliment their starters and gain a post-season advantage—since the 2005 lockout the average Stanley Cup winning goalie has played less than 50 regular season games. Teams can no longer hope to find a franchise goaltender and maintain elite performance by locking them up to a rich, long-term contract without possessing the option of cheaper alternatives. The inability for teams to objectively understand the difference in performance between a goaltender with a $5 million salary and a $1.5 million salary is curious—goaltending is the only position in hockey that performance could be measured in a largely empirical way, analogous to how baseball has managed to successfully employ advanced metrics to better measure player performance. Teams that could use goaltending metrics that more accurately evaluate goaltenders would have an enormous advantage to acquire and retain elite level goaltending at an economical price.
The Estimated Save Percentage Index Model
The most common metric used to measure goaltending performance is save percentage, the number of saves as a percentage of total shots on goal. This metric is fundamentally flawed. To more accurately understand the quality of a particular goaltender, save percentage must be more sophisticated. This is possible because the goaltending position has two important prerequisites that make performance the most quantifiable in hockey. First, the result is absolute: any shot on goal is either stopped or results in a goal. Second, the position is passive: the difficulty to the goaltender is generally dictated by the game in front of him, except for rebound control and puck handling, which can be addressed later in the model.
The Expected Save Percentage (ES% Index) is a predictor of a goaltenders success based on a number of inputs that assigns the individual difficulty of each shot the goaltender faces. The inputs used in the model are shot location, puck visibility, and the rate at which the puck changes angle before or during the shot. The model assumes the goaltender has NHL quality blocking-width, positioning, lateral movement, and reflexes. Then, through an array of formulas, the model determines the expected save percentage for each shot on goal given the inputs. Once these expected save percentages are aggregated over a game, or over a season, we can see how the goaltender’s actual save percentage compares with the expected save percentage and compare them to their peers. The best goaltenders will consistently exceed the predicted save percentage whether they are facing 20 high quality shots or 40 lower quality shots. The Expected Save Percentage Index—the difference between real save percentage and expected save percentage—will measure the proficiency of the goaltender. The index can be tracked game-by-game and season-by-season. Since we are removing much of the fluctuation in team performance we will have a much better idea of a goaltender’s consistency—an attribute critical to NHL success that can be lost in the potentially misleading statistics that are currently employed.
The inputs have been selected for simplicity and versatility. The most obvious is shot location—the closer the shot, the more likely it will be a goal. Assuming the average NHL shot is about 90 miles/hour and a NHL goaltender has a reaction time of .11 seconds, the Expected Save Percentage increases greatly once the shot is from a distance of greater than 15 feet. Inside of 15 feet it assumes the goaltender can cover around 70%- 80% of the net through size and positioning, and the distance model reflects this assumption. Location can also allow the model to determine the shot angle and net available to the shooter, two other factors that are automatically worked into the model. If applicable, visibility is a binary input determining whether the goaltender has a chance to see the puck. Again, since we are assuming NHL quality goaltending, there is no ‘half-screen’ or ‘distraction.’ If the goaltender has an opportunity to see the puck, they are expected to gain a sightline to the puck. If they are completely screened, the expected save percentage is lowered as a function of the net available when the shot is taken—the better angle, the more dangerous the screen. Lastly, the model factors in the rate of the change in the angle of the puck when the shot as taken, if applicable. This way we can discount the expected save percentage if the shot is a one-timer, deke, passing play, or even a deflection to better reflect the difficulty of a shot against. The model assumes NHL quality lateral movement, edge control, and post save recovery. At lower levels, where puck movement is slower, goaltenders will have to put up higher real save percentages to maintain an ES% Index that predicts NHL skills.
These inputs create an admittedly arbitrary, yet sophisticated, expected save percentage. The formulas can be retrofitted as more data is collected to move closer to a universally accurate expected save percentage—ideally the median ES% Index would be 0. The data can be then broken up into three categories, shots with no screen or movement, shots that are screened, and shots where the puck is moving laterally as it is released. Breaking each shot into individual components will make it possible to track and eventually acquire objective data, replacing the placeholder formulas with actual NHL results. However, as it stands now, the expected save percentage is a benchmark, and it is the discrepancy between the realized and expected save percentage that will be the true measure of individual performance. Shot placement may seem like a troublesome omission from the model, however since the model is built on aggregated averages we can account for the complete distribution of shots put on net. NHL quality defense generally takes away time and space from shooters, limiting their ability to place the puck wherever they desire. Teams are not necessarily inclined to giving up shots in a particular place in the net, but weaker teams are prone to giving up shots from more dangerous locations on the ice. In this way shot placement is indirectly built into the expected save percentage: a shot from 10 feet out the shooter has a much greater chance of hitting a target, say high glove, than a shot from 20 feet.
The ES% Index measures goaltender performance in a vacuum, comparing actual performance to how we would expect him to perform in a given situation. However, the goaltender can influence the amount of shots they face through rebound control and effective puck handling. Tracking these occurrences will allow the model to adjust the expected save percentage further. Easier than average shots that result in a rebound will lead to the successive shot not being factored into the model. This is analogous to saying the resulting shot should not have happened. Difficult shots that result in rebounds will take into consideration the difficulty of both shots when assigning expected save percentage to the potentially ‘preventable’ rebound shot. Whenever a goaltender handles the puck and it results in the puck directly clearing the zone, it will be assume the goaltender prevented a shot a certain percentage of the time. By adding the potential shots and removing preventable shots to the actual shot total we will have a good idea of how the goaltender is helping their team and influencing the game.
With the expected save percentage and expected shots against, we can manufacture an expected goals against for each game. We can compare expected goals against to the goal support the goaltender received and determine whether or not the goaltender should have won the game. If the game should have been won based on the actual goals for and expected goals against, but was not, this will be a contributed loss. Conversely, if it was predicted the team should have lost, yet won, this will be a contributed win. So we can remove the bias toward goaltenders on bad teams—who have more opportunity to register contributed wins—we can measure the number of potential contributed wins and losses and compare them to the actual contributed wins and losses.
How does this model predict future goaltending performance?
This analysis allows an NHL team to gain a concise, quantified measurement of goaltending performance across leagues and time. It will more accurately identify goaltending proficiency and consistency. It can be adjusted from league to league as the goaltender advances and will better predict future success as the database grows. The model automatically assumes each goaltender has NHL size, speed, and positioning, so if the goaltender can consistently perform better than his peers, then they will likely continue to outperform them at higher levels. This can apply to a late round pick playing on a weak team in Europe or a college goaltender discredited for being on a strong defensive team. Since the ES% Index can be broken into components—stationary shots, screened shots, and moving shots—it will be easy to identify weaknesses that may be hidden by a specific team. For example, a goalie with poor lateral movement on a team that limits puck movement might perform well by traditional standards, but if the ES% Index on shots with puck movement is below average, chances are they will be exposed at the next level. There is a very real advantage to employing increasingly accurate goaltending metrics that other teams are not using to value goaltenders. It can also be broken up into individual components lending itself to the in-depth analysis of goaltending prospects, opposition goaltenders, and even the performance of other players on the ice. While the ES% Index will likely have limitations, predicting the development and value of goaltenders has not improved during an era when the quality of goaltending has increased dramatically. Therefore, a more accurate metric will almost certainly improve the valuation of each goaltender and offer critical insights into their development.
While advanced goaltending metrics can aid management decisions, they can also lend coaches a helpful perspective when preparing for games. The objective ES% Index will help explain some of the volatility in goaltender performance. Coaches do not always understand the subtleties of the position, their only concern lies in the proficiency of the goaltender in preventing goals—exactly the intent of the ES% Index. It can also be used as a pre-scout for opposing goaltenders. Situational success rates for each NHL goalie are tracked through the season, offering a strategic advantage to the coaching staff and players. If an otherwise successful goaltender is performing below the norm on shots with puck movement, then this is a clear indication to move the puck before shooting. Ability can be judged based on data from an entire season rather than anecdotal observations. This is advantageous because the goaltending position is inconsistent by nature, one bad bounce or mental lapse can be the difference between a good game and a bad game. Watching a select few games of a goaltender will make it difficult to judge their true ability—no doubt part of the reason teams struggle to value goaltenders at the draft. It can also compliment scouting reports. If a scout sees a particular trend or weakness in a goaltenders game, there will be data available which can be used to verify or contradict the scout’s claims.
Additionally, goaltender performance can influence the statistics of players at other positions. Both a defenseman playing if front of poor goaltending and a goal scorer who faced an unlikely sequence of superb goaltending are going to have their statistics skewed. Adjusting these statistics for goaltending performance will give management a clearer idea of why a certain player’s statistics might be deviating from their expectations. For example, the model can be expanded to measure the difference between even-strength expected goals for and expected goals against for each player over the course of the game based on the data already being recorded. This type of analysis is separate from the ES% Index, however having more accurate goaltending statistics would provide an organization another tool properly evaluate players and put the absolute best product on the ice.
No statistical analysis can replace comprehensive subjective evaluation that is performed by the most experienced hockey minds in the world. However, it can offer a fresh perspective and lend objective analysis to a position where contrarians can often be the most successful. The unorthodox goaltending styles of Tim Thomas and Dominik Hasek have remarkably won 8 out of the last 17 Vezina trophies awarded. Not only were they drafted in the 9th and 10th rounds, respectively, they did not even become starting goaltenders until aged 32 and 29 despite their success outside of the NHL. Very few understood how they stopped the puck, but both men clearly prevented goals. It is my hope that employing more advanced goaltending metrics can remove the biases that exist and pinpoint goal prevention, the sole objective of a goaltender. Due to my extensive knowledge of the position as both a student and a coach, the model has been constructed to reflect the complex simplicity of the position—Where is shot from? Can I see it? Can I reach my optimal position?—while deducing the existence of attributes that are critical to NHL success: size, speed, positioning, lateral movement, and consistency. For these reasons, Expected Save Percentage Index and Win Contribution analysis manages to combine the qualitative and quantitative factors that are necessary to properly evaluate goaltenders, benefiting any team that employs these advanced metrics.