Explaining Playoff Goaltending

Are goaltender’s big-time playoff performances best explained by their regular season or historical playoff results? Can some goalies just turn it on when it counts more?

Clutch Off the Bench

The 2018 1st round series featuring the Columbus Blue Jackets and Washington Capitals was an interesting case study in playoff goaltending performance.

Starring for Columbus was Sergei Bobrovsky. The reigning Vezina Trophy winner was coming off another very good season, hoping to continue rolling in the playoffs. However, despite Bobrovsky’s accolades he has never advanced past the 1st round and has been uncharacteristically below average preventing goals in all of his prior playoff appearances.

In another subplot, Washington actually began the series with Philipp Grubauer. He had been excellent in the regular season but relatively untested in the playoffs (he played parts of two clean-up games, neither went great). This decision put Braden Holtby on the bench, who had a very pedestrian regular season but had been at or above average in all 5 of his previous playoff appearances. Cumulatively playoff Holtby has prevented about 1.1% more goals than expected, 2nd only to Jonathan Quick of goalies entering the playoffs with at least 1,000 playoff shots to their names.

Everyone knows what happened next. Grubauer wasn’t great, while Washington dropped their 1st two games at home. Holtby came in and delivered 4 straight above average performances, while Bobrovsky ended the series with 3 straight below average games. Washington took the series 4-2.

A few interesting questions can come out of this series I hope to explore. Was Washington coach Barry Trotz right to go with the ‘hot-hand’ over the ‘proven-vet’ by starting Grubauer? Is it likely that a goalie might be good in the regular season, but below average in the playoffs? More generally, if we are trying to explain goaltending performance in the playoffs, what matters more? Past playoff performances, regular season results, or just career results?

Can someone simply turn it on in the playoffs after a below-average regular season?

High Stakes Noise

Let me preface most of this exploration with the understanding the idea of ‘clutch’ or ‘performance-when-it-matters’ is problematic from a statistical perspective. A few bounces over a playoff series might dictate whether the outcome is perceived as ‘clutch’ or ‘choking.’ In reality, a good game or bad game doesn’t have much effect on the outcome of the next game, but if you flip a few heads in a row (bad games) you are out of the playoffs, while a few tails mean you advance. Someone has to advance, so a ‘clutch’ narrative might be created from chance outcomes alone.

With a small sample like a playoff series, a bounce or two can change the narrative of those outcomes. Analysts can deal with this by framing the outcome with a range of uncertainty. Fewer shots or games mean more uncertainty. Ultimately, we can’t be too sure the outcome of a series reflects the ‘truth.’ Holtby could have come into the playoffs with his game in top condition and his vitals in the optimal range to deliver a clutch performance, but if a few Seth Jones’ shots bounced off of someone’s ass in game 3 or 4, the narrative is completely different. Drilling down further (tied in the 3rd period only, etc) only compounds the problem of insufficient sample size.

Is Winning a Skill?

It’s important to the scientific process that we assume our hypothesis is null then work to prove it with data. A ‘clutchness’ factor is no different, we should assume it doesn’t exist. It might not exist as a differentiator at the NHL-level for good reason, a propensity to fold in critical moments would likely prevent them making it.

However, this doesn’t feel right. I’ve played with the pressure of losing the game 1-0, and it’s certainly easier than winning 2-1. Goaltending can easily be the equalizer between a dominant team and a dominant win, possibly even flipping the script to a loss. Goal prevention is the best way for a goalie to win games. However, it’s possible that some goalies might be consistently better in crunch time than their goal prevention would suggest.

Regardless whether you think being clutch might be an innate skill some have or whether those differences are incredibly tiny at the NHL-level, we have to acknowledge that the finite and imperfect nature of the data will likely be a limiting factor.

What Does the Data Look Like?

The objective of this analysis is to explain goaltender playoff performances using data available prior to round 1, game 1. The target of interest is playoff goal prevention per shot, save % less expected save %. If a goaltender faced 25 expected goals on 250 shots, but only conceded 20 actual goals, this would be a 2% lift (5 / 250 or 92% – 90%). Actual save % may deviate wildly from expected save % in small sample sizes like the playoffs. A few bad goals and/or unlucky bounces against will likely prevent a chance of redemption.

To help explain the selected measure of playoff performance for each season, the save % lift can be calculated for:

  • the regular season performance prior to that playoffs
  • entire career regular season performance prior to that playoffs
  • entire career playoff performance prior to that playoffs
  • a proxy for goaltender workload at the onset of the playoffs

Visualizing the relationship between the save % differences we see a small relationship and correlation between each. As predicted, the variance in playoff results (y-axis) is higher than the explanatory variables (x-axis) with a higher sample size. Initially, it appears regular season results are most correlated with playoff success (a perfect correlation would be equal to 1 with each point falling along the grey diagonal line). Career regular season results have the least variance and lowest correlation.

playoff explained
High Variance Results

Do these any or all of these metrics matter when explaining playoff performance?

The Weight of the Playoffs

In order to understand how each of the explanatory inputs matter we can use a multiple linear regression. This helps us quantify the direction and strength of the relationship between the explanatory variables and playoff performance.

Running a regression of 122 goalie-seasons facing at least 100 shots in the playoffs and 1000 shots in the respective regular season results in the model below.

Variable Coefficient Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.0011 0.00 0.253 0.801
Career Playoffs Sv% Lift 0.2176 0.11 1.987 0.0494*
Regular Season Sv% Lift 0.8485 0.27 3.15 0.002**
Career Regular Season Sv% Lift 0.1203 0.33 0.37 0.712
Weighted Shots in 15 Day Window Prior Playoffs 0.0000 0.00 -1.131 0.261
Playoff Rookie 0.0004 0.00 0.089 0.929
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.01536 on 124 degrees of freedom
Multiple R-squared:  0.1342, Adjusted R-squared:  0.09341
F-statistic: 4.292 on 4 and 124 DF,  p-value: 0.002752

Notably, this is a pretty weak model, confirming the intuition that playoff performance is tough to explain. But directionally, regular season results are more significant and the coefficient is larger than career playoff results. Also noteworthy, career regular season results have no significant effect (though directionally positive) on the playoff results once the current season and career playoff results are controlled for. Workload has a no significant effect, though directionally negative. Being a playoff rookie also has no effect, but is directionally negative too.

Formula For Success

Dropping the insignificant variables and re-running the regression creates the formula below to (loosely) calculate the expected playoff results.

PO \triangle Sv% = -0.3% + (0.21 * Career PO \triangle Sv%) + (0.83 * RS \triangle Sv%)

So, for example, Holtby entered the 2018 playoffs with a save % lift of +1.14% in prior playoffs, but only -0.18% in an uncharacteristically mediocre regular season. The regular season results are weighted about 4 times more important in the formula, resulting in an expected -0.2% save % lift in these playoffs, which he’s exceeded to date.

Bobrovsky’s prior playoff results (-2.27%) pulled down his regular season results (+0.5%), expecting a -0.33% performance. He finished with a -1.22%.

Despite a great regular season, Grubauer’s expected save % was about 0% due to poor prior playoff appearances pulling it down.

playoff projections
Subject to Change

Summary

If there’s anything to take away from this analysis is that explaining playoff performances is difficult. This was likely obvious to anyone who’s watched playoff hockey. Small sample sizes, survivor bias, and out of control narratives, playoff hockey has everything to confound a good analysis.

That said, some things do matter directionally. Entering the playoffs after a good regular season is probably more important than a good playoff track record. Braden Holtby may have bucked this trend playoffs-to-date, but it was probably more likely his regular season results were lower than his true talent suggests.

The results also suggest that waiting for a goalie’s playoff results to regress to a career average is generally fruitless. This makes sense intuitively, a goaltender may change teams and systems. They develop and regress. Regular season results likely give enough of a snapshot of where their game is at that entire career regular season results are unnecessary. Marc-Andre Fleury entered the 2018 playoffs with excellent regular season results, average career regular season results, and below average playoff results. This was a recipe for success based on the basic model (expected +0.5%, chart above) and he’s subsequently delivered with excellent results (he currently has the best save % lift of goalies with over 500 shots in the dataset going back to 2011).

With all of these considerations, there is nothing to suggest a goalie can simply turn it on for the playoffs. Proven experience certainly helps, but it’s more important to have posted good results with the most current team and defensive conditions.

Washington Re-Visited

Was Trotz right to start Grubauer? Probably. Playoff series are short and Grubauer had played excellent during the regular season. However, past playoff results do have a partial explanatory effect, partly because there are other considerations in the playoffs. Playing styles can change, physicality around the net can increase, and facing a well game-planned opposition for 4 to 7 games means that tendencies and tempers can amplify. Holtby had experience in those situations, not enough to completely offset the difference in their regular season, but close.

Bobrovsky can take comfort in the fact that his playoff results should have been better than they turned out this season. There’s likely no use in him re-visiting these playoff letdowns, his best bet is to look forward, focusing on another big season and carrying that performance forward. Either the results will come naturally or maybe he will be carried up by some positive unexplained variance.

 

Thanks for reading! I update goalie-season data using expected goals, it can be downloaded or viewed in my goalie compare app. Any custom requests ping me at @crowdscoutsprts or cole92anderson@gmail.com.

Code for this analysis was built off a scraper built by @36Hobbit which can be found at github.com/HarryShomer/Hockey-Scraper.

I also implement shot location adjustment outlined by Schuckers and Curro and adapted by @OilersNerdAlert. Any implementation issues are my fault.

Leave a Reply

Your email address will not be published. Required fields are marked *