Modelling win probability in shootouts

[Summary for new readers: the takeaway from all this is that you can choose to send out your three best shooters in any order you like for the first three shots; after that, go best-to-worst.]

At any given moment in a shootout, how likely are you to win?

This isn’t a difficult question to answer, because the shootout (in its current NHL form) is a simple process: Teams take penalty shots in turn until there’s a winner. Three shots minimum. Whoever has more goals wins. (Volleyball is also easy. Baseball is way harder. Don’t even get me started on basketball or hockey or football.)

And if we assume that each shot in a shootout is independent of all previous shots in that shootout — not a unreasonable assumption to make — then the process is even simpler.

We’ll model the league-average case, which assumes all shooters have a 37% chance of scoring on their shots. (According to this report, the average success rate is 33% so far this year; we’ll use 37% for now.) I am speaking within an NHL context but this is also applicable to the shootout at other levels of hockey: assuming the teams are evenly-matched, one can substitute 33% or 25% or whatever the success rate is in for the 37% used here, and the win probabilities and leverage indices will change accordingly.

Math stuff

Each state in the shootout can be uniquely described by three things: the round (first, second, third-or-later), the team currently shooting (first or second), and the lead currently held by the first team (-1, 0, 1, etc.). So the shootout begins at the 1/1/0 stage: round 1, team 1 shooting, tie score (team 1 up by 0). If the first team scores, we transition to another state: 1/2/1 (still round 1, team 2 now shooting, and team 1 up by 1). If the first team misses, we go to 1/2/0. The second round begins with either 2/1/0, 2/1/1, or 2/1/(-1), depending on which team (if any) has the lead after the first round.

Assuming that previous shots don’t affect the current shot is what allows us to model the shootout as a Markov chain. Each state in the process is one of those 1/1/0 triplets from the previous paragraph. The transition probabilities between the states are based on the league-average success rate from above. For example, starting from 1/1/0, there’s a 37% chance that Team 1 scores and we go to 1/2/1, and a 63% chance they miss and we go to 1/2/0. At some point the shootout ends, with either a 0% or 100% chance of winning for Team 1, and so eventually we reach what’s called an absorbing state. Then, since every state in the shootout must lead to a win or a loss eventually, we can associate each state with a win probability based on how likely it is that the average Team 1 will end up winning once they reach that state.

To take away the math-talk for a second, we all know goal-miss-miss-miss gives the first team a 1-0 lead with one shot left each and that’s clearly a better position to be in than goal-miss-miss-goal (tied). But how much better? We can find out exactly by connecting all 15 states together based on which ones can be reached from the others.

Note that since we have two league-average teams by assumption, the third round (and any later rounds) will always be a 50-50 shot. In other words, the win probability of the 3/1/0 state is defined to be exactly 0.5.

Results

Here are the results. All probabilities are from the perspective of the team shooting first (Team 1).

Unsurprisingly, the best state to be in, as Team 1, is scoring on your first two shots while having your opponent miss their first (the state 2/2/2). Any miss by them or any goal by you in any of the next three shots will end the game, and so you have a 96% chance of winning.

What’s also interesting is the leverage of each situation. If we define everything relative to the first shot of the shootout, we see that the most crucial shots are those in the third round or later when Team 2 is shooting or Team 1 is shooting while tied — and that these shots are 56% “more important” than the first shot. At some level, every shot is worth the same, of course, but what this means is the difference between scoring and missing matters more in the third round than the first. (Again, that’s obvious, but now we know exactly how much.)

When should you send out your best shooter?

It’s rather obvious that your three shooters should be your best. But in what order?

Well, there’s a chance that you a) won’t even need to take a third shot or b) won’t have the chance to take one, so the best two should definitely go first and second. We can use the leverage numbers in that chart to figure out which order to use.

Say you’re Team 1. Your first shot will always have a leverage index of 1. Your second shot will have either 1.19, 0.86, or 0.67, depending on what happens in the first round. As it happens (note: this is not a coincidence), when you weight those indices by the probability that each corresponding state will occur, you get a leverage index of … exactly 1.

If you’re Team 2, the math is a little more intricate but not impossible. The first shot always has a leverage index of 1, as we can see from the chart. The second shot will have an LI of either 1.19 or 0.36. The probability that it’s 0.36 is the same as the probability that Team 2 is up by one goal (miss-goal-miss) plus the probability that Team 1 is up by two goals (goal-miss-goal), which turns out to be 0.2331. The rest (0.7669) goes with the 1.19 — and, wouldn’t you know it (note: also not a coincidence), 0.7669 * 1.19 plus 0.2331 * 0.36 is exactly 1.

So it doesn’t matter. Both shots are equally important to you, and you should send out your best two shooters first and second in whatever order you like.

This also shows us that it doesn’t really matter if you shoot first or second. This isn’t baseball, where batting last gives you an advantage because you know how many runs you need to win. (The baseball analogy would be a home-run derby, with one pitch to one batter from either team until someone wins, which aside from being stupid would not confer any advantage on the first or second team. Of course, if the NHL ran MLB, they’d probably institute that right away.)

EDITED TO ADD: I should have kept going here, because a helpful fellow points out that the third shot is higher-leverage even if it’s less likely to happen. And sure enough, the average LI of the third shot, not knowing whether you’ll have one, is 1 as well.

Other stuff you can do with this

  • You could figure out how many wins each of your shooters/goalies has been worth in shootouts. If Joe Player scores on the very first shot, that brings his team’s win probability from 50% to 70%, giving him 0.2 of a win or 0.1 standings points. (The absolute nonsense of awarding an extra point to a team based on this process is not lost on me, I trust you to know, and I’ve tried to ignore it until now.) This “win probability added” will mostly correlate with that player’s overall success rate in the shootout, though, unless you have someone on Team 2 who only scores when his team is either up by 1 or down by 2 entering their second shot — which means that his goals don’t really matter, since the game is basically already over.
  • You could graph the win probability of a shootout as it progresses, though this will almost always look roughly the same because there usually aren’t that many data points (five or six) and you can’t ever have big comebacks since neither team can ever have a lead of more than two goals.

Or, you know, anything, really. I just did the math, and I know at least one guy who will take these numbers and run with them soon. (And here he is.)

About these ads
This entry was posted in analysis, hockey, math. Bookmark the permalink.

3 Responses to Modelling win probability in shootouts

  1. Craig Burley says:

    “Of course, if the NHL ran MLB, they’d probably institute that right away”

    Definitely.

    How much variation is there in shootout performance? I guess we probably don’t have enough data (since, you know, penalty shots don’t have much of a role in real hockey) to find out?

    • As I recall, there’s little difference among shooters selected for the shootout. Obviously you don’t send out your AHL callup. With goalies there is some skill but not enough to matter.

      I think that’s the case, anyway, and naturally I am too lazy to find the studies for you now.

  2. Pingback: Vancouver Canucks: Why Chris Tanev is so effective, and other Wednesday questions | The Province

Comments are closed.