Was Belichick's decision justified?
» Posted by Vince on February 4, 2008«
Bill Belichick is one of the most analytical coaches in the NFL. He is also one of the most successful. Yet, nearly everyone thinks he made a big mistake in the third quarter of Super Bowl XLII.
The situation is 6:49 to go in the third ahead 7-3 with a 4th and 13 on the Giants' 31 yard line. The decision is to kick a field goal, punt, or go for the first down. Brady threw an incomplete pass over Jabbar Gafney's head.
Let's use GridIronMine's Victory Forecast (VF) to evaluate this situation and determine whether Belichick's decision to go for the first down was justified. First, we must determine the VF of the various situations. (BTW, in the current situation NE has an 79.8% VF.) We start with the field goal. If the field is good, the VF increases to 83.2%. If missed and spotted at the 38, the VF is 74.0%. (For simplicity we exclude low probability outcomes, such as a blocked field goal and a roughing-the-kicker penalty, that do not have much effect in the final analysis.)
Next, is the punt. This more complicated because there are many more possible outcomes. Barring any unusual event, the worst case is the Giants will take over on the 20, the best is inside the 5. The VF for the worst case is 76.8%. It is 79.1% if the Giants take over on the 5.
The last situation, going for the first down, is the most complicated. Again, we do not consider unusual cases like a pick-six or a roughing-the-passer penalty. The most likely failure case is an incomplete pass, yielding a VF of 75.2%. A successful conversion will be between 13 and 31 yards. Let's suppose the average conversion is a 15-yard gain. This increases NE's VF to 86.7%.
Here is a summary so far.
| Victory Forecast | ||
|---|---|---|
| Situation | Worst case | Best case |
| FG | 74.0 | 83.2 |
| Punt | 76.8 | 79.1 |
| Pass | 75.2 | 86.7 |
So an overly pessimistic person would punt because it has the best worst-case VF, whereas an overly optimistic person would pass because it has the best best-case VF. However, a rational person must weigh the good and bad. To do that we need to know the likelihood of these outcomes. For example, to evaluate the choice of a field goal, we need to know the probability of kicking a 48-yard field goal. Gostkowski has not kicked enough 48-yard field goals to use only his statistics. The GridIronMine VF estimates the NFL average as 58%. It is not clear what Gostkowski's probability is, so we compute the result of an attempted field goal for several values. The field goal VF is calculated as:
probability x best_case_VF + (1 - probability) x worst_case_VF
| Probability | VF |
|---|---|
| 0 | 74.0 |
| 10 | 74.9 |
| 20 | 75.8 |
| 30 | 76.8 |
| 40 | 77.7 |
| 50 | 78.6 |
| 60 | 79.5 |
| 70 | 80.4 |
| 80 | 81.4 |
| 90 | 82.3 |
| 100 | 83.2 |
The current VF is 79.8% (and of course a successful field goal raises the VF and missed field goal lowers it). Interestingly, on average attempting to kicking the field goal will lower a team's VF. This does not mean it is a bad idea to try to kick a field goal, just that it is a bad situation to be in.
The above table shows that punting is not a good idea. The best-case punt is about the same as a field goal with a 54% chance. That means best-case punt is better if the field goal will be made less than 54% of the time. On the other hand, the worst-case punt is about the same as a field goal with a 29% chance. Basically, in this situation at this time of the game, punting is a worse choice than kicking a field goal. (Later in the game there is a different story.)
So now we turn our attention to going for the first down. We must look at the probability of converting. There are not enough 4th and 13 situations, so we consider 3rd and 13 as well because teams are trying to get all 13 yards on either down. Also, one or two yards difference probably doesn't change the play call. So we look at the 3rd and 4th downs with between 11 and 15 yards to go (excluding weird time-crunch situations at end of each half). NE had this situation on 3rd down 21 times and converted 5 times. They had 8 4th down situations and attempted none (before the Super Bowl). Thus, NE has an 23.8% average conversion this season. Of course, that was not against the Giants and was not in the Super Bowl. Teams converted 21.1% of their 3rd and long situations against the Giants, which is almost exactly the NFL average of 21.0%. None of this data tells us the precise probability of a conversion. So once again, we look at several conversion probabilities. The VF is calculated in the same way as for the field goal table.
| Probability | VF |
|---|---|
| 0 | 75.2 |
| 10 | 76.4 |
| 20 | 77.5 |
| 30 | 78.7 |
| 40 | 79.8 |
| 50 | 81.0 |
| 60 | 82.1 |
| 70 | 83.3 |
| 80 | 84.4 |
| 90 | 85.6 |
| 100 | 86.7 |
Using the last two tables, we can evaluate the choice between a field goal and a pass. It is a push (neither choice is better) if we choose best-cases (or worst-cases) from each table. For example, 66% field goal chance is essentially the same as 42% conversion. To make the case against Belichick one has to suppose that Gostkowski will make the field goal at higher than the NFL average (say 62% for a VF of 79.7%) and NE will convert at less than their season average (say 18% for a VF of 77.3%). In this case, the decision was only 2.4% VF worse. That is not a large VF change. For example, the previous play (a sack) cost the Patriots 5% VF. Interestingly, supposing NE had a 0% chance of converting and a 100% chance of making the field goal, the decision to go for it has a VF 8% worse than kicking the field goal. This last fact shows how close the call really is.
Note: This analysis would be different if the score were different. For example, if Patriots led by 6 points or trailed by 2 the field goal would have a greater value, making going for it less justifiable.
The above analysis makes many assumptions. One can argue against these and provide other assumptions. However, unless the new assumptions are significantly different, the new analysis will be similar. Basically, Belichick made a reasonable and easily justifiable decision.
Side note
We had assumed that NE was the best, or nearly the best, third-down team in the league. In gathering data for this article, we were surprised to find that NE is a pedestrian 63rd percentile (12th best) at converting third and long. Below is the data for 3rd and long (11-15 yards) not including situations at the end of each half.
| team | conversions | attempts | rate |
|---|---|---|---|
| SD | 12 | 27 | 0.444 |
| CIN | 8 | 20 | 0.400 |
| NO | 5 | 14 | 0.357 |
| DEN | 9 | 27 | 0.333 |
| JAC | 9 | 30 | 0.300 |
| WAS | 6 | 21 | 0.286 |
| NYJ | 8 | 28 | 0.286 |
| IND | 5 | 18 | 0.278 |
| CLE | 5 | 19 | 0.263 |
| PHI | 5 | 20 | 0.250 |
| OAK | 8 | 33 | 0.242 |
| NE | 5 | 21 | 0.238 |
| NFL | 157 | 746 | 0.210 |
Update
Some have commented that this article doesn't justify Belichick's decision.
In which case, the original title ("Belichick's decision was justified") was misleading.
We have edited the title.
This article has had some typos fixed and we extended the range of two tables.