Volutpat Sed Etiam
16-0: The myth of perfection.
The Fount tells you how the New York Giants could defeat the New England Patriots in Superbowl XLII. Or more appropriately, why everyone was wrong to doubt the Giants.
Statistical analysis first published January, 2008.
The top win total from the NFL the past five years has been, in succession, 14, 15, 14, 14 and now 16 games. Over the previous 23 years in the league, if you combine the best five teams' records overall you can scrape together a total that equals that many wins. Why has a sudden upswing in wins, punctuated by the unprecedented 16-0 performance by the Patriots this year, been seen for top NFL teams? Your Fount experts decided to get to the root of this question: Does 16-0 in 2007 mean what it would have meant two decades ago?
To do this, we started our analysis in 1978, the first year teams played a 16 game regular season schedule. Throughout the analysis, we excluded the strike years 1982 and 1987, which left 28 relevant years. Or, as we found to be the case, 22 relevant years from 1978-2001, and 6 years from 2002-2007. Because what we found was a clear pattern – teams at the top of the league are winning more games than they used to, and the change starts in 2002:
Why the change in 2002? In 2002, the NFL went through a major realignment, and made changes to its scheduling methods. The league's expansion from 28 teams (1978-1994) to 31 teams in 2001 had created divisions as large as six teams, and when the round number of 32 teams was reached in 2002, a move was made to eight divisions of four teams each.
What impact has this had on the league's win totals? Prior to 2002, the average number of wins for the best team in the league was 13.5. Since 2002, it has increased to 14.2. The average number of wins for the team with the second best record league wide has increased from 12.4 to 13. That's an increase of about two-thirds of a win per team at the top levels.
From 1978-2001, three teams went 15-1 or better, or about one per seven years. In the six years since the realignment, two teams have gone 15-1 or better – about twice the rate – at one in three years. Though a 16-0 regular season record is the first in the 28 years under discussion, viewed through another prism it can be seen as a record achieved in one out of six years since the realignment – or about the rate of teams going 15-1 previously.
Is 16-0 the new 15-1?
In the 1980's, two teams achieved regular season records of 15-1, and went on to 3-0 playoff runs. As Steve Mariucci said, "These other teams that were 18-1, the Bears and the 49ers, finished with strong superbowl wins, dominating playoff runs. These are the best teams ever." The two teams that have gone 15-1 since then haven't even made it to the Superbowl. The one team that went 16-0 made it but didn't win. So what changed?
Dilution of talent due to expansion is one probable explanation. At the core of the improvements in the top win totals, though, you can more likely blame changes in scheduling. For many of the years prior to 2002, the top teams in each division not only played top teams from other divisions, they played a whole schedule that was weighted heavily based on their previous success. Though scheduling practices varied over the years, many schedules that were heavily dependent on weighting must have resulted in weighting down the win totals of top teams from the previous year.
Added to that was the fact that teams played in larger divisions, and the key result of that was that each team had to play four other teams twice. An analysis of the poorer records of top teams when playing within their divisions is not shown here, but historically these division rivalries have demonstrated a much higher potential for upsets (think, in 2007, of Week 15 Eagles over Cowboys and Week 16 Bears over Packers). How, though, does the division size change directly affect win totals? Simply by creating a situation where a team facing three other teams twice instead of four teams twice has a greater chance of having all those teams come from the bottom of the league's talent that year. If you randomly distribute the numbers 1-32 around all teams in the league, there's a much greater chance of having three numbers in the 24-32 range in one division than for having four.
The 2007 AFC East's historic achievement: best overall record all time and worst record all time for 2-4 teams
This effect plays into the 2007 regular season success of the Patriots as well. The Pats played, and racked up a 6-0 record against, a historically weak set of teams at 2-4 in their division. The win totals of those teams, 7, 4, and 1 game, for a total of 12, are the absolute lowest win totals for a set of teams 2-4 in any division in NFL history back to 1978. Realize that because these three teams play six games against each other, the lowest possible number is 6 wins. Outside of their division these teams were a tragically poor 6-24, or an average of 2-8. This three team accomplishment is as historic as the Patriots 16-0 record, and in some ways it's more impressive. Don't think that the dominance of the Patriots contributed significantly to their weakness; 5-1 or better division records are common for top teams and even one win over the Pats would only bring these basement teams up to 13 wins total. We have not looked into the historical ranking of teams at 2-4 within divisions, but many rankings have the 2007 Buffalo Bills around 20th, and ESPN's final power rankings had the New York Jets and Miami Dolphins as the 30th and 32nd ranked teams in the league. One system had all three teams at 24th or below.
So why did the Patriots lose the Superbowl? A better question is why did a team that was later proven not to be dominant ever went 16-0. The answer has a lot to do with the teams that they had to face, and most especially the teams they had to face twice. They faced three very poor teams twice in the regular season, and went 6-0. In the playoffs, they faced two very good teams each for the second time, and ended up going 1-1.
When thinking historically about 16-0, it can be considered to have been considerably harder to run the table back in the days when 5 team divisions were typical, when the other four teams drafted and built their teams to offset a top team's strength, and most especially in the days when the schedule for a top team consisted not just of playing all #1 teams in their conference, but an overall strength of schedule of as high as 40-24 for the four at large teams they played.
How robust are the changes we're detecting, statistically speaking? To assess this, we focused on the fraction of teams, rather than the absolute number, achieving a certain number of wins in a given year. Failing to do this would unfairly bias the results in favor of higher numbers in the later years, as more teams in the league would result in more with better records. We then calculated p values via t-test as well as assessed the sample size one would choose if designing the t-test in a controlled experiment. Since the sample values we derived were on the order of n=25 for each group, and yet there are only 22 and 6 data points in the populations for comparison here, one should not reasonably expect statistical significance at the 95% level, and one should realize that absence of significance at that level is not evidence against the observation due to low sample size. The experiment will continue, unless scheduling practices change again, and a final answer will emerge. However, based on the current data, a wise bettor would bet on the effect we're describing being true – there being only a one in six chance of it emerging due to random chance alone. For the record of the second best team each year, the trend is even stronger, with only a one in ten chance of seeing as strong a trend by random chance. To check this result, we also ran a monte carlo analysis, creating a population from the entire set of 28 data points, and repeatedly randomly selecting six samples. The results were consistent, with random selection of six points resulting in higher win totals than the actual data from 2002-2007 about one in six times. The odds in favor of the trend are roughly the same as those for a team favored to win a game by two touchdowns.
Paul Tagliabue, NFL Commissioner, touted the 2002 realignment and the new scheduling formula. "The new formula guarantees that NFL fans will see every team play each other on a regular, rotating basis. The formula will eliminate the many aberrations of the past in which teams either did not play for long periods of time or did not play in another team's stadium for many years." This has proven to be true, and an AFC basement team that never matched up against an NFC playoff perennial under the old scheduling system now meets them once every four years. Another byproduct of the new system is a better assessment of how teams really stack up within their division, as division rivals share 14 games of common opponents. But the former mechanisms that served to pull top teams back to the pack are gone. The 15-1 and 16-0 regular season record has become more common, and another team could go 16-0 within a decade. But we should no longer be deceived that a 16-0 or 15-1 record means dominance. The average win totals of the top teams have gone up by almost a full game per season, and no one should be surprised if we see another team with a top record losing a big playoff game. The current scheduling can easily fail to test teams enough to know their true strength from record alone.