Who is the better quarterback – David Carr or Michael Vick? According to the NFL’s quarterback rating system, Carr is currently the top rated signal caller in the league. Meanwhile Vick is ranked 27th out of 31 quarterbacks.
To understand the validity of this ranking, let’s quickly review how the NFL evaluates quarterbacks.
As we note in The Wages of Wins, this is the step-by-step process:
First one takes a quarterback’s completion percentage, then subtracts 0.3 from this number and divides by 0.2. You then take yards per attempts, subtract 3 and divide by 4. After that, you divide touchdowns per attempt by .05. For interceptions per attempt, you start with .095, subtract from this number interceptions per attempt, and then divided this result by .04. To get the quarterback rating, you add the values created from your first four steps, multiply this sum by 100, and divide the result by 6. Oh, and by the way, the sum from each of your first four steps cannot exceed 2.375 or be less than zero.
All in all, this is a pretty intuitive and simple measure of player performance.
Hmmm…not really. The formula is not only complicated and unclear, it also ignores sacks, yards lost from sacks, rushing yards, rushing attempts, and fumbles.
QB Score, the simple formula we report in The Wages of Wins, is meant to improve upon the NFL’s metric by considering what the quarterback does with both his arm and feet. For example, Carr has been sacked 15 times, second only to Daunte Culpepper. Additionally he has lost three fumbles. When these weaknesses are taken into account, as one can see in the Week Four Overall QB Score Rankings, Carr is only ranked 20th in the league.
In contrast, the NFL’s metric appears to undervalue Vick. Vick is towards the bottom in the NFL’s rankings. But Vick has gained 333 yards rushing. And when this is noted, the numbers associated with Vick are good enough to rank the signal caller 9th in the league.
It is important to note that player statistics tracked in football are not like the statistics tracked in baseball and basketball. In the b-ball sports there is good reason to believe that many of the statistics tracked reflect the ability of the individual. On the gridiron, though, this is not exactly the case. The stats a quarterback accumulates do reflect his ability, but also the ability of his teammates, his coaches, and the players the quarterback faces. Consequently, we want to be very careful looking at these stats and saying that this shows this player is “better” than that player.
Of course, if a player keeps appearing towards the top of the rankings we might suspect something is going on. As one can see with the Week Four Rankings, there are now only two quarterbacks who have been above average every week they have played. And as we saw with the overall rankings, the two survivors – Donovan McNabb and Peyton Manning – are currently the top two quarterbacks in the league in terms of QB Score per play.
Given the performances of McNabb and Manning – both this season and in recent years – I think we can argue that these players are blessed with some skills. And this is true even if these players never win a title.
Before ending this post I wish to ask one more question about the NFL’s metric. Will there ever come a day when broadcasts of NFL games do not report each quarterback’s rating?
The short answer — I think that is unlikely. Even if people believed QB Score — or other metrics like those developed by Football Outsiders — were better, I doubt we will ever see a broadcast of an NFL game without reference to a quarterback’s rating. Why do I say this? In watching the playoffs in baseball I still see reference to each player’s batting average. And that stat was only found to be inferior several decades ago.
There is a lesson in the persistence of batting average for anyone who takes the time to develop new statistics for sports. Ultimately, broadcasters have to stick with what the majority of fans recognize. And the majority of fans are not economists or sabermetricians. So expect to continue seeing batting average and quarterback ratings. And given that these games are ultimately about entertainment, I think that’s probably okay.