Over the years, I’ve focused a lot of my original statistical analysis on the quarterback position. The idea behind this is the more I understand about the most complex job in professional sports, the more I understand about sports. Most of this analysis isn’t ever part of a game. It’s more like a hobby. If I gain an insight that helps my work, that’s great.
I’ve put together a fairly extensive spreadsheet of quarterback performance going back to 1974 (insert oft-repeated explanation about NFL passing rules changes here). It was a different game before the rules changes. None of the timing routes or combination routes that have defined modern NFL offenses would have been possible under the old rules.
Still, the game constantly changes. Back in the late ’70s, the league’s interception rate was around 5.2%. Then 4.2% in the ’80s, 3.4% in the ’90s, 3.2% in the ’00s and 2.6% in the ’10s. Completion percentage has risen in a similar manner and yards per catch has dropped.
The most interesting question I’d like to answer is what makes an NFL quarterback? I’ve written a lot about this as well, but in 1998, the year I left the corporate megalith world and began Front Office Football, there was a critical question for Bill Polian, as General Manager of the Indianapolis Colts. The Colts were drafting first overall and needed a quarterback. Two quarterbacks were scouted as being worthy of a number-one pick.
Polian picked correctly, and he’s in the NFL Hall of Fame. Much has been written about Polian’s decision, and I think it would be a fantastic game to recreate that process and have people run drafts and make similar decisions. The problem there is that while we know the answer to Polian’s question, without any trace of doubt, we don’t know exactly what, in all the information he had available, truly determined the outcome.
In order to create a simulation, you have to model something. The more you incorporate into your model, the more engrossing the simulation. But that means you have to make more decisions about what you’re modeling. Some random chance is necessary – you can’t have a sports simulation without some elements of randomness. But somewhere in the data you present to GMs is a piece of information that Polian would see and would lead him to the correct decision.
And once you know, you know. The uncertainty of knowing whether a certain player fails is no longer a mystery. You see attribute X on a rookie’s card, and you know that there’s a high probability he will never develop into a good player. So I try to model that part of the decision process as little as possible. There’s no “well, he’s checking his phone and wearing headphones instead of interacting with his teammates” rating. Scouting error is higher for draftees and there’s a known variable, volatility, which, when triggered, forces a huge change in a player’s ratings. It sure doesn’t feel great (or realistic) when your top pick suddenly gets the volatility drop, but the alternative is a map of Polian’s black box which could never be unseen once revealed.
A while back, I created my own quarterback rating system. I took the major statistical categories, and determined how much they correlated with “winning” football games. This varies a bit from year to year. Since the ultimate intent of this data is to study career paths, I wanted a data set that’s normalized from year to year. This is unlike the oft-published quarterback rating, which has risen considerably as offensive strategies improve.
The system I came up with seemed pretty good, but it doesn’t offer enough credit for the quarterbacks who are more of a threat to run the ball. Now part of that effect takes care of itself. If defenses have to keep a linebacker or a nickel in a short zone as a “spy” in order to protect against a long quarterback run, presumably receivers will have an easier time getting open. But that doesn’t take into account the rushing yardage the quarterback gains, which may be more valuable because quarterback-rushers have a much higher yardage per carry than running backs. Just the nature of these plays.
I’ve spent a lot of trial-and-error time determining what stats correlate best to winning. I found that quarterback rushing yardage was enough of a positive to warrant breaking out on its own. I also hadn’t built sack numbers into the system, and getting sacked is also pretty bad for winning percentage.
I needed to improve my quarterback database, so I decided to take sack and quarterback running numbers back to 1998. I also tried to remove plays where the quarterback takes a knee at the end of a half, since that muddies the quarterback rushing picture. I also removed quarterback spikes from passing attempts.
From there, I put together a list of quarterbacks who have made about 50 or more starts since 1998, adding in top draft picks since 1998 and quarterbacks of significance who were still active in 1998, but had fewer games played later. I ended up with a list of 109 quarterbacks and I included their full careers, modifying the formula to remove fumbles from the rating before 1994 (it’s hard to find consistent fumble data before then).
Each performance is then normalized based on the correlations for that season. I use rolling seven-year averages to smooth out the correlations, because there are wide year-to-year swings. These allow the rating to change gradually, on the hypothesis that each season is one trial – one that could be skewed to one side of the expected result distribution – on an unknowable league mean.
As it turns out, Peyton Manning is ranked #1 of 109 in average quarterback rating over his career and Ryan Leaf is ranked #109. Polian not only made a great decision; it might objectively be the best decision ever made in sports drafting. Imagine – pick correctly and you get the best quarterback of his generation. Pick wrong, and you not only waste the pick, but you’re starting the worst quarterback of his generation for a couple of years.
Another reason I wanted to make sure I was more effectively crediting quarterbacks who run the ball a lot was that I wanted to separate running quarterbacks from those who rarely run the ball and see what I could learn. So I divided the 109 quarterbacks into quintiles based on the percentage of plays they either run the ball or get sacked, or throw the ball.
I found that the rating was almost flat through the top three quintiles – those who ran/were sacked the most. In fact, the top quintile saw a bit of an improvement over the next two. But the fourth quintile was considerably higher than the first and the fifth quintile (those who run/were sacked the least) a huge improvement over those numbers. This was reflected in winning percentages as well.
Keeping in mind that there are only about 22 quarterbacks in each quintile, and Manning and Tom Brady are in that fifth quintile, that might not mean all that much.
I noticed two other important pieces of data. First, quarterbacks in the first quintile reached their performance peaks after less than two years of starting. That’s about a year ahead of every other group. And second, quarterbacks in the first quintile had the shortest careers on average – about seven years. That rises to eight for the second and third quintiles, eleven for the fourth and twelve years for the fifth.
Starting is about opportunity and opportunity comes from winning. With small samples, just a couple of players can make a difference. It doesn’t mean that running quarterbacks can’t win. The first quintile includes Hall of Famer Steve Young, and Russell Wilson seems to be having a very similar career (he’ll be in Canton, I have no doubt). Of today’s young stars, Deshaun Watson and Lamar Jackson are in the first quintile – Watson and Patrick Mahomes (third quintile) have average ratings already at a Hall of Fame level.
Interestingly enough, college performances from the first quintile were by far the best in terms of college quarterback rating. College defenses simply can’t handle a fast quarterback with a good arm who is always a threat to run the ball. The rest of the group had similar college performances, but the fifth quintile included quarterbacks who had a lot more experience in college throwing the ball. This group threw about 25% more passes than the other four quintiles, on average, even though Brady himself was rather inexperienced coming out of college.
My takeaway? It’s hard to find a good quarterback in the NFL, so you have to take the best one when you need one. A good runner is a bonus at the position, and they tend to develop faster but they have shorter careers. As quarterbacks develop, if they need to stay in the pocket, they need to learn to get the ball out quickly. You’ll find all five quintiles represented among the twelve of the 109 quarterbacks studied who are either in the Hall of Fame or definitely Canton-bound.
However, five of the twelve are in the fifth quintile, as are the next three most likely to receive serious consideration. Ideally, if you want your number-one draft pick to develop into a 16-year franchise guy, you want someone who makes quick decisions and not end plays with the ball in his hands.