Football Research Appendix

By Max Mulitz

I’ll keep this updated as I make new posts with a quick summary of the big idea of each post and links to posts by category:

Mission Statement:

What is football analytics?/Blueprint

Game Strategy Research:

The Role of Analytics in Game Planning

First down probability and when to accept/decline penalties.

Running out of shotgun on 3rd & 1 is more efficient than running from under center. Though under center plays have a higher average conversion percentage because teams run more often from under center.

Game theory can help tendency research with respect to the order in which teams make decisions for a given play

Solving the problem of the Two Point Conversion up 7 late in the game

When to call Timeout on defense at the end of the First Half

Individual Player Value:

A significant majority of NFL Teams will have to give at least 6 offensive linemen significant snaps each season. Teams will commonly give significant snaps to as many as 8 different offensive linemen per season.

Run Stops Added is our foundational metric for understanding how individual players contribute to run defense

Expected Points forms the basis of our fundamental metric for understanding how much value a receiver provided when he was targeted.

Projecting Quarterback success in the NFL Draft

Valuing Free Agent Receivers using Production and Age

How to use expected points to estimate Wide Receiver value

How valuable is Earl Thomas? He’s probably worth 2-3 points per game to the Seahawks

A mental model for evaluating risk in free agency

A Running backs’ expected success rate varies wildly depending on the backs role. Expected success rate is far more stable year to year than actual performance, meaning role is more consistent than rushing performance for running backs.

Injury Study: There’s about a 40-50% chance a given player will miss time with an injury in a given season. It’s likely some players are injury prone and have a 70%+ chance of being injured an some players are particularly resilient to injury and only have about a 25-30% chance of being injured in a given season.

The most valuable kicker you could reasonably expect to have is worth about 0.1 wins above average per season, which is  worth about 0.66% of the Salary Cap (about 1.1 Million dollars/season)

An extra 1% of the cap spent in a given year correlates with 0.15 more wins

NFL Draft/Scouting

Calculating Sparq for Edge Rushers

Arm Length almost always matters more than height

Jumping tests at the combine are a valid measure of leg power once you adjust for weight.

Measuring Grip Strength to gain an advantage selecting linemen

Using decision trees to form an inclusive model of which players are worth scouting

Indicators of Team Strengths and Weaknesses: 

How are different team efficiency statistics correlated with winning? Each extra sack per hundred snaps is worth about an extra quarter of a win. An increase of a yard per attempt passing or a decrease of a yard per attempt allowed each correlate with about 1.6 extra wins.

Passing efficiency is at least a little over twice as important as rushing efficiency, both offensively and defensively. Team passing efficiency predicts team offense quality far better than rushing efficiency.

Point differential is highly correlated with wins. Every 36 points of added point differential is worth an expected win

How to handle being in Field Goal attempt range when tied or trailing by 1-2 points at the end of a game

 

Advertisements

Measuring Grip Strength to Gain an Advantage when Selecting Offensive Linemen

By Max Mulitz

At the 1:30:50 mark of this clinic,  Alex Gibbs, the godfather of the modern zone blocking scheme, explains that hand strength is one of the most important athletic traits for an offensive lineman. He says, “this is their tool right here, this is what they have to have, they have to have hand strength…” Gibbs goes on explain that hand strength is a more accurate indicator of functional strength than the bench press.

Former Pro Bowl Lineman LeCharles Bentley, who runs a premier offensive line school, also believes grip strength is very important.

If measurable grip strength is a key trait for offensive linemen, then any attempt to project offensive lineman athleticism using data without grip strength is going to be considerably less accurate than a model that includes a measure of grip strength. Unfortunately, none of the NFL combine tests directly measure this trait, so there’s no way to know the exact nature of the relationship between measured grip strength and offensive line performance.

NFL teams should collect grip strength measurements on any offensive linemen that comes through the franchise to help examine how grip strength affects performance. If some of the great minds of offensive line play are correct, then a team will be able to get an advantage on their offensive line by using data to study the relationship between grip strength and performance.

A Mental Model For Valuing Players In Free Agency

by Max Mulitz

Adam Harstad has a fantastic piece explaining that players do not  gradually decline as they age, they continue performing at more or less their average level of output until they hit a wall. This probably isn’t precisely true. For instance we know speed declines as players age, but they also improve by gaining experience and technical skill. Larry Fitzgerald is an example of a player whose style of play has changed as he’s gotten older and lost some of his speed, but who has maintained a very high level of production. Still, given that our model of player aging should be some balance of year to year decline and risk of falling off a cliff, Adam’s data is a strong case that the risk of “death” has a far larger impact.

When teams consider how to structure a players’ contract, they must consider the possibility the player maintains his approximate level of production through the life of the contract as well as the possibility the player stops producing after each year of the contract. Every year of a players’ career carries some risk that the player will, either due to injury, loss of athleticism, or off field issues, no longer be a viable NFL player. A teams’ risk in each year is simply how costly it would be to the franchise in terms of percentage of the salary cap paid over the life of the contract if the player stopped being productive in that year. Good contracts (for the franchise) balance saving salary cap space with protection in the event of a players’ drop off. There is some utility in estimating a players’ risk, but becoming overconfident in a specific individuals’ durability creates fragility.

This mental model is limited to established starters going on their 2nd or 3rd contract. Players who have not yet been given the opportunity to play significant snaps do not have an established baseline of production and are not likely to get relatively large guaranteed contracts.

 

How Valuable is Earl Thomas to the Seahawks’ Defense?

by Max Mulitz

Over the past couple seasons, numerous articles have been written explaining why Earl Thomas is the most important player in the Seahawks defense. Basically, the entire scheme of the Seahawks defense relies on Earl Thomas being able to cover the deep middle third of the defense on the majority of the snaps.

Passes to the deep middle of the field are the most dangerous and efficient passes in the game, so Thomas is charged with defending the most important area of the field on a down to down basis. As Numberfire points out, the Seahawks deep passing defense went from allowing only about one expected point per game with Thomas to about three per game without him, for a net increase of two points per game allowed. Football Outsiders data also evidences that the Seahawks defense of the deep middle of the field has gone from elite to very poor with the loss of Thomas.

Based on Numberfire’s analysis the Seahawks pass defense was about 2.5 points a game worse without Earl Thomas. This would value Thomas at a little over a win per season.

Thomas’ salary of  $10 Million per year (6% of the Cap) values him at about .9 win a year, though Numberfire’s valuation of him would put his value in the 12-14 million/year range, which seems reasonable.

What is Football Analytics/Blueprint for integrating Football Science into an NFL Franchise

by Max Mulitz

 

Analytics is the process of applying the scientific method to football. Scientific knowledge is knowledge that makes testable predictions and is hard to vary. All knowledge is conjectural and comes from within us through creativity to solve problems, not to summarize data. The scientific method is an endless iterative process of moving closer to the truth of the world, especially when studying complex systems such as NFL football.

Untitled.png

A successful Football Science department requires the following characteristics and priorities:

  1. Be centralized in such a way that the department can communicate with all areas of the franchise, because all areas of an NFL franchise overlap in their needs and uses for data.
  2. Integrate the football knowledge and theories of Scouts, General Management, and Coaches into testable theories that can enhance the total football knowledge of the organization. This framework can be used to create comprehensive theory of roster construction, projecting college players to the NFL, and understanding how different players contribute to winning games All models and knowledge created by the Football Science department should be rooted in the organizations’ foundational philosophy of football.
  3. Work with the coaches to create a system of game management that is agreed upon and understood by the Head Coach and play callers and can be continuously updated and improved as new situations are recognized and analyzed and more knowledge is created regarding the topics of game and clock management.
  4. Utilize the coaching staff’s knowledge of play calling and how coaches think to enhance pre-scouting opponents and game planning with the best, most relevant, and most useful data. The role of analytics in game preparation is to generate information that aids the coach in forming his gameplan, not to summarize the data as many ways as possible.
  5. Always be in the process of revising and building on foundational models and systems to continuously learn about each area of football the department interacts with. Perpetual growth and improvement in every area Football Science touches is a necessity.

A Simple Box Score Metric for Valuing Wide Receivers

One of the projects I’ve been working on is creating a model of player value by position based on converting box score based metrics to a single number of points added. This model is by no means the be all end all of player value, but it would provide an evidence-based starting point for estimating the value of players across positions. In a previous post, I explained why measures of a players contribution should always be in terms of Expected Points.

For now I’m only going to deal with the value a receiver adds by catching passes. It is undoubtedly true that an NFL wide receiver can contribute to his offense scoring points by drawing defensive attention away from other players or by blocking well in the running game, but these contribution are probably an order of magnitude less important to a receivers full contribution than his pass catching and they are much harder to measure, so I’m going to set these considerations aside for now.

Receivers with under 20 targets in a given season over 2011-2015 averaged about 0.5 NEP/Target, so this can be considered the per target value of a replacement level player. Adam Harstad has written about the fact that for receivers, simply being targeted is an efficient play for the receiver because receivers who can’t get open aren’t targeted at all. This implies that if we forced a replacement level player to see a high share of targets, their efficiency would suffer, so we would expect the NEP/Target for a replacement level player thrust into a starting role to be marginally lower than 0.5. The 25th percentile of receivers is about 0.45 NEP/Target over the past 5 years, which strikes me as being about right. To calculate the value a receiver added in a given season over a replacement player, simply take the players’ total Net Expected Points and then penalize the player 0.45 expected points for each target they receive.  This method leaves approximately 90-100 receivers a season more than 5 points above replacement level, which is probably about right as well, generally players don’t stick around the league too long if they aren’t one of their teams three best receivers.

A weakness of this method is it does not account for quarterback quality. I’ll get back to this in a later post though it is very difficult to separate out a quarterbacks’ influence on his receivers, so it is best to leave quarterback quality out of the initial model.The top 10 Receivers from the 2016 Season and their Value over Replacement are presented below:

Name Targets NEP/Target Expected Points Added
Mike Evans 171 0.84 67.27
T.Y. Hilton 155 0.84 60.09
Julio Jones 129 0.89 57.06
Jordy Nelson 152 0.81 54.75
Brandin Cooks 117 0.84 46.21
Adam Thielen 92 0.95 45.7
Pierre Garcon 114 0.84 43.93
Antonio Brown 154 0.71 40.4
Tyrell Williams 119 0.78 39.07
Doug Baldwin 125 0.75 37.78

For the most part, the players at the top of the list got their on high volume and efficiency and are all part of the NFL’s elite passing attacks. Adam Thielen and Pierre Garcon probably are not top 10 receivers at this point in their career, but we would expect there to be significant variation between the 10 most skilled receivers and the 10 receivers who add the most value in a given season just due to luck and random variation. In other words, the observation that Adam Thielen was extremely efficient when he was targeted last season is a fact, but it is not by itself an argument to extrapolate that he is one of the 10 best wide receivers in the NFL. As stated, this is a descriptive metric meant to summarize a receivers’ entire stat line into a single value, so it is by definition going to suffer the same problems of any analysis drawn from the box score. Nevertheless, it is a far more useful starting point for evaluating the value a receiver added when targeted than simply using Yards, Touchdowns, Yards/Target, raw Net Expected Points, or any other box score driven statistic.

Using Decision Tree Modeling to Determine Who To Scout

The first question a scouting department must answer is which players to devote time and effort to scouting. Time is a finite resource, so while the goal of thoroughly vetting every college football player in the nation is admirable, it’s not realistic. By asking a few simple, objective questions about a college player we can quickly determine if he is worth scouting. A decision tree on whether or not to devote resources to scouting a college football player would look something like the following:

Screen Shot 2017-01-12 at 7.14.40 PM.png

General Theory: Because it is possible to scout a lot of players (but not every player) it is worth being fairly inclusive and looking into players who show signs of having at either the athleticism or football skill to succeed at the NFL level.

Node 1 Explanation: If a player is a starter at a top Division I school it is a sign that the player is good enough to play with top competition. Also, top Division I schools usually recruit the best high school athletes, so a player starting at a school like Alabama is a signal that the player likely has above average athletic traits.

Node 2 Explanation: The best college football players are most likely to go on to become the best NFL players. If a player is a dominant player in college football, their influence on the game will be reflected in their box score statistics at some point in the players collegiate career. Players who are unable to make an impact in college football usually aren’t good enough to play in the NFL either.

Node 3 Explanation: Sometimes, players with unique athletic traits can switch positions from college to the NFL and excel at their new position in spite of their lack of collegiate experience. Julian Edelman was a quarterback in college football but did not have the skills to play quarterback in the NFL. However, Edelman’s elite agility  allowed him to eventually make the transition to being a successful receiver in the NFL (though he didn’t break out as a receiver until his fifth year in the league.) Terrell Pryor is another example of a freakishly gifted athlete who eventually transitioned from quarterback to receiver with success. Runningbacks Jerrick McKinnon and Denard Robinson are collegiate quarterbacks with strong athletic traits who’ve had some success in the NFL. Future Hall of Fame Tight End Antonio Gates never played college football at all and two time All Pro Tight End Jimmy Graham only played a year of college football after a college basketball career. Anyway, there are enough cases of great athletes who haven’t had the opportunity to develop their technical football knowledge and who lock a strong collegiate resume who were be able to learn the requisite skills and who’s innate football talent allowed them to eventually succeed at the NFL level that it is worth at least evaluating anyone with elite athleticism.

It is extremely rare for below unproductive college football players with marginal tested athleticism to make an impact at the NFL level, so it is not particularly wise to waste time and energy scouting these players.

The Fundamental Metric for Valuing Receivers

by Max Mulitz

 

The best way to measure a receivers total value is some version of Net Expected Points (NEP) or Expected Points Added. These models basically take the expected point value of the down and distance situation after a play where the receiver was targeted and then subtracts the expected point value before the play. For example a 10 yard reception on 3rd and 15 at a teams own 25 would have a slightly negative value as the team would usually be forced to punt, as would an incompletion. A 20 yard reception would have a high positive value as 3rd and 15 at your own 25 is significantly less valuable than 1st and 10 at your own 45 in terms of expected points. For more on how Net Expected Points are calculated click the links above.

Why Use Expected Points instead of other Metrics

The benefit of expected points compared to other receiver metrics like yards per target or catch rate is that expected points maps directly to wins and therefore could eventually allow us to meaningfully compare players across positions. Contrast this with Yards per Target or Catch Rate, which would need to be converted into a universal metric before any sort of comparison between receivers and say, Linebackers could be made. Because scoring points is the ultimate goal of football, expected points added is universal in a way other receiving statistics are not.

Another benefit of expected points is that it answers the question “when player x was thrown the ball, did good things happen?” It is possible to have a low yards per reception and still be a valuable receiver, a player like Wes Welker who ran the vast majority of his routes near the line of scrimmage and had an excellent catch rate is an example. On the other extreme, Vincent Jackson had  poor catch rate most of his career because he ran primarily deep routes (which are completed less often) but his huge production on his receptions still allowed him to become an elite performer. Expected Points isn’t flexible, if a player has very poor expected points per target it means when they were thrown the ball good things didn’t happen and if the player has high expected points it means that when they were thrown the ball good things did happen on the whole. A metric like Yards Per Target, which combines yards per catch and catch rate, does a much better job of measuring a receiver’s ability than simple catch rate, but there’s no reason to believe an 18 Yards Per Catch Receiver with a 50% Catch Rate (9 Yards Per Target) and a 12 Yards Per Catch Receiver with a 75% catch rate (9 Yards Per Target) are necessarily equally valuable. It is necessarily the case however that if two players receive equal targets and each produce 20 Net Expected Points then they did in fact get equal production out of their targets.

Weaknesses and Limitations of Expected Points

Net Expected Points tells us if good things happened when a receiver was targeted. They do not tell us how responsible the receiver is for that benefit. Targets coming from Aaron Rodgers are likely to be more accurate (and therefore valuable) than targets from a career backup QB. Some of this can be alleviated by adjusting Expected Points for quarterback by comparing to how the Quarterback does throwing to other receivers, but this method would underestimate receivers with excellent teammates and would overestimate players with very poor teammates.

Expected Points also don’t tell us what happened when a player is not targeted on a play. For instance, if a player fails to get separation all game and is only targeted once on forty routes, where he coverts a 3rd & 3 by gaining 7 yards, his Net Expected Points for the game will be positive, in spite of the fact that he clearly hurt his team during the game by consistently failing to become open.

On every pass play there is some risk of a sack, which is a negative result, but plays where a receiver is targeted obviously are never sacks, so by only looking at pass plays where a receiver is targeted we are ignoring some of the inherent risk in deciding to pass the ball. By subtracting a players NEP/Target from the average for players at their position (for receivers the average is right around .6 most years) and then multiplying by total number of targets, we can get a better sense of the players value relative to the average player.

Another issue is Net Expected Points doesn’t account for route quality or tell us anything about player style. A receiver who runs the highest value routes and is regularly targeted downfield may have greater opportunity to return value than a player who only runs short routes near the line of scrimmage, but the short route player may be outperforming how the types of targets they receive by a greater amount. Expected Points doesn’t tell us anything about the types of routes a receiver is asked to run.

Net Expected Points also doesn’t measure a receivers blocking ability or soft skills (running pick routes well to help teammates get open and drawing safety help, for example.)

Because of these failings, Expected Points do not, by themselves, predict how a player would do in a different system or with different usage than they have received. This problem is endemic to all box score metric, which can only measure what actually happened, not what might have happened had a player been used differently.

Conclusions:

Expected Points tells us the value of what happened when a receiver was targeted, though it is not the end-all-be-all of receiver quality. Expected Points benefits from being a universal metric, that is, if done correctly it would allow us to accurately compare players across positions. Expected Points can be adjusted to account for certain aspects of a players situation, though these adjustments are imperfect.

While other metrics may be necessary to supplement expected points in order to understand a receivers particular talents and abilities, some version of expected points (possibly transformed for situation or role) should be the foundational Key Performance Indicator for evaluating  the value of a receivers’ contribution.