Using Decision Tree Modeling to Determine Who To Scout

The first question a scouting department must answer is which players to devote time and effort to scouting. Time is a finite resource, so while the goal of thoroughly vetting every college football player in the nation is admirable, it’s not realistic. By asking a few simple, objective questions about a college player we can quickly determine if he is worth scouting. A decision tree on whether or not to devote resources to scouting a college football player would look something like the following:

Screen Shot 2017-01-12 at 7.14.40 PM.png

General Theory: Because it is possible to scout a lot of players (but not every player) it is worth being fairly inclusive and looking into players who show signs of having at either the athleticism or football skill to succeed at the NFL level.

Node 1 Explanation: If a player is a starter at a top Division I school it is a sign that the player is good enough to play with top competition. Also, top Division I schools usually recruit the best high school athletes, so a player starting at a school like Alabama is a signal that the player likely has above average athletic traits.

Node 2 Explanation: The best college football players are most likely to go on to become the best NFL players. If a player is a dominant player in college football, their influence on the game will be reflected in their box score statistics at some point in the players collegiate career. Players who are unable to make an impact in college football usually aren’t good enough to play in the NFL either.

Node 3 Explanation: Sometimes, players with unique athletic traits can switch positions from college to the NFL and excel at their new position in spite of their lack of collegiate experience. Julian Edelman was a quarterback in college football but did not have the skills to play quarterback in the NFL. However, Edelman’s elite agility  allowed him to eventually make the transition to being a successful receiver in the NFL (though he didn’t break out as a receiver until his fifth year in the league.) Terrell Pryor is another example of a freakishly gifted athlete who eventually transitioned from quarterback to receiver with success. Runningbacks Jerrick McKinnon and Denard Robinson are collegiate quarterbacks with strong athletic traits who’ve had some success in the NFL. Future Hall of Fame Tight End Antonio Gates never played college football at all and two time All Pro Tight End Jimmy Graham only played a year of college football after a college basketball career. Anyway, there are enough cases of great athletes who haven’t had the opportunity to develop their technical football knowledge and who lock a strong collegiate resume who were be able to learn the requisite skills and who’s innate football talent allowed them to eventually succeed at the NFL level that it is worth at least evaluating anyone with elite athleticism.

It is extremely rare for below unproductive college football players with marginal tested athleticism to make an impact at the NFL level, so it is not particularly wise to waste time and energy scouting these players.

Advertisements

The Fundamental Metric for Valuing Receivers

by Max Mulitz

 

The best way to measure a receivers total value is some version of Net Expected Points (NEP) or Expected Points Added. These models basically take the expected point value of the down and distance situation after a play where the receiver was targeted and then subtracts the expected point value before the play. For example a 10 yard reception on 3rd and 15 at a teams own 25 would have a slightly negative value as the team would usually be forced to punt, as would an incompletion. A 20 yard reception would have a high positive value as 3rd and 15 at your own 25 is significantly less valuable than 1st and 10 at your own 45 in terms of expected points. For more on how Net Expected Points are calculated click the links above.

Why Use Expected Points instead of other Metrics

The benefit of expected points compared to other receiver metrics like yards per target or catch rate is that expected points maps directly to wins and therefore could eventually allow us to meaningfully compare players across positions. Contrast this with Yards per Target or Catch Rate, which would need to be converted into a universal metric before any sort of comparison between receivers and say, Linebackers could be made. Because scoring points is the ultimate goal of football, expected points added is universal in a way other receiving statistics are not.

Another benefit of expected points is that it answers the question “when player x was thrown the ball, did good things happen?” It is possible to have a low yards per reception and still be a valuable receiver, a player like Wes Welker who ran the vast majority of his routes near the line of scrimmage and had an excellent catch rate is an example. On the other extreme, Vincent Jackson had  poor catch rate most of his career because he ran primarily deep routes (which are completed less often) but his huge production on his receptions still allowed him to become an elite performer. Expected Points isn’t flexible, if a player has very poor expected points per target it means when they were thrown the ball good things didn’t happen and if the player has high expected points it means that when they were thrown the ball good things did happen on the whole. A metric like Yards Per Target, which combines yards per catch and catch rate, does a much better job of measuring a receiver’s ability than simple catch rate, but there’s no reason to believe an 18 Yards Per Catch Receiver with a 50% Catch Rate (9 Yards Per Target) and a 12 Yards Per Catch Receiver with a 75% catch rate (9 Yards Per Target) are necessarily equally valuable. It is necessarily the case however that if two players receive equal targets and each produce 20 Net Expected Points then they did in fact get equal production out of their targets.

Weaknesses and Limitations of Expected Points

Net Expected Points tells us if good things happened when a receiver was targeted. They do not tell us how responsible the receiver is for that benefit. Targets coming from Aaron Rodgers are likely to be more accurate (and therefore valuable) than targets from a career backup QB. Some of this can be alleviated by adjusting Expected Points for quarterback by comparing to how the Quarterback does throwing to other receivers, but this method would underestimate receivers with excellent teammates and would overestimate players with very poor teammates.

Expected Points also don’t tell us what happened when a player is not targeted on a play. For instance, if a player fails to get separation all game and is only targeted once on forty routes, where he coverts a 3rd & 3 by gaining 7 yards, his Net Expected Points for the game will be positive, in spite of the fact that he clearly hurt his team during the game by consistently failing to become open.

On every pass play there is some risk of a sack, which is a negative result, but plays where a receiver is targeted obviously are never sacks, so by only looking at pass plays where a receiver is targeted we are ignoring some of the inherent risk in deciding to pass the ball. By subtracting a players NEP/Target from the average for players at their position (for receivers the average is right around .6 most years) and then multiplying by total number of targets, we can get a better sense of the players value relative to the average player.

Another issue is Net Expected Points doesn’t account for route quality or tell us anything about player style. A receiver who runs the highest value routes and is regularly targeted downfield may have greater opportunity to return value than a player who only runs short routes near the line of scrimmage, but the short route player may be outperforming how the types of targets they receive by a greater amount. Expected Points doesn’t tell us anything about the types of routes a receiver is asked to run.

Net Expected Points also doesn’t measure a receivers blocking ability or soft skills (running pick routes well to help teammates get open and drawing safety help, for example.)

Because of these failings, Expected Points do not, by themselves, predict how a player would do in a different system or with different usage than they have received. This problem is endemic to all box score metric, which can only measure what actually happened, not what might have happened had a player been used differently.

Conclusions:

Expected Points tells us the value of what happened when a receiver was targeted, though it is not the end-all-be-all of receiver quality. Expected Points benefits from being a universal metric, that is, if done correctly it would allow us to accurately compare players across positions. Expected Points can be adjusted to account for certain aspects of a players situation, though these adjustments are imperfect.

While other metrics may be necessary to supplement expected points in order to understand a receivers particular talents and abilities, some version of expected points (possibly transformed for situation or role) should be the foundational Key Performance Indicator for evaluating  the value of a receivers’ contribution.

How NFL Teams Should Play the Field Goal Choke Hold

by Max Mulitz

 

Brian Burke coined the term field goal choke hold to describe the situation where a team that is tied or trailing by 1-2 points and has the ball near the opponents End Zone may opt to intentionally not score a Touchdown (and therefore have to kick off to the opponent) to instead run the clock down and kick a Field Goal as time expires. I’m going to break down this situation from the offenses’ perspective to  create a rule of thumb.

First, let’s consider the situation where intentionally not scoring is most attractive. Say we faced First & Goal at the opponents’ 1-yard line with the opponent having no timeouts coming out of the Two-Minute Warning trailing by 2 points. By kneeling three times, we can attempt a game-winning Field Goal as time expires from the opponents 5 yard-line. In this situation our win probability is simply the chance you convert our Field Goal Attempt. 22-yard Field Goal Attempts have converted ~98% of the time from 2011-2015. It is conceivable that end of game field goals could convert at a lower rate, as the opponent will be selling out for a block, so perhaps the probability is slightly lower. Kicks in clutch situations have done about 2% worse than in regular situations, so 96% is probably a reasonable estimate for Field Goal success rate in this situation (presuming the team has an NFL quality kicker and long snapper available). On the other hand, consider the case where we score on First Down, fail our Two Point Conversion, and have to kick off with 1:58 remaining leading by 4. Teams with one to two minutes remaining on their final drive needing a Touchdown have scored 10% of the time from 2011-2015 when starting inside their own 25. Because 96% is greater than 90%, we can see that barring unusual circumstances, it is better to kneel and kick a Field Goal to win as time expires rather than to give the opponent the ball with over a minute left.

On the other hand, consider the case where you face First and Goal at the opponent’s 1 with 0:15 remaining and three timeouts trailing by 1 point. It is at least reasonable to consider running to try to score a Touchdown. Drives starting on a teams own side of the 50 with under 30 seconds remaining when the team needed a touchdown scored 0/46 times from 2011-2015, and the touchdown rate is also less than 1% over a large sample for drives in the first half starting with under 0:31 remaining. Of course we would expect a teams chance of scoring to be higher in this situation than in the first half (when they were likely to kneel) but combined with the 0/46 statistic, it seems likely the opponents chance of going 75+ yards to score in less than 30 seconds is probably about 2%. The decision to kneel and kick in this situation or run and try to score a Touchdown (and attempt a Field Goal on 4th down if you fail) is a judgement call.

A team may also opt to play a mix of the two strategies. In our first example we could  choose to kneel on 1st and 2nd down and then try to score on our 3rd down play with about 0:38 left and then kick on 4th down if we fail to score. For teams that are unconfident in their kicking operation, this may be a preferable option to scoring on 1st down and giving the opponent the ball back with a lot of time remaining.

The more difficult class of situation is when kneeling and kicking a Field Goal will leave our opponent with time on the clock. For instance, if we face First and Goal at our Opponents’ 2 with 1:45 remaining trailing by 2 and our opponent has 1 timeout, then if we kneel three times, our opponent will get the ball back with about 20 seconds remaining after our Field Goal Attempt.In this situation, our chance of winning by kneeling and kicking is simply the chance our kick is successful and our opponent does not score on their ensuing drive.

For the situation where kneeling and kicking will give our opponent a final drive to try to score a game winning Field Goal of their own, win probability can be modeled as:

Win Probability (Kneeling and Kicking) = Probability (Kick is Successful) * Probability (We stop opponent after kicking successfully)

Win Probability (Kneeling and Kicking) = .96*P(Stop Opponent)

We earlier defined the Win Probability of scoring with 1-2 minutes remaining to take a less than 1 touchdown lead as being about 90%.

Therefore, for kneeling and kicking to be preferable to scoring and giving the ball back:

Win Probability (Kneeling and Kicking) > Win Probability (Scoring)

.96*P(Stop Opponent) > .9

P(Stop Opponent) > .94

So kneeling and kicking is only preferable to scoring if the probability of our opponent scoring on their ensuing drive is less than 6%.

As stated, drives with 1:00-2:00 remaining score  about 10% of the time and drives with less than 0:31 remaining seldom score. Due to small samples, it’s not really clear how quickly the probability changes with 0:59 to 0:31 remaining. A reasonable guideline that isn’t too cute might be to kneel and kick only if it will leave our opponent under 0:25 to score on their ensuing drive.

Implementation

From an offensive perspective, the decision to kneel and kick instead of trying to gain yards when tied or trailing by 1-2 points is only relevant when the offense is within five yards of the opponents goal line. A team facing First & Ten at the opponents’ 25 should still try to gain yards, even if they can kneel and kick as time expires, as  43-yard field goals convert considerably less often than shorter attempts.

On the other hand, that same team may break a long play from the 25 in a situation where it is preferable to go down at the 1 yard line and then kneel and kick. As seen above, these situations can occasionally have significant impact in terms of win probability.

It is not fair to the players to expect the to be able to derive when to and not to score in the end of a game. Instead, teams should have a one word call to add to the end of their playcall to alert players when it is preferable not to score on the next play even if they break a long play. This call could be used prior to any play where a first down inside the opponents five would allow us to kneel and kick a Field Goal to take the lead with 0:25 remaining or less, which could be derived from a simple chart based on time remaining and opponent timeouts.

When Should NFL Teams Call Timeouts on Defense at the End of The First Half?

By Max Mulitz

 

Using the Pro Football Reference Drive Finder, I compiled the scoring rate for drives starting during the last five minutes of the first half from 2011-2015. I broke the data into two tables, one for drives starting inside Own 25 and one for drives starting between Own 25 and Midfield.

At/Inside Own 25
Time Remaining (Seconds) Number of Drives Drive Scoring Rate Drive Touchdown Rate Points Per Drive
0 to 30 365 1% 0% 0.02
31 to 60 291 11% 3% 0.43
61 to 90 202 24% 5% 0.95
91 to 120 307 22% 9% 1.05
121 to 150 136 30% 15% 1.52
151 to 180 135 32% 13% 1.46
181 to 210 127 28% 19% 1.58
211 to 240 133 27% 11% 1.23
241 to 270 111 25% 14% 1.30
271 to 300 120 30% 19% 1.67

 

Own 26-Midfield
Time Remaining (Seconds) Number of Drives Drive Scoring Rate Drive Touchdown Rate Points Per Drive
0 to 30 184 8% 1% 0.25
31 to 60 130 34% 6% 1.26
61 to 90 88 38% 16% 1.76
91 to 120 136 43% 24% 2.25
121 to 150 54 33% 20% 1.81
151 to 180 70 37% 19% 1.86
181 to 210 55 42% 22% 2.13
211 to 240 56 50% 32% 2.79
241 to 270 69 35% 16% 1.68
271 to 300 58 40% 21% 2.29

First let’s deal with the implications for running clock 4th Downs and then we will get into defensive timeout usage on Downs 1-3.

 

4th Down

0:07 less remaining: If the opponent is outside their own 40-45 or so they should forgo punting for a Hail Mary play. An example of this can be seen in Week 10 of 2015 where the Houston Texans faced 3rd & 10 with 11 seconds remaining in the first half at their own 41 against the Cincinnati Bengals. The Texans ran for 3 yards and the Bengals called timeout with 6 seconds remaining in the half hoping the Texans would punt. The Texans recognized the Bengals mistake and attempted a Hail Mary pass. Although the Hail Mary was intercepted, it is clear that the Bengals put themselves in a negative freeroll situation. With less than about 7 seconds remaining it’s better to just let the half end if the opponent is outside their own 40.

If the opponent is inside their own 40 there’s not a very good chance they will be able to throw the ball to the end zone. A smart opponent who faces 4th down with five seconds or less in the half may choose to max protect and intentionally throw the ball out of bounds rather than risking a punt. An example of this can be seen from the Patriots in Week 12 of the 2015 Season from their own 38 with 5 seconds left in the half against the Broncos. This strategy is unlikely to lead to a score and the opponent may opt to punt, so calling timeout or not is a judgement call.

0:08 to 0:30 remaining: If the opponent is in Field Goal Attempt range and does not have a timeout it is best to allow the clock to run and hope the opponent is unable to get their Field Goal unit on the field in time.

If the opponent is outside of Field Goal Attempt range but inside our 50, it is usually best to just let the clock run unless the opponent faces a very long 4th Down. I’ll get into the nitty gritty of this particular situation from the offenses perspective in a future post.

If the opponent is on their own side of the field it is usually best to call timeout and force them to punt. Even though drives starting with less than 30 seconds remaining in the half rarely score, it is always possible to get a blocked punt or score on the punt return, and there’s little downside to forcing the punt.

0:31 to 2:00 remaining: Points Per Drive increases with time remaining up until Two to Three minutes remaining, so any running clock 4th down where you expect the opponent to either punt or attempt a Field Goal warrants an immediate timeout in this range. If the opponent is in a position to go for it on 4th Down (such as 4th & 1 on our 39) we may want to just let the clock run, as we do not want to save time for our opponents’ drive if they convert.

2:01 to 2:20 remaining: timeouts used on offense save about 16 seconds, so calling one with less than 2:16 remaining doesn’t make sense (as you are only saving the amount of time remaining until the Two Minute Warning.) Being able to choose when to use a timeout makes it a little bit more valuable than 16 seconds with this much time remaining, so it’s best to just wait until the Two Minute Warning.

2:21 or more remaining: Expected points per drive increases with time remaining up until about three minutes remaining, and then it flattens out. Calling timeout with more than three minutes remaining increases the risk that your own drive will end quickly and the opponent will have one more drive before the half, while not increasing your own chance of scoring, so it is a mistake to call timeouts too early. Calling timeout before an expected kick (Field Goal or Punt) with 3:20 or less and a running clock is reasonable but not mandatory.

1st-3rd Down

If the opponent is backed up (inside their own 15): There’s really no need to call timeouts ahead of the Two-Minute Warning after 1st or 2nd Down, as an immediate stop will force a punt that will probably net 40-50 yards. Two minutes is enough time for a drive that starts between Midfield and Own 40, and the data doesn’t suggest calling timeouts to save time at this point is necessary, even for drives starting on average slightly further back than this range. On the other hand, if your opponent does get a First Down, they still will likely be inside their own 25-30, so the extra time may be valuable for their drive.

If the opponent is near their own 25-35, then a punt will likely put us at our own 25-35. Because the field position we would get from a  stop is approximately equal to our opponents’ current field position, calling timeouts on Downs 1-3 under two minutes makes sense if our opponent has a less than 50% chance of gaining a first down. Using the table from our First Down Likelihood Post, we can see this corresponds to 2nd & 10 or more and 3rd & 5 or more. In a real game these probabilities would vary slightly depending on relative team strength, wind direction, kicker quality, and field position within that range, but this is the appropriate anchor point. Calling timeouts ahead of the Two-Minute Warning on Downs 1-3 is probably not advised unless the opponent is in a very poor down and distance situation (3rd & 20 for example).

If the opponent is between their own 36 and our 25 or so, it is probably never advisable to call timeout on 1st – 3rd down on defense except in very unfavorable down and distances. Even if we stop the opponent and force a punt we will likely be backed up following the punt, while a first down for our opponent following our timeouts would be a disaster if it left them time to attempt to score a Touchdown when the clock otherwise would have run out,  forcing a Field Goal Attempt. Because the cost of timeouts when the opponent achieves a first down is significantly greater than the benefits of saving time with a timeout, only very poor down and distances situations justify a timeout.

Once the opponent enters our 25, there reaches a point where their chance of running out of time is essentially zero. For instance, if the opponent has 1st & Goal at our 10 with two minutes remaining in the half and three timeouts, we should begin using our own timeouts, as it is extremely unlikely our opponent will run out of time. By calling timeouts, we can ensure getting the ball back with ~1:30 remaining in the half, whereas if we don’t call timeout our opponent can run down the clock it’s possible we don’t get the ball back at all before the half. Essentially, once  our opponent is close enough to our End Zone that they are not under any time pressure, we should focus on getting the ball back for our own drive with an optimal amount of time (2:00-3:00) before the half. With under 40 seconds or so it is unwise to call timeout before the half, as the value of a drive starting in your own territory with 0:30 or less remaining is essentially  zero and even if your opponent has timeouts there is some chance they mismanage the clock and cost themselves downs in the final 40 seconds of a half.

There are a a few unique constellations of situations I haven’t addressed here where this basic outline is suboptimal, but this should provide a basic theoretical baseline for evaluating timeout usage in end of half defensive situations. This is already running fairly long, but in a future post I’ll distill this information into a simple rule chart that would be easy for a team to implement.

A Game Theory Approach to the Problem of Going for Two When Up by Seven Late in a Game

By Max Mulitz

 

This post is arguably a month late, but good game management is always timely and I don’t think anyone on the internet really got at the problem from a game theory perspective.

On the one hand, in the article linked above, Pete Carroll notes that being up 7 gave his team about a 79% chance of winning, he further suggested that making the field goal to go up 8 would give his team an 85/86% chance of winning and that making the two point conversion would raise his teams chances to 90-92%. These numbers are more or less aligned with Brian Burkes old Win Probability model from Advanced NFL Stats.

On the other hand, win probability estimates can get pretty noisy in the endgame, especially when you are talking about crossing over meaningful thresholds of points. For instance, weather a team is trailing by 4 or 5 points with 0:30 left makes no practical difference to the teams chance of winning, but the one point difference between leading by 6 and 7 has an enormous impact on win probability. For this reason, relatively  time remaining, it makes sense to model a game in terms of number of score differential rather than point differential.

It’s time for some game theory. I’m going to take the view that with less than 5 minutes remaining, number of score differential is more important than points.

The first question is if an 8 point lead constitutes a one or two score game. If you believe leading by 7 points and 8 points are each one score games and a 9 point lead is a two score game, then going for two up 7 is risk free proposition! Of course this is absurd because a team is much more likely to fail on a two point conversion than they are on a PAT.

The chance that an 8 point game is a two score game is simply the probability that  the opponent will convert their two point conversion if they score. So a team with a 25% chance of converting a two point conversion is in a Two Score game 75% of the time when they are down 8, but a team with a 75% chance of converting only has a 25% chance of trailing by two scores. In reality, the vast majority of teams are probably between 40-60% to convert a two point conversion in a given game, with the mean right around 50%.

The chance that a 9 point game is a two score game is obviously 100%, since there is no three point conversion.

The idea that a 7 point game is always a one score game is a parochial misconception. In fact, with the new PAT rules, teams only make about 94% of their PATs. So when trailing by 7 there is actually a 6% chance you trail by two scores.

Therefore, the chance of leading by Two Scores when going for Two when leading by 7 is equal to the chance you fail to convert multiplied by .06 (the chance your opponent scores but misses the PAT) plus the chance you do convert to go up multiplied by 1 (as a 9 point lead is always a Two Score game), as show below.

Two Score Game Probability (Go For It)=Probability you fail on your two point attempt and your opponent misses their PAT to tie + probability you succeed on your two point attempt.

Two Score Game Probability (Go For It)= .06*P(Fail) + 1*P(Success)

Two Score Game Probability (Go For It)= .06*(1-P(Success))+P(Success)

Two Score Game Probability (Go For It)= .94*P(Success) + .06.

On the other hand, if you attempt a PAT to go up 8, the chance you are in a  two score game is .06*the chance you miss your PAT (i.e. you miss your PAT and are still up 7, then the opponent misses theirs and you still lead by 1) plus the chance the opponent makes a two point conversion after you make your PAT. If we assume a teams own chance of making the PAT is 94%, we can model this situation the following way.

Two Score Game Probability (Attempt PAT)= Probability both teams miss their PAT attempt + Probability you make your PAT and the opponent fails on their Two Point Conversion.

Two Score Game Probability (Attempt PAT)= .06*.06 + .94*P(Opponent Fails)

Two Score Game Probability (Attempt PAT)==.036 + .94 & P(Opponent Fails)

A team should Go For It if their probability of leading by two scores is at least as high as it would be if they chose to kick the PAT. That is, they should go for it if

Two Score Lead Probability (Attempt PAT) > Two Score Lead Probability (Go For It) which is equivalent to saying to go for it if

.94*Probability of success + .06>.036 + .94*Probably opponent fails

This can be rewritten as:

.94*Probability of success -.94*Probability Opponent Fails>.024

Which can be further simplified to

Probability of Success -Probability Opponent Fails>2.6%

So if a teams probability of converting a Two Point Conversion is a little higher than their chance of stopping their opponent on a Two-Point Attempt then Going For Two up 7 is justified.

Of course, some kickers are better or worse than others. I made both kickers league average so I could easily show the difference in success rates when going for two, but in an applied setting it would be easy to change these numbers slightly on a situation by situation basis.

 

Conclusions:

It seems, the Go For Two vs. Kick a PAT decision late in a 7 point game is close enough to essentially be a toss up, so there’s no clear right or wrong answer in a given situation. Essentially, the decision a team must make is if they think they are more likely to convert their own Two Point Conversion attempt or if they are more likely to stop their opponent from converting a Two Point Conversion. At some point I will post about at what happens if we expect our opponent to go for Two when they are trailing by 7, though the math is a little more complicared and the conclusion (it’s a judgment that comes down to a teams relative offensive and defensive strength) stays the same.

 

Understanding The Jumping Tests at the NFL Combine by Adjusting for Weight

By Max Mulitz

 

Except in very rare cases (possibly defensive backs and receivers) the actual height of a players’ vertical jump is not directly relevant to their ability to play football. The broad jump is even less immediately applicable. That said, both the broad and vertical jump are important metrics for predicting football success because they measure leg power.

This study from the NIH validates these formulas which use body weight and jump height to predict leg power.

Bill Walsh also points this concept out in his book, Finding The Winning Edge where he states, “Although the vertical jump is a reliable indicator of explosive leg power, this test has a few limitations as an evaluative tool. For example, vertical jump scores are affected by body weight. All factors considered, a 260-pound offensive lineman, who would be considered light by NFL standards for his position, would normally be able to jump higher than his 300 pound teammate. Even if both players were in comparable physical condition, the 300 pounder has to do more work than the 260 pound player because he has to move more weight over the same distance.”

Pat Kirwan demonstrates the importance of Jumping Metrics with his   Explosion Index from his book Take Your Eye Off the Ball where he shows Vertical Jump + Broad Jump + Bench Press Reps is a meaningful predictor of which pass rushing prospects will be successful. While the Bench Press has largely been shown to be uncorrelated with success, I like that Kirwan uses both Broad and Vertical Jump because, as Zach Whitman points out in the FAQ for his site using more than one test reduces bias that can result from measurement error/a player having an randomly particularly good or bad day at one of the tests, so by combining two leg body power tests you are likely to get a more accurate estimate of a players true leg power than by using either test individually.  Based on the above studies, we can expect metrics such as Pat Kirwans, which are already useful for predicting successful NFL players,  to become even more powerful if we use weight adjusted jumping and instead of simple jump height, because weight adjusted jumping is a more accurate measure of leg power than jump height alone.

 

Arm Length is Probably Universally More Important than Height in Sports

by Max Mulitz

Studies of the NBA have shown that wingspan is more important than height when predicting blocked shots (and player success in general.) Of the five players on the All NBA team from last season, Kawhi Leonard has remarkably long arms, while Lebron James, Russell Westbrook, and Deandre Jordan also have unusually long arms relative to their height. Only Stephen Curry’s arm length is unexceptional.

It is also true that that the top 3 Pound For Pound Fighters in the UFC, Conor McGregor, Demetrius Johnson, and Jon Jones, all have long arms for their height. At 76 inches tall,  Jon Jones 84.5 inch reach is famously the best in the UFC. Mcgregors 74 inch reach at 5’9 is also unusual.

Perhaps the greatest Olympian of all time, Swimmer Michael Phelps, also has abnormally long arms.

At the NFL level arm length and wingspan data can be hard to come by, but it is worth noting that Julio Jones, Deandre Hopkins, and Odell Beckham all have exceptionally long arms for their height. Neither Hopkins nor Beckham are particularly tall for NFL receivers, but both players seem to have enormous catch radii, so it seems reasonable to suspect that their arm length is causing them to be able to play bigger than their height would suggest.

I will build to looking at Arm Length among NFL players, but first I wanted to establish some theory and priors. Across a range of sports, we have evidence that the top performers tend to have unusually long arms relative to their height. It would be very strange if height was  more important than arm length/wingspan, especially for receivers and cornerbacks, given the fact that arm length is more important than height when predicting NBA players and the fact that the top performers across a fairly broad range of sports all have unusually long arms for their height.

It’s unclear exactly how height and arm length combine to determine a players “size” or “length”, but it should be clear that any explanation that only uses height at the expense of arm length is inadequate.

How Many Linemen do NFL Teams need to use in a season?

By Max Mulitz

 

Using some snap count data from Football Outsiders, I wanted to look at how many offensive linemen play meaningful snaps for the average team in a season. I looked at each team season from 2012 to 2015 (For instance the 2014 Green Bay Packers is a team season, the 2013 Packers are a different team season) and how many offensive linemen played at least 20% of the teams offensive snaps.

Number of Linemen Seasons
5 11%
6 39%
7 26%
8 22%
9 2%

Only 11% of teams got through the season with only 5 linemen playing 20%+ of snaps. Teams had a 50/50 chance of only having to give 5 or 6 linemen 20%+ of snaps. Teams had a 76% chance of only having to give 20%+ of snaps to 7 or fewer linemen, though to be truly safe a team seems to need to have 8 players they are willing to put in the game at offensive lineman.

Because some Offensive Linemen can only play certain positions (Center/Guard, Guard/Tackle, Guard Only, Tackle Only, etc.) assembling an offensive line is complex and depth needs depend on the versatility of a teams’ personal. Nevertheless, the idea that 6-8 linemen are going to see meaningful playing time can provide a framework for beginning to determining roster depth needs.

 

How Runningback Success Rate is affected by Situation

By Max Mulitz

 

Background Research: Brian Burke, who heads the football analytics department for ESPN (the worldwide leader in sports) showed Runningback Success Rate is more strongly correlated with winning than Yards per Carry. However, it is unclear how much the down and distance situations where teams choose to run effects success rate, compared to the actual talent of the players.ArmchairAnalysis defines success in the following manner: “Generally, a play is deemed ‘Successful’ when the following occurs: 40% of yards-to-go are gained on 1st down; 60% of yards-to-go on 2nd down; or 100% of yards-to-go on 3rd & 4th down.”

Using data from Armchair Analysis from the 2011 through 2015 seasons, I was able to construct the following chart showing expected rushing success rate by down and distance.

Screen Shot 2016-12-15 at 11.29.54 PM.png

We can see that rushing success rate clearly decreases as yards to go increases and that rushing success rate is generally lower on 3rd down than on the other downs.

By looking at the differential between a players success rate and the success rate that would be expected based on the down and distance situations where they got their carries, we can look at a running backs success rate independent of how they re being used.

The percentage of a players carries that are successful is  the players Success Rate. The average success rate for rushes over all the situations where a player gets the ball is the players Expected Success Rate.  We will futherdefine the difference between a players actual success rate and their expected success rate as their Success Rate Differential, which is success rate compared to average.

Findings: The Year to Year Correlation for Success Rate for Runningbacks with consecutive 100 carry seasons between 2011 and 2015 (Sample Size=106) was .26 and the correlation for Success rate differential was only .20, however the Year to Year Correlation for Expected Success Rate was .43, meaning a runningbacks’ success rate as a function of their role is more stable year to year than the success rate experienced as a function of their talent (as measured by differential from expected success rate.)

It is also worth noting that Expected Success Rate ranged from 41% to 51% in the sample, a significant distribution, so if you compare running backs based only on success rate you are capturing a lot of situational effects.

While differential only weakly predicts itself, previously years differential is significant at P=.05 when trying to predict current years differential. The coefficient for previous years differential is .21, so a RB who performs 5% above average would be expected to regress to just over 1% above average the following year.

Case Study: Mark Ingram of the New Orleans Saints was a 1st Round pick (28th Overall in 2011)

Ingram.png*Ingram did not receive 100 Carries in 2013 due to injury

The average success rate for a 100+ carry RB was 45.5%, so Ingram was above average in 3 of the 4 years. However, once you adjust for the expected success rate based on the situations in which Ingram got the ball, we see his success rate is approximately average to slightly below average over the four year sample. Success rate overrates how successful the Saints have been when rushing with Mark Ingram.

Case Study 2: The New England Patriots have had a 100+ Carry back with a 49%+ Expected Success Rate each year.  3 different players have served this lead back role over the timespan.

Patriots.png*In 2014 only 60 of Blounts 125 Carries came with the Patriots, early in the season he played for the Steelers

It is unclear if the relationship between Saints and Patriots high Win % in the sample (56% and 76%, respectively) and their above average rushing success rate is driven more by runningback talent or choosing to run in high success rate situations.

Conclusions: After adjusting for situation, a runningbacks previous years Success Rate is a weak but still statistically significant predictor of the next years Success Rate. Only about 20% of the difference between how successful a running back is compared to expectation persists from year to year.

These findings disprove the idea that rushing success rate is a pure or approximately pure measure of running back talent. These findings also strongly suggest that at least part of the relationship between rushing success rate and winning is caused by teams choosing to run in advantageous situations and not just teams having more rushing talent than others.

 

Piecing Together the Injury Puzzle

By Max Mulitz

 

There’s been some really great work from a variety of sources on injury rates.  I’m gonna try to synthesize the current research on injury rate and hopefully further the discussion by addressing a couple areas that have so far gone unnoticed.

Football Outsiders and Pro Football Logic both peg the percentage chance of a player missing at least one regular season game with an injury in a given season at ~38%, meaning 62% of players go uninjured. However, when you limit your sample to players who were on a team for the whole 16 game season, only 54% of players are able to avoid injury the entire season. Football Outsiders also notes that rate of players missing time with injuries is generally increasing over time, though Pro Football Logic’s data is from 2015. In total, a base rate of about a 58% chance of not missing time for the average player in a given season is probably about right. Another way to address this problem would be to look at the per game injury rates and then take the probability of not getting injured in a given game and extrapolate it to a full season (15 games, since injuries suffered during week 17 can’t cause the player to miss any regular season games.) Using Pro Football Logic’s injury rate per game of 4.1%, we can extrapolate an expected 47% chance of a player missing at least one game with an injury.

One question might be why the implied chance of a healthy season is a little lower than we’d expect, 53% instead of 58%. One contributing might be injury recurrence. If players who miss games at some point in the season with an injury are more likely to get re-injured later in the season than the average player, than the total injury rate will cause us to expect a greater number of players to sustain at least one injury than is prudent. A player with a 58% chance of staying healthy to dress for 16 games has a 3.6% chance of being injured in a given game (assuming all games carry an equal risk of injury).

 

Injury Proneness

Injury Predictor is an NFL injury analysis prediction website that has done research demonstrating that “injury proneness” seems to be predictable and documenting the increased risk of injury recurrence over a season.

It strikes me as inherently obvious that some players are injury prone. To give an extreme example, it became clear that Arian Foster was more likely than the average player to get injured in his final few seasons, which led to his in-season retirement. While I don’t agree with all of Injury Predictors’ methodology (predicting Drew Brees only has a 1% chance of missing a game with an injury this year is silly) they do claim to have demonstrated the ability to predict which players are at the highest risk of injury (66-75% chance of missing at least one game) and which players are at the lowers risk (~30% chance of missing time).

It may seem strange given a base rate of about 40% injuries to see injury predictability of injury prone players 20-30% above the baseline but for lower risk players to only be injured 10% less than average, but actually I think it makes intuitive sense. If you imagine me, or any person of average athleticism, playing in an NFL game where I either receive 20 carries or play until I am too injured to continue, the chance I would actually be healthy enough to touch the ball 20 times are vanishingly small. This should make it clear that injury likelihood can scale up to essential 100% depending on situation. On the other hand, even the healthiest player is one awkward helmet to the side of the knee away from being out for the rest of the season, football is a brutal game on the human body, and there simply aren’t people who are impervious to it’s risks.

Anyway, my sense right now is that it is possible to identify high injury likelihood, but players with an extremely low rate of injury are probably mostly lucky. Also, because injury likelihood increases with age, by the time you identify a player as being particularly unlikely to get injured, his increased age has probably offset the benefit. Actually that’s exactly what we see in the Football Outsiders piece, where the injury rate is flat across all ages, but when you look at players with long careers their probability of getting injured increases each year of their career. Research suggests the negative physical effects of aging begin at around age 23, so each year of a players career is going to bring accumulated damage and increased risk.

Closing Thoughts

In an upcoming post, we’ll look at average injury length across position and then combine injury rate with injury lengths to look at total games lost due to injury. Understanding how many injuries a team can expect to have is integral to team building and achieving balance between acquiring top-end starters and maintaining quality depth through the lineup so the team can continue to function if injuries do occur.