Jump to content

S&P+ rankings: Overperformance or overcorrection?


satyanash

Recommended Posts

Updated S&P+ rankings: we're now at #41, up one spot from last week. Other quick notes:

- Only two Big 12 schools are featured in the top 15: Oklahoma at #5 and WVU at #12. We're the 6th-best team in the Big 12, behind the aforementioned two plus Iowa State, Tech, and Okie lite.

- USC is now ranked at #36, 5 spots ahead of us O_o They jumped 7 spots after losing to Cal. Tech also jumped 7 spots after losing to us.

- Among in-state schools, we're behind A&M (#21), North Texas (#24), and Texas Tech (#26). We're ahead of Houston (#43) and TCU (#67).

- Our expected win total is now 8.4. The Iowa State game is a tossup thanks in part to our home-field advantage, while we're heavily favoured to beat Kansas.

Edited by satyanash
Link to comment
Share on other sites

5 hours ago, satyanash said:

- USC is now ranked at #36, 5 spots ahead of us O_o They jumped 7 spots after losing to Cal. Tech also jumped 7 spots after losing to us.

I like what Bill is trying to do, but I've never followed his rankings this closely week-to-week. How is this possible? Is our ability to give up big leads late short circuiting his system?

Link to comment
Share on other sites

I think looking at week to week movement is silly in computer rankings. One of their major benefits relative to human polls is they don’t simply move teams up and down based on the prior week’s games.

For example, a human poll gives LSU a major boost for beating #8 Miami and #7 Auburn over the first three weeks of the season and then just moves LSU up or down based on their performance week to week, without ever really looking back and reconsidering the value of those wins over what turned out to be fairly mediocre teams.

Link to comment
Share on other sites

24 minutes ago, DanRydell said:

I think looking at week to week movement is silly in computer rankings. One of their major benefits relative to human polls is they don’t simply move teams up and down based on the prior week’s games.

For example, a human poll gives LSU a major boost for beating #8 Miami and #7 Auburn over the first three weeks of the season and then just moves LSU up or down based on their performance week to week, without ever really looking back and reconsidering the value of those wins over what turned out to be fairly mediocre teams.

I guess I'm saying, I don't understand what data you could get from the Tech game that moves them up 7 spots (and still ahead of us). We entered the 4th quarter, on the road, with a lead against a team that is better than us (according to his rankings). 

Link to comment
Share on other sites

5 hours ago, GabrielsHorn said:

He really undervalues turnovers and we gave up a shit ton of yards to Tech. 

Again, I’ll repeat that this system is given way too much credence on the internet. FPI, FEI, and Saragin are all much higher on us, and when you average the systems, S&P is towards our lowest ranks. 

turnovers are luck in his model. if you win close games due to turnover advantage his model basically assumes you lost.

Link to comment
Share on other sites

18 minutes ago, dcar00 said:

turnovers are luck in his model. if you win close games due to turnover advantage his model basically assumes you lost.

Texas loses to TCU, OU, and Tech without unforced errors leading to turnovers so this isn't necessarialy a bad way to look at things. I would be curious to know whether or not the model views a team that consistently finishes games with no turnovers differently than one that has a string of 2-3 turnover games from opponents. Texas has lucked into a fair bit of turnovers this year but taking care of the ball like they have is a virtue. Turning the ball over only twice in seven conference games is not luck.

Link to comment
Share on other sites

25 minutes ago, gmr548 said:

Texas loses to TCU, OU, and Tech without unforced errors leading to turnovers so this isn't necessarialy a bad way to look at things. I would be curious to know whether or not the model views a team that consistently finishes games with no turnovers differently than one that has a string of 2-3 turnover games from opponents. Texas has lucked into a fair bit of turnovers this year but taking care of the ball like they have is a virtue. Turning the ball over only twice in seven conference games is not luck.

Ok but what unforced errors?  this is the problem with his "I'm accounting for luck" model.  all our turnovers against tech were forced.  does he consider a 4th down stop a turnover?  

an unforced turnover might be a snap over a QB/Punter that is immediately recovered but there are very few unforced turnovers.

Link to comment
Share on other sites

13 minutes ago, dcar00 said:

Ok but what unforced errors?  this is the problem with his "I'm accounting for luck" model.  all our turnovers against tech were forced.  does he consider a 4th down stop a turnover?  

an unforced turnover might be a snap over a QB/Punter that is immediately recovered but there are very few unforced turnovers.

1. Not all of them were forced. The interception from Duffey doesn't happen without the terrible snap that caused him to scramble.

2. Bill calculates an 'expected turnover margin' for each game based on the number of forced fumbles and pass breakups. He doesn't use actual turnover margin because that figure can rarely be predicted (turnovers are highly dependent on luck). So essentially it's considered an anomaly when your actual turnover margin is out of line with what's expected based on the above two factors.

Link to comment
Share on other sites

Just now, satyanash said:

1. Not all of them were forced. The interception from Duffey doesn't happen without the terrible snap that caused him to scramble.

2. Bill calculates an 'expected turnover margin' for each game based on the number of forced fumbles and pass breakups. He doesn't use actual turnover margin because that figure can rarely be predicted (turnovers are highly dependent on luck). So essentially it's considered an anomaly when your actual turnover margin is out of line with what's expected based on the above two factors.

I knew someone would come back with that.  Duffey could have thrown it away earlier but we kept on him and he threw it up late.  if we don't stay on on him or don't get a rush he might actually complete a pass.

I realize its a predictive model but it pretty much sucks as it relates to turnovers.  turnovers are rarely "unforced" regardless of what you hear the talking head idiot announcers say.

  • Like 2
Link to comment
Share on other sites

3 minutes ago, satyanash said:

1. Not all of them were forced. The interception from Duffey doesn't happen without the terrible snap that caused him to scramble.

2. Bill calculates an 'expected turnover margin' for each game based on the number of forced fumbles and pass breakups. He doesn't use actual turnover margin because that figure can rarely be predicted (turnovers are highly dependent on luck). So essentially it's considered an anomaly when your actual turnover margin is out of line with what's expected based on the above two factors.

Disagree. Duffey still had plenty of time after catching the snap to decide to throw it away, run it, or try to salvage the play.

Link to comment
Share on other sites

8 minutes ago, dcar00 said:

I knew someone would come back with that.  Duffey could have thrown it away earlier but we kept on him and he threw it up late.  if we don't stay on on him or don't get a rush he might actually complete a pass.

I realize its a predictive model but it pretty much sucks as it relates to turnovers.  turnovers are rarely "unforced" regardless of what you hear the talking head idiot announcers say.

Spot on. All except one of the turnovers in the TCU game were not flukes and a result of good defensive play. We won that game by two touchdowns and were in complete control of it in Q4. We don't lose that game without the "fluke" turnover.

The only potentially "fluke" turnover was when Murray tried to use the ball to prop himself up while under duress and lost it. I still don't completely consider that a fluke, since the defensive pressure contributed to that mental error.

Bottom line: turnovers are a part of the game. Defenses take pride and celebrate forcing turnovers. Announcers and analysts make note of a teams turnover margin  when discussing match-ups. It's much more often skill than luck that results in a plus turnover margin.

TL/DR: Stats are for losers. Take the win, pussies.

Edited by USNALonghorn
Link to comment
Share on other sites

32 minutes ago, dcar00 said:

Ok but what unforced errors?  this is the problem with his "I'm accounting for luck" model.  all our turnovers against tech were forced.  does he consider a 4th down stop a turnover?  

an unforced turnover might be a snap over a QB/Punter that is immediately recovered but there are very few unforced turnovers.

Shawn Robinson derp-fumbling in Texas territory or missing a TD pass to instead check down to a wide open Brandon Jones. Kyler Murray's ground fumble thing. Duffey INT as mentioned. I am not ragging on Texas but rather pointing out they have gotten some breaks. As I also pointed out, they have consistently taken good care of the football which is definitely not a product of luck. That is why I am curious on how exactly the model incorporates turnovers.

Edited by gmr548
Link to comment
Share on other sites

4 minutes ago, gmr548 said:

Shawn Robinson derp-fumbling in Texas territory or missing a TD pass to instead check down to a wide open Brandon Jones. Kyler Murray's ground fumble thing. Duffey INT as mentioned. I am not ragging on Texas but rather pointing out they have gotten some breaks. As I also pointed out, they have consistently taken good care of the football which is definitely not a product of luck. That is why I am curious on how exactly the model incorporates turnovers.

If our opponents making shitty plays is a break, well we have given them plenty of breaks as well.

Link to comment
Share on other sites

3 minutes ago, gmr548 said:

 As I also pointed out, they have consistently taken good care of the football which is definitely not a product of luck.

That's a good point to bring up, because many people see turnover margin as a solely defensive statistic. Turnover margin is TEAM statistic. It's a product of forcing turnovers with good defensive play (stripping the ball, jumping routes, reading the QB), taking advantage of opposing offense's mental errors, and taking care of the football on offense. 

Link to comment
Share on other sites

4 minutes ago, Valmy77 said:

If our opponents making shitty plays is a break, well we have given them plenty of breaks as well.

We've actually done a great job of not committing turnovers. For the season we're +7 in turnover margin, good for 15th in the nation. But like I said, part of that is luck as well.

Edited by satyanash
Link to comment
Share on other sites

13 minutes ago, satyanash said:

We've actually done a great job of not committing turnovers. For the season we're +7 in turnover margin, good for 15th in the nation. But like I said, part of that is luck as well.

Yeah but we have done plenty of horribad non-turnover things. Like Brandon Jones fielding that punt on the one yard line...

Link to comment
Share on other sites

Claiming that a team executing on the defensive side by creating turnovers is luck is absurd. Luck is a human construct that does not have any basis in fact. Literally what the people who make the S&P rankings are saying when they call turnovers "luck" is "this is a variable that we don't know how to or dont want to calculate." Either way, its fucking bush league.

Link to comment
Share on other sites

8 hours ago, gmr548 said:

Shawn Robinson derp-fumbling in Texas territory or missing a TD pass to instead check down to a wide open Brandon Jones. Kyler Murray's ground fumble thing. Duffey INT as mentioned. I am not ragging on Texas but rather pointing out they have gotten some breaks. As I also pointed out, they have consistently taken good care of the football which is definitely not a product of luck. That is why I am curious on how exactly the model incorporates turnovers.

kyler murray was under serious duress .  if he doesn't fall we could have just as easily stripped the ball from him.

Actually the way you are making the point, taking care of the football is a product of chance.  a guy fumbling while running wide open is chance as is jumping a route to get an INT.

The dudes model sucks as it relates to turnovers. its fine, it doesn't mean shit, but make no mistake it sucks.

  • Like 1
Link to comment
Share on other sites

17 hours ago, Ricky Butler said:

I guess I'm saying, I don't understand what data you could get from the Tech game that moves them up 7 spots (and still ahead of us). We entered the 4th quarter, on the road, with a lead against a team that is better than us (according to his rankings). 

You can't predict how well a team performed week-to-week in his metrics based on score.  The bulk of S&P+ is success rate/play, which may be masked in the final score by other factors (turnovers, field position, points/opportunity, etc.).  

Link to comment
Share on other sites

5 hours ago, HOOK'EMHOOAH said:

Claiming that a team executing on the defensive side by creating turnovers is luck is absurd. Luck is a human construct that does not have any basis in fact. Literally what the people who make the S&P rankings are saying when they call turnovers "luck" is "this is a variable that we don't know how to or dont want to calculate." Either way, its fucking bush league.

Does he actually claim that turnovers are created by "luck"?  S&P+ is designed as a predictive metric, so it makes sense to discount stats that have little reliability, like turnovers, as factoring them in will decrease S&P+'s own reliability.  

Link to comment
Share on other sites

Others have said he uses pass breakups which seems like it may have an issue as a proxy for interceptions. Lots of interceptions aren't caused by tight coverage but by confusing the quarterback (think about Brandon Jones's INT against OU). Then there are those caused by quarterback pressure (but not sacks), etc.

Link to comment
Share on other sites

Although Bill's not entirely clear in the formula behind S&P+, I believe he includes turnover margin in the S&P+, but gives it little weight relative to success rate due to turnover margin's limited reliability relative to success rate.  I believe he also uses adjusted turnover margin as well to account for variance in fumble recovery rate and the fact that the ratio of interceptions/passes defended has extremely little year-to-year reliability.  

Link to comment
Share on other sites

His system doesn't say that turnovers are luck. His system says that creating turnover opportunities is skill (as measured by fumbles forced and passes defensed) but whether you actually turn those opportunities into turnovers is largely luck. According to his numbers, teams will generally revert to the mean of recovering 50% of fumbles forced and 22% of passes defensed.

So a team who has 4 forced fumbles and 18 passes defensed is treated the same (as an expected 6 turnovers), whether those opportunities actually resulted in zero turnovers or 22 turnovers.

  • Like 2
Link to comment
Share on other sites

1 hour ago, Huckleberry said:

Fumble recovery rates, on the other hand, are random and will eventually average out to 50%.

...except that this is not necessarily true. There are elements of luck involved, of course, because which direction the ball bounces is completely out of anyone's control, but there are other aspects of fumble recovery that are very much determined by factors which are more easily predicted. Bigger, stronger, faster players who practice turnover recovery on a regular basis will consistently outperform smaller, weaker, slower players who do not have turnover drills regularly.

Hell, scramble drills in basketball are life or death matters for a lot of coaches. I see no reason why that wouldn't apply to football, as well.

Link to comment
Share on other sites

1 hour ago, Fozzz said:

You can't predict how well a team performed week-to-week in his metrics based on score.  The bulk of S&P+ is success rate/play, which may be masked in the final score by other factors (turnovers, field position, points/opportunity, etc.).  

So if we ran a more aggressive passing attack that led to Sam throwing more INTs but resulted in more explosive plays, would S&P+ like us more?

Link to comment
Share on other sites

11 minutes ago, Ricky Butler said:

So if we ran a more aggressive passing attack that led to Sam throwing more INTs but resulted in more explosive plays, would S&P+ like us more?

Success rate isn't a measure of explosiveness. It's actually the opposite.

It's a measure of the percentage of time a team gained at least 50% of yards to go on 1st down, at least 70% of yards to go on 2nd down, and at least 100% of yards to go on 3rd and 4th downs.

So a team that gains 5 yards on 1st, 4 yards on 2nd, and 1 yard on 3rd will have a higher success rate than a team who throws incomplete on 1st and 2nd and then for 20 yards on 3rd.

Link to comment
Share on other sites

5 minutes ago, DanRydell said:

Success rate isn't a measure of explosiveness. It's actually the opposite.

It's a measure of the percentage of time a team gained at least 50% of yards to go on 1st down, at least 70% of yards to go on 2nd down, and at least 100% of yards to go on 3rd and 4th downs.

So a team that gains 5 yards on 1st, 4 yards on 2nd, and 1 yard on 3rd will have a higher success rate than a team who throws incomplete on 1st and 2nd and then for 20 yards on 3rd.

Interesting. I think I saw a tweet that our offensive ranking has moved up into the 20s over the past few weeks which would align with what I've seen watching the games.

Link to comment
Share on other sites

New S&P+ rankings. We're up eight spots to #33. We're also 26th in Bill's adjusted metric Resume S&P+.

- We're now the 5th-highest team in the Big 12, behind Oklahoma, West Virginia, Okie Lite, and TTU.

- We're the fourth-highest team in the state behind A&M, North Texas, and Tech.

- Our expected regular season win total is now 8.9 wins, with an 87.1% chance of beating Kansas this week.

texas%20-%20sp+%20-%20red-green.png

Link to comment
Share on other sites

3 minutes ago, Drew said:

Just crazy we will most likely have at least 9 wins.  And if they play well in the Big 12 title game and the Bowl game they have a shot at winning 11 games.  11 fucking games?!?  After the last decade?  Just crazy.

I agree with the “crazy” part of this. It was crazy ever getting this program to a point where anyone could think it was crazy for us to win a lot of games.

Link to comment
Share on other sites

On 11/13/2018 at 4:37 AM, HOOK'EMHOOAH said:

Claiming that a team executing on the defensive side by creating turnovers is luck is absurd. Luck is a human construct that does not have any basis in fact. Literally what the people who make the S&P rankings are saying when they call turnovers "luck" is "this is a variable that we don't know how to or dont want to calculate." Either way, its fucking bush league.

Luck is what happens when opportunity meets preparation.

Link to comment
Share on other sites

The formula for offensive success under Herman is to take care of the ball and minimize penalties.  We know that S&P doesn't value the former very much and I suspect the later isn't really accounted for either.  Texas had ZERO penalties on offense against ISU (1 defensive, 2 ST).  There were 2 offensive penalties against TTU and neither one killed the drive.  That's especially impressive when one was a PF that resulted in 1st and 25.  Not sure how a stat based system could measure that.

Link to comment
Share on other sites

On 11/19/2018 at 2:19 PM, Drew said:

Just crazy we will most likely have at least 9 wins.  And if they play well in the Big 12 title game and the Bowl game they have a shot at winning 11 games.  11 fucking games?!?  After the last decade?  Just crazy.

It isn't crazy when you read the fucking name on the jersey, hombre. 

  • Like 1
Link to comment
Share on other sites

What's crazy is us being the 4th best team in Texas. No one honestly believes we wouldn't beat the other 3 teams at least 80% of the time. 


I absolutely don’t believe Texas would beat A&M, Tech, and UNT 80% of the time. Hell, I’m not sure I’d take anyone other than maybe Bama and Clemson to beat them 80% of the time.
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.



×
×
  • Create New...