Jump to content

S&P+ rankings: Overperformance or overcorrection?


satyanash

Recommended Posts

44 minutes ago, sushihorn said:

It's hard to tell where his model ends and his personal opinion starts.  It's very unprofessional and a perfect example of the Appeal to (false) Authority Fallacy.

honestly, it really isn't and if you think that you aren't doing any research or paying attention to it.

it's pretty clearly based around the Five Factors (link) - in the past he released the underlying information in a google sheet but think that has been mostly stopped by ESPN. I went back a little ways and thought I saw one of the docs after like week 3, but cant find it. He is pretty damn transparent as to what goes into his model and historically has released weekly profiles for each school in an enormous google sheet.

here is some additional/updated info on that in this tweet thread about where things are YTD (the first tweet has more 5 factors stuff in it):

 

 

  • Like 1
  • Fuck You 1
Link to comment
Share on other sites

Sushi is right.  I've been following him for years and it's pretty true.  He'll make decisions based on what he thinks should be true instead of where the data takes him (and posts about it in his blog).  He ended up with a weird amalgamation of statistical modeling and opinion, and it's pretty consistent about overvaluing the style of football he prefers and undervaluing others.

He was in a better place 10 years ago when he was focused on S&P (success and points earned), rather than the five factors.  Fremeau's system ended up better (FEI - bcftoys.com) while using less granular data because he stated true to that philosophy.  Connelly should have come close to creating what amounts to DVOA for college, but he went the opposite direction from pretty much the entire FO community.

Edited by JBJ
  • Like 1
Link to comment
Share on other sites

1 hour ago, NoName said:

honestly, it really isn't and if you think that you aren't doing any research or paying attention to it.

it's pretty clearly based around the Five Factors (link) - in the past he released the underlying information in a google sheet but think that has been mostly stopped by ESPN. I went back a little ways and thought I saw one of the docs after like week 3, but cant find it. He is pretty damn transparent as to what goes into his model and historically has released weekly profiles for each school in an enormous google sheet.

here is some additional/updated info on that in this tweet thread about where things are YTD (the first tweet has more 5 factors stuff in it):

 

 

Sushi was talking about his tweets and replies, and he’s correct. Like when Bill commented on the OSU game where his own system said we had a 89% win expectancy and said the game just came down to who finished drives the most. 

  • Like 1
Link to comment
Share on other sites

14 hours ago, JBJ said:

Sushi is right.  I've been following him for years and it's pretty true.  He'll make decisions based on what he thinks should be true instead of where the data takes him (and posts about it in his blog).  He ended up with a weird amalgamation of statistical modeling and opinion, and it's pretty consistent about overvaluing the style of football he prefers and undervaluing others.

He was in a better place 10 years ago when he was focused on S&P (success and points earned), rather than the five factors.  Fremeau's system ended up better (FEI - bcftoys.com) while using less granular data because he stated true to that philosophy.  Connelly should have come close to creating what amounts to DVOA for college, but he went the opposite direction from pretty much the entire FO community.

generally with you, FEI is good as well - not that I am some S&P+ fanboy here, but this isn't a 538 situation where you don't really know what tweaks they are doing on it. Bill C is very up front about what goes into S&P+

what y'all are saying about his decisions based on the data and his conclusions - with you there. after re-reading what Sushi said, i read it as you can't really tell what he is using and that was not what he said

FWIW

S&P+ has Texas as the #20 overall team, with the #4 Offense, #66 Defense and #7 Special Teams

FEI has Texas as #16, with the #8 Offense, #48 Defense

F/+ (combined version of the two) has Texas at #18 

here is FEI's garbage time rules btw

 

Spoiler

 

Garbage and Non-Garbage Possessions


Unless otherwise noted, all ratings and supporting data found on this site are calculated only after first filtering out garbage possessions, defined as of July 2018, as follows:

  • An offensive possession of two plays or fewer that runs out the clock to conclude the first half and does not result in a turnover, score, or field goal attempt
  • An offensive possession of two plays or fewer that runs out the clock to conclude the second half with the score tied and does not result in a turnover, score, or field goal attempt
  • A possession in the second half of a game in which eight times the number of the losing team's remaining possessions plus one is less than the losing team's scoring deficit at the start of the possession
  • An offensive possession or non-offensive scoring possession by the winning team leading by eight points or fewer at the start of the possession that runs out the clock to conclude the game

Since 2007, 10.5 percent of all possessions have been classified as garbage possessions according to these criteria. FBS games have averaged 23.8 non-garbage possessions per game (11.9 non-garbage possessions and 11.2 non-garbage offensive drives per team per game) in that span.

 

 

 

Link to comment
Share on other sites

FEI has been way off on their projected margins on Texas all year. I'd be curious to see how far off they are on PM overall, since they have literally not been close once in 5 tries on Texas games.

Game 1 PM 8       Actual 23
Game 2 PM -11.6   Actual -7
Game 3 PM 23.9    Actual 35
Game 4 PM .4      Actual 6
Game 5 PM 4.3     Actual 11

The closest they got was within 4.6 on the LSU game. In 5 tries, that just doesn't strike me as all that good.

Link to comment
Share on other sites

FEI has been way off on their projected margins on Texas all year. I'd be curious to see how far off they are on PM overall, since they have literally not been close once in 5 tries on Texas games.
Game 1 PM 8       Actual 23
Game 2 PM -11.6   Actual -7
Game 3 PM 23.9    Actual 35
Game 4 PM .4      Actual 6
Game 5 PM 4.3     Actual 11
The closest they got was within 4.6 on the LSU game. In 5 tries, that just doesn't strike me as all that good.


Generally speaking, an average error of 8.6 is quite impressive. Over the 2018 season, the final Vegas line for all FBS v. FBS games had an average error of 12.65.

That said, Texas has played insanely close to the line so far this year, with an average error of 4.1, so in that context, a system with an average error more than 2x greater than the line probably doesn’t have Texas all that pegged.
  • Like 2
Link to comment
Share on other sites

5 minutes ago, DanRydell said:

a system with an average error more than 2x greater than the line probably doesn’t have Texas all that pegged.

I know nothing about that system but if it’s pure number crunching it can’t adjust real time to injury data like Vegas can.  I would also be surprised if it has a way to account for the effect of teams coming off emotional hard fought games. 

Link to comment
Share on other sites

6 hours ago, Pato del Muerto said:

Is FEI a betting application?  Or a score predictor?  Because as is said ad naseum the Vegas line can be effected by the money and will move to balance the money on both sides and isn’t a true outcome predictor. 

While the final line is not a pure scoring predictor, I was comparing them because the final line finishes basically every year as having the smallest absolute error of the many different systems tracked by ThePredictionTracker.com.

http://www.thepredictiontracker.com/ncaaresults.php?year=18

  • Like 1
Link to comment
Share on other sites

On 10/11/2019 at 10:11 PM, Walden Ponderer said:

FEI has been way off on their projected margins on Texas all year. I'd be curious to see how far off they are on PM overall, since they have literally not been close once in 5 tries on Texas games.

Game 1 PM 8       Actual 23
Game 2 PM -11.6   Actual -7
Game 3 PM 23.9    Actual 35
Game 4 PM .4      Actual 6
Game 5 PM 4.3     Actual 11

The closest they got was within 4.6 on the LSU game. In 5 tries, that just doesn't strike me as all that good.

i get what you are saying but FEI is doing quite well overall this year overall using ThePredictionTracker - 3rd overall, picked games correctly at a 78.17% clip and 2nd overall almost in line with the Updated Line weekly at 54.68%

Quote

The ideal system would be one that has the highest correct game decisions,
has the smallest mean error(deviation from the actual game result), and has
a bias of zero.

Mean Error = average[abs(prediction-actual)]

      Bias = agerage(prediction - actual)

      Std. = Standard Deviation of individual game biases

 

Edited by NoName
Link to comment
Share on other sites

On 10/13/2019 at 12:32 AM, ClubWhatever said:

I know nothing about that system but if it’s pure number crunching it can’t adjust real time to injury data like Vegas can.  I would also be surprised if it has a way to account for the effect of teams coming off emotional hard fought games. 

not disagreeing with you but right now there are 12 ranking systems with "better" SU results (by 1-4 games out of 339) this year on ThePredictionTracker

http://www.thepredictiontracker.com/ncaaresultsphp?orderby=wpct desc&type=1&year=19

Link to comment
Share on other sites

On 10/27/2019 at 10:03 AM, Fozzz said:

I guess this is just gonna be the Missouri thread now that no one can conceivably argue that SP+ was underrating Texas. 

Texas State #108?  This guy doesn't know his ass from a hole in the ground!  No way the Bobcats should be lower than 105!  Fuck Bill Connelly!

Link to comment
Share on other sites

On 10/27/2019 at 10:03 AM, Fozzz said:

I guess this is just gonna be the Missouri thread now that no one can conceivably argue that SP+ was underrating Texas. 

I’m just impressed his model predicted massive injuries to the Texas defense. Kind of amazing it knew we’d be starting a true freshman with no game reps instead of sterns in the 8th game.

Link to comment
Share on other sites

1 hour ago, bigcigar said:

I’m just impressed his model predicted massive injuries to the Texas defense. Kind of amazing it knew we’d be starting a true freshman with no game reps instead of sterns in the 8th game.

If it is true that but for injuries Texas would be a top ten team then I don't know why the hoopleheads are clamoring for firings.  Seems like the coaches are doing their jobs and the fans need to grow up.

Link to comment
Share on other sites

16 minutes ago, Fozzz said:

If it is true that but for injuries Texas would be a top ten team then I don't know why the hoopleheads are clamoring for firings.  Seems like the coaches are doing their jobs and the fans need to grow up.

I’m fine with conceding top 10, but it’s not a stretch to think they were comfortably a 9 win team even with coaching issues if early season trends held.

Link to comment
Share on other sites

  • 2 weeks later...
40 minutes ago, Fozzz said:

It is so infuriating having to monitor his twitter to find the box scores and any other information he chooses to release.  Why doesn't ESPN have a page dedicated to this shit on their site?  

it's a shitty CMS i assume, people ask Keith Law about this all the time as well. 

if it was do-able I assume they would do it. but due to rolling out a shitty CMS i am guessing it isn't.

Link to comment
Share on other sites

Just now, NoName said:

it's a shitty CMS i assume, people ask Keith Law about this all the time as well. 

if it was do-able I assume they would do it. but due to rolling out a shitty CMS i am guessing it isn't.

ESPN's site is almost unusable.  Way too many ads and way too many visuals/media that cause your browser to shit itself.  

  • Like 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.



×
×
  • Create New...