So this post got delayed. I had planned to have this post drop going into week 1 but one thing led to another and here we are. This post is certainly still relevant and hopefully, it can help convince you about where to spend your time in fantasy. This will be the 4th and final part of my 3-year review series. if interested, please check out the previous posts below
Week 1: Snake Draft Strategy
Week 4: Are you smarter than ESPN: Lineup Decisions
Two years ago, I made my first two blog posts about fantasy football on how accurate ESPN's projections are and whether we set better lineups than ESPN's suggestions. I followed up on the coach comparison, adding comparisons to Fantasy Pros and Reddit Consensus Rankings and adding some better controls for data quality. It was the response to those initial posts that set me down this path of fantasy football analysis. Today, I want to look back at those results, adding in data from the last 2 years to provide a better assessment of just how accurate are ESPN's projections? Do we lowly managers outperform ESPN's lineups? Should we bother thinking about lineups at all? Let's find out.
The Data
I have expanded the dataset from 2 years ago. Now I have over 470,000 lineups from almost 7000 ESPN leagues spanning the last 3 years (2019, 2020, and 2021). All lineups have also been checked to make sure the manager was paying attention, removing any lineup where either the player or their opponent had a player projected for 0 in the lineup. I then went and calculated what the fantasy manager's lineup scored, what ESPN would have scored based on the players on the roster and ESPN's player projections (their actual score, not the projected score), and the best possible score given the roster. I also recorded the projected score of the fantasy manager's lineup. With that, let's dive into the data.
How accurate are ESPN's Team Projections
Scatter plot with a regression line. Each dot represents the odds of winning a matchup given ESPN's projected score difference at the start of the matchup.
As I will do throughout a lot of the analysis today, rather than looking at the raw accuracy (a team is projected for 90 and scores 90, a player is projected for 12 and scores 12) I am focused on functional accuracy (A team projected for 90 normally beats a team projected for 70, a player projected for 12 outscored a player projected for 10).
Given that, the graph actually looks like what I would hope. When comparing the projected difference between a coach and their opponent vs how often they actually won the matchup we see the characteristic S shape that I would expect. Large projected deficits (-40 for instance) have a near zero percent chance to win, the slope increases before crossing the 50% chance of a win when the two teams are projected for the same value and finally, a larger projected advantage wins near 100% chance.
The chart is not perfect, in that person projected to win does not always win and I think better projections would help our own sanity by producing a steeper slope, but overall its not bad. While I wish projections like a 20-point advantage have a better than ~70% chance at winning, that randomness does lead to some of the fun.
Are we smarter than ESPN?
Box Plots of how many points the coach's chosen lineup, ESPN's suggested lineup, and the hindsight is 20/20 Best possible lineup scores
Point-wise, we can see that there is not much difference between ESPN and our Coach Lineups on average in terms of points scored. There may be a little more variance in the coach lineups (both positive and negative as represented by the larger box and wider "whiskers" in the plot). We can also see that neither the coach nor the ESPN lineup is optimal, instead leaving some 20 points on the bench each week on average.
Another way we can look at this is by looking at the distribution of scores and in particular the lineup efficiency of each.
Distributions of lineup efficiency for both coach (blue) and ESPN (orange) lineups
Here we start to see a difference between the ESPN lineups and the Coach Lineups, though still small. Lineup efficiency is simply the total points scored by a lineup divided by the points the best lineup would have scored (with the benefit of hindsight). While the general shape is the same for both coach and ESPN driven lineups, we can see that the ESPN Lineups had more lineups scoring a ~85% efficiency and higher.
That difference probably is why those that followed ESPN won more. Specifically, If the manager followed ESPN, you won 52.5% of your games, compared to only 49.2% of games if the manager did not follow ESPN.
ESPN vs Manager Win Totals
Distribution of the difference in wins across the season between the Coach's lineups each week and ESPN's lineup each week. Blue vertical line marks the point of no difference (0)
That being said, the difference over the course of a season is negligible. We see that nearly 60% of teams would not have changed their win totals if they had followed ESPN the entire season compared to whatever lineups they went with. Note: this does include weeks when they followed ESPN along with weeks when they did not. though the result doesn't change when we remove those weeks.
Do better managers follow ESPN more?
Binned scatterplot and regression line of Coach Efficiency vs ESPN Follow Rate. Data is binned into 25 points of equal underlying size. Each plotted point represents the mean and 95% confidence interval of the bin. The regression line represents the mean and 95% confidence interval of the regression.
Looking at the graphs, I think the answer is mostly yes, better managers follow ESPN More often. Now that being said, the very best managers do not follow ESPN as much. Now this is on a given week, which I doubt will hold when we average across the season.
Does beating ESPN correlate with success?
Binned Scatterplot with regression line. Each point is the mean and 95% confidence interval of its bin. Line is mean and 95% confidence interval.
Kinda surprising given the last graph, we see the teams that finish better also beat ESPN's lineup more often than those who finish in last place. Interestingly, (and importantly) no standing beats ESPN more than 50% of the time on average.
Binned scatterplot with regression of standing vs win difference vs ESPN. 12 bins are created of equal size with the mean and 95% confidence interval (though this is smaller than the dot itself in most cases) shown.
And finally, yes, the person who finishes in first place tends to get all of 1 more win than if they followed ESPN all year, but that advantage quickly drops to basically no advantage by the 3rd or 4th place finisher. My interpretation of this is that yes, you should not treat ESPN as gospel, but you need to be very confident, like greater than 95% sure, that your decision is better than ESPN's suggestion. Alternatively, to win it all you may need to steal a win so playing a player projected for less points but with more variance who hits is important for finishing in 1st place.
Limitations
The common limitations apply. These are mostly public ESPN leagues and while I have tried to account for whether managers continue to pay attention my methods are not perfect, nor do they account for managerial skill (just attention). As some of the plots have shown, the most skilled managers here do seem to outperform ESPN, but they appear to be outliers, not the norm. Personally, I don't think I am that much smarter than average when it comes to Fantasy Football, and will spend my time worrying about who is on my roster, rather than who to start each week.
Conclusions
Taken all of this together, we average humans actually come out better than in my previous work. While I still think you should follow ESPN on average, it is actually closer than I expected. That being said, I still don't recommend deviating from ESPN's suggested lineup. Just spend that time and energy thinking about the waiver wire and making trades with league-mates and let ESPN make your lineup decisions.
One closing note, if debating your lineup decisions each week is an aspect of fantasy football you love, please, continue to make your own lineup decisions. This is a game and should be a fun one at that.
Questions? Comments? Let me know at ac@alexcates.com. Want to read more breakdowns like this? sign up for my newsletter. Finally, like what I do? Consider supporting me on buy me a coffee or by signing up for fantasyleaguereport.com.
Comments