Sports
The Stats Led Our Brackets Astray This March Madness. That Doesn’t Happen Often. – FiveThirtyEight
College Basketball
By Neil Paine
PHOTO ILLUSTRATION BY EMILY SCHERER / GETTY IMAGES
This article is part of our March Madness series.
Since Ken Pomeroy began releasing his team ratings in 2004, his site has become the go-to source for advanced metrics in men’s college basketball. It has helped win bracket pools for countless people — myself included — and firmly established the value of efficiency and tempo-free statistics in the minds of pundits and coaches alike. Sometimes the ratings from Pomeroy and his ilk have been so useful that it’s easy to forget why March has a reputation for madness.
And then there are years like 2022.
Although the men’s Final Four wound up including four blue-blood programs — Kansas, North Carolina, Duke and Villanova — and was ultimately won by the most successful team in college basketball history, this was as unpredictable a tournament for the stats as we’ve seen in recent memory. Very few of the advanced metrics’ pre-tournament darlings ended up having a strong showing, while a number of the teams that looked doubtful by the numbers did better than expected. Compared with any other tournament in the past handful of seasons, this was a year to forget for all of us NCAA number-crunchers.
Let’s take Pomeroy’s ratings as our example. That’s not to pick on him — far from it, in fact. Rather, it’s because his site is regarded as the gold standard in college basketball analytics for consistently doing so many things right. (His data also goes all the way back to the 2001-02 season, which is useful for historical comparisons.) From the 2002 through 2021 men’s tournaments, the average top-five team in KenPom’s pre-tournament ratings won 0.19 more games than we would expect from its seed,1 beating those seed-based expectations 13 times in 19 years (including four out of five from 2016 through 2021). That wasn’t just true of the very highest-rated teams, either; the KenPom top 10 and top 20 also traditionally beat their seed expectations most years, with particular hot streaks running in recent seasons.
This year, however, the highest-rated teams tended to miss their marks more often than not. The average team ranked in KenPom’s top five won 1.08 fewer games in 2022 than its seed would have predicted — its second-worst showing since 2002 (ahead of only 2011) — and the top 10 and top 20 suffered similar fates. In terms of performance versus seed, this was one of the worst years for the metrics in the past two decades of men’s NCAA Tournaments.
Worst performances versus seed expectation for teams ranked in KenPom’s top five, top 10 and top 20 in the men’s NCAA Tournament, 2002-22
Seed expectations are calculated as the average number of tournament wins (excluding play-in games) for teams with each seed number.
Sources: KenPom.com, Sports-Reference.com
Of the top five teams in Pomeroy’s 2022 pre-tournament rankings, four undershot seed expectations by at least one full win, with No. 3 Kentucky (upset in the first round) and No. 5 Baylor (out in the second) missing by more than two wins apiece. (Only fourth-ranked Houston, which won three games as a No. 5 seed, actually met its expectations.) While that is somewhat misleading because the national champion, Kansas, was No. 6 in the rankings — just missing the top five — and it obviously beat its expectations, the rest of the top 10 included Auburn and Tennessee, two more picks that looked promising on paper but flamed out early in reality.
Of course, the computer ratings weren’t alone in overvaluing the likes of Gonzaga, Arizona, Kentucky and Baylor. Running the same exercise on, say, the AP poll would yield a similar disappointing result for 2022. But there is an even clearer indication that this was a tough year for the stats in particular. Collectively, teams that were considered “overseeded” by the KenPom ratingsS-curve of the rankings) were worse than the actual seeds they were given by the selection committee.
RECOMMENDED
Culture
The Definitive Analysis Of ‘Love Actually’
Read Now
Recommended
The Definitive Analysis Of ‘Love Actually’
What Americans Think Of The Biden Impeachment Inquiry
The Second GOP Debate Could Be Smaller, With Or Without Trump
If we look at the net performance of “underseeds” (teams the data liked more than the committee) versus “overseeds” (teams the committee liked more than the data), 2022 was neck-and-neck with 2011 – when 11th-seeded VCU and eighth-seeded Butler both made the Final Four, and relatively unheralded UConn (No. 15 in the pre-tournament rankings) won the championship – as the worst tournament of the KenPom era for us statheads.
Fewest average wins over seed expectations for teams that were underseeded versus overseeded (relative to KenPom rankings) in the men’s NCAA Tournament, 2002-22
Sources: Kenpom.com, Sports-Reference.com
So does this mean we should ignore the metrics when filling out our brackets next year? I don’t think so. Although college basketball is undergoing many wholesale changes at the moment, there isn’t any clear structural reason that KenPom and company would be less reliable predictors than they’ve been in the past. And there are still plenty of opportunities to find value in the bracket: More teams were misseeded (on either the high or low side) this season than in the average tournament since 2002, despite the NCAA overhauling its preferred team rating system just a few years ago.
No, if anything, a down year for the metrics reveals just how spoiled we’ve gotten by their accuracy in recent seasons. According to the net underseed-versus-overseed numbers above, the six best seasons of the entire KenPom era for statistical darlings all came during the period from 2013 through 2021. Underseeds outperformed overseeds in every single tournament of that span, often significantly so. In recent years, ratings like Pomeroy’s were such a secret weapon that it was easy to regard them as automatic tickets to bracket-pool success.
But the NCAA Tournament is designed to create losers, not winners. It is cruel and chaotic, even when the last teams standing are basketball royalty. It’s not supposed to be predictable; the madness is the point. For a brief time, stats like Pomeroy’s helped us forget that — and perhaps it takes a year of reduced accuracy to help us appreciate how useful those ratings have been the rest of the time.
Based on the average tournament wins (excluding play-in games) for each seed number from 2002 through 2021, and excluding the 2020 seedings as that year’s tournament was canceled due to the coronavirus pandemic.
In the sense that their expected seeds based on KenPom’s rankings (found by reverse-engineering seeds off an S-curve of the rankings) were worse than the actual seeds they were given by the selection committee.
Neil Paine was the acting sports editor at FiveThirtyEight. @Neil_Paine
Filed under
College Basketball (231 posts) March Madness (225) Men's College Basketball (76) 2022 NCAA Tournament (16) Brackets (8) March Madness Predictions (3)
2023 NCAA Tournament
2023 NCAA Tournament
2023 NCAA Tournament
College Basketball
Impeachment
2024 Republican Debates
2024 Election
2024 Election
© 2023 ABC News Internet Ventures. All rights reserved.