CBCR2 Sam is a sophisticated statistical system for predicting outcomes of CFB games. CBCR2 Sam uses advanced statistics summarizing prior team performance, team-specific home-field advantage, team-specific away performance, climatic differences for away teams, field surface, and matchup-specific information to predict team performance. CBCR2 Sam is a perfect example of a projection model adhering to the R2 Sports Metrics Philosophy.
CBCR2 Sam uses machine learning methods that allow us to test hundreds of different model specifications and select the best amongst them by comparing how they perform on certain diagnostic statistics. Because CBCR2 Sam uses matchup-specific information, it would be inappropriate to use Roy to create an index of relative team strength along the lines of SP+, ESPN FPI, or our very own original CBCR2 Power Index.
Like the CBCR2 Power Index, CBCR2 Sam begins with a "preseason model" - a model using past performance, returning production, recruiting data, and other information available at the beginning of the season to predict performance over the course of the year. The role of the preseason ratings declines through the course of the year. However, unlike index-based models, Sam includes features that allows it to estimate the amount of weight that should be given to the preseason model natively - meaning that no human judgment used in this determination.
CBCR2 Sam also uses our own Simple Rating System (SRS) that we believe is superior to other simple rating systems available. SRS is essentially opponent-adjusted points. It uses a system-of-equations approach to solve for how many more points a team scores above an average team. For more information about SRS, see the resources at sports reference. Unlike the preseason model that should fade over time, SRS initially provides a very weak signal of team performance, but as the number of mutual competitors grows as the season advances, it becomes sharper. Thus Sam includes features that will natively scale up the emphasis of SRS variables over time as it becomes sharper.
In developing CBCR2 Sam, we tested the model on the 2021 season. In this test, the results reported are out-of-sample "projections." This means that only information that was known prior to kickoff was used to develop these estimates.
Summary Statistics from 2021 Testing:
Picks Against the Spread: 303-257 (54.1%)
Picks Straight-Up: 394-166 (70.3%)
Absolute Error: 12.9
Given that this post lands in the middle of the 2022 season, I'll also provide this season's performance to date:
Summary Statistics from 2022, weeks 2-8:
Picks Against the Spread: 169-128 (56.9%)
Picks Straight-Up: 225-79 (74%)
Absolute Error: 11.84
As you can see, CBCR2 Sam is performing at a very high level ATS. This performance currently leads the CFBD Computer Pick'em Challenge (https://predictions.collegefootballdata.com/leaderboard) in ATS picks (minimum picks = 200). it is performing significantly better than any model listed on thepredictiontracker.com, including popular models such as ESPN FPI and Sagarin.
Like R2 Roy, CBCR2 Sam performs better ATS the more confident it is -- though the data is admittedly a bit noisy, so far. It will be interesting to see whether these trends continue.
CBCR2 Sam ATS Performance by Divergence with the Vegas Point Spread
As noted in the R2 philosophy, the biased-but-efficient Vegas line will have an edge over the less-efficient/less-biased R2 system in picking winner's "straight-up." I do not recommend CBCR2 Sam for making moneyline picks or for use in your pick-em contest (unless it is a ATS pickem - in which case you should use CBCR2 Sam). If you participate in a confidence pick'em league, you should use R2 products to set the confidence levels, but I would not pick upsets unless attempting a hero call to overcome a deficit.
CBCR2 Sam does not currently project all games. Sam uses 1000s of variables and sometimes some information is missing. Rather than publish inferior projections, we omit certain games. We hope to correct this issue by the 2023 season.
Big thanks to the whole team that helped me build this system along the way. They are the @ChapelBellCurve team, Nathan Lawrence (@nathanjlawrence), Justin Bray (@TheJustinBray), Dr. Stephen Joiner (@StephenJoiner), Dr. Stephen Chaudoin (stephenchaudoin.com), and Ryan Moore (@ryanmre).