Hello and welcome to a new series I am working on that I am calling Board Game Breakdown. The idea is simple, try to use data science to better understand the strategy and mechanics that underlie some of our favorite board games (similar to the type of posts I have been creating with regards to fantasy football. For those of you solely focused on that, do not worry, I still plan to write more posts on the topic).
The first board game I want to tackle is the big one. Settlers of Catan. Within this series, I plan to analyze the game of Catan through a number of different angles, such as resource value, placement strategies, dev card values, and more, but today I want to define some basics. How long is a game on average? How are the resources distributed? Is there an advantage in placement order? Among others. I plan to do deeper dives into many of these topics in future posts, but we will lay the groundwork today.
Finally, all of the data I am present data was collected from games I played on colonist.io. I am actively looking for more people to contribute to this dataset so if you want to add your game data, you can do so (and see some stats about your own game) at https://settlersofcatan.anvil.app/
The Data
The data comes from 251 four-player games of the base game of Settlers of Catan. All games were to 10 points and used the base map with random tile and number placement along with random dice rolls. Some of the games included bots as opponents (which is something I plan to explore in detail in a later post) but all the games are included in today's post to increase the sample size as we define our Catan basics.
How Long is a Game of Catan?
Violin plot showing the distribution of the number of turns in a game of Catan. The thick inner bar represents the 25th to 75th quartiles while the white dot and red line mark the median number of turns (71).
To start off lets we can see that a game of Catan consists of 71 turns on average. That means the dice will be rolled 71 times to produce the resources you need to win. That means you have a total of 17 to 18 turns to build what you need to win the game.
How do you win?
In this great post by David at boardgameanalysis.com, he details the 143 different combinations of point generating assets that can achieve the 10 points needed to win. So what were the most common ways in my dataset?
Number of Settlements | Number of Cities | Number of Dev Card VPs | Longest Road? | Largest Army? | Number of Games won (% of dataset) |
1 | 4 | 1 | No | No | 19 (7.5%) |
2 | 3 | 0 | Yes | No | 16 (6.3%) |
2 | 3 | 2 | No | No | 15 (6.0%) |
2 | 4 | 0 | No | No | 15 (6.0%) |
1 | 3 | 1 | No | Yes | 14 (5.6%) |
The biggest thing that stands out is the importance of cities. All of the top 5 methods of winning include at least 3 cities. looking beyond the top 5, 9 of the top 11 combinations include at least 3 cities, with number 12 being the first instance of only 1 city. In fact, only 2 games were won without any cities. Also interesting, 92 games (36.6%) were won with the longest road while 69 games (27.5%) were won with the largest army (Note these are not conditional probabilities, but rather just descriptive. We will look at conditional probabilities involved with these, ie what are your odds of winning if you have the longest road, in a future post)
Placement Order
The first thing you do is place 2 settlements, which obviously has an impact on how the rest of the game goes. Unfortunately, I do not have a way to figure out where anyone builds settlements yet from the colonist.io data (if you have ideas on how to do this, please contact me!). But one thing we can look at is placement order. When on average do you want to place your settlements?
Pie chart showing the percent of games won by the person who placed their first settlement first, second, third, and fourth.
If we assume that everyone should start with even odds at 25%, we can see a slight advantage with 3rd and 4th while a slight disadvantage from 1st and 2nd. My guess is that this is largely due to being able to coordinate your 2 settlements better, such as grabbing a great source of one resource with one settlement, and that resource's port with the second. (Note: I have a feeling that the impact of bots and player skill is a significant limiting factor, so take this result with a grain of salt and expect a future post to dive into this more).
Resources
Obviously, resource acquisition and management is the point of Settlers of Catan. So how many resources does it take to win?
Distribution of the total resources gained by the average player (blue) vs the winning player (orange) by the end of the game.
We can see that the winner gets more resources normally (unsurprisingly). I think it is interesting that the distribution is narrower for the winning player, though that is likely a consequence of the game ending when they get to 10 points (capping both the bottom (they need enough to produce 10 points) and the top (eventually you have to win). To put some numbers to it, an average player received 64 resources (with a standard deviation of 17), whereas the winning player received 76 resources (with a standard deviation of 14), so 12 resources on average are the difference between winning and average.
We can see that as well if we look at the resource total vs the number of points built by a player.
Box plots of the total number of resources gained (x-axis) vs the total number of points a player earned (y-axis)
I find this graph both obvious and fascinating. First, the obvious, as you get more resources, your point total increases (the graph moves up and to the right). That makes sense, more resources lead to better outcomes.
But fascinating to me are the clumps at the top and bottom. A player that finishes with 2, 3, or 4 points all likely had the same amount of resources. Even more interesting is that a player with 8, 9, or 10 points likely all had the same amount of resources. That really shows how close some of these games are and mirrors the feeling that the end of games often comes down to luck between a player or two. (now getting to 11 points takes more resources than 8,9,10.
This also demonstrates how it's not just the total number of resources that matters, but which resources you get, so let's break that down.
Bar plot of the total amount of each resource gained by the average player (blue) vs the winning player (green). Error bars represent the 95% confidence intervals.
As expected, the winning player receives more resources than the average losing player. I think the stark difference comes in ore, where the winning player has almost twice as much ore on average. I think this along with the increase in grain really speaks to the importance of cities as we noted earlier to building a winning collection of assets. I also find the comparison on wool (sheep) interesting, where winning players actually get less sheep than losing players on average. This breaks my heart as I personally love to win as a sheep lord.
Limitations
There are plenty of limitations with this analysis. First, it's only 251 games, which is a relatively small sample size. On top of that, a lot of games have bots in them which may skew any results. That said, a lot of the results align with our intuition and give me the confidence to pursue future posts. Also important is that these games all used the base map. Other maps will lead to different strategies and should be considered.
Also to address a potential limitation before it is mentioned, a lot of people feel that the dice on colonist.io are unbalanced and while each individual game will have some randomness, when we look across games we get a near-perfect distribution of rolls. (shown on the right) Additionally, if we test each game for how well it aligns to the ideal (using a chi-squared test), we find 14 games with a p-value less than 0.05 (the standard line for statistical significance), suggesting that 14 games differed significantly from the ideal. Except that what we mean by statistical significance is that there is a less than 5% chance of seeing this randomly. Stated differently that if we repeated this a large number of times, we would expect 5% of them to significantly differ from the expected. So given 251 repetitions, we would expect 5% to be significant. 5% of 251 is 12.55, which is pretty darn close to the 14 we see. given that, I would say that you could not ask for a better representation of dice rolls. (Note: not accounting for the number of times you test something is a common form of p-hacking, which is often thrown out as a problem in science right now.)
Conclusions
I think the biggest (even if it is obvious) conclusion is the importance of gathering as many resources as you can, with a specific interest in ore. Cities are important clearly important, and if you have a choice, more often than not it is better to place third or fourth.
Want to contribute your data to future posts? You can do so at colonist_data (anvil.app)
Questions? Comments? Let me know at ac@alexcates.com. Want to read more breakdowns like this? sign up for my newsletter here. Finally, like what I do? Consider supporting me on buy me a coffee.
Comments