What Poker Can Teach Us About Post-Mortems

by Cory Chamblin

In this talk titled "What Poker Can Teach Us About Post-Mortems," Cory Chamblin presents insights on how principles from poker can improve the effectiveness of post-mortem analysis in software development. Chamblin begins by emphasizing the prevalence of reactive and unsatisfying post-mortems that often unfairly assess decision-making based on negative outcomes.

Key Points Discussed:
- Understanding Poker: Chamblin provides a primer on poker, specifically focusing on No Limit Hold'em, where players make strategic decisions based on hidden information and the actions of others at the table.
- Decision-Making in Poker: He explains essential poker strategies that revolve around betting, understanding risk, and making informed decisions based on positions and odds.
- Learning Through Losses: A significant takeaway from poker is that making the right decisions does not always lead to success. Chamblin parallels this to incident post-mortems, stressing that just because an outcome was negative does not mean the decisions leading to it were poor.
- Critique of Conventional Post-Mortems: The talk critiques standard post-mortem practices, which often focus too much on outcomes rather than the decision-making process. Chamblin encourages teams to ask intentional questions that promote valuable insights rather than reinforcing a blame narrative.
- Practical Applications: He discusses practical strategies for conducting more effective post-mortems, urging teams to recognize accepted risks and learn from the outcomes without falling into the trap of results-oriented thinking.
- Long-term Thinking and Resilience: Chamblin advocates for resilience in decision-making processes, suggesting that like in poker, where players manage bankrolls and risks, teams should also prepare for potential setbacks in their projects.

In conclusion, Chamblin empowers attendees to draw parallels between poker and real-life decision-making, advocating for refined approaches to post-mortems that foster learning and continuous improvement. By adopting a poker mindset, teams can mitigate risk aversion and recognize the skill involved in making sound decisions, regardless of outcomes. The talk serves as a call to rethink failure and learning in both poker and life, encouraging attendees to better manage risks and support sound strategies moving forward.

00:00:17.060 Hello everyone, my name is Cory Chamblin. I'm a software engineer at PagerDuty. Today, I'm going to talk to you about poker, post-mortems, and life. How many of you here have ever played poker before? Okay, most of you have played. Hopefully, this talk will be helpful for you, and it's also catered to those who have never played before.

00:00:35.640 I am not a professional poker player; I work in software. However, I have taken a few courses, read some books, and played a lot of poker—it's a serious hobby for me. I am, however, an expert on post-mortems, having attended and caused quite a number of them in my travels.

00:00:43.829 My conclusion is that post-mortems would be a lot better if they were run and attended by people who understood poker. Today, I'm going to help you become those people. I submit that most post-mortems, as they exist today, make us risk-averse, and that learning about poker can help us mitigate that.

00:01:06.840 First, we're going to talk about poker and the rules of the game, as well as basic strategy—this is why most of you are probably here. Next, we're going to discuss post-mortems and how we can mess them up in unintuitive ways because we're not thinking like a poker player might. Finally, we'll explore other areas of life where thinking like a poker player can be beneficial.

00:01:20.159 By the end of this talk, I hope you'll have a few ideas to take with you to your next poker game and some insights you can apply to your next post-mortem. So, poker. Poker is a game of numerous variants, typically played with a standard 52-card deck and often involves some number of hidden cards and rounds of betting. The goal is to determine which player has the best hand. Players may call bets or fold, and when the betting is all done, the player remaining in the hand with the strongest five-card hand wins all the money in the pot.

00:02:02.850 Here are the hand ranks in poker, from the weakest to the best: a high card, one pair, two pairs, three of a kind, a straight (which is five cards in sequential order), a flush (which is five cards of the same suit), a full house (a pair plus three of a kind), a four of a kind, a straight flush, and finally, the best possible hand, which is the royal flush (10, J, Q, K, A of the same suit). You're guaranteed to win if you have a royal flush.

00:02:32.030 Within the framework of these rules, many different variants of poker are played. For the purpose of this talk, we're going to focus on just one—No Limit Hold'em, which is the most popular variant of poker today. It is often referred to as the Cadillac of Poker; if you buy a beginner book on poker, take a course, or watch poker on TV, odds are you'll see No Limit Hold'em.

00:03:10.760 No Limit Hold'em is played with a round of blind bets. Players called the big blind and the small blind put in bets dictated by the table before they receive their cards. These bets are made before each hand to encourage action at the table, so players don't just wait for the best cards to play. The blinds along with the betting order rotate around the table each hand.

00:03:38.480 Next, everyone receives two cards, known as their hole cards. The player to the left of the big blind starts the first betting round, followed by subsequent betting rounds after community cards are revealed. The first community cards, called the flop, consist of three cards dealt face-up on the table. Following the flop, another round of betting occurs. Then the turn is dealt, which introduces another community card, followed by another round of betting. Finally, the river is dealt with yet another community card, and the last betting round occurs before a possible showdown.

00:04:38.900 In most hands, one player is usually able to convince the others they have a stronger hand, causing them to fold before the showdown occurs. The mechanics of poker are simple, making it possible to learn the rules quickly—usually within about ten minutes. The real choices you make in poker are to call, raise, or fold when it's your turn. However, these choices can have profound implications over time.

00:05:56.240 What's the skill in poker? The skill of poker lies in making better bets. A player who consistently makes better bets than their opponents will ultimately win their opponents' money over time. There are several strategies to improve the quality of your bets: most beginners play far too many hands. The goal in poker is not to win the most hands but to win the most money. You should play hands that play well post-flop.

00:06:58.320 A good player will fold around 80% of the time before they commit any money to the pot. As you improve, you can learn to outplay your opponents with a looser but still aggressive style of play, but you'll likely still be playing far fewer hands than most beginning players. At the end, I'll mention a few courses and books that provide recommended hand ranges for opening hands.

00:08:07.710 It's important to remember that most beginning players often like to call too much. They take weak hands and call instead of raising with stronger hands. When you have a narrow range of quality hands, you usually want to raise against their wider range of weak hands because they're likely to call.

00:08:28.830 For example, many beginners like to slow-play a very strong hand pre-flop, like aces or kings. Instead, you want to raise until you have exactly one opponent left. Your strong hand has great odds against individual hands, but when you face multiple opponents, your odds of winning decrease significantly. Another critical aspect of poker is understanding your position in the betting order.

00:09:05.420 The players who bet last have the most information. Therefore, you should play fewer hands in earlier positions and be more aggressive in later positions. It's hard to overstate the power of the advantage gained from acting last. Consider a scenario where you hold a hand like 8-7 suited; it performs very well in a pot with multiple callers but poorly against a single opponent.

00:09:35.130 If you're the first player to act before the flop, you might call the big blind with this hand. But if a player in a later position raises significantly, it's likely you should fold. In live poker, playing in position by acting after your opponents gives you critical insight into their likely actions, based on their earlier decisions.

00:10:58.300 This understanding leads to better bets and improves the expected value of your decisions. Expected value is the sum of all possible outcomes of a decision, weighted by their individual likelihoods. For example, if you flip a coin, you can calculate your expected value based on the potential payout for heads versus the loss for tails.

00:12:03.900 In poker, you can apply similar calculations. If you believe your opponent has a strong hand, you need to weigh your chances of winning versus the amount you're betting and the odds offered by the pot. Understanding your odds and calculating whether you are making a profitable bet is essential for long-term success.

00:13:09.690 It's important to differentiate between implied odds and fold equity. Implied odds consider how much more you can win after hitting your hand. On the other hand, fold equity estimates the percentage of the time your opponent may fold a better hand if you raise. By calculating all these factors, you can make informed decisions in your poker game.

00:14:20.170 Notably, the outcomes of particular hands aren’t the critical element in evaluating decision-making. If you make good decisions over time, the results will usually take care of themselves. This philosophy aligns with the way poker teaches us about life; sometimes, you can do everything right and still lose.

00:15:05.240 Now, let’s discuss how this applies to post-mortems. In incident response documentation, a post-mortem is a thorough, blame-free description of what went wrong in an incident. It typically involves root-cause analysis, follow-up tickets, and developing communications for stakeholders. However, I’ve found that many post-mortems are trapped in the mindset of assuming that if we just prepared harder or followed the right processes, we could’ve prevented the incident.

00:16:40.760 This dogmatic faith in prevention can lead to shortsighted conclusions. If we analyze poker play this way, we wouldn't be able to continue playing poker for long. In the software world, it might seem easier, given the apparent safety nets like venture capital or waiting for others to catch up. However, this neglects a painful truth—sometimes outages occur despite sound choices.

00:17:32.400 For example, consider the coin flip scenario where it's statistically profitable to play. If you play and lose, the mindset persuades you to believe you made a bad decision, while in reality, this is a misinterpretation. Results-oriented thinking stems from an illusion of control, confusing correlation with causation. We’re often tempted to judge the quality of decisions by their outcomes instead of considering the context available at the time.

00:18:41.600 This leads to an oversimplified view of risk taking. While historic thinking may seem reasonable in some situations, the stakes are considerably lower in modern enterprise, where calculated risks lead to real advancement. An excellent example from pop culture is from Star Trek: The Next Generation. In one episode, an android named Data experiences a narrative where he struggles after an unexpected defeat in a strategy game.

00:19:51.570 Data questions his performance since he saw no mistakes yet still lost. The captain reminds him that it’s possible to commit no mistakes and still lose, which reflects how we should view decisions during post-mortems. We need to resist results-oriented thinking that presumes a direct link between choices we make and their outcomes.

00:20:52.30 At PagerDuty, we once had an elaborate monitoring system that ultimately generated false alarms rather than meaningful data. When a related outage occurred, the initial post-mortem suggested the system's decommissioning was a significant mistake. Yet the system, plagued with operational issues, led to a miscalculation regarding the balance between the risk of outages and the cost of maintenance.

00:21:35.330 Results-oriented thinking can mislead teams into making reactive decisions that overlook the true context of past decisions. To conduct effective post-mortems, teams should ask intentional questions to maximize learning potential and mitigate the probability of misinterpretation.

00:22:26.400 Should we have prevented this? Many post-mortems presuppose that all problems are worth preventing, which introduces risk aversion. Instead, if you recognize reasonable risks were accounted for when making a decision, you should feel assured about your choices.

00:22:52.160 Asking whether you should try to prevent this in the future also holds significance. After learning from past events, decisions must adapt to reflect new knowledge, and different stakeholders will expect changes in processes. Teams need to balance the potential risks of reoccurrences with the priority of addressing new incidents effectively.

00:23:37.990 Another critical question concerns whether risks were previously understood or if they emerged. If a risk was known and accepted, the blame narrative shifts. It's imperative to assess whether there were reasonable due diligence efforts to recognize lurking risks.

00:24:20.100 Finally, consider how your decision-making should evolve going forward. Sometimes there isn’t a new lesson to learn, but instead, an acknowledgment that you can make sound decisions and still run the risk of loss. Identifying overlooked dependencies and weighing the aggregation of risks can adjust future strategies.

00:25:05.200 Learning from poker can reframe our understanding of post-mortems. The game teaches us about tilt, constructive failure management, and long-term thinking. Unlike video games where you get reset points, poker builds continuously, where each hand contributes to overarching success. Avoiding catastrophic risks allows you to keep playing through downswings and variance.

00:26:23.520 In the classic movie Rounders, the main character struggles with bankroll management, ultimately losing everything in a single hand due to poor risk management. A general rule of thumb is to maintain at least twenty buy-ins for the stakes you play. In life, this reflects preparing against significant risks by having backup strategies.

00:27:37.150 Whether through insurance or strategic planning, ensuring room to absorb shocks while continuing forward is truly crucial. In conclusion, I hope you feel empowered to apply these tools and insights to your work, your poker games, and your life. We’ve discussed poker, its strategies, and how to conduct post-mortems without falling into the pitfalls of results-oriented thinking.

00:28:57.140 To further your poker education, I recommend exploring resources such as Alton Hardin's online poker school, and various classic poker books, as well as engaging in the poker community for shared wisdom.

00:29:26.650 I’d like to thank my employer, PagerDuty, for supporting me, and I encourage anyone interested in joining our team to apply. All the slides from this talk, as well as resources, are available at chamberlain.net/rubyconf18. You can find me on Twitter @chamberlain or read my blog where I mainly write about programming and technology.