Our experience is our best resource when it comes to incident prevention, so why are we so resistant to learning from our past mistakes?
More like this
In engineering, incident is not just a scare word, it’s a reality we live daily. Given the often costly nature of incidents, it makes sense to invest some time and effort in their prevention. The recipe is simple: Do a blameless analysis of the root cause and identify actions to address it. Yet, the execution is often sloppy, and as a result, our learnings turn out to be surprisingly shallow.
Treated as bureaucratic exercises, postmortems often become checkbox tasks once the dust settles. Many of them lead to superficial conclusions, finger-pointing, or even acts of virtue signaling, where individuals confess to errors to project accountability. Human error is often put front and center, diverting attention from weaknesses in the system, sending us down that spiral of repeating the same mistakes again and again — those same mistakes we identified as the root cause of the incident.
This behavior is not a novel phenomenon, and identifying it is not an exciting discovery. It’s a cliché, a commonplace of our guild, but this makes the fact that we keep doing it even more intriguing. In this talk, we’ll dive into a rabbit hole of hopefully relatable anecdotes from our engineering experience and identify some (anti-)patterns in how we handled incidents. Along the way, we’ll try to offer some fresh perspective without being too judgy (postmortems are supposed to be blameless after all!), and by the end, we’ll hopefully find our way out with a clearer understanding of how slightly adjusting to from a reporting to a learning mindset can help us move beyond annoying paperwork — from surface-level problem-solving to addressing the heart of recurring issues.
Key takeaways
- Postmortems are learning tools, not just reporting tools.
- Shift focus from human error to addressable weaknesses in systems and processes.
- As an engineering leader, it is your responsibility to ensure your teams live by the above principles.
