How Do You Keep Runbooks up to Date After an Incident?
At 2:30 a.m. on a Tuesday, an alert fires for a service nobody has touched in six months. The on-call engineer opens the runbook, finds a kubectl command, pastes it into the terminal, and hits enter. The command fails. The cluster configuration changed three weeks ago, but the runbook still references the old namespace. The engineer spends the next forty minutes reverse-engineering the new architecture while the service remains degraded.
This is how runbooks die. Not because teams refuse to write them. They die because teams update them right after an incident (when memory is fresh and guilt is high) and then never touch them again. Six months later, the runbook references deprecated services, outdated commands, or people who left the company. The next on-call engineer finds instructions that don't work and loses trust in the entire library.
The answer to keeping runbooks current isn't a better calendar reminder. It's tying runbook updates to events that already happen: deployments, config changes, and the postmortems that follow incidents. When runbook validation is cheap enough to do during normal work, it survives contact with reality. When it requires scheduling a separate meeting, it doesn't.
Why the Scheduled Review Doesn't Work
Most post-incident processes assume documentation happens once. They tell you to "schedule regular reviews" or "assign an owner." Those tasks compete with roadmap work, and roadmap work always wins.
Documentation technical debt is a specific, virulent strain of the broader problem. A qualitative study of a large software development organization found 35 distinct causes of documentation debt, with organizational culture and competing priorities at the top of the list. The researchers identified a consistent pattern: well-defined documentation processes existed on paper, but team members' day-to-day pressures meant those processes were rarely followed. The runbook that nobody updates is not a failure of intention. It's a predictable outcome of a system that treats documentation as optional.
When a runbook is outdated, it doesn't just fail to help. It actively misleads the responder, increasing MTTR and adding cognitive load to an already stressful situation. The Google SRE book's chapter on postmortem culture describes a culture where "an unreviewed postmortem might as well never have existed." The same logic applies to a runbook that hasn't been validated since the system it describes was last changed.
You need a system that ties runbook updates to existing forcing functions. Runbook updates survive when they're required to close an incident ticket, part of the deploy checklist for risky changes, or embedded in the postmortem template as a mandatory field.
Small Friction at Decision Points Beats Big Planning Sessions
Lightweight validation beats scheduled review. Instead of a massive quarterly audit (which no one does), build quick checks into existing workflows.
When someone deploys a change that touches alerting logic, the PR template should ask: "Does this affect any runbooks? If yes, link to the update or mark N/A." This is the same principle behind effective pull request templates: they reduce ambiguity and speed reviews by standardizing what authors explain at the moment of change. The Financial Times engineering team ran a project called RUNBOOK.md that embedded runbook authoring directly into the same repository as a system's source code, coupling runbook changes to code changes so that a pull request that fundamentally changes an application's architecture would naturally prompt a runbook update. The insight was that editing a runbook in a separate CMS, detached from the process of making code changes, relied on engineers remembering to do it. Coupling the two removed that dependency.
Similarly, when an on-call shift ends, the handoff should ask: "Did you use any runbooks this week? Did they work?" Effective on-call handoffs prevent dropped context and missed incidents during shift transitions. The Google SRE Workbook describes on-call handoff emails as a standard practice, where the outgoing engineer documents what happened during their shift. Adding a single question about runbook quality to that handoff costs almost nothing and surfaces failures before the next incident.
If a runbook failed, the outgoing engineer doesn't have to rewrite it immediately. They have to flag it. That flag is what keeps the problem from being invisible.
Some staleness is also detectable without human intervention. If a runbook references a kubectl command and your cluster config changed, a script can flag it. If it links to a wiki page that 404s, that's automatable. If it mentions an API endpoint and your OpenAPI spec changed, you can surface a warning. Automated policy checks are essential for scaling operations without sacrificing quality. Not every runbook can be programmatically validated, but the mechanical failures — broken links, outdated commands, deprecated tools — can be caught before a human needs them at 2 AM.
The Postmortem Is the Best Documentation Session You're Not Using
The best time to update a runbook is during the postmortem, when you know exactly what failed and what should have been documented.
Build it into the template. The Atlassian postmortem framework asks teams to document corrective actions with named owners and deadlines. Add two more questions: "What runbook should have existed?" and "If one existed, what was wrong with it?" An ACM Queue piece on SRE documentation describes a recognizable scenario: an on-call engineer discovers a runbook that references an old version of a tool, and the postmortem discussion reveals that half the team didn't know a new version existed. The runbook update that would have prevented this is obvious in retrospect. It just never got assigned.
Assign the runbook update as an action item with the same weight as a code fix. Post-mortem action items die for four predictable reasons: no named owner, wrong tracking tool, vague wording, and no follow-up cadence — a pattern well-documented in incident response research. "Improve documentation" is a wish. "Alex: update the runbook for auth service failover procedure by next Friday" is an action item. The difference is that one can be verified as done and the other cannot.
If the team culture treats runbook debt like technical debt, it gets prioritized. If it's treated as a chore, it gets ignored. The Google SRE postmortem culture guidance is explicit that postmortems should identify "effective preventive actions" and that action items should be tracked at appropriate priority. A runbook update is a preventive action. It belongs in the same tracker as the code fix.
What This Looks Like in Practice
The teams that maintain runbooks well share a common structure. They don't have a dedicated runbook maintainer. They have a set of lightweight checkpoints that distribute the work across the people who are already doing incidents and deployments.
None of these require a separate meeting. None of them require a dedicated headcount. They require the discipline to add four questions to documents your team already fills out.
The Writing Problem
After an incident, engineers know what happened. Writing it down is the hard part, and it happens when they're burned out and already behind on the work that piled up while the incident was running.
If a system can generate a runbook draft from incident logs, Slack transcripts, and the commands that actually worked, the engineer's job becomes validation and editing rather than writing from scratch. That's faster, and it's more likely to happen before momentum dies. Research on post-mortem reconstruction found that manual reconstruction alone wastes 60 to 90 minutes per incident as teams review Slack threads, monitoring data, and call recordings to rebuild the timeline. The same problem applies to runbook creation: the raw material exists, but assembling it is tedious.
This is where Doc Holiday fits into the picture. It connects to your code commits, Slack, and engineering workflows, and generates documentation drafts from that material directly. The engineer's job becomes reviewing and validating the output, not producing it from scratch. For lean SRE teams that can't afford a documentation org, that's the difference between a runbook that gets written and one that gets deferred until the next incident makes it urgent again.
