CardsCategoriesDaily setsGlossaryHistory

SLOs, error budgets and burn-rate alerting

Category: reliability and observability Difficulty: 4 Asked: 2026-10-31 · reliability-observability-01 · Reliability and observability · Edit

Tip: answer aloud for ~5 minutes before reading the solution.

Question

As CTO you inherit a platform with hundreds of noisy CPU/latency alerts and constant tension between product velocity and stability. How do you define reliability targets, alert on them, and use them to govern release decisions?

Step-by-step solution

  1. Pick user-centric SLIs per critical journey (availability = good requests / valid requests; latency = % under threshold at the load balancer or client).
  2. Set SLOs below 100% based on user need and history (e.g. 99.9% over 28 days); 100% is the wrong target.
  3. Error budget = 1 − SLO (99.9% → ~40 min/28d). Agree an error budget policy with product: budget exhausted → freeze risky launches, prioritize reliability work.
  4. Replace cause-based alerts with multi-window, multi-burn-rate alerts (e.g. page at 14.4x burn over 1h confirmed by 5m; ticket at lower burn rates over longer windows).
  5. Keep cause metrics as dashboards for debugging, not pages.
  6. Review SLOs quarterly; report budget consumption to execs.

Final answer

Define SLIs on user journeys, set realistic SLOs, derive error budgets and an executive-backed policy that ties budget burn to release velocity, and page only on multi-window burn-rate alerts against the SLO.

Trade-offs & common mistakes

Key terms

SLI, SLO, error budget, burn rate, error budget policy

References

Deeper learning

Video