Skip to main content

Checklists / chaos-experiment-safety

Chaos Experiment Safety Checklist

The four things that separate an experiment from a self-inflicted outage: a written hypothesis, a bounded blast radius, a tested abort, and responders who know it is happening.

8 steps · 7 required · last reviewed (next review 2027-03-20) · updated 2026-09-20

Related focus areas:Chaos Engineering

Roadmaps covering the same ground:Incident Management MaturityResilience and DR Improvement

Run this before every experiment, including the small ones and especially the ones you have run before. An experiment without these four properties is an outage you caused on purpose, and the difference is entirely in the preparation.

If the system has known unowned alerts or no incident process, stop: injecting faults there produces incidents rather than learning.

  1. A hypothesis about steady-state behaviour is written down before injecting anything

    required

    Predicting the outcome first is where most of the value is; without it you get an anecdote.

  2. Steady state is defined as a measurable signal, not a feeling

    required

    The experiment must be able to conclude something, which requires a number to compare against.

  3. Blast radius is bounded to the smallest useful scope

    required

    One instance or a traffic fraction first; widen only after the small version behaves as predicted.

  4. An abort condition is defined and the stop mechanism has been tested

    required

    Discovering that the stop button does not work is the worst possible finding to make mid-experiment.

  5. The people on call know the experiment is running and when

    required

    A surprise experiment is indistinguishable from an incident and wastes a night proving that.

  6. The experiment runs at a time chosen for recovery capability, not convenience

    required

    The point of choosing the moment is having the right people available and awake.

  7. The result will be recorded whether or not it is surprising

    required

    Unrecorded boring results are how the same experiment gets run three times by three teams.

  8. A follow-up owner is named for any finding before the experiment starts

    optional

    Findings without an owner become a list, and the list is the reason the practice gets cancelled.

Run it against your real gaps

A checklist is most useful when you know which controls you are actually missing. A free OpsRoadmaps assessment scores your controls with explainable findings and builds a roadmap from the results.

Assess my stack

Checklists document self-reported practice. Items were last reviewed 2026-09-20 — adapt them to your own environment.