Chaos Engineering

WHAT IT IS Chaos Engineering is experimentation on a system to build confidence in how it behaves under adverse conditions. A real experiment has four parts: a written hypothesis about steady-state behaviour, an injected condition, a bounded blast radius, and an abort condition. Anything without those is an outage you caused on purpose. **Not this:** Chaos Engineering is not Resilience Engineering: it is the narrower experimental practice of injecting specific faults to test a stated hypothesis, not the broader discipline of how systems and the people running them absorb surprise. WHY IT EXISTS Distributed systems fail in combinations nobody designed for, and the failure modes that matter are usually emergent rather than component-level. Design reviews and tests exercise the failures you imagined; the ones that cause outages are the ones you did not. Running the experiment converts an assumption — "we fail over cleanly" — into an observation, and it does so at a time you chose, with the right people awake. WHEN TO USE IT Adopt it when the system has redundancy whose behaviour has never been observed, when failover is documented but untested, or after an incident revealed an assumption that was wrong. Do not adopt it while known reliability work is outstanding: injecting faults into a system with unowned alerts and no incident process produces incidents, not learning. WHO OWNS IT The team that owns the service owns its experiments and their consequences — chaos run on someone else's system by a central team generates resentment and little learning. A reliability function may own the tooling, the standards and the review of results. Whoever is on call at the time must know it is happening; a surprise experiment is indistinguishable from an incident, and treating it as one wastes everybody's night. WHAT GOOD LOOKS LIKE Every experiment has a written hypothesis and a defined steady-state metric, so the result is a measurement rather than an impression. Blast radius starts small — one instance, one dependency, a fraction of traffic — and widens only after the small version behaves as predicted. An abort condition is defined in advance and the stop mechanism is tested before the experiment. Results are recorded whether or not they were surprising, including the boring ones. Findings turn into changes, which is the only reason to run the experiment. HOW TO IMPLEMENT Start in a non-production environment to prove the tooling and the abort path, then move to production, because a non-production system rarely has production's failure modes. Write the hypothesis down before injecting anything; the discipline of predicting the outcome is where most of the value is. Keep the first experiments trivially small. Announce them. Record the result and the resulting change. Build the practice on top of the alerting and ownership in /observability-engineering and the response process in /incidentops — chaos without those two amplifies confusion instead of producing knowledge. HOW TO ASSESS YOURSELF Ask when your failover last actually happened, and whether anyone watched. Ask whether your last experiment had a written hypothesis and an abort condition. Ask what changed as a result. Take the scored self-assessment at /assess and sequence the findings with /roadmaps.