Skip to main content
Software Delivery & Engineering OpsLast reviewed 2026-09-20

QAOps

The quality function as an operating discipline: deciding what to test at which layer, running defect triage, analysing escaped defects, and setting the gates that a change must pass.

Overview

Read the full QAOps guidance
WHAT IT IS QAOps is the operating discipline of the quality function: deciding what is worth testing and at which layer, defining what a defect is and how it is triaged, analysing what escaped to production and why, and setting the gates a change passes before release. **Not this:** QAOps is not TestOps: it owns the strategy and the judgment about quality — what to test, what a defect means, what gates apply — not the infrastructure that executes tests. WHY IT EXISTS Test coverage is a measure of execution, not of judgment; a suite can be large, fast and green while systematically failing to exercise the thing that breaks. Somebody has to decide where quality effort goes, and in the absence of that decision effort follows whatever is easy to automate. The other reason is feedback: escaped defects are the highest-quality information an organisation gets about its own testing, and most organisations discard it. WHEN TO USE IT Adopt it when defects reach production in a category nobody was looking at, when triage depends on who picks the ticket up, or when "is this ready?" is answered by feel. The need grows with the number of teams, because consistency of judgment is what erodes first. WHO OWNS IT A quality function or quality-focused engineers own the strategy, the defect taxonomy and the escaped-defect analysis. Delivery teams own the quality of their own work — quality assured by a separate group at the end is the anti-pattern this discipline replaced. The product owner owns the acceptance criteria and the risk appetite that gates encode. WHAT GOOD LOOKS LIKE There is a written view of what is tested at each layer and why, and it is revisited rather than inherited. Defects have a severity definition that two people apply consistently. Every escaped defect is examined for the gap that let it through, and that analysis changes something. Release gates are explicit and proportionate to risk, and exceptions are recorded rather than negotiated silently. Quality signals from production — errors, support contacts, user reports — feed back into where testing effort goes. HOW TO IMPLEMENT Start with escaped-defect analysis on the last quarter, because it tells you where the real gaps are rather than where you imagine them. Write the defect severity definition and test it by having two people triage the same ten issues. Map what is tested at which layer and find the categories nobody covers. Define the gates, keep them proportionate, and make exceptions visible. Work with /testops on making the resulting suite fast and reliable enough to run, and connect production signals through /observability-engineering. HOW TO ASSESS YOURSELF Take the last five production defects and ask, for each, which layer should have caught it. Ask whether two people triage the same bug the same way. Ask who decides a release is ready and on what basis. Take the scored self-assessment at /assess, then plan with /roadmaps.

Capabilities

  • Defect triage

    A severity definition specific enough that two people classify the same issue the same way.

  • Escaped defect analysis

    Examining every production defect for the gap that let it through, and changing something as a result.

  • Layered test strategy

    A written, revisited view of what is tested at which layer and why, rather than coverage following whatever is easy to automate.

  • Production quality feedback

    Routing errors, support contacts and user reports back into where testing effort is spent.

  • Release quality gates

    Explicit, risk-proportionate conditions a change must meet, with exceptions recorded rather than negotiated silently.

Practices

  • Escaped defect review

    A regular session examining what reached production and which layer should have caught it.

  • Triage calibration

    Having two people triage the same issues periodically to check the severity definition still means one thing.

Metrics

  • Change Failure Rate

    Share of deployments causing degraded service requiring remediation.

    Unit: %. Good direction: down.

  • Escaped defect rate

    Defects found in production per release, the direct measure of what testing did not catch.

    Unit: defects per release. Good direction: down.

  • Triage agreement rate

    Share of issues two people independently assign the same severity, testing whether the definition means anything.

    Unit: percent. Good direction: up.

  • Triage latency

    Time from a defect being reported to it being classified and assigned.

    Unit: hours. Good direction: down.

Tools

  • Allure Report

    Test reporting that aggregates history, making flake and failure patterns visible across runs.

    Visit site ↗
  • Jira

    Issue tracker commonly used for defect triage, severity workflow and escaped-defect analysis.

    Visit site ↗
  • Playwright

    Browser automation framework with tracing and retry controls useful for diagnosing and containing flakiness.

    Visit site ↗

Architecture patterns

Reference architectures and their trade-offs live in the blueprints library.

Maturity

  1. L1 ReactiveQuality is whatever the last person checkedrequired stage

    There is no written view of what is tested where, and readiness is judged by feel.

  2. L2 RepeatableDefect severity is defined and applied consistentlyrequired stage

    Two people classify the same issue the same way, which makes every downstream number mean something.

  3. L3 StandardizedTesting strategy per layer is written and revisited

    Effort follows a decision about risk rather than whatever was easiest to automate.

  4. L4 MeasuredEvery escaped defect is analysed for the gap

    Production defects are treated as the highest-quality feedback available and change where effort goes.

  5. L5 OptimizedGates are explicit, proportionate and rarely bypassed

    Release conditions match risk, exceptions are recorded, and production signals feed the strategy continuously.

Maturity for qaops is measured, not guessed — every score traces to your answers. See how maturity is scored.

Put it to work

Sources