Checklists / data-pipeline-production-readiness
Data Pipeline Production Readiness Checklist
Before a pipeline becomes something other teams depend on: ownership, freshness expectations, quality gates, and a known answer for when it breaks.
9 steps · 8 required · last reviewed (next review 2027-03-20) · updated 2026-09-20
Related focus areas:DataOps
Run this before a pipeline moves from "someone's job" to a dependency. The failure mode being defended against is silence — a pipeline that half-works produces a dashboard that is merely wrong, and the business acts on it for a week.
Work the checklist against the pipeline as it actually runs, not as the design document describes it.
The pipeline has a named owner who knows they own it
requiredOwnership recorded somewhere nobody reads is not ownership; the test is whether the person would answer a page about it.
Each output dataset states a freshness expectation
required"How current should this be?" must have a number before anything can detect that it is stale.
An automated check compares actual freshness against that expectation
requiredFreshness reported by a confused analyst is the most expensive monitoring you can buy.
Data quality assertions run on every load and can block publication
requiredA quality test that annotates bad data rather than stopping it just documents the incident you are about to have.
Failures alert the owner with enough context to act
requiredAn alert that says only "job failed" costs the responder the first twenty minutes of every incident.
Schema expectations for upstream inputs are written and agreed with the producer
requiredUpstream schema change is the single most common cause of downstream breakage, and it is preventable by agreement.
Backfill and replay are documented and have been performed at least once
requiredThe time to discover that reprocessing double-counts is not during the incident that requires it.
Pipeline code is in version control and changes are reviewed
requiredOrchestration layered on unreviewed logic simply fails more efficiently.
Lineage is recorded so downstream impact can be assessed before a change
optionalImpact analysis after the fact is called an incident review.
Run it against your real gaps
A checklist is most useful when you know which controls you are actually missing. A free OpsRoadmaps assessment scores your controls with explainable findings and builds a roadmap from the results.
Assess my stackChecklists document self-reported practice. Items were last reviewed 2026-09-20 — adapt them to your own environment.