Skip to main content
OpsRoadmaps

Checklists / llm-production

LLM Production Launch Checklist

Ship an LLM feature to production with known failure modes: evals, grounding, cost caps, privacy, and human fallback.

10 steps · 10 required · last reviewed (next review 2027-03-19) · updated 2026-09-21

Related focus areas:LLMOpsMLOpsSRE

Roadmaps covering the same ground:Incident Management MaturityObservability MaturityPlatform Engineering AdoptionProduction-Ready KubernetesResilience and DR ImprovementSRE Adoption

Sign in to run this checklist

LLM features fail differently from software: outputs are stochastic, failures are soft (plausible-but-wrong), and cost scales with usage in ways infrastructure does not. This checklist is the pre-production gate that turns those unknowns into managed risks.

The non-negotiable items: an eval suite that runs in CI, and a human fallback path when the model is wrong. Everything else manages cost and reputation.

  1. Evaluation suite exists with golden examples, and runs in CI on every prompt/model change

    required

    Without evals, prompt tweaks are vibes; a regression in accuracy ships silently and is discovered by users.

  2. Prompts, model versions, and parameters are versioned like code

    required

    Model behavior changes when ANY of these change; unversioned prompts are undebuggable by construction.

  3. Outputs are grounded (RAG over approved content) or marked advisory where they can't be

    required

    Ungrounded generation is confident fabrication; grounding scope decides what the system is even allowed to claim.

  4. Human fallback / override path exists for when the model is wrong or unavailable

    required

    Degraded LLM output must never trap the user; the escape hatch is the difference between an annoyance and an incident.

  5. Cost controls: per-request caps, usage limits, alerts on spend anomalies

    required

    Token costs scale with success — your viral feature is also your bill; caps turn a spike into an error instead of a bankruptcy.

  6. PII/sensitive data policy is enforced on inputs AND outputs (no training on user data, retention defined)

    required

    User data flowing to a third-party model is a data-processing event; treat it with the same care as a database export.

  7. Prompt-injection exposure is reviewed (user input never concatenated into instructions)

    required

    Instructions built from user text let users rewrite your product; treat model-visible input as data, never as code.

  8. Latency and timeout behavior is defined (streaming? fallback answer on timeout?)

    required

    A 30-second generation with no timeout is a hung request that eats your connection pool.

  9. Monitoring captures quality signals (thumbs, escalations, refusal rates), not just uptime

    required

    A 200-response with wrong answers is a silent failure; only human feedback closes that loop.

  10. Scope honesty: the feature says what it is (assistant, not authority) and no consequential action is fully autonomous

    required

    Rule 3 discipline — AI never scores, decides, or authorizes; the UI must make the same promise the architecture keeps.

Run it against your real gaps

A checklist is most useful when you know which controls you are actually missing. A free OpsRoadmaps assessment scores your controls with explainable findings and builds a roadmap from the results.

Assess my stack

Checklists document self-reported practice. Items were last reviewed 2026-09-19 — adapt them to your own environment.