Skip to main content
OpsRoadmaps

Checklists / prompt-change-review

Prompt Change Review Checklist

The review a prompt change passes before it reaches users — because editing one word changes behaviour for everyone, and no compiler objects.

8 steps · 7 required · last reviewed (next review 2026-12-20) · updated 2026-09-20

Related focus areas:PromptOps

Sign in to run this checklist

Run this for every change to a production prompt. A prompt is code with none of code's safety net: no types, no compiler, and an effect that is often only visible in aggregate across many outputs.

Keep this checklist short enough that it is actually used on every change, rather than thorough enough that it is skipped.

  1. The prompt is stored as a versioned artifact, not an inline literal

    required

    Everything else on this list depends on being able to say what changed.

  2. A second person has read the change

    required

    The author is structurally blind to the assumption they just encoded; that is what review is for.

  3. The regression set ran against both the candidate and the incumbent

    required

    Absolute quality scores drift; the comparison against what is currently live is the meaningful signal.

  4. Cases the change was meant to fix are in the regression set

    required

    Otherwise the next change will break them again and nothing will notice.

  5. Outputs were checked by someone with the domain knowledge to judge them

    required

    Whether an output is correct is a domain judgment, not an engineering one.

  6. The change can be reverted without a deploy

    required

    Reverting a prompt should be faster than diagnosing it.

  7. The prompt version is recorded with outputs so a complaint can be traced

    required

    Support cannot investigate "the answer was wrong last Tuesday" without knowing which prompt served it.

  8. The change is rolled out to a subset before everyone

    optional

    Aggregate effects are the ones that surprise you, and a subset surfaces them cheaply.

Run it against your real gaps

A checklist is most useful when you know which controls you are actually missing. A free OpsRoadmaps assessment scores your controls with explainable findings and builds a roadmap from the results.

Assess my stack

Checklists document self-reported practice. Items were last reviewed 2026-09-20 — adapt them to your own environment.