Skip to main content
OpsRoadmaps

Checklists / retrieval-readiness

Retrieval Readiness Checklist

Before a grounded system answers from your content: measured retrieval quality, versioned embeddings, permissions enforced in the query path, corpus freshness, and citations that hold up.

8 steps · 7 required · last reviewed (next review 2026-12-20) · updated 2026-09-20

Related focus areas:RAGOps

Sign in to run this checklist

Run this before a retrieval-augmented feature is user-facing, and repeat it after any change to chunking, embeddings or the corpus. Most bad answers from a grounded system are retrieval failures wearing a model's clothes, and teams routinely spend weeks tuning the wrong half.

The permission item is not a maturity nicety. Retrieval is where an authorization bug becomes a disclosure.

  1. Retrieval quality is measured in isolation on a labelled set of real questions

    required

    This number tells you whether to work on retrieval or generation, and most teams guess wrong.

  2. Permissions are enforced inside the query path against the requesting user

    required

    Filtering after generation means the model already saw the document the user may not.

  3. Chunking and embedding configuration are versioned and recorded with the index

    required

    A silently mixed-embedding index degrades quality in a way that is very hard to diagnose.

  4. Changing chunking or embeddings triggers a full rebuild rather than an incremental write

    required

    Partial reindexing across configurations is the defect this item exists to prevent.

  5. Corpus freshness is monitored and the ingestion lag is a known number

    required

    Confidently outdated answers are the characteristic failure of an unattended corpus.

  6. A reindex procedure is documented and has been rehearsed without serving partial results

    required

    A rebuild you have not practised will be attempted for the first time under pressure.

  7. Sampled answers are checked to confirm the citations support the claims made

    required

    A fluent answer with links looks grounded whether or not it is.

  8. Retrieved context is logged (with appropriate controls) so a bad answer can be diagnosed

    optional

    Without the retrieved context you are debugging the generation step blind.

Run it against your real gaps

A checklist is most useful when you know which controls you are actually missing. A free OpsRoadmaps assessment scores your controls with explainable findings and builds a roadmap from the results.

Assess my stack

Checklists document self-reported practice. Items were last reviewed 2026-09-20 — adapt them to your own environment.