Retrieval-Augmented Generation System

ARCHITECTURE An ingestion pipeline pulls source documents on a schedule, chunks them under a versioned strategy, embeds them with a pinned model, and writes to a vector index alongside the metadata needed to filter by entitlement. At query time the user's question is embedded, candidates are retrieved with the requesting user's permissions applied in the query, and the selected passages are passed to the generation step with instructions to cite them. Retrieval quality is measured separately from answer quality. This is the single most important structural decision: without it, teams tune prompts for weeks to fix a retrieval problem. Store the chunking and embedding configuration with the index. Changing either requires a full rebuild, and a partially re-embedded index degrades in ways that are very hard to diagnose. ASSUMPTIONS You have a corpus worth grounding on, content owners who will keep it current, and an access model you can express at query time. It assumes eventual consistency between source and index is acceptable, with a known lag. If the corpus is public and small, a simpler design may serve. If entitlements are complex and undocumented, fix that first — retrieval will otherwise enforce a permission model nobody has written down. SECURITY This is where a retrieval bug becomes a disclosure. Filter by the requesting user's entitlements inside the query, not after generation: post-filtering means the model already saw content the user may not access, and anything it summarised has already leaked. Re-check entitlements at answer time for long-lived sessions. Treat retrieved content as untrusted — a document in the corpus can carry injected instructions, so the generation step must not grant retrieved text authority over its own behaviour. OBSERVABILITY Recall against a labelled question set is the headline measure: how often does a passage containing the answer appear in the top k. Track corpus freshness lag, ingestion failures, citation support rate on sampled answers, and retrieval latency separately from generation latency. Log the retrieved context (with appropriate controls) or you are debugging generation blind. AVAILABILITY Index rebuilds must not serve partial results — build alongside and switch atomically. An ingestion outage should surface as staleness you can see, not as confidently outdated answers. Define behaviour when retrieval returns nothing: saying "I do not have a document covering that" is a correct answer and a better one than an ungrounded guess. COST CONSIDERATIONS Embedding the corpus is a one-off per rebuild and re-embedding everything is the expensive event — which is another reason to version the configuration deliberately. Ongoing cost is dominated by retrieval-inflated context: retrieving twenty passages when five suffice multiplies generation tokens on every request. Tune k against measured recall rather than intuition. Vector storage in an existing PostgreSQL via pgvector avoids a separate managed service at small and medium scale. IAC GUIDANCE Index definitions, ingestion schedules and the embedding model pin belong in code. Keep the labelled evaluation set in version control beside the pipeline — it is the asset that makes every future change safe, and it is the one most often kept in a spreadsheet. PRODUCTION READINESS Before users: retrieval recall measured on real questions, permissions enforced in the query path, chunking and embedding versions recorded with the index, a rehearsed atomic reindex, corpus freshness monitored, citations spot-checked for support, and a defined answer for "nothing retrieved". The retrieval readiness checklist at /checklists is the gate; /ragops is the discipline.