# RAG Service Backend Course: [Retrieval-Augmented Generation](https://www.devtutorials.ai/learn/rag) · Section: Development Workflows Source: https://www.devtutorials.ai/learn/rag/dev-workflows/rag-service-backend > Building the backend service behind a RAG assistant: API design, access filters from the session, validation and errors, streaming with checked citations, safe caching, and ingestion and deletion as background jobs. The chat window is the part of a RAG application that users see. Behind it, a backend service manages the work needed to process each question safely and reliably. The backend service checks access, validates requests, retrieves information, sends it to the model, verifies the answer, updates the index, and manages system performance. A clear interface is important, but it cannot protect the application by itself. The backend must prevent unauthorised access, handle errors, and remain reliable when many users send questions at the same time. **The main idea:** a RAG backend is an **API service** like any other, with a few special parts. Design a small API (a streaming **`POST /api/ask`**, plus feedback, traces and document endpoints). Take **access filters from the session**, never from the request body. **Validate** input before spending money. **Stream** with server-sent events and send checked citations in the final event. **Cache** by question, tenant and index version. Run **ingestion as background jobs** that re-index only what changed, and honour **deletion** everywhere. ## A Real-World Example Brightgrid’s first backend used one simple process: receive a question, search for relevant information, send it to the model, and return the answer. During the first month, three problems appeared: The service trusted access details sent by the user. A user changed the request to claim administrator access and retrieved an internal pricing document. During an outage, many users asked the same question. The backend sent every request to the model, creating the same answer repeatedly and increasing the cost. When one help article changed, the system recreated embeddings for all 4,000 documents. The search index remained outdated until this work finished. Brightgrid improved the backend by: Taking user roles and permissions from the secure signed-in session Checking every request before processing it Reusing a saved answer when the same question is asked and the index has not changed Placing document updates in a queue Creating new embeddings only for documents that changed The language model and prompt stayed the same, but the improved backend made the application more secure, efficient, and up to date. **In simple words:** treat RAG as a backend service: secure the filters, validate the input, stream the output, cache wisely, and keep the index in sync. ## What You Will Learn In this lesson, you will learn: 1. How to design the RAG service's API. 2. Why access filters must come from the session. 3. How to validate requests and shape errors. 4. How streaming and citation checks fit into the response. 5. How to cache answers safely. 6. How to run ingestion and deletion as background jobs. Open the RAG Lab's **App view** and switch to **API stream (backend)**. It shows the exact request the frontend sends and the server-sent events the backend returns for your last question: `sources`, then `token` events, then `done` with the checked citations. ## Designing the API ```js runnable title="The RAG service's endpoints" // The RAG service API: one streaming endpoint plus a few support endpoints. const api = [ ["POST", "/api/ask", "question -> SSE stream (sources, token, done)", "user"], ["POST", "/api/feedback", "trace id + vote -> 204", "user"], ["GET", "/api/traces/{id}", "full trace for debugging", "admin"], [ "POST", "/api/documents", "add or update a document -> 202 (queued for indexing)", "editor", ], [ "DELETE", "/api/documents/{id}", "remove a document from every index -> 202", "editor", ], ]; for (const [method, path, what, role] of api) { console.log(`${method.padEnd(6)} ${path.padEnd(22)} ${role.padEnd(6)} ${what}`); } ``` Expected output: ```text POST /api/ask user question -> SSE stream (sources, token, done) POST /api/feedback user trace id + vote -> 204 GET /api/traces/{id} admin full trace for debugging POST /api/documents editor add or update a document -> 202 (queued for indexing) DELETE /api/documents/{id} editor remove a document from every index -> 202 ``` | Endpoint | Notes | | --- | --- | | `POST /api/ask` | Body has the question and optional non-security options (topK, category). Responds with `text/event-stream`. | | `POST /api/feedback` | Trace id and vote. Returns 204. Feeds evaluation. | | `GET /api/traces/{id}` | Full trace (retrieved chunks, prompt, timings). Admin only, because it may contain document text. | | `POST /api/documents` | Queues an add or update. Returns **202 Accepted**, because indexing happens later. | | `DELETE /api/documents/{id}` | Queues removal from chunks, vectors, caches and logs. | Follow your normal API conventions: plural nouns, standard status codes, and a consistent error body such as `{ "code": "...", "message": "..." }`. ## Access Control ### Filters From the Session The single most important backend rule: **who may see what** is decided by the server, from the signed-in session, and applied **before** retrieval (see **Metadata Filtering** and **RAG Security**). ```js runnable title="Ignore access fields sent by the client" // Access filters come from the signed-in session, never from the request body. function buildFilter(session, body) { const filter = { status: "current", tenant: session.tenant, roles: session.roles }; if (body.filters?.roles || body.filters?.tenant) { console.log(" ignored client-sent access fields:", JSON.stringify(body.filters)); } if (body.filters?.category) { filter.category = body.filters.category; // non-security filters may come from the client } return filter; } const session = { user: "u-42", tenant: "acme", roles: ["customer"] }; const body = { question: "refund rules", filters: { category: "billing", roles: ["admin"], tenant: "other" }, }; console.log("filter used:", JSON.stringify(buildFilter(session, body))); ``` Expected output: ```text ignored client-sent access fields: {"category":"billing","roles":["admin"],"tenant":"other"} filter used: {"status":"current","tenant":"acme","roles":["customer"],"category":"billing"} ``` | Filter | Source | | --- | --- | | Tenant, roles, team, region | The verified session or token | | Document status (current only) | Server default | | Category, product, date range | May come from the client (they narrow results, they don't grant access) | Never let the model enforce access. A prompt that says "don't reveal internal documents" is not a security control. If a chunk the user may not see reaches the prompt, it can leak. Filter in the retrieval query, using the session. ## Validation and Errors Validate the request before any retrieval or model call. It's cheaper, and it shrinks the attack surface: ```js runnable title="Validate the request first" // Validate the request before spending any money on retrieval or the model. function validate(body) { const errors = []; if (typeof body.question !== "string" || body.question.trim().length < 3) { errors.push("question: at least 3 characters"); } if (body.question?.length > 1000) { errors.push("question: at most 1000 characters"); } if ( body.topK !== undefined && (!Number.isInteger(body.topK) || body.topK < 1 || body.topK > 10) ) { errors.push("topK: integer 1-10"); } return errors.length ? { status: 400, body: { code: "INVALID_REQUEST", message: errors.join("; ") } } : { status: 200 }; } console.log(JSON.stringify(validate({ question: "How long do refunds take?", topK: 3 }))); console.log(JSON.stringify(validate({ question: "hi", topK: 50 }))); ``` Expected output: ```text {"status":200} {"status":400,"body":{"code":"INVALID_REQUEST","message":"question: at least 3 characters; topK: integer 1-10"}} ``` | Situation | Response | | --- | --- | | Missing or too-short question | 400 `INVALID_REQUEST` | | Not signed in | 401 | | Signed in but not allowed (for example, traces) | 403 | | Too many requests | 429 with `retry-after` | | Model or index unavailable | 503, and the UI shows Retry | | Mid-stream failure | An `error` event, then close the stream | ## Streaming the Answer ### The Event Sequence 1. **Authenticate** and **validate**. 2. **Retrieve** with the session's filters; send a `sources` event (ids, titles, sections, dates). 3. **Generate**; forward each chunk of text as a `token` event. 4. **Check** citations and evidence (see **Citation Tracing**). 5. Send **`done`** with valid citations, a refusal flag and the trace id. The frontend renders tokens immediately and chips only after `done` (previous lesson). After asking, open **App view → API stream** and match each event to the steps above. ## Caching ### Cache Safely Identical questions are common: during outages, after a product launch, from onboarding flows. A cache saves cost and time, but it must never leak answers across users who see different documents: ```js runnable title="Cache by question, tenant and index version" // A small answer cache: key by normalized question + tenant + index version. const cache = new Map(); let calls = 0; const indexVersion = "idx-2026-10-09"; function ask(question, tenant) { const key = `${tenant}|${indexVersion}|${question .toLowerCase() .replace(/[^a-z0-9 ]/g, "") .trim()}`; if (cache.has(key)) { return `${cache.get(key)} (cache hit)`; } calls += 1; const answer = `answer #${calls}`; cache.set(key, answer); return answer; } console.log(ask("How long do refunds take?", "acme")); console.log(ask("how long do refunds take", "acme")); console.log(ask("How long do refunds take?", "globex")); console.log(`model calls: ${calls}`); ``` Expected output: ```text answer #1 answer #1 (cache hit) answer #2 model calls: 2 ``` - Key by **normalized question**, **tenant or permission set**, and **index version**. - Clear or bump the version when documents change, so stale answers expire. - Don't cache answers built from personal or restricted documents unless the key includes the user. Cache **retrieval results** as well as whole answers. Retrieval is cheap to cache and safe to reuse with a different prompt or model, and it removes most of the latency from repeat questions. ## Ingestion and Deletion ### Background Jobs Indexing is slow, so the API returns **202** and a background worker does the work. Compare a content **hash** to re-embed only what changed: ```js runnable title="Re-index only what changed" // Ingestion as a background job: queue changes, re-index only what changed. const indexed = { "returns-policy": "h1", "shipping-times": "h2", warranty: "h3" }; const incoming = [ { id: "returns-policy", hash: "h1" }, { id: "shipping-times", hash: "h9" }, { id: "gift-cards", hash: "h4" }, ]; const deleted = ["warranty"]; const jobs = []; for (const d of incoming) { if (!indexed[d.id]) { jobs.push(`add ${d.id}`); } else if (indexed[d.id] !== d.hash) { jobs.push(`re-embed ${d.id}`); } } for (const id of deleted) { jobs.push(`delete ${id} (chunks, vectors, cache)`); } console.log(jobs.join("\n")); console.log( `${incoming.length - jobs.filter((j) => !j.startsWith("delete")).length} unchanged documents skipped`, ); ``` Expected output: ```text re-embed shipping-times add gift-cards delete warranty (chunks, vectors, cache) 1 unchanged documents skipped ``` Deletion must reach **every** copy: chunks, vectors, caches, traces that contain document text, and backups on their normal schedule. That's both a correctness rule (no answers from deleted docs) and often a legal one. ### Index Versions Build a new index version next to the old one, run the gold set against it, and switch traffic only when it passes. That's how you change chunking or embedding models without breaking production (see **Production Operations**). ## Common mistakes - **Trusting client-sent filters.** Access comes from the session. - **Letting the model enforce permissions.** Filter before retrieval. - **No input validation.** Reject bad requests before paying for retrieval or tokens. - **Caching across tenants.** Include tenant and index version in the key. - **Re-embedding everything on every change.** Use hashes and a job queue. - **Deleting only from the vector store.** Remove from chunks, caches and logs too. ## Check Your Understanding ## Exercises Each exercise ends with a value on the last line. Press **Check** to compare it with the expected answer. ## What to Remember - A RAG backend is an **API service**: `POST /api/ask` (SSE), feedback, traces, documents. - **Access filters come from the session** and are applied **before retrieval**. - **Validate** input first, and use clear status codes and error bodies. - Stream `sources`, `token` and `done`; send **checked citations** in `done`. - **Cache** by question, tenant and index version. - Run **ingestion** as background jobs with hashes, and delete **everywhere**. Next: **Engineering Knowledge Assistant**, where you secure and test a RAG assistant built on internal engineering docs.