RAG Service Backend

By the Syncfusion DevTutorials team · Last updated October 19, 2026

Building the backend service behind a RAG assistant: API design, access filters from the session, validation and errors, streaming with checked citations, safe caching, and ingestion and deletion as background jobs.

The chat window is the part of a RAG application that users see. Behind it, a backend service manages the work needed to process each question safely and reliably. The backend service checks access, validates requests, retrieves information, sends it to the model, verifies the answer, updates the index, and manages system performance. A clear interface is important, but it cannot protect the application by itself. The backend must prevent unauthorised access, handle errors, and remain reliable when many users send questions at the same time.

The main idea: a RAG backend is an API service like any other, with a few special parts. Design a small API (a streaming POST /api/ask, plus feedback, traces and document endpoints). Take access filters from the session, never from the request body. Validate input before spending money. Stream with server-sent events and send checked citations in the final event. Cache by question, tenant and index version. Run ingestion as background jobs that re-index only what changed, and honour deletion everywhere.

A Real-World Example

Brightgrid’s first backend used one simple process: receive a question, search for relevant information, send it to the model, and return the answer.

During the first month, three problems appeared:

The service trusted access details sent by the user. A user changed the request to claim administrator access and retrieved an internal pricing document. During an outage, many users asked the same question. The backend sent every request to the model, creating the same answer repeatedly and increasing the cost. When one help article changed, the system recreated embeddings for all 4,000 documents. The search index remained outdated until this work finished.

Brightgrid improved the backend by:

Taking user roles and permissions from the secure signed-in session Checking every request before processing it Reusing a saved answer when the same question is asked and the index has not changed Placing document updates in a queue Creating new embeddings only for documents that changed

The language model and prompt stayed the same, but the improved backend made the application more secure, efficient, and up to date.

In simple words: treat RAG as a backend service: secure the filters, validate the input, stream the output, cache wisely, and keep the index in sync.

What You Will Learn

In this lesson, you will learn:

1. How to design the RAG service's API.
2. Why access filters must come from the session.
3. How to validate requests and shape errors.
4. How streaming and citation checks fit into the response.
5. How to cache answers safely.
6. How to run ingestion and deletion as background jobs.

Designing the API

The RAG service's endpoints

// The RAG service API: one streaming endpoint plus a few support endpoints.
const api = [
  ["POST", "/api/ask", "question -> SSE stream (sources, token, done)", "user"],
  ["POST", "/api/feedback", "trace id + vote -> 204", "user"],
  ["GET", "/api/traces/{id}", "full trace for debugging", "admin"],
  [
    "POST",
    "/api/documents",
    "add or update a document -> 202 (queued for indexing)",
    "editor",
  ],
  [
    "DELETE",
    "/api/documents/{id}",
    "remove a document from every index -> 202",
    "editor",
  ],
];
for (const [method, path, what, role] of api) {
  console.log(`${method.padEnd(6)} ${path.padEnd(22)} ${role.padEnd(6)} ${what}`);
}

Expected output:

POST   /api/ask               user   question -> SSE stream (sources, token, done)
POST   /api/feedback          user   trace id + vote -> 204
GET    /api/traces/{id}       admin  full trace for debugging
POST   /api/documents         editor add or update a document -> 202 (queued for indexing)
DELETE /api/documents/{id}    editor remove a document from every index -> 202

Follow your normal API conventions: plural nouns, standard status codes, and a consistent error body such as { "code": "...", "message": "..." }.

Access Control

Filters From the Session

The single most important backend rule: who may see what is decided by the server, from the signed-in session, and applied before retrieval (see Metadata Filtering and RAG Security).

Ignore access fields sent by the client

// Access filters come from the signed-in session, never from the request body.
function buildFilter(session, body) {
  const filter = { status: "current", tenant: session.tenant, roles: session.roles };
  if (body.filters?.roles || body.filters?.tenant) {
    console.log("  ignored client-sent access fields:", JSON.stringify(body.filters));
  }
  if (body.filters?.category) {
    filter.category = body.filters.category; // non-security filters may come from the client
  }
  return filter;
}

const session = { user: "u-42", tenant: "acme", roles: ["customer"] };
const body = {
  question: "refund rules",
  filters: { category: "billing", roles: ["admin"], tenant: "other" },
};
console.log("filter used:", JSON.stringify(buildFilter(session, body)));

Expected output:

  ignored client-sent access fields: {"category":"billing","roles":["admin"],"tenant":"other"}
filter used: {"status":"current","tenant":"acme","roles":["customer"],"category":"billing"}

Validation and Errors

Validate the request before any retrieval or model call. It's cheaper, and it shrinks the attack surface:

Validate the request first

// Validate the request before spending any money on retrieval or the model.
function validate(body) {
  const errors = [];
  if (typeof body.question !== "string" || body.question.trim().length < 3) {
    errors.push("question: at least 3 characters");
  }
  if (body.question?.length > 1000) {
    errors.push("question: at most 1000 characters");
  }
  if (
    body.topK !== undefined &&
    (!Number.isInteger(body.topK) || body.topK < 1 || body.topK > 10)
  ) {
    errors.push("topK: integer 1-10");
  }
  return errors.length
    ? { status: 400, body: { code: "INVALID_REQUEST", message: errors.join("; ") } }
    : { status: 200 };
}

console.log(JSON.stringify(validate({ question: "How long do refunds take?", topK: 3 })));
console.log(JSON.stringify(validate({ question: "hi", topK: 50 })));

Expected output:

{"status":200}
{"status":400,"body":{"code":"INVALID_REQUEST","message":"question: at least 3 characters; topK: integer 1-10"}}

Streaming the Answer

The Event Sequence

1. Authenticate and validate.
2. Retrieve with the session's filters; send a sources event (ids, titles, sections, dates).
3. Generate; forward each chunk of text as a token event.
4. Check citations and evidence (see Citation Tracing).
5. Send done with valid citations, a refusal flag and the trace id.

The frontend renders tokens immediately and chips only after done (previous lesson).

After asking, open App view → API stream and match each event to the steps above.

Caching

Cache Safely

Identical questions are common: during outages, after a product launch, from onboarding flows. A cache saves cost and time, but it must never leak answers across users who see different documents:

Cache by question, tenant and index version

// A small answer cache: key by normalized question + tenant + index version.
const cache = new Map();
let calls = 0;
const indexVersion = "idx-2026-10-09";

function ask(question, tenant) {
  const key = `${tenant}|${indexVersion}|${question
    .toLowerCase()
    .replace(/[^a-z0-9 ]/g, "")
    .trim()}`;
  if (cache.has(key)) {
    return `${cache.get(key)} (cache hit)`;
  }
  calls += 1;
  const answer = `answer #${calls}`;
  cache.set(key, answer);
  return answer;
}

console.log(ask("How long do refunds take?", "acme"));
console.log(ask("how long do refunds take", "acme"));
console.log(ask("How long do refunds take?", "globex"));
console.log(`model calls: ${calls}`);

Expected output:

answer #1
answer #1 (cache hit)
answer #2
model calls: 2

• Key by normalized question, tenant or permission set, and index version.
• Clear or bump the version when documents change, so stale answers expire.
• Don't cache answers built from personal or restricted documents unless the key includes the user.

Ingestion and Deletion

Background Jobs

Indexing is slow, so the API returns 202 and a background worker does the work. Compare a content hash to re-embed only what changed:

Re-index only what changed

// Ingestion as a background job: queue changes, re-index only what changed.
const indexed = { "returns-policy": "h1", "shipping-times": "h2", warranty: "h3" };
const incoming = [
  { id: "returns-policy", hash: "h1" },
  { id: "shipping-times", hash: "h9" },
  { id: "gift-cards", hash: "h4" },
];
const deleted = ["warranty"];

const jobs = [];
for (const d of incoming) {
  if (!indexed[d.id]) {
    jobs.push(`add ${d.id}`);
  } else if (indexed[d.id] !== d.hash) {
    jobs.push(`re-embed ${d.id}`);
  }
}
for (const id of deleted) {
  jobs.push(`delete ${id} (chunks, vectors, cache)`);
}
console.log(jobs.join("\n"));
console.log(
  `${incoming.length - jobs.filter((j) => !j.startsWith("delete")).length} unchanged documents skipped`,
);

Expected output:

re-embed shipping-times
add gift-cards
delete warranty (chunks, vectors, cache)
1 unchanged documents skipped

Deletion must reach every copy: chunks, vectors, caches, traces that contain document text, and backups on their normal schedule. That's both a correctness rule (no answers from deleted docs) and often a legal one.

Index Versions

Build a new index version next to the old one, run the gold set against it, and switch traffic only when it passes. That's how you change chunking or embedding models without breaking production (see Production Operations).

Common mistakes

• Trusting client-sent filters. Access comes from the session.
• Letting the model enforce permissions. Filter before retrieval.
• No input validation. Reject bad requests before paying for retrieval or tokens.
• Caching across tenants. Include tenant and index version in the key.
• Re-embedding everything on every change. Use hashes and a job queue.
• Deleting only from the vector store. Remove from chunks, caches and logs too.

Check Your Understanding

Exercises

Each exercise ends with a value on the last line. Press Check to compare it with the expected answer.

What to Remember

• A RAG backend is an API service: POST /api/ask (SSE), feedback, traces, documents.
• Access filters come from the session and are applied before retrieval.
• Validate input first, and use clear status codes and error bodies.
• Stream sources, token and done; send checked citations in done.
• Cache by question, tenant and index version.
• Run ingestion as background jobs with hashes, and delete everywhere.

Next: Engineering Knowledge Assistant, where you secure and test a RAG assistant built on internal engineering docs.