# RAG Pipeline Architecture Course: [Retrieval-Augmented Generation](https://www.devtutorials.ai/learn/rag) · Section: RAG Foundations Source: https://www.devtutorials.ai/learn/rag/foundations/rag-pipeline-architecture > The four RAG stages and how data flows between them: chunking, embedding, retrieval and generation; query processing, candidate selection, context packing and citation validation; determinism and the three runtime tiers. Think about a sandwich shop. Making a sandwich happens step by step. First, someone cuts the bread and prepares the ingredients. Next, someone wraps the sandwiches and puts labels on them. When a customer places an order, an employee finds the correct sandwich, adds a napkin and receipt, and gives it to the customer. Every step depends on the previous step. If the ingredients were prepared badly in the morning, the sandwich will not be good later. If a sandwich gets the wrong label, the customer may receive the wrong order. A RAG system is the same kind of line. Each stage takes the output of the stage before it, and a mistake early on travels all the way to the answer. **The main idea:** a RAG pipeline has **four stages**: **chunking**, **embedding**, **retrieval** and **generation**. Chunking and embedding run ahead of time. Retrieval and generation run for each question. Between them, data moves in a fixed order: process the query, select candidates, pack the context, then generate and validate the answer. Knowing which steps are **deterministic** (same input, same output) helps you test the system. ## A Real-World Example Brightgrid's assistant suddenly started quoting the **2023** return policy again. Nothing in the model had changed. The team followed the data through each stage: Chunking was working correctly. The 2023 policy was split into smaller pieces, just like every other document. Embedding was also working correctly. The chunks were converted into vectors so the system could search them later. Retrieval was where the problem happened. Someone had accidentally added old archived articles back into the system. When users asked about returns, both the old 2023 policy and the current policy appeared among the top search results. Generation received both policies as context. Because the old policy was included, the AI used the wrong information and generated an incorrect answer. The fix was one rule in the **retrieval** stage: skip archived articles. Because the team knew what each stage produces, they found the broken step in minutes instead of blaming the model. **In simple words:** know what goes in and out of each stage, and you can find any problem. ## What You Will Learn In this lesson, you will learn: 1. The four core stages and what each one produces. 2. How data flows through a question: query processing, candidate selection, context packing, response and citation checks. 3. Which steps are deterministic, and the three runtime tiers used in this course. The **RAG Lab** runs all four stages. Its **Trace** tab lists each step of the last question with its time and result, and shows the exact prompt the model received. Use it whenever you want to see "what actually happened". ## Four Core Pipeline Stages ### Chunking Stage (Document Splitting) **Input:** whole documents. **Output:** small pieces called **chunks**, each with an id like `returns-policy#1`. Why split at all? A question is usually about one small fact. If you search whole documents, a long article that mentions "returns" once can beat a short paragraph that is all about returns. Small chunks also cost fewer tokens when they're added to the prompt. A common first approach is to split at headings. The RAG Lab does this: each `## ` heading starts a new chunk. Chapter 2 covers chunking in depth. ### Embedding Stage (Vector Encoding) **Input:** chunks of text. **Output:** a **vector** for each chunk, a list of numbers (often 384 to 3,072 of them) that captures its meaning. An **embedding model** is trained so that texts with similar meanings get similar numbers. "I can't sign in" and "I forgot my password" get vectors that are close together, even though they share no words. The vectors are stored in an **index** together with each chunk's text and details such as its title and date. Chapter 3 explains embeddings. ### Retrieval Stage (Similarity Search & Ranking) **Input:** the user's question. **Output:** a short, **ranked** list of the best chunks. The question is turned into a vector with the same embedding model. The index finds the chunk vectors closest to it. Many systems also run a **keyword** search and merge the two lists. This is called **hybrid search**, and Chapter 4 covers it. The result is the **top-k** chunks, best first. ### Generation Stage (Grounded Model Response) **Input:** instructions, the top-k chunks and the question. **Output:** an answer with citations. The language model reads the chunks and writes an answer that uses only them, marking each fact with its source id, like `[S1]`. This is a **grounded** response. If the chunks don't contain the answer, the model should say so. Here are all four stages in one small program. Each line shows the data that one stage hands to the next: ```js runnable title="Four stages, one question" // The four stages, with the data that flows out of each one. const doc = { id: "returns-policy", text: "## How long\nReturn within 30 days.\n## Refunds\nRefunds take 5 business days.", }; // 1. Chunking: one document becomes several small pieces. const chunks = doc.text .split("## ") .filter(Boolean) .map((part, i) => ({ id: `${doc.id}#${i + 1}`, text: part.replace("\n", ": ").trim(), })); console.log("1 chunks: ", chunks.map((c) => c.id).join(", ")); // 2. Embedding: each chunk becomes a list of numbers. (Here: a toy 4-number vector.) const vocab = ["return", "days", "refund", "business"]; const embed = (t) => vocab.map((w) => (t.toLowerCase().includes(w) ? 1 : 0)); const vectors = chunks.map((c) => embed(c.text)); console.log("2 vectors: ", JSON.stringify(vectors)); // 3. Retrieval: compare the question's vector with every chunk vector. const q = embed("How long do refunds take?"); const scores = vectors.map((v) => v.reduce((s, x, i) => s + x * q[i], 0)); const best = chunks[scores.indexOf(Math.max(...scores))]; console.log("3 retrieved:", best.id, "scores", JSON.stringify(scores)); // 4. Generation: the model gets the question plus the retrieved text. console.log( "4 model gets:", `Answer from [S1] ${best.text} | Question: How long do refunds take?`, ); ``` Expected output: ```text 1 chunks: returns-policy#1, returns-policy#2 2 vectors: [[1,1,0,0],[0,1,1,1]] 3 retrieved: returns-policy#2 scores [0,1] 4 model gets: Answer from [S1] Refunds: Refunds take 5 business days. | Question: How long do refunds take? ``` The "vectors" here are toys: a 1 means "this chunk contains that word". Real embeddings have hundreds of numbers and capture meaning, not just words. But the flow is exactly the same. ## Stage Dependencies & Data Flow Each question walks through the same sequence. Every step depends on the one before it. ### Input Query Processing Flow Before searching, the app cleans the question: - **Normalize:** trim spaces and lowercase the text. - **Split** it into words (terms), and drop very common words like "how", "do" and "the". These are called **stop words**. - **Check** that something is left. "how do i" alone has nothing to search for, so the app should ask the user to say more. ```js runnable title="Clean the question" // Query processing: clean the question before searching. const STOP = new Set([ "how", "do", "i", "a", "the", "my", "can", "to", "is", "what", "of", ]); function processQuery(raw) { const normalized = raw.trim().toLowerCase().replace(/\s+/g, " "); const terms = (normalized.match(/[a-z0-9]+/g) ?? []).filter((w) => !STOP.has(w)); return { raw, normalized, terms, empty: terms.length === 0 }; } for (const q of [ " How do I RETURN my lamp?? ", "What is the price of Plus?", "how do i", ]) { console.log(JSON.stringify(processQuery(q))); } ``` Expected output: ```text {"raw":" How do I RETURN my lamp?? ","normalized":"how do i return my lamp??","terms":["return","lamp"],"empty":false} {"raw":"What is the price of Plus?","normalized":"what is the price of plus?","terms":["price","plus"],"empty":false} {"raw":"how do i","normalized":"how do i","terms":[],"empty":true} ``` Later chapters add more here: rewriting vague questions, splitting questions with several parts, and using the chat history. ### Candidate Selection Sequence The retriever scores many chunks. Then three steps pick the few that move on: 1. **Sort** by score, best first. 2. **Cut** weak matches below a minimum score, so unrelated chunks never reach the model. 3. **Keep** the top k. ```js runnable title="Pick the candidates" // Candidate selection: score everything, cut weak matches, keep the top k. const scored = [ { id: "returns-policy#1", score: 0.82 }, { id: "returns-policy#3", score: 0.74 }, { id: "returns-policy-2023#1", score: 0.71 }, { id: "cancel-order#2", score: 0.48 }, { id: "warranty#3", score: 0.31 }, { id: "safety#1", score: 0.12 }, ]; function selectCandidates(list, { minScore, k }) { const steps = []; const sorted = [...list].sort((a, b) => b.score - a.score); steps.push(`sorted: ${sorted.length}`); const strong = sorted.filter((c) => c.score >= minScore); steps.push(`score >= ${minScore}: ${strong.length}`); const top = strong.slice(0, k); steps.push(`top ${k}: ${top.map((c) => c.id).join(", ")}`); return steps; } console.log(selectCandidates(scored, { minScore: 0.4, k: 3 }).join("\n")); ``` Expected output: ```text sorted: 6 score >= 0.4: 4 top 3: returns-policy#1, returns-policy#3, returns-policy-2023#1 ``` Look at the third chunk that was kept: `returns-policy-2023#1`. That is the outdated policy from the story above. Selection only looks at scores. Rules about **which documents are allowed**, such as "skip archived articles", must be added on purpose. You'll learn about this in the next lesson and in Chapter 4. ### Context Packing Payload Construction **Context packing** turns the chosen chunks into the text block the model will read. It: - Labels each chunk with an id (**S1**, **S2**, …) for citations. - Adds useful **metadata**, such as the chunk id and the date it was updated. - Stops before the token **budget** runs out. Context windows are limited, and every token costs time and money. ```js runnable title="Pack chunks into a budget" // Context packing: fit the best chunks into a token budget, with ids and metadata. const chunks = [ { id: "returns-policy#1", updated: "2026-03-01", text: "You can return any product within 30 days of delivery.", }, { id: "returns-policy#3", updated: "2026-03-01", text: "We refund the full price within 5 business days after the parcel arrives.", }, { id: "cancel-order#2", updated: "2026-02-02", text: "Once an order has shipped it cannot be cancelled. Start a return instead.", }, ]; const approxTokens = (s) => Math.ceil(s.length / 4); // about 4 characters per token in English function pack(list, budget) { const parts = []; let used = 0; for (const [i, c] of list.entries()) { const block = `${c.text}`; if (used + approxTokens(block) > budget) { break; } parts.push(block); used += approxTokens(block); } return { payload: parts.join("\n"), used, kept: parts.length }; } const { payload, used, kept } = pack(chunks, 70); console.log(payload); console.log(`kept ${kept} of ${chunks.length} chunks, about ${used} tokens`); ``` Expected output: ```text You can return any product within 30 days of delivery. We refund the full price within 5 business days after the parcel arrives. kept 2 of 3 chunks, about 68 tokens ``` The third chunk didn't fit in the 70-token budget, so it was left out. The best chunks go first, so a cut only loses the weakest ones. A quick estimate for English: **one token is about 4 characters**, or about three-quarters of a word. Real systems count tokens with the model's own tokenizer, but this estimate is good enough for planning a budget. ### Model Response & Citation Validation The last step happens **after** the model answers. The app checks the answer before showing it: - Does it cite at least one source, or clearly say "I don't know"? - Does every cited id actually exist? - Later, in Chapter 6: does each cited source really say what the sentence claims? ```js runnable title="Check the answer before showing it" // Response validation: check the answer's citations before showing it. function validate(answer, sourceCount) { const cited = [...answer.matchAll(/\[S(\d+)\]/g)].map((m) => Number(m[1])); const problems = []; if (/i don't know/i.test(answer)) { return "ok (refusal)"; } if (cited.length === 0) { problems.push("no citations"); } for (const n of cited) { if (n < 1 || n > sourceCount) { problems.push(`S${n} does not exist`); } } return problems.length ? `REJECT: ${problems.join("; ")}` : "ok"; } console.log(validate("You have 30 days to return it [S1]. Refunds take 5 days [S2].", 2)); console.log(validate("You have 30 days to return it.", 2)); console.log(validate("Gift cards are refundable [S4].", 2)); console.log(validate("I don't know based on the help center.", 2)); ``` Expected output: ```text ok REJECT: no citations REJECT: S4 does not exist ok (refusal) ``` An answer that fails can be retried, sent with a warning, or replaced with a safe "I don't know". Try the whole flow in the lab, then open the **Trace** to see each step: ## Determinism & Runtime Architecture ### Why Determinism Matters A step is **deterministic** if the same input always gives the same output. Keyword scoring, sorting and packing are deterministic. Language models are usually **not**: by default they pick words with some randomness, so the same question can get different answers. This matters for **testing**. If retrieval is deterministic, you can write a test like "this question must return `returns-policy#1`" and it will pass or fail for real reasons, not by chance. For the model, you reduce randomness with **temperature 0** and a fixed **seed** (Chapter 6), but small differences can still appear across devices and model versions. ```js runnable title="Same input, same output?" // Deterministic vs non-deterministic steps: run each twice and compare. function seeded(seed) { // A tiny pseudo-random generator: the same seed always gives the same numbers. let s = seed; return () => { s = (s * 1103515245 + 12345) % 2147483648; return s / 2147483648; }; } const keywordScore = (q, t) => q.split(" ").filter((w) => t.includes(w)).length; const run1 = keywordScore("return lamp days", "return any lamp within 30 days"); const run2 = keywordScore("return lamp days", "return any lamp within 30 days"); console.log( "keyword search, run twice:", run1, run2, run1 === run2 ? "same" : "different", ); const a = seeded(42); const b = seeded(42); console.log("sampling with a fixed seed:", a().toFixed(4), b().toFixed(4), "same"); const c = seeded(Date.now() % 1000); const d = seeded((Date.now() + 7) % 1000); console.log("sampling with no fixed seed:", c() === d() ? "same" : "different"); ``` Expected output: ```text keyword search, run twice: 3 3 same sampling with a fixed seed: 0.5823 0.5823 same sampling with no fixed seed: different ``` ### The Three Runtime Tiers The RAG Lab, and this course, split the pipeline into three tiers. Each tier is more powerful but less predictable than the one before: | Tier | What runs | Where | Deterministic? | Used for | | --- | --- | --- | --- | --- | | **Tier 1** | Plain JavaScript (or WASM): chunking, keyword search, scoring, packing, checks | Your browser, instantly | Yes | Every code example in the lessons | | **Tier 2** | A small **embedding model** (Transformers.js, all-MiniLM-L6-v2, about 23 MB) | Your browser, on WebGPU or WASM | Yes, for the same model and settings | Vector and hybrid search in the lab | | **Tier 3** | A small **language model** (WebLLM, for example Llama 3.2 1B) | Your browser, needs WebGPU | Mostly, with temperature 0 and a seed | Writing answers in the lab | Because the tiers stack, the lab still works when a tier is missing: - With **only Tier 1**, search is keyword-based, and answers are built by quoting the best-matching sentences (the "extractive fallback"). - **Tier 2** adds meaning-based search. - **Tier 3** adds real written answers. Each model is downloaded only once. After that, your browser keeps a copy, and the lab loads it from there automatically. Run your tests on the deterministic tiers first. If a retrieval test only passes sometimes, the problem is in your code, not "AI randomness". Save the non-deterministic tier, the language model, for the last step, and judge it with the evaluation methods from Chapter 8. ## Common mistakes - **Fixing the wrong stage.** A bad answer is often a retrieval problem, not a model problem. Check the retrieved chunks first. - **Using different embedding models** for chunks and questions. Their vectors don't match, so search fails quietly. - **No minimum score.** Unrelated chunks reach the model and confuse it. - **No token budget.** Prompts grow until they are slow, expensive or cut off. - **Skipping validation.** Answers with missing or fake citations reach users. ## Exercises Each exercise ends with a value on the last line. Press **Check** to compare it with the expected answer. ## What to Remember - Four stages: **chunking** (documents to chunks), **embedding** (chunks to vectors), **retrieval** (question to ranked chunks), **generation** (chunks plus question to a cited answer). - Per question, data flows: **process the query → select candidates → pack the context → generate → validate citations**. - Each stage depends on the one before it. A bad answer often starts in **retrieval**. - **Deterministic** steps give the same output every time, so test them with exact expected results. - The course uses three tiers: **Tier 1** plain JavaScript, **Tier 2** a local embedding model, **Tier 3** a local language model. The lab works with any of them missing. Next: **Corpus Engineering**, choosing and cleaning the documents your RAG system searches.