# Chunk Optimization Course: [Retrieval-Augmented Generation](https://www.devtutorials.ai/learn/rag) · Section: Document Processing Source: https://www.devtutorials.ai/learn/rag/document-processing/chunk-optimization > Tuning chunks for retrieval: sliding-window overlap and ratios, over- vs under-chunking, sizing to queries, standalone completeness, orphans, pronoun resolution, benchmarking, contextual headers and parent-child chunks. Think about taking several photos of a long wall painting. If one photo ends exactly where the next photo begins, an important detail at the edge may be split between them. To avoid this, each photo can include a small part of the previous one. This shared area is called an overlap. RAG chunks can work in the same way. A small overlap between neighbouring chunks helps preserve information that appears near a chunk boundary. The size of each chunk also matters: A chunk that is too small may lose important context. A chunk that is too large may contain unrelated information. A useful chunk should contain enough information to make sense on its own. Chunk optimisation means choosing a suitable chunk size and overlap, then testing whether the resulting chunks help the system retrieve useful information. The team can test the chunking approach using a benchmark, which is a prepared set of questions and expected source content. This shows whether the correct chunks are being retrieved. The main idea: Choose a chunk size that preserves complete ideas, add a small overlap where needed, and test whether the correct information is retrieved. ## A Real-World Example Brightgrid's gold set showed one question that always failed: "What do I do when the lamp blinks blue?". The answer spans two sentences: "It blinks blue when it is ready to pair. Then open the app and press Add device." Their chunker had cut exactly between those two sentences. The team made three changes: 1. **One sentence of overlap**, so the boundary sentence appears in both chunks. The failing question passed. 2. A **quality check** found chunks that started with "It" and "This" or were only a few words long. They were merged with their neighbour or given a header. 3. **Parent-child chunks** for long articles: small chunks for precise search, and the whole section sent to the model. Hit rate on the gold set went from 82% to 93%. **In simple words:** overlap the edges, choose a sensible size, and make every piece complete. ## What You Will Learn In this lesson, you will learn: 1. Overlap mechanics: sliding windows, preventing boundary loss, and good overlap ratios. 2. Granularity tuning: the risks of chunks that are too small or too large, matching size to questions, and size vs vector precision. 3. Quality checks: standalone completeness, orphan chunks, pronoun resolution and benchmarking. 4. Two useful patterns: contextual chunk headers and parent-child chunks. In the RAG Lab's **Chunker** tab, choose **fixed** or **sentence** and drag the **Overlap** slider. Chunks with quality problems get a red edge and labels like "very short", "starts mid-sentence" or "starts with a pronoun". ## Chunk Overlap Mechanics ### Sliding Window Overlap Setup A **sliding window** moves along the text. Each window is one chunk. With **overlap**, the window moves forward by less than its own size, so neighbouring chunks share some text. Overlap can be measured in characters, tokens or whole sentences. Sentences are best when you can use them, because they never cut a sentence in half. ### Context Boundary Loss Prevention Without overlap, any fact that crosses a boundary is split. Neither chunk holds the whole answer, so neither scores well. With overlap, at least one chunk contains both halves. ```js runnable title="Overlap saves an answer at the boundary" // Sliding-window overlap: without it, an answer that crosses a boundary is cut in half. const sentences = [ "Plug in the lamp.", "It blinks blue when it is ready.", "Then open the app.", "Press Add device to pair it.", ]; function windows(list, size, overlap) { const out = []; for (let i = 0; i < list.length; i += size - overlap) { out.push(list.slice(i, i + size).join(" ")); if (i + size >= list.length) { break; } } return out; } // The answer to "What do I do when the lamp blinks blue?" spans sentences 2 and 3. const answerFound = (chunks) => chunks.some((c) => c.includes("blinks blue") && c.includes("open the app")); for (const overlap of [0, 1]) { const chunks = windows(sentences, 2, overlap); console.log( `overlap ${overlap}: ${chunks.length} chunks, answer in one chunk: ${answerFound(chunks)}`, ); chunks.forEach((c) => console.log(" ", c)); } ``` Expected output: ```text overlap 0: 2 chunks, answer in one chunk: false Plug in the lamp. It blinks blue when it is ready. Then open the app. Press Add device to pair it. overlap 1: 3 chunks, answer in one chunk: true Plug in the lamp. It blinks blue when it is ready. It blinks blue when it is ready. Then open the app. Then open the app. Press Add device to pair it. ``` ### Optimal Overlap Ratios Overlap is not free. Every overlapping part is stored, embedded and searched twice. The **overlap ratio** is the overlap divided by the chunk size: ```js runnable title="What overlap costs" // Overlap has a cost: more chunks to embed, store and search. Find a balance. const docChars = 100000; // a 100,000-character manual const size = 800; console.log("overlap ratio chunks stored chars extra storage"); for (const overlap of [0, 80, 160, 240, 400]) { const step = size - overlap; const chunks = Math.ceil((docChars - overlap) / step); const stored = chunks * size; const extra = Math.round((stored / docChars - 1) * 100); console.log( `${String(overlap).padStart(7)} ${(overlap / size).toFixed(2).padStart(5)} ${String(chunks).padStart(6)} ${String(stored).padStart(12)} ${String(extra).padStart(12)}%`, ); } ``` Expected output: ```text overlap ratio chunks stored chars extra storage 0 0.00 125 100000 0% 80 0.10 139 111200 11% 160 0.20 156 124800 25% 240 0.30 179 143200 43% 400 0.50 249 199200 99% ``` A common starting point is **10–20%** overlap, or one sentence. Above about 30%, storage and duplicate results grow fast, while quality rarely improves much. Header-aware chunking often needs **no** overlap, because sections are already complete units. ## Chunk Granularity Tuning ### Over-Chunking vs Under-Chunking Risks - **Over-chunking** (chunks too small): pieces lose their subject ("It costs $3.99"), facts are spread across many chunks, and the index grows huge. - **Under-chunking** (chunks too large): the matching sentence is buried among unrelated text, embeddings become a blurry average of many topics, and fewer chunks fit in the prompt. ### Matching Chunk Size to Retrieval Queries Look at the questions your users ask: - **Short factual questions** ("How much is Plus?") work best with **smaller** chunks: one fact, one chunk. - **"How do I…" questions** need complete steps, so **medium** chunks, like one section. - **"Explain" or "compare" questions** need wider context, so **larger** chunks or parent-child retrieval. If your users ask all three kinds, use medium chunks plus parent-child retrieval (below). ### Chunk Size vs Vector Space Precision An embedding squeezes a whole chunk into one vector. A short, focused chunk gets a **sharp** vector that sits close to questions about its topic. A long chunk about five topics gets an **average** vector that sits somewhere in the middle, near none of them. Word density shows the same effect: ```js runnable title="Small, medium and large chunks for one question" // Chunk size vs precision: a big chunk dilutes the match, a tiny one loses context. const question = ["firmware", "update"]; const options = { tiny: "Update the lamp from Device settings.", medium: "Old firmware can also cause flickering. Update the lamp from Device settings, then Firmware.", huge: "Brightgrid lamps must not be plugged into a wall dimmer. A dimmer causes flickering. Use the app to dim instead. " + "Old firmware can also cause flickering. Update the lamp from Device settings, then Firmware. " + "Lamps and hubs update themselves at night. Keep the device powered during the update. The lamp stays cool to the touch.", }; for (const [name, text] of Object.entries(options)) { const words = text.toLowerCase().match(/[a-z]+/g); const hits = words.filter((w) => question.includes(w)).length; const covers = question.every((q) => words.includes(q)); console.log( `${name.padEnd(6)} ${String(words.length).padStart(3)} words, density ${(hits / words.length).toFixed(3)}, has both terms: ${covers}`, ); } ``` Expected output: ```text tiny 6 words, density 0.167, has both terms: false medium 14 words, density 0.214, has both terms: true huge 55 words, density 0.091, has both terms: true ``` The tiny chunk misses a key word. The huge chunk has both words, but they make up a much smaller share of its text. The medium chunk is the best match. ## Chunk Quality Verification ### Standalone Information Completeness The test for every chunk: **if someone read only this chunk, would it make sense?** It should name its subject, contain whole sentences, and not depend on a table or sentence somewhere else. ### Removing Fragmented Orphan Chunks **Orphans** are tiny leftovers, like "schedules." or "a minute.", usually from the end of a fixed-size cut. They match almost nothing and confuse the model when they do. **Merge** them into the previous chunk, or drop them if they are truly empty. ### Pronoun & Anaphora Resolution in Chunks **Anaphora** is a word that points back to something said earlier: "it", "this", "they", "the plan above". Inside a chunk that starts with "It also adds energy reports", nobody knows what "it" is. Fixes: - **Replace** the pronoun with its subject: "Brightgrid Plus also adds energy reports". - Add a **contextual header**: "[Brightgrid Plus > What you get]". - Keep the sentence with the one before it, so the subject is in the same chunk. ```js runnable title="Find and fix weak chunks" // Quality checks: find chunks that cannot stand on their own, then fix them. const chunks = [ "Plus costs $3.99 a month or $39 a year.", "It also adds energy reports and unlimited", "schedules.", "Cancel any time in Account, then Subscription.", ]; function problems(c) { const p = []; if (c.length < 20) { p.push("orphan (too short)"); } if (/^[a-z]/.test(c)) { p.push("starts mid-sentence"); } if (/^(it|this|that|they)\b/i.test(c)) { p.push("pronoun with no subject"); } return p; } chunks.forEach((c, i) => console.log(`#${i + 1} ${problems(c).join(", ") || "ok"} | ${c}`), ); // Fixes: merge orphans into the chunk before, and replace a leading pronoun with the subject. const fixed = []; for (const c of chunks) { if (c.length < 20 && fixed.length) { fixed[fixed.length - 1] += " " + c; } else { fixed.push(c.replace(/^It\b/, "Brightgrid Plus")); } } console.log("after fixes:"); fixed.forEach((c, i) => console.log(`#${i + 1} ${problems(c).join(", ") || "ok"} | ${c}`), ); ``` Expected output: ```text #1 ok | Plus costs $3.99 a month or $39 a year. #2 pronoun with no subject | It also adds energy reports and unlimited #3 orphan (too short), starts mid-sentence | schedules. #4 ok | Cancel any time in Account, then Subscription. after fixes: #1 ok | Plus costs $3.99 a month or $39 a year. #2 ok | Brightgrid Plus also adds energy reports and unlimited schedules. #3 ok | Cancel any time in Account, then Subscription. ``` ### Chunk Integrity Benchmarking Measure, do not guess. For each chunking setup, record: - **Gold-set hit rate:** does the right chunk appear in the top k? - **Warnings rate:** the share of chunks that are orphans, start mid-sentence or start with a pronoun. - **Size stats:** the number of chunks, and the average and largest size in tokens. Compare setups on the same **frozen snapshot**. Keep the setup with the best hit rate, as long as its size and cost stay acceptable. ## Two Useful Patterns ### Contextual Chunk Headers Add a short header with the document title and section path to each chunk before embedding it. You saw this in Lesson 5. It's the cheapest fix for pronoun and subject problems. Some teams go further and ask a language model to write a one-sentence summary of the chunk's place in the document. That works well, but it costs one model call per chunk. ### Parent-Child Chunks Search works best with **small** chunks, but the model answers best with **more context**. Parent-child chunking gives you both: 1. Split each section (the **parent**) into small **children**. 2. Embed and search the **children**. 3. When a child matches, send its **parent** to the model. ```js runnable title="Search small, send the parent" // Parent-child chunks: search small, precise children, but send the whole parent section. const parents = { "wifi#causes": "Most failures happen because the network is 5 GHz only, the password was typed wrong, or the lamp is too far from the router.", "wifi#fix": "Move the lamp within 5 metres of the router, check that a 2.4 GHz network is on, and reset the lamp by switching it off and on 5 times.", }; const children = [ { id: "c1", parent: "wifi#causes", text: "the network is 5 GHz only" }, { id: "c2", parent: "wifi#causes", text: "the password was typed wrong" }, { id: "c3", parent: "wifi#fix", text: "reset the lamp by switching it off and on 5 times", }, { id: "c4", parent: "wifi#fix", text: "check that a 2.4 GHz network is on" }, ]; const question = "how do i reset the lamp"; const q = new Set(question.split(" ")); const best = children .map((c) => ({ ...c, score: c.text.split(" ").filter((w) => q.has(w)).length })) .sort((a, b) => b.score - a.score)[0]; console.log("matched child:", best.id, `"${best.text}"`); console.log("sent to model:", parents[best.parent]); ``` Expected output: ```text matched child: c3 "reset the lamp by switching it off and on 5 times" sent to model: Move the lamp within 5 metres of the router, check that a 2.4 GHz network is on, and reset the lamp by switching it off and on 5 times. ``` Parent-child retrieval needs one more step: if two children from the same parent both match, send that parent **once**. Otherwise the prompt fills up with duplicates. Every change to chunking changes the chunk ids. Re-check your gold set's **reference chunks** after each change (Lesson 3), or your benchmark will report wrong results. ## Common mistakes - **No overlap with fixed-size chunks**, so answers that cross a boundary are lost. - **Too much overlap**, which doubles storage and fills results with near-duplicates. - **Chunks without a subject**, because of pronouns or missing headers. - **Leftover orphans** from the end of each document. - **Tuning without measuring.** Chunk settings must be judged with a gold set, not by eye. ## Exercises Each exercise ends with a value on the last line. Press **Check** to compare it with the expected answer. ## What to Remember - **Overlap** repeats the end of one chunk at the start of the next, so facts at a boundary stay whole. Start with **10–20%** or one sentence. - Too-small chunks lose their subject. Too-large chunks blur their meaning. **Match size to the questions** users ask. - Every chunk should **stand on its own**: merge **orphans**, and resolve leading **pronouns** with the subject or a header. - **Contextual headers** and **parent-child chunks** give precise search and enough context. - **Benchmark** chunk settings with a gold set on a frozen snapshot, and re-map reference chunks after each change. Next: **Chapter 3, Vector Embeddings**, how text becomes numbers that capture meaning.