# Chunking Strategies

Course: [Retrieval-Augmented Generation](https://www.devtutorials.ai/learn/rag) · Section: Document Processing
Source: https://www.devtutorials.ai/learn/rag/document-processing/chunking-strategies

> The main ways to split documents for RAG: fixed-length by characters or tokens with sentence fallbacks, sentence, paragraph, section and header-aware chunking, and semantic chunking at topic shifts.

Cutting a pizza is easy: same-size slices, straight lines. Cutting a cake with layers and decorations is harder. If you slice without looking, one piece gets all the cream and another gets half a strawberry. A careful baker cuts along the natural lines of the cake, so every piece is complete and looks right on the plate.

**Chunking** is cutting documents into pieces for search. You can cut by size, like the pizza, or along the document's natural lines, like the cake.

**The main idea:** there are three families of chunking. **Fixed-length** cuts by characters or tokens. It's simple and predictable, but it cuts through sentences. **Structural** chunking cuts at sentences, paragraphs and headings. It follows the writer's own boundaries. **Semantic** chunking cuts where the topic changes, measured by similarity between sentences. The right choice depends on your documents and questions.

## A Real-World Example

Brightgrid first divided every article after a fixed number of characters. This sometimes cut useful information in the middle of a sentence or even in the middle of a word.

For example, a customer asked:

What should I do when the lamp blinks blue?

The retrieved text ended with:

In the app, press Add dev...

The rest of the instruction appeared in another chunk. Neither chunk contained the complete answer on its own.

To fix this, Brightgrid started dividing articles by headings and sentences. This approach is called structure-aware chunking.

Each section stays together when possible. If a section is too long, it is divided at a sentence boundary, and its heading is added to each resulting chunk.

The setup instructions now remain complete, making them easier for the search system to find and for the language model to understand.

In simple words:
 Split a document where one complete idea ends, not simply after a fixed number of characters.

## What You Will Learn

In this lesson, you will learn:

1. Fixed-length chunking by characters and tokens, its trade-offs, and sentence fallbacks.
2. Structural chunking: sentence boundaries, paragraphs and sections, and header-aware chunking.
3. Semantic chunking: topic segmentation, similarity boundaries and content-shift detection.

The RAG Lab's **Chunker** tab runs four strategies on any article or on the raw web page: **fixed**, **sentence**, **section** and **semantic**. Each chunk shows its length and warnings like "starts mid-sentence". Change the size and overlap sliders and watch the chunks change.

## Fixed-Length Chunking

### Character & Token Count Boundaries

The simplest method: take the next N characters, or the next N **tokens**, and repeat. Tokens matter more than characters, because models and costs are measured in tokens. For English, one token is about 4 characters, or 0.75 words.

```js runnable title="Fixed chunks of 50 characters"
// Fixed-length chunking by characters: simple, but it cuts words and sentences.
const text =
  "Plug in the lamp. It blinks blue when it is ready to pair. In the app, press Add device and follow the steps. Pairing usually takes under a minute.";

function fixedChunks(t, size) {
  const out = [];
  for (let i = 0; i < t.length; i += size) {
    out.push(t.slice(i, i + size));
  }
  return out;
}

fixedChunks(text, 50).forEach((c, i) => console.log(`#${i + 1} |${c}|`));
```

Expected output:

```text
#1 |Plug in the lamp. It blinks blue when it is ready |
#2 |to pair. In the app, press Add device and follow t|
#3 |he steps. Pairing usually takes under a minute.|
```

Look at the cuts: "ready |to pair", "follow t|he steps". Each piece is incomplete.

### Fixed Chunk Size Tradeoffs

| Size | Good | Bad |
| --- | --- | --- |
| **Small** (100–200 tokens) | Precise matches, cheap to send | Pieces lack context, more chunks to store |
| **Medium** (200–500 tokens) | A good default for help articles | Can mix two topics |
| **Large** (500–1,000+ tokens) | Full context, fewer chunks | Diluted matches, costly prompts, fewer chunks fit |

Fixed-length chunking is still useful. It's fast, predictable, and works on any text, even text with no structure, like transcripts or logs.

### Sentence Splitting Fallbacks

A common improvement: aim for a size, but **only cut at sentence ends**. Keep adding sentences until the next one would go over the limit, then start a new chunk. If a single sentence is longer than the limit, then cut it, as a last resort.

```js runnable title="Token limit, but never mid-sentence"
// Token-count chunks, with a fallback: never cut inside a sentence if you can avoid it.
const text =
  "Plug in the lamp. It blinks blue when it is ready to pair. In the app, press Add device and follow the steps. Pairing usually takes under a minute.";
const tokens = (s) => Math.ceil(s.split(/\s+/).length * 1.3); // rough: about 1.3 tokens per English word

function tokenChunks(t, maxTokens) {
  const sentences = t.match(/[^.!?]+[.!?]+/g).map((s) => s.trim());
  const out = [];
  let cur = "";
  for (const s of sentences) {
    const next = cur ? `${cur} ${s}` : s;
    if (cur && tokens(next) > maxTokens) {
      out.push(cur);
      cur = s;
    } else {
      cur = next;
    }
  }
  if (cur) {
    out.push(cur);
  }
  return out;
}

tokenChunks(text, 20).forEach((c, i) =>
  console.log(`#${i + 1} (${tokens(c)} tokens) ${c}`),
);
```

Expected output:

```text
#1 (17 tokens) Plug in the lamp. It blinks blue when it is ready to pair.
#2 (13 tokens) In the app, press Add device and follow the steps.
#3 (8 tokens) Pairing usually takes under a minute.
```

## Structural & Sentence Chunking

### Sentence-Boundary Chunking

Splitting into sentences sounds easy: cut at `.`, `?` and `!`. Real text has traps: "2.4 GHz", "e.g.", "Dr. Lee", "$3.99", and website names. Use a careful splitter: one that only cuts when the full stop is followed by a space or the end of the text, or a library built for sentence splitting.

### Paragraph & Section-Based Splitting

Writers already group ideas into **paragraphs** and **sections**. Splitting there keeps related sentences together. Paragraph chunks are often small; section chunks are often a good size for help articles.

```js runnable title="By paragraph and by section"
// Paragraph and section splitting: use the writer's own breaks.
const doc = `## Before you start
You need the Brightgrid app and a 2.4 GHz Wi-Fi network.

The Lumo Lamp does not support 5 GHz networks.

## Steps
Plug in the lamp. It blinks blue when it is ready.

## Troubleshooting
If it does not blink, switch it off and on 5 times.`;

const byParagraph = doc.split(/\n\s*\n/).map((p) => p.replace(/^## .*\n/, "").trim());
console.log("paragraphs:", byParagraph.length);
byParagraph.forEach((p, i) => console.log(`  #${i + 1} ${p}`));

const bySection = doc.split(/\n(?=## )/).map((s) => {
  const [head, ...rest] = s.split("\n");
  return { section: head.slice(3), text: rest.join(" ").replace(/\s+/g, " ").trim() };
});
console.log("sections:", bySection.length);
bySection.forEach((s, i) => console.log(`  #${i + 1} [${s.section}] ${s.text}`));
```

Expected output:

```text
paragraphs: 4
  #1 You need the Brightgrid app and a 2.4 GHz Wi-Fi network.
  #2 The Lumo Lamp does not support 5 GHz networks.
  #3 Plug in the lamp. It blinks blue when it is ready.
  #4 If it does not blink, switch it off and on 5 times.
sections: 3
  #1 [Before you start] You need the Brightgrid app and a 2.4 GHz Wi-Fi network. The Lumo Lamp does not support 5 GHz networks.
  #2 [Steps] Plug in the lamp. It blinks blue when it is ready.
  #3 [Troubleshooting] If it does not blink, switch it off and on 5 times.
```

### Header-Aware Structural Chunking

**Header-aware chunking** combines the best of both:

1. Split by **heading** first, so a chunk never mixes two sections.
2. If a section is too long, split it at **sentence** ends.
3. Add the **heading path** to every piece, so "Check that a 2.4 GHz network is on" still says it's from "Lamp will not connect to Wi-Fi > What to try".

The RAG Lab's index uses this strategy: one chunk per section, long sections split by sentences.

```js runnable title="Split by headings, then by sentences"
// Header-aware chunking: split by section, then split long sections by sentences,
// and keep the heading with every piece.
const doc = {
  title: "Lamp will not connect to Wi-Fi",
  sections: [
    {
      h: "Common causes",
      text: "Most failures happen because the network is 5 GHz only. The password may be wrong. The lamp may be too far away.",
    },
    {
      h: "What to try",
      text: "Move the lamp within 5 metres of the router. Check that a 2.4 GHz network is on.",
    },
  ],
};

function headerAware(d, maxChars) {
  const out = [];
  for (const s of d.sections) {
    let cur = "";
    for (const sent of s.text.match(/.+?\.(?=\s|$)/g).map((x) => x.trim())) {
      if (cur && (cur + " " + sent).length > maxChars) {
        out.push(`${d.title} > ${s.h}: ${cur}`);
        cur = sent;
      } else {
        cur = cur ? `${cur} ${sent}` : sent;
      }
    }
    out.push(`${d.title} > ${s.h}: ${cur}`);
  }
  return out;
}

headerAware(doc, 80).forEach((c, i) => console.log(`#${i + 1} ${c}`));
```

Expected output:

```text
#1 Lamp will not connect to Wi-Fi > Common causes: Most failures happen because the network is 5 GHz only.
#2 Lamp will not connect to Wi-Fi > Common causes: The password may be wrong. The lamp may be too far away.
#3 Lamp will not connect to Wi-Fi > What to try: Move the lamp within 5 metres of the router. Check that a 2.4 GHz network is on.
```

## Semantic Chunking

### Topic-Based Semantic Segmentation

Some text has no headings but still changes topic: a long FAQ page, a meeting transcript, a support chat. **Semantic chunking** reads it sentence by sentence and starts a new chunk when the **topic shifts**.

### Embedding Similarity Chunk Boundaries

To spot a topic shift, compare each sentence with the one before it. If they're similar, they belong together. If similarity drops sharply, cut there. Real systems compare **embeddings** (Chapter 3). The cell below uses shared words, which is simpler but works the same way.

### Content Shift Detection Mechanics

The steps:

1. Split the text into sentences.
2. Compute the similarity between each pair of neighbouring sentences.
3. Cut where similarity falls below a **threshold**, or where it drops much lower than usual. That second rule adapts to each document.
4. Merge pieces that come out too small, and split pieces that come out too large.

```js runnable title="Cut where the topic shifts"
// Semantic chunking: start a new chunk where the topic shifts.
// Here, "similar" means sharing words. Real systems compare embeddings instead.
const sentences = [
  "The Lumo Lamp needs a 2.4 GHz Wi-Fi network.",
  "Lumo Lamp pairing fails on 5 GHz Wi-Fi networks.",
  "Move the lamp closer to the Wi-Fi router.",
  "Brightgrid Plus costs $3.99 a month.",
  "Plus also adds energy reports.",
];
const stop = new Set(["the", "a", "on", "to", "also", "adds"]);
const words = (s) =>
  new Set(
    s
      .toLowerCase()
      .match(/[a-z0-9.]+/g)
      .filter((w) => !stop.has(w)),
  );

function similarity(a, b) {
  const A = words(a);
  const B = words(b);
  const shared = [...A].filter((w) => B.has(w)).length;
  return shared / Math.min(A.size, B.size);
}

const chunks = [[sentences[0]]];
for (let i = 1; i < sentences.length; i++) {
  const s = similarity(sentences[i - 1], sentences[i]);
  console.log(
    `sentence ${i} -> ${i + 1}: similarity ${s.toFixed(2)}${s < 0.2 ? "  <- topic shift, new chunk" : ""}`,
  );
  if (s < 0.2) {
    chunks.push([]);
  }
  chunks[chunks.length - 1].push(sentences[i]);
}
chunks.forEach((c, i) => console.log(`chunk ${i + 1}: ${c.join(" ")}`));
```

Expected output:

```text
sentence 1 -> 2: similarity 0.63
sentence 2 -> 3: similarity 0.50
sentence 3 -> 4: similarity 0.00  <- topic shift, new chunk
sentence 4 -> 5: similarity 0.33
chunk 1: The Lumo Lamp needs a 2.4 GHz Wi-Fi network. Lumo Lamp pairing fails on 5 GHz Wi-Fi networks. Move the lamp closer to the Wi-Fi router.
chunk 2: Brightgrid Plus costs $3.99 a month. Plus also adds energy reports.
```

Semantic chunking costs more, because every sentence must be embedded before you can chunk. It's worth it for long text without structure. For documents with good headings, header-aware chunking is usually just as good and much cheaper.

### Which Strategy to Choose

| Your documents | Good first choice |
| --- | --- |
| Help articles, docs and policies with headings | **Header-aware** (sections, then sentences) |
| Plain text, transcripts and logs | **Sentence-based** with a token limit, or **semantic** |
| Code | Split by **function** and **class**, never in the middle |
| Tables and spreadsheets | One row or a small group of rows per chunk, with column names |
| Anything, when you need a quick baseline | **Fixed-length** with overlap (next lesson) |

There's no single best chunk size. Measure it: run your **gold set** (Lesson 3) with two or three strategies and sizes, and keep the one that puts the right chunks in the top k most often.

## Common mistakes

- **Cutting mid-sentence** with fixed sizes, so answers are split across chunks.
- **Mixing two sections** in one chunk, which blurs its meaning and its embedding.
- **A naive sentence splitter** that breaks decimals, abbreviations and URLs.
- **Too-large chunks**, so the matching sentence is buried among unrelated text.
- **Too-small chunks**, which lose the subject, like "It costs $3.99."

## Exercises

Each exercise ends with a value on the last line. Press **Check** to compare it with the expected answer.

## What to Remember

- **Fixed-length** chunking (characters or tokens) is simple and works on anything, but cuts through sentences. Add a **sentence fallback**.
- **Structural** chunking follows sentences, paragraphs and sections. **Header-aware** chunking splits by heading, then by sentence, and keeps the heading path.
- **Semantic** chunking cuts where similarity between neighbouring sentences drops, which is good for long text without headings.
- Chunk size is a trade-off between **precision** and **context**. Measure it with a gold set.
- Use a careful sentence splitter, and choose a strategy per document type.

Next: **Chunk Optimization**, overlap, size tuning and checking chunk quality.