Context Windows in Context Engineering

By the Syncfusion DevTutorials team · Last updated October 20, 2026

How text becomes tokens, how big the context window is, why the reply needs its own budget, why facts in the middle of a long context get missed, and how sampling settings work with the context.

Think about packing a suitcase for a flight. The suitcase has a fixed size. You can't add more space by pushing harder. If you throw everything in, the one thing you need ends up buried at the bottom.

A model's context window works the same way. It has a fixed size. Everything must fit inside it: the instructions, the documents, the question, and the reply. Where you put each thing changes how easily the model finds it.

The main idea: the context window is a fixed space. You must plan what fits and where each fact goes.

A Real-World Example

Think about a teacher reading 30 student essays in one evening.

• The first few essays get full attention.
• The last few are fresh in the teacher's mind the next morning.
• The essays in the middle of the pile are easiest to forget.

The teacher read every essay. But some essays were remembered better. Just because of where they sat in the pile.

Language models behave in a similar way. A fact at the start or end of the context is usually used well. A fact buried in the middle is more likely to be missed.

In simple words: a fact needs room in the window and a good place in it.

What You Will Learn

In this lesson, you will learn:

1. How text is split into tokens and how to estimate a token count.
2. What a token limit is and why the reply needs its own budget.
3. How to share the window between the parts of a context.
4. Why facts in the middle of a long context are easier to miss.
5. What context rot is and the four ways a context can fail.
6. Where to put the most important facts.
7. How sampling settings work together with the context.
8. How to check that a context fits before you send it.

Context Window Basics

Tokenization

A model does not read letters or whole words. It reads tokens. A token is a small piece of text. It could be a whole short word, part of a long word, a space, or a symbol.

Tokenization is the step that splits text into tokens. Each token becomes a number called a token ID. The model only ever sees these numbers.

Try it. The box below uses a real tokenizer. Press Load the tokenizer. Then type any text and see how it is split.

Notice a few things:

• Common words are usually one token each.
• Rare words are split into several pieces.
• Spaces are often joined to the word after them.

You learned this in more detail in the Large Language Models course. Here, what matters is the count. Every token takes space in the window.

BPE Encoding

Most tokenizers today use byte pair encoding, or BPE. BPE builds its list of tokens from real text. One step at a time:

1. Start with single characters.
2. Find the pair of tokens that appears next to each other most often.
3. Join that pair into one new token.
4. Repeat many thousands of times.

The next box runs three steps of BPE on the words "low lower lowest".

BPE joins the most common pair again and again

let tokens = ["l", "o", "w", " ", "l", "o", "w", "e", "r", " ", "l", "o", "w", "e", "s", "t"];

function mostCommonPair(list) {
  const counts = {};
  for (let i = 0; i < list.length - 1; i++) {
    if (list[i] === " " || list[i + 1] === " ") continue;
    const pair = list[i] + "|" + list[i + 1];
    counts[pair] = (counts[pair] ?? 0) + 1;
  }
  return Object.entries(counts).sort((a, b) => b[1] - a[1])[0][0];
}

function merge(list, pair) {
  const [a, b] = pair.split("|");
  const out = [];
  for (let i = 0; i < list.length; i++) {
    if (list[i] === a && list[i + 1] === b) {
      out.push(a + b);
      i++;
    } else {
      out.push(list[i]);
    }
  }
  return out;
}

console.log("Start:", tokens.filter((t) => t !== " ").length, "tokens");
for (let step = 1; step <= 3; step++) {
  const pair = mostCommonPair(tokens);
  tokens = merge(tokens, pair);
  console.log("Merge " + step + ": " + pair.replace("|", " + ") + " -> " + tokens.filter((t) => t !== " ").join(" "));
}

Expected output:

Start: 14 tokens
Merge 1: l + o -> lo w lo w e r lo w e s t
Merge 2: lo + w -> low low e r low e s t
Merge 3: low + e -> low lowe r lowe s t

The 14 single letters became 6 tokens after three merges. "low" is common. So it became one token. A real tokenizer does this on huge amounts of text. That is why common English words become single tokens. Rare words or other languages need more tokens.

Token Counts

You often need a quick guess of how many tokens a text uses. A common rule of thumb for English is:

One token is about 4 characters.

A quick token estimate

const estimate = (text) => Math.ceil(text.length / 4);

const samples = [
  "Why did my export fail?",
  "Order 80000 failed at 2026-10-08 14:20.",
  "Brightgrid exports up to 5,000 rows on the Free plan.",
];

for (const text of samples) {
  console.log(text.length + " characters -> about " + estimate(text) + " tokens");
}

Expected output:

23 characters -> about 6 tokens
39 characters -> about 10 tokens
53 characters -> about 14 tokens

This is only an estimate. Numbers, code, dates, and non-English text often use more tokens than the rule says. Paste the same sentences into the tokenizer box above and compare. When the count matters, use the real tokenizer of your model.

Token Limits

The token limit is the size of the context window. It is the most tokens the model can handle in one request.

Limits differ a lot between models. The small model in your sandbox has a window of about 4,096 tokens. Large hosted models can handle hundreds of thousands. But every model has a limit.

The limit covers input and output together. The reply is written into the same window as the context. A full window leaves no room to answer.

Output Budgets

An output budget is the number of tokens you keep free for the reply. You set it with the max tokens setting.

Plan it like this:

room for input = window size − output budget

Does the input fit next to the reply?

const windowSize = 4096;
const outputBudget = 512;

function check(inputTokens) {
  const room = windowSize - outputBudget;
  if (inputTokens <= room) {
    return inputTokens + " input tokens: fits, " + (room - inputTokens) + " tokens to spare";
  }
  return inputTokens + " input tokens: too big by " + (inputTokens - room) + " tokens";
}

console.log("Room for input:", windowSize - outputBudget);
console.log(check(1200));
console.log(check(3900));

Expected output:

Room for input: 3584
1200 input tokens: fits, 2384 tokens to spare
3900 input tokens: too big by 316 tokens

3,900 tokens fit inside 4,096 on their own. But then only 196 tokens are left for the reply. A long answer would be cut off in the middle of a sentence.

Context Boundaries

A context boundary is the edge of the window. Anything outside the boundary doesn't exist for the model.

When a context is too big, something must be left out. Some apps cut the oldest messages without telling you. Others refuse the request with an error. Either way, you should decide what goes. Not leave it to chance.

The next box keeps the system message and the question. Then it adds chat history from the newest message back. It adds history until the budget is used.

Keep the newest history that fits

const tokens = (text) => Math.ceil(text.length / 4);

const system = "You are the support assistant for Brightgrid.";
const question = "Why did my export fail?";
const history = [
  "User: Hi, I need help with my invoice from March.",
  "Assistant: Sure. Your March invoice was paid on 2 April.",
  "User: Thanks. Now my export is failing.",
  "Assistant: I can help. How many rows are you exporting?",
  "User: About 80,000 rows.",
];

function fit(budget) {
  const kept = [];
  let used = tokens(system) + tokens(question);
  for (let i = history.length - 1; i >= 0; i--) {
    const cost = tokens(history[i]);
    if (used + cost > budget) break;
    kept.unshift(history[i]);
    used += cost;
  }
  return { kept: kept.length, dropped: history.length - kept.length, used };
}

console.log("Budget 80:", JSON.stringify(fit(80)));
console.log("Budget 40:", JSON.stringify(fit(40)));

Expected output:

Budget 80: {"kept":5,"dropped":0,"used":75}
Budget 40: {"kept":2,"dropped":3,"used":38}

With a budget of 40, the three oldest messages are dropped. Here that works well: they were about an invoice, not the export. The rules and the question are never dropped.

Resource Allocation

Resource allocation means deciding in advance how much of the window each part may use. It is like a budget for a school trip: so much for travel, so much for food, so much for tickets.

Share the window between the parts

const budget = 3000;

const plan = [
  { part: "System and safety rules", share: 0.10 },
  { part: "Retrieved documents", share: 0.50 },
  { part: "Conversation history", share: 0.25 },
  { part: "Tool outputs", share: 0.10 },
  { part: "Question", share: 0.05 },
];

let total = 0;
for (const p of plan) {
  const tokensForPart = Math.round(budget * p.share);
  total += tokensForPart;
  console.log(p.part + ": " + tokensForPart + " tokens");
}
console.log("Total:", total);

Expected output:

System and safety rules: 300 tokens
Retrieved documents: 1500 tokens
Conversation history: 750 tokens
Tool outputs: 300 tokens
Question: 150 tokens
Total: 3000

These shares are only an example. A document question app gives more to documents. A long chat app gives more to history. The point is to have a plan so one part can't quietly take all the space.

Attention Mechanics

Lost in the Middle

You might expect a model to use every token in its window equally well. Research shows it does not.

A 2023 paper by Liu and others called "Lost in the Middle" gave models a long list of documents. Only one document had the answer. The researchers moved that document to different places in the list. Models found the answer best when it was near the start or near the end. They did worst when it was in the middle.

In simple words: a big window is not the same as perfect attention. The model "has" a middle fact but it may not use it.

Information Positioning

Information positioning means choosing where in the context each part goes. It is a free way to improve answers. You don't add a single token. You only change the order.

Try it with the real model. The card below hides one fact among 80 filler lines and asks for it. It runs three times: with the fact at the start, in the middle, and at the end.

Look at the results table:

• Did the model find the code in all three places?
• Now make the context longer with the slider and run again. Does the middle fail first?

Your results depend on the model and the length. A small model may miss the code in several places. A strong model may find it everywhere at this length. That is the honest lesson: test position on your own model.

Position Effects

Two effects explain the pattern:

• Primacy effect. Things at the start get strong attention. That is why system instructions go first.
• Recency effect. Things just before the reply get strong attention too. The model writes its reply right after them.

The middle gets neither effect. The longer the context the bigger that weak middle becomes.

Recall Zones

Recall means how well the model finds and uses a fact. A recall zone is a part of the context where recall is good or bad.

Context Allocation

Context allocation joins two ideas: how much space each part gets and where it goes.

A good default order is:

1. Rules and instructions at the start.
2. Supporting documents in the middle, least useful first.
3. The most useful evidence near the end.
4. The question last.

Recall Optimization

Recall optimization means arranging the context so the model is most likely to use the right facts.

The next box sorts documents by a relevance score, a number from 0 to 1 that says how well each one matches the question. The best document goes last, right next to the question.

Put the best evidence next to the question

const docs = [
  { id: "doc-1", text: "Brightgrid themes can be changed in Grid settings.", score: 0.21 },
  { id: "doc-2", text: "Free plan: export up to 5,000 rows.", score: 0.93 },
  { id: "doc-3", text: "Keyboard shortcut Ctrl+F searches the grid.", score: 0.12 },
  { id: "doc-4", text: "Pro plan: export up to 1,000,000 rows.", score: 0.81 },
];

const ordered = [...docs].sort((a, b) => a.score - b.score);

console.log("Order sent to the model (question comes right after the last line):");
for (const d of ordered) console.log(d.id + " (" + d.score + ")");

Expected output:

Order sent to the model (question comes right after the last line):
doc-3 (0.12)
doc-1 (0.21)
doc-4 (0.81)
doc-2 (0.93)

The two weak documents sit in the middle where a miss costs little. The two strong ones sit in the recall zone at the end. Even better: leave the weak ones out entirely. Lesson 3 covers that.

Other ways to improve recall:

• Remove duplicates so the same text does not use space twice.
• Keep each document short and on one topic.
• For very long contexts, repeat the most important rule near the end.

Context Rot

Context rot is what happens to answer quality as a context gets longer and messier. Each extra token is one more thing that competes for the model's attention. Studies of long-context models report that accuracy often drops as input length grows even on simple tasks and well before the window is full.

So a 100,000-token window does not mean a 100,000-token context is a good idea. Keep contexts as short as the task allows. Lesson 3 shows how to choose.

Context Failure Types

When a context goes wrong, it usually goes wrong in one of four ways. Knowing their names helps you spot them.

You will meet a fix for each one in the coming lessons: selection in Lesson 3, formatting in Lesson 4, conflict rules in Lesson 6, and compact memory in Lesson 7.

Sampling Controls

The context decides what the model can say. Sampling settings decide how it picks each next token. You learned these in the Large Language Models course. Here is a short recap with one question in mind: how do they work with the context?

Temperature

Temperature controls how much randomness is used when the model picks each token. Low temperature makes the model pick its top choice almost every time. High temperature spreads the chances out.

Temperature changes the chances, not the context

function probabilities(logits, temperature) {
  const scaled = logits.map((x) => x / temperature);
  const max = Math.max(...scaled);
  const exps = scaled.map((x) => Math.exp(x - max));
  const sum = exps.reduce((a, b) => a + b, 0);
  return exps.map((x) => Math.round((x / sum) * 100) + "%");
}

const words = ["5,000", "10,000", "unlimited"];
const logits = [3.0, 1.5, 0.5];

console.log(words.join(" | "));
console.log("Temperature 0.2:", probabilities(logits, 0.2).join(" | "));
console.log("Temperature 1.0:", probabilities(logits, 1.0).join(" | "));
console.log("Temperature 2.0:", probabilities(logits, 2.0).join(" | "));

Expected output:

5,000 | 10,000 | unlimited
Temperature 0.2: 100% | 0% | 0%
Temperature 1.0: 77% | 17% | 6%
Temperature 2.0: 57% | 27% | 16%

Imagine the context says "5,000 rows" so the model scores "5,000" highest. At temperature 2.0 it still picks a wrong number about 4 times in 10. For factual answers from a context use a low temperature.

Top-P

Top-p keeps only the most likely tokens whose chances add up to p then picks from those. With top-p 0.9 the rare tokens that make up the last 10% are never picked.

Top-K

Top-k keeps only the k most likely tokens. With top-k 40 the model picks from its top 40 choices only.

Repetition Penalties

A repetition penalty lowers the chance of a token that was already used. It stops the model from repeating the same phrase in a loop. Set it too high and the model avoids words it really needs like a product name. The sandbox does not have this setting but most hosted APIs do.

Fixed Seeds

A seed is a number that fixes the random choices. The same seed the same context and the same settings give the same reply.

Deterministic Outputs

An output is deterministic when the same input always gives the same output. Use temperature 0 or a fixed seed while you test contexts. Then when a reply changes you know the context caused it not chance.

Context Limit Detection

Context limit detection means checking whether a context fits before you send it. If you don't check you may get an error, a cut-off reply, or an app that quietly drops part of your context.

A good check does three things:

1. Count the tokens of every part with the model's real tokenizer if possible.
2. Add the output budget.
3. Compare the total with the window size and decide what to trim if it is too big.

The exercises below build each step.

Common mistakes

Exercises

Each exercise ends with a value on the last line. Press Check to compare it with the expected answer.

What to Remember

• Text becomes tokens. Most tokenizers use BPE which joins common pairs into single tokens.
• A quick estimate is 4 characters per token in English. Use the real tokenizer when it matters.
• The token limit covers input and output together. Always keep an output budget.
• Plan how much of the window each part may use and decide yourself what to trim.
• Models use facts at the start and end best. The middle is the weak zone.
• Put rules first, the best evidence last, and the question at the very end.
• Long, messy contexts rot. Watch for poisoning, distraction, confusion, and clash.
• Use low temperature or a fixed seed when testing and check the size before you send.

Next: Context Selection, how to decide what to include, leave out, summarize, or fetch only when needed.