State, Memory and Context Limits

By the Syncfusion DevTutorials team · Last updated October 27, 2026

An AI helper can only read so much of a long chat. Learn how a memory window, a summary, a sticky note and customer memory keep the important facts close, and keep them with the right customer.

State, Memory and Context Limits

Asha has been chatting with Shop Helper for a long time.

At message 3 she gave her order number.

"My order number is 1044, the rain jacket."

Then came small talk, a returns question and more small talk.

At message 60 she asks to send it somewhere new.

"Can you send it to 9 Hill Road instead?"

Shop Helper replies that the address is updated.

But it changed nothing in the shop. It no longer knew which order she meant, so it guessed.

This lesson teaches why a helper forgets, and how to keep the important facts close.

Simple lesson: the helper can only see so much, so keep the key facts where it can see them.

A Real-World Example

Think about a waiter on a busy night.

• A table orders drinks, then starters, then talks for an hour.
• The waiter cannot remember every word.
• So they write the key things on a small pad: table 4, no nuts, pay by card.
• At the end, the pad has the facts. The chat about the weather is gone.

A good waiter does not need every word. They need the right few facts.

Shop Helper works the same way. It needs the order number, not the small talk.

Simple lesson: write down the few facts you will need later.

What You Will Learn

In this lesson, you will learn:

1. Why a helper forgets the start of a long chat.
2. Why sending the whole chat every time gets expensive.
3. How a summary keeps old facts in a short note.
4. How a sticky note keeps key facts in every call.
5. How customer memory works between visits, and how it can leak.
6. What personal data memory may keep, and what it must never keep.

Old Notes Fall Off

Every time the customer writes, the app makes a new model call. The model does not remember the last call. The app must send the chat again.

But the model can only read so much at once.

Memory window = the recent part of the chat that the app sends on each call.

Real models have a size limit called the context window. You can read about it in Large Language Models. In this lab, the window holds the last 20 messages, and older ones drop out.

Press Play and watch the frame move along the chat.

• The small squares: each one is a message. The orange outline marks the key ones.
• The faded squares: they fell out of the frame. The helper cannot read them.
• Message 3: the order number drops out at message 23, and nobody notices.
• Message 60: the helper says it changed the address. The shop log shows nothing changed.

This is rule 5 of the pretend model. When the chat gets too long, the oldest messages drop out. Then rule 2 kicks in. If a fact is missing, the model guesses.

Simple lesson: what falls out of the window is forgotten, with no warning.

Why Not Send Everything?

Here is one quick fix. Send the whole chat with every call.

It works for a while. But every new message is added to every later call. You pay for the old small talk again and again.

Press Play and watch the bars grow.

• Whole chat: the bar gets longer with every call, because the chat keeps growing.
• Window plus sticky note: the bar stays about the same size.
• The answer: both get message 60 right. Only one gets cheaper as the chat goes on.

Real models also stop at their size limit. A very long chat may not fit at all.

A big call is also slower, because the model has more to read before it answers.

Many model providers offer prompt caching. When a call starts with the same text as an earlier call, that repeated start may cost less. It helps the bill. It does not let the model see more, because the size limit stays the same.

Simple lesson: sending everything costs more on every call, and it still has a limit.

A Summary Saves It

You do not need every old word. You need the facts inside them.

Summary = a short note that replaces old messages. It keeps the facts and drops the small talk.

The app asks the model to write the summary. Then it sends the summary plus the recent messages.

Press Play and watch forty old notes turn into one card.

• Before: forty messages are gone, and the order number went with them.
• The card: one short note now stands in for those forty messages.
• The answer: the helper finds order 1044 in the card and changes the right order.

A summary costs one extra model call to write. Many apps make a new one every few dozen messages.

A summary can also miss a detail, because a model wrote it. So check that it keeps the facts you care about.

Simple lesson: a summary trades many old messages for one short note.

A Sticky Note Saves It

A summary is good, but it can miss a detail. Some facts are too important to leave to chance.

Sticky note = a few key facts the app sends with every call, no matter how long the chat gets. It is also called working memory.

The app pins a fact as soon as it appears. Then the fact stays in sight even after its message drops out.

Press Play and watch the yellow note at the top.

• Pin it: at message 3 the app pins "Order: 1044".
• Message 30: message 3 is out of the frame. The note is still there.
• The answer: the helper changes the right order, and the shop log proves it.
• The cost: one short line adds only a few tokens to each call.

This one rule covers feelings and wishes too. Take the words "I am in a hurry". Is a final reply still to come? Then that step needs it, so pin it. If the only step left is a lookup, nothing uses it, so let it go.

The fact is the same. What changes is whether a step still to come will use it.

Simple lesson: pin the facts a later step will need, and only those.

Customer Memory

So far, everything is gone when the chat ends. Sometimes you want the helper to remember a customer next week too.

Customer memory = facts the app saves about one customer and loads again on their next visit. It is also called long-term memory.

Picture a filing cabinet with one drawer per customer. When a chat ends, the app saves facts into that customer's drawer. When a new chat starts, it opens the drawer again.

Press Play and watch the drawers.

• Save: Asha's chat ends. Her new address goes into her drawer.
• Ravi arrives: the app reads his customer id and opens his drawer only.
• The answer: Ravi asks about his last order and gets his own desk lamp.

Customer memory makes chats faster. The customer does not have to repeat themselves.

Save only what helps next time, such as a usual address or a last order. Do not save a whole chat, because the drawer would fill up with small talk.

Simple lesson: customer memory lasts between visits, one drawer per customer.

The Memory Leak Bug

Customer memory has one big danger. The app must load the drawer of the customer who is chatting, and nobody else's.

Here is a loader with a one-line bug. It never looks at the customer id. It always opens the top drawer.

Press Play and watch which drawer opens for Ravi.

• Wrong drawer: the top drawer is Asha's, so her facts load into Ravi's chat.
• The leak: Ravi's chat now holds Asha's home address.
• The answer: Ravi asks about his order and hears about Asha's rain jacket.

This app saved Asha's drawer before she moved. So it still says 12 Lake Street, not 9 Hill Road.

Personal Data in Memory

A customer's drawer holds facts about a real person. Privacy laws in many places set rules for that. The details differ by country, so check the rules where your shop sells.

Some safe habits:

• Keep only what helps. An order number is fine. So is a wish the customer shared, such as replies by email.
• Never keep secrets. Card numbers, passwords and ID numbers do not belong there. A leak would cause real harm.
• Tell customers. Say what the helper remembers about them.
• Delete on request. When a customer asks you to forget them, empty their drawer.
• Do not keep it forever. Keep a fact only as long as it is needed.

Simple lesson: load memory only for the customer who is chatting, and keep only what they would expect.

Where This Is Used

Many real AI helpers mix these ideas:

• Support chats often keep a short window plus a few pinned facts, such as the order number.
• Long assistants may write a summary every so often, so a long chat still fits.
• Shopping helpers may save preferences, such as sizes, in customer memory.

Each one has to choose what to keep close, what to shrink and what to let go.

A good test is a very long test chat. Put a key fact near the start and ask about it near the end. Then check what the helper did in the shop, not only what it said.

Simple lesson: real helpers usually combine a window, a summary, a sticky note and customer memory.

In Code

This code builds the window, the sticky note and customer memory. It uses the same Shop Helper names: Asha, Ravi, order 1044.

The first cell builds the prompt for a 60-message chat. The order number is only in message 3. Rough tokens are counted as words × 1.3.

Build the prompt with a window and a sticky note

type Msg = { who: "customer" | "helper"; text: string };
type Sticky = { [key: string]: string };
type Options = { window: number; sticky?: Sticky; summary?: string };

// A 60-message chat. The order number is in message 3 only.
const history: Msg[] = [];
for (let i = 1; i <= 60; i++) {
  let text = "Thanks, that makes sense, one more small question about the shop";
  if (i === 3) text = "My order number is 1044, the rain jacket";
  if (i === 60) text = "Can you send it to 9 Hill Road instead?";
  history.push({ who: i % 2 === 1 ? "customer" : "helper", text });
}

function roughTokens(text: string): number {
  const words = text.split(/\s+/).filter((w) => w.length > 0).length;
  return Math.round(words * 1.3);
}

function buildPrompt(all: Msg[], opts: Options): string[] {
  const lines: string[] = [];
  if (opts.sticky) lines.push("Sticky note: " + JSON.stringify(opts.sticky));
  if (opts.summary) lines.push("Summary: " + opts.summary);
  // Keep only the last N messages (the memory window).
  for (const m of all.slice(-opts.window)) lines.push(m.who + ": " + m.text);
  return lines;
}

function show(label: string, lines: string[]): void {
  const text = lines.join("\n");
  const sees1044 = text.indexOf("1044") >= 0;
  const size = lines.length + " lines, about " + roughTokens(text) + " tokens";
  console.log(label + ": " + size + ", sees 1044: " + sees1044);
  console.log("  first line -> " + lines[0]);
}

show("Everything", buildPrompt(history, { window: 60 }));
show("Window only", buildPrompt(history, { window: 6 }));
show("With sticky", buildPrompt(history, {
  window: 6,
  sticky: { customer: "C-101", order: "1044" },
  summary: "Asha asked about returns. No changes made yet.",
}));

Expected output:

Everything: 60 lines, about 930 tokens, sees 1044: true
  first line -> customer: Thanks, that makes sense, one more small question about the shop
Window only: 6 lines, about 91 tokens, sees 1044: false
  first line -> customer: Thanks, that makes sense, one more small question about the shop
With sticky: 8 lines, about 107 tokens, sees 1044: true
  first line -> Sticky note: {"customer":"C-101","order":"1044"}

• The window: slice(-opts.window) keeps the last few messages. Message 3 drops out, and the model never sees 1044.
• Sending everything: it keeps 1044, but it costs about ten times more on every call.
• The sticky note: two small facts ride along on every call. The prompt stays small, and 1044 is back.
• The summary line: one sentence stands in for the 50 or so messages that dropped out.

The second cell is the memory leak bug from above, in code. Ravi (C-102) is chatting.

Load customer memory by id

type Facts = { name: string; address: string; lastOrder: string };

// One drawer per customer, keyed by customer id.
const drawers = new Map<string, Facts>();
drawers.set("C-101", { name: "Asha", address: "12 Lake Street", lastOrder: "1044 rain jacket" });
drawers.set("C-102", { name: "Ravi", address: "4 Park Lane", lastOrder: "1043 desk lamp" });

// Bug: ignores the id and opens the first drawer.
function loadBuggy(customerId: string): Facts | undefined {
  for (const facts of drawers.values()) return facts;
  return undefined;
}

// Fixed: opens only this customer's drawer.
function loadScoped(customerId: string): Facts | undefined {
  return drawers.get(customerId);
}

function memoryLine(facts: Facts | undefined): string {
  if (!facts) return "Memory: none";
  return "Memory: " + facts.name + ", " + facts.address + ", last order " + facts.lastOrder;
}

const chattingNow = "C-102"; // Ravi
console.log("Buggy load  -> " + memoryLine(loadBuggy(chattingNow)));
console.log("Scoped load -> " + memoryLine(loadScoped(chattingNow)));
console.log("Unknown id  -> " + memoryLine(loadScoped("C-999")));

Expected output:

Buggy load  -> Memory: Asha, 12 Lake Street, last order 1044 rain jacket
Scoped load -> Memory: Ravi, 4 Park Lane, last order 1043 desk lamp
Unknown id  -> Memory: none

• The bug: loadBuggy never uses the id. It opens the first drawer, which is Asha's.
• The fix: loadScoped opens the drawer for this id only. An unknown id gets no memory, not someone else's.

In a Real App

A real app keeps customer memory in a database, and asks the model to write the summary. The code below is simplified. Names differ between providers, databases and SDK versions, so it uses small interfaces of its own.

interface ModelClient {
  complete(prompt: string): Promise<string>;
}

interface MemoryStore {
  get(customerId: string): Promise<CustomerFacts | null>;
  put(customerId: string, facts: CustomerFacts): Promise<void>;
  delete(customerId: string): Promise<void>; // "forget me" requests
}

type CustomerFacts = { usualAddress?: string; lastOrder?: string; notes: string[] };

// Ask the model to shrink old messages into one short note.
async function summarise(model: ModelClient, oldMessages: string[]): Promise<string> {
  return model.complete(
    "Summarise this support chat in 2 sentences. Keep order numbers, " +
      "addresses and anything promised.\n\n" + oldMessages.join("\n"),
  );
}

// Never store card numbers: mask long digit runs before saving.
function redact(text: string): string {
  return text.replace(/\b(?:\d[ -]?){13,19}\b/g, "[card removed]");
}

async function saveNote(store: MemoryStore, customerId: string, note: string): Promise<void> {
  const facts = (await store.get(customerId)) ?? { notes: [] };
  facts.notes.push(redact(note));
  await store.put(customerId, facts);
}

async function forgetCustomer(store: MemoryStore, customerId: string): Promise<void> {
  await store.delete(customerId);
}

The customer id comes from the login, not the chat text. A customer could type any id. A simple pattern like this one catches most card numbers, but not every secret. So save only the few fields you need.

Simple lesson: in code, memory is a prompt builder plus a store read by the logged-in customer's id.

TL;DR

1. Memory window = the recent part of the chat the app sends on each call.
2. Old messages = drop out of the window, and the helper forgets them with no warning.
3. Sending everything = costs more on every call and still hits a size limit.
4. Summary = a short note that replaces old messages and keeps their facts.
5. Sticky note (working memory) = key facts sent with every call.
6. Customer memory (long-term memory) = facts saved per customer and loaded on the next visit.
7. The safety rule = load memory only for the customer who is chatting.
8. Personal data = keep only what helps, never secrets, and delete it when the customer asks.

Common mistakes

Check Your Understanding

Exercises

What to Remember

• The helper only reads its memory window. Older messages drop out with no warning.
• Sending the whole chat gets more expensive with every call.
• A summary keeps old facts in one short note.
• A sticky note keeps key facts in every call. Pin only what a later step needs.
• Customer memory lasts between visits, one drawer per customer.
• Always load memory for the customer who is chatting, and nobody else.
• Never keep secrets in memory. Tell customers what is kept, and delete it when they ask.

Next: Guardrails, Permissions and Human Approval, which actions are never allowed, and which need a person's OK.