Model Limits

By the Syncfusion DevTutorials team · Last updated October 20, 2026

The real limits of an LLM: the context window, output length, knowledge cutoff, model size and memory, speed, and cost, with small calculations for each.

Every tool has limits. A phone has a battery. A suitcase has a size. An LLM has limits too, and most of them come from one simple fact: the model can only read so much text at one time.

This lesson covers the limits you will meet most often. Each one comes with a small calculation, so you can check it yourself.

The main idea: an LLM works inside a fixed space. Your messages and its reply must fit in that space, and every token costs time and money.

A Real-World Example

Think about a whiteboard in a meeting room.

You write the meeting notes on it. As the meeting goes on, the board fills up. When it is full, you must erase the oldest notes to make room for new ones. Anything you erase is gone. Nobody in the room can read it any more.

An LLM has a whiteboard like that.

• The whiteboard is the model's context window.
• The notes are the system message, the chat history, your question, and the reply.
• Erasing old notes is what an app does when a chat gets too long.
• Gone for good is what happens to erased messages. The model cannot see them any more.

In simple words: the model can only think about what fits on its whiteboard right now.

What You Will Learn

In this lesson, you will learn:

1. What a context window is, and what fills it.
2. How to work out the room left for a reply.
3. What happens when a chat gets too long.
4. What the knowledge cutoff means for your answers.
5. How the size of a model decides how much memory it needs.
6. How to estimate the time and cost of a reply.

The Context Window

The context window is the largest number of tokens the model can read at one time. Everything must fit inside it.

Four things share the window:

• The system message. The app's rules for the chat.
• The chat history. Every earlier message, sent again on each turn.
• Your new question.
• The reply. The reply's tokens need room too.

Window sizes are different for each model. Small models often have 4,000 to 32,000 tokens. Large models can have 200,000 tokens or more.

Words and Tokens

Window sizes are measured in tokens, not words. For English, a useful rule of thumb is about 4 tokens for every 3 words.

Estimate tokens from words

const text = "Large language models read text as small pieces called tokens.";
const words = text.split(" ").length;
const tokens = Math.round(words * 4 / 3);

console.log("Words:", words);
console.log("About", tokens, "tokens");

Expected output:

Words: 10
About 13 tokens

This is only an estimate. Code, numbers, and other languages often use more tokens per word.

Room Left for the Reply

The reply must fit in what is left of the window. So the more you send, the less room the model has to answer.

How much room is left for the reply?

const contextWindow = 4096;

const systemTokens = 200;
const historyTokens = 2500;
const questionTokens = 300;

const used = systemTokens + historyTokens + questionTokens;
const left = contextWindow - used;

console.log("Used:", used, "tokens");
console.log("Left for the reply:", left, "tokens");

Expected output:

Used: 3000 tokens
Left for the reply: 1096 tokens

Only 1,096 tokens are left. That is enough for a short answer, but not for a long document.

In simple words: a long question leaves a short space for the answer.

When the Chat Gets Too Long

What if everything does not fit? Apps usually do one of two things.

1. Show an error, and ask you to shorten the text or start a new chat.
2. Drop the oldest messages until the rest fits. This is the whiteboard being erased.

The next box drops the oldest messages until the history fits in 2,000 tokens. Each number is the token count of one message, from oldest to newest.

Drop the oldest messages until it fits

const messageTokens = [400, 900, 1200, 800, 600];
const limit = 2000;

const kept = [...messageTokens];
let total = kept.reduce((sum, n) => sum + n, 0);

while (total > limit) {
  const removed = kept.shift();
  total -= removed;
  console.log("Dropped a message of", removed, "tokens");
}

console.log("Kept", kept.length, "messages, total", total, "tokens");

Expected output:

Dropped a message of 400 tokens
Dropped a message of 900 tokens
Dropped a message of 1200 tokens
Kept 2 messages, total 1400 tokens

The first three messages are gone. If your first message had an important detail, the model can no longer see it.

Max Output Tokens

There is a second limit for the reply alone. It is called max output tokens, or just max tokens. You met it in the Generation Flow lesson.

• It is set by the app, or by you when you call the model from code.
• The reply stops when it reaches this limit, even mid-sentence.
• The reply can never be longer than the room left in the context window.

The Knowledge Cutoff

The model learned from text collected up to a certain date. That date is its knowledge cutoff.

• It does not know about events after the cutoff.
• It may still answer questions about them, with a guess that sounds sure.
• If you need recent facts, put them into your message. The model can read and use text you give it.

Model Size and Memory

A model's size is the number of parameters it has. You will see sizes written like 0.5B, 7B, or 70B. The B means billion. So 7B means 7 billion parameters.

Every parameter is a number, and every number takes up memory. The memory needed is:

parameters × bytes per parameter

The bytes per parameter depend on how precisely each number is stored.

• 16-bit numbers use 2 bytes each. This is a common full-size format.
• 4-bit numbers use half a byte each. Storing numbers with fewer bits is called quantization. It makes the model about 4 times smaller, with a small loss in quality.

How much memory does a model need?

function memoryGB(billions, bits) {
  const bytesPerParameter = bits / 8;
  return billions * bytesPerParameter;
}

console.log("0.5B at 16-bit:", memoryGB(0.5, 16), "GB");
console.log("0.5B at 4-bit: ", memoryGB(0.5, 4), "GB");
console.log("7B at 16-bit:  ", memoryGB(7, 16), "GB");
console.log("7B at 4-bit:   ", memoryGB(7, 4), "GB");

Expected output:

0.5B at 16-bit: 1 GB
0.5B at 4-bit:  0.25 GB
7B at 16-bit:   14 GB
7B at 4-bit:    3.5 GB

The model you will run later in this course is small and stored in 4-bit, so it fits in a normal laptop's browser. A 7B model at 16-bit needs 14 GB just for its parameters. That is more than most laptops can spare.

In simple words: bigger models usually give better answers, but they need much more memory.

Speed and Cost

Every token takes time to make, and on paid services every token costs money.

Speed

Speed is measured in tokens per second. A small model in a laptop browser might make about 12 tokens per second. A big cloud service can make 50 to 100 or more.

How long will the reply take?

const replyTokens = 300;
const tokensPerSecond = 12;

const seconds = replyTokens / tokensPerSecond;
console.log("About", seconds, "seconds");

Expected output:

About 25 seconds

Long replies take longer. This is one more reason to ask for short answers when you do not need long ones.

Cost

Paid AI services charge per token. They usually charge a price per million tokens, and output tokens often cost more than input tokens.

The prices in the next box are made up for this example. Real prices change often, so always check the provider's price page.

How much does one request cost?

const inputPricePerMillion = 0.5;   // example price in dollars
const outputPricePerMillion = 1.5;  // example price in dollars

const inputTokens = 2000;
const outputTokens = 500;

const cost =
  (inputTokens / 1000000) * inputPricePerMillion +
  (outputTokens / 1000000) * outputPricePerMillion;

console.log("One request: $" + cost.toFixed(5));
console.log("10,000 requests: $" + (cost * 10000).toFixed(2));

Expected output:

One request: $0.00175
10,000 requests: $17.50

One request costs very little. Many requests add up. Remember that the whole chat history is sent again on every turn, so long chats cost more each time.

Common mistakes

Exercises

Each exercise ends with a value on the last line. Press Check to compare it with the expected answer.

What to Remember

• The context window is the most tokens a model can read at once.
• The system message, history, question, and reply all share that window.
• English is about 4 tokens for every 3 words.
• Long chats may drop old messages, so the model loses what was in them.
• Max tokens limits the length of the reply.
• The knowledge cutoff is the date the model's training text ends.
• Memory needed is parameters × bytes per parameter. 4-bit storage is about 4 times smaller than 16-bit.
• Every token costs time, and on paid services, money.

Next: Tokens, the small pieces of text that every LLM reads and writes.