Generation Flow

By the Syncfusion DevTutorials team · Last updated October 20, 2026

The full path from the prompt you type to the reply you read: chat messages, tokens, the model, picking a token, the loop, stopping, and streaming.

You type a question and press Enter. A short time later, the answer starts to show up, one small part at a time.

A lot is happening in that moment. This lesson will walk with your question through every step, from when you send it to the very last word of the reply.

The main idea: a reply is built in a loop. The model reads all the text, picks the next piece, adds it, and then reads everything again.

A Real-World Example

Imagine a friend who helps you finish a story, one word at a time.

You read the story out loud. Your friend hears the whole thing and says one new word. You write that word down. Then you read the whole story out loud again, with the new word added at the end. Your friend says the next word.

You keep doing this until your friend says "The end", or until the page is full.

An LLM works just like that friend.

• The story so far is your question plus the reply written so far.
• Your friend saying one word is the model picking the next piece.
• Writing it down is adding that piece to the text.
• "The end" or a full page are the two ways the reply can stop.

In simple words: the model writes a reply by repeating "read everything, add one piece" again and again until it is told to stop.

What You Will Learn

In this lesson, you will learn:

1. The six steps between your question and the reply.
2. How chat messages are joined into one piece of text.
3. Why the text is split into small pieces called tokens.
4. How the generation loop works, and the two ways it stops.
5. Why replies show up word by word.
6. Why the whole chat is sent again every time.

The Six Steps at a Glance

Here is the full path of one reply. Each step has its own section below.

1. Messages. Your question, plus any instructions, are put together as a list of messages.
2. Chat format. The messages are joined into one long piece of text.
3. Tokens. That text is split into small pieces called tokens.
4. Model. The model reads all the tokens and gives a chance to every possible next token.
5. Pick and add. One token is picked and added to the end. Then steps 4 and 5 run again.
6. Text out. When the loop stops, the tokens are turned back into text that you can read.

Step 1: Messages

A chat is more than just your question. It is a list of messages. Each message has a role that tells who wrote it.

• system: instructions from the app, like "You are a helpful support assistant. Keep answers short." You usually do not see this message.
• user: what you type.
• assistant: what the model replied earlier in the chat.

A chat is a list of messages

const messages = [
  { role: "system", content: "You are a helpful assistant. Keep answers short." },
  { role: "user", content: "What is an LLM?" },
];

for (const message of messages) {
  console.log(message.role + ":", message.content);
}

Expected output:

system: You are a helpful assistant. Keep answers short.
user: What is an LLM?

The system message sets the rules for the whole chat. The user message is the question.

In simple words: before anything else, your question is put into a list, right next to the app's instructions.

Step 2: Chat Format

The model cannot read a list. It can only read one long piece of text. So the messages are joined together, with special markers that show where each message begins and ends.

This joining is called the chat format, or chat template. Each model family uses its own markers. The idea is always the same.

Join the messages into one piece of text

const messages = [
  { role: "system", content: "Keep answers short." },
  { role: "user", content: "What is an LLM?" },
];

let text = "";
for (const message of messages) {
  text += "<|" + message.role + "|>" + message.content + "<|end|>\n";
}
text += "<|assistant|>";

console.log(text);

Expected output:

<|system|>Keep answers short.<|end|>
<|user|>What is an LLM?<|end|>
<|assistant|>

Look at the last line. The text ends with the assistant marker and nothing after it. That is an empty space. The model's job is to fill it in.

In simple words: the chat becomes a script with a blank line at the end for the assistant to "speak".

Step 3: Tokens

The model does not read letters or whole words. It reads tokens. A token is a small piece of text, often a whole word or a part of a word. Every token has a number, called its token ID.

The next box uses a tiny made-up list of tokens. It splits a sentence into words and looks up each word's ID.

Turn text into token IDs

const vocabulary = { "what": 1, "is": 2, "an": 3, "llm": 4, "?": 5 };

const text = "what is an llm ?";
const pieces = text.split(" ");
const ids = pieces.map((piece) => vocabulary[piece]);

console.log("Pieces:", pieces.join(" | "));
console.log("IDs:   ", ids.join(", "));

Expected output:

Pieces: what | is | an | llm | ?
IDs:    1, 2, 3, 4, 5

Real tokenizers are smarter than this. They split rare words into smaller parts, and a real list has tens of thousands of tokens. The Tokens chapter shows a real one.

In simple words: the model sees your text as a list of numbers.

Steps 4 and 5: The Generation Loop

Now the model gets to work. This is the loop from the LLM Basics lesson.

1. The model reads all the tokens so far.
2. It gives every possible next token a chance.
3. One token is picked.
4. That token is added to the end.
5. Go back to step 1.

Each pass through the loop adds exactly one token. A reply of 200 tokens needs 200 passes.

The Two Ways It Stops

The loop cannot run forever. It stops in one of two ways.

• The end token. The model has a special token that means "I am done". When the model picks it, the reply is finished. This is the normal way to stop.
• The maximum length. The app sets a limit on how many tokens the reply may have. This is often called max tokens. If the limit is reached, the reply stops, even in the middle of a sentence.

The next box has a pretend model. It always gives the same next word. The loop stops at the end token.

Loop until the end token

const nextWord = {
  "<|assistant|>": "An",
  "An": "LLM",
  "LLM": "predicts",
  "predicts": "text.",
  "text.": "<end>",
};

let last = "<|assistant|>";
const reply = [];

for (let step = 1; step <= 10; step++) {
  const word = nextWord[last];
  if (word === "<end>") {
    console.log("Step " + step + ": end token, stop");
    break;
  }
  reply.push(word);
  console.log("Step " + step + ": added", word);
  last = word;
}

console.log("Reply:", reply.join(" "));

Expected output:

Step 1: added An
Step 2: added LLM
Step 3: added predicts
Step 4: added text.
Step 5: end token, stop
Reply: An LLM predicts text.

The loop could have run 10 times. It stopped after step 5 because the model picked the end token.

Now look at the other way to stop. Here, max tokens is set to 2.

Stop at max tokens

const nextWord = {
  "<|assistant|>": "An",
  "An": "LLM",
  "LLM": "predicts",
  "predicts": "text.",
  "text.": "<end>",
};

const maxTokens = 2;
let last = "<|assistant|>";
const reply = [];

while (reply.length < maxTokens) {
  const word = nextWord[last];
  if (word === "<end>") break;
  reply.push(word);
  last = word;
}

console.log("Reply:", reply.join(" "));
console.log("Stopped because max tokens is", maxTokens);

Expected output:

Reply: An LLM
Stopped because max tokens is 2

The reply was cut off in the middle. If you ever see a reply that ends halfway through a sentence, the max tokens limit is the most likely reason.

Step 6: Text Out and Streaming

The tokens are numbers, so they are turned back into text. This is called decoding.

Apps do not wait for the whole reply. Each new token is shown as soon as it is picked. This is called streaming. It is why replies appear word by word, like someone typing.

Streaming does not make the model faster. It only lets you start reading sooner.

Why the Whole Chat Is Sent Again

Here is something that surprises many people. The model does not remember your chat. Every time you send a message, the app sends the whole chat so far again.

Each turn sends the full history

const history = [{ role: "system", content: "Keep answers short." }];

function sendTurn(question, answer) {
  history.push({ role: "user", content: question });
  console.log("Sending", history.length, "messages to the model");
  history.push({ role: "assistant", content: answer });
}

sendTurn("What is an LLM?", "A program that predicts text.");
sendTurn("Who makes them?", "Many companies and research groups.");
sendTurn("Can I run one at home?", "Yes, small ones.");

Expected output:

Sending 2 messages to the model
Sending 4 messages to the model
Sending 6 messages to the model

The list grows with every turn. The model reads all of it each time. This is why a very long chat can get slow, and why it can run out of room. The next lesson, Model Limits, explains that room.

In simple words: the model's "memory" is just the app sending the old messages again.

Common mistakes

Exercises

Each exercise ends with a value on the last line. Press Check to compare it with the expected answer.

What to Remember

• A chat is a list of messages with roles: system, user, and assistant.
• The messages are joined into one text with the chat format.
• The text is split into tokens, and each token has a number called a token ID.
• The generation loop reads all tokens, picks one, adds it, and repeats.
• The loop stops at the end token or at max tokens.
• Streaming shows each token as soon as it is picked.
• The model has no memory. The app sends the whole chat again on every turn.

Next: Model Limits, the size of the model's working space, and the other limits every LLM has.