# Token Prediction

Course: [Large Language Models](https://www.devtutorials.ai/learn/llm) · Section: Predictions
Source: https://www.devtutorials.ai/learn/llm/predictions/token-prediction

> Run a real language model in your browser and see its real next-token guesses, how context changes them, and how sure the model is.

So far, every model on this course was a small, made-up one. In this lesson, you will run a **real** language model, right inside your browser, and look directly at what it guesses.

You will type the start of a sentence. The model will tell you which tokens it thinks should come next, and how likely each one is.

**The main idea:** at every step, a real LLM gives a chance to every token in its vocabulary. The reply is built from those chances.

## A Real-World Example

Think about a weather forecast.

The forecaster does not just say "it will rain tomorrow". They say "70% chance of rain, 20% chance of cloud, 10% chance of sun". This is more useful, because it shows how sure they are.

A language model gives the same kind of forecast, but for text.

- **Tomorrow's weather** is the next token.
- **The forecast** is the list of tokens along with their chances.
- **"70% rain"** is a confident guess. **"35% rain, 33% sun"** is an unsure one.
- **Today's weather report** is the text you have already typed. It changes the forecast.

**In simple words:** an LLM does not know the next word. It forecasts it.

## What You Will Learn

In this lesson, you will learn:

1. How to run a real model in your browser.
2. How to read a list of chances for the next token.
3. How the text before changes the prediction.
4. How to tell a confident prediction from an unsure one.
5. How a chat model is different from plain text continuation.

The model runs inside the **AI sandbox**, which is a small panel at the bottom of every page in this course. You can open it with the **AI sandbox** button at the top of the lesson, or by pressing Ctrl and the backtick key together. It needs **WebGPU**, a feature found in recent desktop browsers like Chrome and Edge. The model downloads once, and every lesson reuses that same copy. If your browser does not have WebGPU, the sandbox will tell you and will not run anything. The code boxes on this page will work in any browser.

## The Model You Will Run

The sandbox starts with **Qwen2.5-0.5B-Instruct**. Here is what that name means:

- **Qwen2.5** is the name of the model family.
- **0.5B** means it has about half a billion parameters. That is small enough to run on a laptop.
- **Instruct** means it was fine-tuned to follow instructions, as you saw in the Model Training lesson.

The model runs using a library called **WebLLM**, which uses your computer's graphics chip through WebGPU. Nothing you type ever leaves your computer.

## Try the Model

Press the button below. The AI sandbox will open on its **Next token** tab. The first time you do this, press **Download and start the model** there. Once it is ready, the sandbox will predict the next token for the text shown below.

Here is how to read the **Next token** tab:

- Each row is one token the model might write next.
- The percentage is the model's chance of that token.
- The bar shows the chance, compared to the top token.
- The highlighted row is the token the model picks when it always chooses the most likely one.
- A dot (·) at the start of a token means a space, like `·Paris`.

## Things to Try

### Change the Context

Change the text and predict again. Try these:

- `The capital of France is`
- `The capital of Japan is`
- `My favourite fruit is`
- `def add(a, b):`

The list changes completely each time. The model reads **all** of the text before it makes a guess.

### Look at the Spaces

Most of the top tokens begin with a space, like `·Paris`. This is because the space belongs to the next word, as you saw in the Tokens chapter. Now end your text with a space, like `The capital of France is `, and predict again. The top tokens will change, because the space is already there.

### Find an Unsure Prediction

Try `I went to the`. Many places are possible, so the chances are spread out. No single token gets a big share.

## Working With Predictions

The sandbox shows real numbers from a real model. The code boxes below use small lists of chances, so you can practise the same ideas one step at a time.

### Pick the Most Likely Token

```js runnable title="Pick the token with the highest chance"
const chances = { " Paris": 0.82, " the": 0.05, " a": 0.03, " located": 0.02 };

let best = "";
let bestChance = 0;
for (const token in chances) {
  if (chances[token] > bestChance) {
    best = token;
    bestChance = chances[token];
  }
}

console.log("Most likely:", JSON.stringify(best), "with", bestChance * 100 + "%");
```

Expected output:

```text
Most likely: " Paris" with 82%
```

Always picking the most likely token is called **greedy decoding**. You will see other ways to pick a token in the Sampling chapter.

### Sort the Forecast

```js runnable title="Sort the tokens from most to least likely"
const chances = { " a": 0.03, " Paris": 0.82, " located": 0.02, " the": 0.05 };

const sorted = Object.entries(chances).sort((a, b) => b[1] - a[1]);
for (const [token, chance] of sorted) {
  console.log(JSON.stringify(token).padEnd(11), (chance * 100).toFixed(0) + "%");
}
```

Expected output:

```text
" Paris"    82%
" the"      5%
" a"        3%
" located"  2%
```

The sandbox shows its list in this same order.

### The Chances Do Not Add Up to 100% Here

The sandbox only shows the top 10 tokens. The real model has about 150,000 tokens in total. All the other tokens share whatever chance is left.

```js runnable title="What the other tokens share"
const top = [0.82, 0.05, 0.03, 0.02];

const shown = top.reduce((sum, chance) => sum + chance, 0);
const rest = 1 - shown;

console.log("Top 4 together:", (shown * 100).toFixed(0) + "%");
console.log("All other tokens:", (rest * 100).toFixed(0) + "%");
```

Expected output:

```text
Top 4 together: 92%
All other tokens: 8%
```

Across the **whole** vocabulary, the chances always add up to exactly 100%.

**In simple words:** the sandbox shows the top of the list, but every token still gets a small chance.

### How Sure Is the Model?

A simple way to measure how sure the model is: look at the chance of the top token.

```js runnable title="Confident or unsure?"
function describe(topChance) {
  if (topChance >= 0.7) return "confident";
  if (topChance >= 0.3) return "fairly sure";
  return "unsure";
}

console.log("The capital of France is ->", describe(0.82));
console.log("My favourite fruit is     ->", describe(0.21));
console.log("I went to the             ->", describe(0.09));
```

Expected output:

```text
The capital of France is -> confident
My favourite fruit is     -> unsure
I went to the             -> unsure
```

These chances are just examples. Check the real ones in the sandbox. A spread-out forecast often means many answers are possible. It can also mean the model does not really know the answer.

**Easy Way to Remember:** the model makes a **forecast**, not a decision. A tall top bar means "sure". A row of short bars means "not sure".

### Context Changes Everything

```js runnable title="Same last word, different forecast"
const forecasts = {
  "I drank a glass of": { " water": 0.41, " milk": 0.22, " wine": 0.12 },
  "I read a page of": { " the": 0.35, " a": 0.20, " my": 0.10 },
};

for (const text in forecasts) {
  const [top] = Object.entries(forecasts[text]).sort((a, b) => b[1] - a[1]);
  console.log(text, "->", JSON.stringify(top[0]));
}
```

Expected output:

```text
I drank a glass of -> " water"
I read a page of -> " the"
```

Both texts end with the word "of". The forecast is completely different, because the model reads everything that comes before it.

## Plain Text and Chat

The sandbox gives the model your text exactly as you typed it, and asks it to continue. This is called **plain text completion**.

A chat app works differently. It wraps your message inside the chat template from the Tokenization lesson, adding special tokens around it. The model then predicts the start of an **answer**, not the next word of your sentence.

- **Plain completion:** the model continues your sentence, often with something like " Paris".
- **Chat:** the model starts a reply to you, for example by finishing the fact in a full sentence, or by asking what you mean.

The model and the forecast mechanism are the same in both cases. Only the text the model sees is different.

Try it: type `The capital of France is` in the sandbox's **Chat** tab, and compare the reply with the **Next token** tab.

A small model like this one makes mistakes more often than large models. It may give a wrong fact a high chance. That is the whole point of this course: you are seeing a real model's real guesses, not a polished answer. Never treat a high chance as proof that something is true.

## Common mistakes

| Mistake | Wrong idea | Correct idea | What goes wrong |
|---|---|---|---|
| Treating the top token as a fact | "82% means it is 82% true." | 82% is how likely the token is to come next in text like this. | You trust a wrong but likely-sounding answer. |
| Expecting 100% in the top list | "The chances shown should add up to 100%." | Only the top tokens are shown. The rest of the vocabulary shares what is left. | You think the sandbox or the model is broken. |
| Ignoring the trailing space | "`is` and `is ` with a space give the same forecast." | The space is a token. It changes what can come next. | Predictions look strange after a trailing space. |
| Thinking the model sees only the last word | "It only looks at the word before." | It reads all the text before. Context changes the forecast. | You cannot explain why the same word gives different guesses. |
| Mixing up chat and completion | "A chat app sends my text as it is." | Chat apps add a template, so the model starts an answer. | You get different results in the sandbox and in a chat app. |
| Expecting big-model quality | "A 0.5B model should know everything." | Small models are fast and private but make more mistakes. | You judge all LLMs by a small model's errors, or the reverse. |

## Exercises

Each exercise ends with a value on the last line. Press **Check** to compare your answer with the expected one.

## What to Remember

- A real LLM gives a **chance to every token** at every step.
- The sandbox runs **Qwen2.5-0.5B-Instruct** in your browser using **WebLLM** and **WebGPU**.
- Always picking the top token is called **greedy decoding**.
- The chances of the **whole vocabulary** always add up to 100%.
- A tall top bar means **confident**. Short, even bars mean **unsure**.
- The model reads **all** of the text before, so the context changes the forecast.
- A high chance means **likely text**, not a **true fact**.

Next: **Logits**, the raw scores the model produces before they are turned into chances.