# Top-P

Course: [Large Language Models](https://www.devtutorials.ai/learn/llm) · Section: Sampling
Source: https://www.devtutorials.ai/learn/llm/sampling/top-p

> How top-p (nucleus) sampling keeps the smallest set of tokens whose chances add up to p, why it adapts to how sure the model is, and how all the settings work together.

Top-k keeps the same number of tokens at every step. That is too many when the model is sure, and too few when it is unsure. **Top-p** fixes this. Instead of a fixed number, it keeps a fixed **amount of chance**.

**The main idea:** top-p keeps the fewest tokens whose chances add up to at least **p**. When the model is sure, that is just one or two tokens. When it is unsure, it is many tokens.

## A Real-World Example

Think about packing a suitcase for a trip, where you must pack "enough clothes for 90% of the weather you might get".

For a trip to a desert, one kind of weather covers almost everything: hot and dry. You can pack light. For a trip to a rainy mountain town, the weather could be anything. You need clothes for sun, rain, wind, and cold.

The rule never changes: cover 90%. But the size of the suitcase changes with how sure the forecast is.

Top-p works in the same way.

- **"Cover 90% of the weather"** is top-p with p = 0.9.
- **The desert trip** is a step where the model is sure: just a few tokens cover 90%.
- **The mountain trip** is a step where the model is unsure: many tokens are needed to reach 90%.
- **The suitcase** is the shortlist of tokens that may be picked.

**In simple words:** top-p keeps "enough tokens to cover p of the chance", however many that turns out to be.

## What You Will Learn

In this lesson, you will learn:

1. How to add up chances from the top of the list.
2. How top-p picks the shortlist.
3. Why top-p adapts to how sure the model is.
4. How temperature, top-k, and top-p work together, and in what order.
5. How to choose settings for common tasks.

Top-p is a real WebLLM setting, so it works in the AI sandbox's **Chat** tab (you set it in **Settings**). The **Sampling** tab applies it to the real model's scores while you drag the slider, so you can see exactly which tokens it keeps.

## Adding Up the Chances

Top-p is also called **nucleus sampling**. The "nucleus" is the small core of tokens that holds most of the chance.

To find it, sort the tokens from most to least likely, and keep a running total.

```js runnable title="A running total of chances"
const sorted = [
  [" blue", 0.32], [" green", 0.20], [" red", 0.15], [" purple", 0.10],
  [" black", 0.08], [" the", 0.06], [" yellow", 0.05], [" probably", 0.04],
];

let total = 0;
for (const [token, chance] of sorted) {
  total += chance;
  console.log(JSON.stringify(token).padEnd(11), (chance * 100).toFixed(0).padStart(2) + "%", "running total", Math.round(total * 100) + "%");
}
```

Expected output:

```text
" blue"     32% running total 32%
" green"    20% running total 52%
" red"      15% running total 67%
" purple"   10% running total 77%
" black"     8% running total 85%
" the"       6% running total 91%
" yellow"    5% running total 96%
" probably"  4% running total 100%
```

## How Top-P Works

1. Sort the tokens from most to least likely.
2. Keep adding tokens until the running total reaches **p**.
3. Drop everything after that.
4. **Renormalize** the kept chances so they add up to 100% again.

```js runnable title="Top-p with p = 0.8"
const sorted = [
  [" blue", 0.32], [" green", 0.20], [" red", 0.15], [" purple", 0.10],
  [" black", 0.08], [" the", 0.06], [" yellow", 0.05], [" probably", 0.04],
];

function topP(sorted, p) {
  const kept = [];
  let total = 0;
  for (const [token, chance] of sorted) {
    if (total >= p) break;
    kept.push([token, chance]);
    total += chance;
  }
  return kept.map(([token, chance]) => [token, chance / total]);
}

for (const [token, chance] of topP(sorted, 0.8)) {
  console.log(JSON.stringify(token).padEnd(9), (chance * 100).toFixed(1) + "%");
}
```

Expected output:

```text
" blue"   37.6%
" green"  23.5%
" red"    17.6%
" purple" 11.8%
" black"  9.4%
```

After four tokens, the total is 77%, which is still below 80%. So " black" is added too, which brings the total to 85%. Then the loop stops.

**In simple words:** keep adding tokens until you have covered p of the chance, then stop.

## Top-P Adapts

This is why top-p is so popular. The same p keeps a different number of tokens, depending on how sure the model is.

```js runnable title="The same p on a sure step and an unsure step"
function countKept(chances, p) {
  const sorted = [...chances].sort((a, b) => b - a);
  let total = 0;
  let kept = 0;
  for (const c of sorted) {
    if (total >= p) break;
    total += c;
    kept++;
  }
  return kept;
}

const sure = [0.93, 0.04, 0.02, 0.01];
const unsure = [0.14, 0.13, 0.12, 0.11, 0.10, 0.10, 0.09, 0.08, 0.07, 0.06];

console.log("Sure step (The capital of France is):", countKept(sure, 0.9), "token kept");
console.log("Unsure step (My favourite food is):  ", countKept(unsure, 0.9), "tokens kept");
```

Expected output:

```text
Sure step (The capital of France is): 1 token kept
Unsure step (My favourite food is):   9 tokens kept
```

With p = 0.9:

- On the **sure** step, only the top token is kept. Nonsense cannot get in.
- On the **unsure** step, nine tokens are kept. Good variety is not cut.

A fixed top-k could not do both of these things at once.

**Easy Way to Remember:** **top-k is a fixed number of seats. Top-p is a fixed share of the votes.** Top-p's shortlist grows when the model is unsure and shrinks when it is sure.

## All the Settings Together

When several settings are used at once, they run in this order:

1. **Temperature** reshapes the logits.
2. **Softmax** turns them into chances.
3. **Top-k** keeps at most k tokens.
4. **Top-p** keeps the smallest set that reaches p.
5. The kept chances are **renormalized**, and one token is **sampled**.

```js runnable title="The full pipeline"
function pipeline(tokens, logits, temperature, k, p) {
  const exps = logits.map((l) => Math.exp(l / temperature));
  const total = exps.reduce((a, b) => a + b, 0);
  let list = tokens
    .map((token, i) => [token, exps[i] / total])
    .sort((a, b) => b[1] - a[1])
    .slice(0, k);

  const kept = [];
  let sum = 0;
  for (const [token, chance] of list) {
    if (sum >= p) break;
    kept.push([token, chance]);
    sum += chance;
  }
  return kept.map(([token, chance]) => token + " " + Math.round(chance / sum * 100) + "%");
}

const tokens = [" blue", " green", " red", " purple", " black", " the"];
const logits = [3.0, 2.5, 2.2, 1.8, 1.6, 0.5];

console.log("T 0.7, k 40, p 0.9:", pipeline(tokens, logits, 0.7, 40, 0.9).join(", "));
console.log("T 1.2, k 40, p 0.9:", pipeline(tokens, logits, 1.2, 40, 0.9).join(", "));
console.log("T 1.2, k 3,  p 0.9:", pipeline(tokens, logits, 1.2, 3, 0.9).join(", "));
```

Expected output:

```text
T 0.7, k 40, p 0.9:  blue 50%,  green 25%,  red 16%,  purple 9%
T 1.2, k 40, p 0.9:  blue 35%,  green 23%,  red 18%,  purple 13%,  black 11%
T 1.2, k 3,  p 0.9:  blue 46%,  green 30%,  red 24%
```

- At **T 0.7**, top-p keeps four tokens. " black" and " the" are cut.
- At **T 1.2**, the chances are flatter, so top-p needs five tokens to reach 90%.
- With **k = 3**, top-k cuts the list first, so only three tokens are left for top-p to look at.

Each line adds up to about 100%, give or take rounding.

## Try It in the AI Sandbox

Press the button to open the **Sampling** tab. Set **Top-k** to off (0) and drag **Top-p** from 1.0 down to 0.5. Watch how many tokens stay. Then try a different prompt where the model is sure, like "The capital of France is", and compare the two.

Then try it for real in **Chat**: open **Settings**, set temperature to 1.0, and compare replies at top-p 1.0 and top-p 0.5.

## Choosing Settings

These are common starting points. Always test with your own prompts.

| Task | Temperature | Top-p | Why |
|---|---|---|---|
| Data extraction, JSON, tests | 0 | 1.0 | Greedy and repeatable. Top-p does nothing at temperature 0. |
| Facts and code | 0.2 | 0.9 | A little room for variety, but no tail tokens. |
| Everyday chat | 0.7 | 0.9 | Natural variety, with a safe shortlist. |
| Stories and ideas | 1.0 | 0.95 | More variety, still no nonsense tail. |

The same top-p values, used on an unsure step, keep very different numbers of tokens:

```js runnable title="How many tokens each preset keeps on an unsure step"
function countKept(chances, p) {
  let total = 0;
  let kept = 0;
  for (const c of chances) {
    if (total >= p) break;
    total += c;
    kept++;
  }
  return kept;
}

const unsure = [0.14, 0.13, 0.12, 0.11, 0.10, 0.10, 0.09, 0.08, 0.07, 0.06];
const presets = { facts: 0.9, stories: 0.95, tight: 0.5 };

for (const name in presets) {
  console.log(name.padEnd(8), "p =", presets[name], "->", countKept(unsure, presets[name]), "tokens");
}
```

Expected output:

```text
facts    p = 0.9 -> 9 tokens
stories  p = 0.95 -> 10 tokens
tight    p = 0.5 -> 4 tokens
```

A common tip from AI providers is to change **either** temperature **or** top-p, not both at once. They do similar jobs, and changing both at the same time makes results hard to understand.

A very low top-p, like 0.1, keeps only the top token on most steps. That is almost greedy decoding, and long replies can start to repeat themselves. A top-p of 1.0 keeps every token, which means top-p is switched off.

## Common mistakes

| Mistake | Wrong idea | Correct idea | What goes wrong |
|---|---|---|---|
| Reading p as a share of tokens | "p = 0.9 keeps 90% of the tokens." | It keeps the fewest tokens that hold 90% of the chance. | You expect thousands of tokens, but often only a few are kept. |
| Very low p for stories | "p = 0.2 keeps it safe." | That is almost greedy. Text gets dull and may loop. | Stories repeat the same phrases. |
| Tuning temperature and p together | "Change both to get the best result." | They do similar jobs. Change one at a time. | You cannot tell which change helped. |
| Expecting p to matter at T = 0 | "Lower p makes temperature 0 safer." | At temperature 0, the top token always wins. Top-p changes nothing. | You tune a setting that has no effect. |
| Forgetting the order | "Top-p runs before temperature." | Temperature comes first, then top-k, then top-p. | Your own code gives different results from the library. |
| p = 1.0 as a filter | "p = 1.0 is the strictest." | p = 1.0 keeps everything. It switches top-p off. | The tail is not cut at all. |

## Exercises

Each exercise ends with a value on the last line. Press **Check** to compare your answer with the expected one.

## What to Remember

- **Top-p**, also called **nucleus sampling**, keeps the fewest top tokens whose chances add up to **p**.
- It works using a **running total** of chances from the top of the sorted list.
- The shortlist **grows** when the model is unsure and **shrinks** when it is sure.
- The order is **temperature → softmax → top-k → top-p → sample**.
- **p = 1.0** switches top-p off. A very low p is almost greedy.
- Usually change **temperature or top-p**, not both.
- At **temperature 0**, top-k and top-p make no difference.

Next: **Hallucinations**, where you will learn why a model can sound sure and still be wrong.