Top-K

By the Syncfusion DevTutorials team · Last updated October 20, 2026

How top-k sampling keeps only the k most likely tokens, cuts off the long tail of unlikely ones, and how it works together with temperature.

Temperature changes the shape of the chances, but every token still keeps a small chance. With about 150,000 tokens in the vocabulary, all those tiny chances add up. From time to time, sampling picks one of them, and the reply suddenly contains an odd word.

Top-k is the simplest fix: only the k most likely tokens are allowed. Everything else is cut out.

The main idea: top-k keeps the k best tokens, throws the rest away, and shares out their chance among the ones that are left.

A Real-World Example

Think about a talent show with a hundred singers.

The judges do not let all hundred into the final. They keep the top 5. Only those 5 can win. A singer in 6th place cannot win, even with a lucky night.

Inside the final, the public still votes. The best singer out of the 5 is likely to win, but the others still have a real chance.

Top-k works in the same way.

• The hundred singers are all the tokens in the vocabulary.
• Keeping the top 5 is top-k with k = 5.
• The public vote is sampling among the tokens that are left.
• 6th place never winning is the long tail being cut off.

In simple words: top-k says "only the best k tokens may be picked".

What You Will Learn

In this lesson, you will learn:

1. What the long tail is, and why it causes odd words.
2. How top-k keeps the k most likely tokens.
3. How the kept chances are shared out again.
4. Why k = 1 is the same as greedy decoding.
5. How top-k works together with temperature.

The Long Tail

Here is a forecast for the text "My favourite colour is". These are example chances.

• blue 32%
• green 20%
• red 15%
• purple 10%
• black 8%
• the 6%
• yellow 5%
• probably 4%

The tokens at the bottom of the list are the long tail. Each one by itself is unlikely. But together, they are not unlikely at all.

How much chance sits in the tail?

const chances = {
  " blue": 0.32, " green": 0.20, " red": 0.15, " purple": 0.10,
  " black": 0.08, " the": 0.06, " yellow": 0.05, " probably": 0.04,
};

const sorted = Object.entries(chances).sort((a, b) => b[1] - a[1]);
const top3 = sorted.slice(0, 3).reduce((sum, [, c]) => sum + c, 0);
const tail = sorted.slice(3).reduce((sum, [, c]) => sum + c, 0);

console.log("Top 3 together:", Math.round(top3 * 100) + "%");
console.log("All the rest:  ", Math.round(tail * 100) + "%");

Expected output:

Top 3 together: 67%
All the rest:   33%

About a third of the time, sampling picks a token outside the top 3. In a real model, the tail is about 150,000 tokens long, and some of those tokens make no sense in this place.

How Top-K Works

1. Sort the tokens from most to least likely.
2. Keep the first k tokens. Drop all the others.
3. Renormalize: divide each kept chance by their total, so they add up to 100% again.

Keep the top 3 and share out the chance

const chances = {
  " blue": 0.32, " green": 0.20, " red": 0.15, " purple": 0.10,
  " black": 0.08, " the": 0.06, " yellow": 0.05, " probably": 0.04,
};

function topK(chances, k) {
  const kept = Object.entries(chances).sort((a, b) => b[1] - a[1]).slice(0, k);
  const total = kept.reduce((sum, [, c]) => sum + c, 0);
  return kept.map(([token, c]) => [token, c / total]);
}

for (const [token, chance] of topK(chances, 3)) {
  console.log(JSON.stringify(token).padEnd(8), (chance * 100).toFixed(1) + "%");
}

Expected output:

" blue"  47.8%
" green" 29.9%
" red"   22.4%

The three kept tokens stay in the same order. Their chances grow, because the chance of the dropped tokens is shared out among them.

In simple words: cut the list at k, then make the rest add up to 100% again.

k = 1 Is Greedy

If you keep only one token, there is nothing left to choose. The top token wins every time. So top-k with k = 1 is greedy decoding.

Top-k with k = 1

const chances = { " blue": 0.32, " green": 0.20, " red": 0.15 };

function topK(chances, k) {
  const kept = Object.entries(chances).sort((a, b) => b[1] - a[1]).slice(0, k);
  const total = kept.reduce((sum, [, c]) => sum + c, 0);
  return kept.map(([token, c]) => [token, c / total]);
}

console.log(topK(chances, 1).map(([t, c]) => t + " " + c * 100 + "%").join(", "));

Expected output:

 blue 100%

The Effect on Sampling

The next code box samples 1,000 times, once without top-k and once with k = 3, and counts how often a tail token was picked.

How often does a tail token get picked?

function seededRandom(seed) {
  let a = seed >>> 0;
  return () => {
    a = (a + 0x6d2b79f5) >>> 0;
    let t = a;
    t = Math.imul(t ^ (t >>> 15), t | 1);
    t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
  };
}

const sorted = [
  [" blue", 0.32], [" green", 0.20], [" red", 0.15], [" purple", 0.10],
  [" black", 0.08], [" the", 0.06], [" yellow", 0.05], [" probably", 0.04],
];

function tailPicks(k) {
  const kept = sorted.slice(0, k);
  const total = kept.reduce((sum, [, c]) => sum + c, 0);
  const random = seededRandom(9);
  let tail = 0;
  for (let i = 0; i < 1000; i++) {
    let r = random() * total;
    let j = 0;
    while (r > kept[j][1]) { r -= kept[j][1]; j++; }
    if (j >= 3) tail++;
  }
  return tail;
}

console.log("No top-k (k = 8):", tailPicks(8), "tail picks out of 1000");
console.log("Top-k with k = 3:", tailPicks(3), "tail picks out of 1000");

Expected output:

No top-k (k = 8): 339 tail picks out of 1000
Top-k with k = 3: 0 tail picks out of 1000

With top-k = 3, a tail token can never be picked.

Top-K With Temperature

Temperature and top-k work together. The usual order is:

1. Apply temperature to the logits.
2. Run softmax.
3. Apply top-k.

A higher temperature spreads the chances out, which makes the tail bigger. Top-k then cuts that bigger tail. So you can raise the temperature for variety, and use top-k as a safety net.

Temperature 1.5, then top-k 3

const tokens = [" blue", " green", " red", " purple", " black"];
const logits = [3.0, 2.5, 2.2, 1.8, 1.6];

function chances(temperature, k) {
  const exps = logits.map((l) => Math.exp(l / temperature));
  const order = exps.map((e, i) => i).sort((a, b) => exps[b] - exps[a]).slice(0, k);
  const total = order.reduce((sum, i) => sum + exps[i], 0);
  return order.map((i) => tokens[i] + " " + Math.round(exps[i] / total * 100) + "%").join(", ");
}

console.log("T 1.5, all:  ", chances(1.5, 5));
console.log("T 1.5, k = 3:", chances(1.5, 3));

Expected output:

T 1.5, all:    blue 32%,  green 23%,  red 19%,  purple 14%,  black 13%
T 1.5, k = 3:  blue 43%,  green 31%,  red 25%

Try It in the AI Sandbox

Press the button to open the Sampling tab. Set Temperature to 1.5, then drag Top-k from 0 (off) down to 3, and then to 1. Watch the greyed rows grow as tokens are cut.

The Weakness of Top-K

Top-k always keeps exactly k tokens, no matter what the situation is.

• When the model is sure, like after "The capital of France is", most of the k tokens are bad choices. Keeping 40 of them lets nonsense in.
• When the model is unsure, like after "My favourite food is", many good choices exist. Keeping only 5 cuts good ones out.

A fixed k cannot be right for both cases. The next lesson, Top-P, fixes this by changing the size of the shortlist at every step.

Common mistakes

Exercises

Each exercise ends with a value on the last line. Press Check to compare your answer with the expected one.

What to Remember

• The long tail is the many unlikely tokens. Together, they can hold a lot of chance.
• Top-k keeps only the k most likely tokens.
• The kept chances are renormalized so they add up to 100%.
• k = 1 is greedy decoding.
• The usual order is temperature, then top-k.
• Top-k keeps the same number of tokens at every step, which is its weakness.
• WebLLM has no top_k setting. The sandbox applies it in the browser.

Next: Top-P, a shortlist that grows and shrinks based on how sure the model is.