Logits

By the Syncfusion DevTutorials team · Last updated October 20, 2026

What logits are, how softmax turns them into chances, why only the gaps between logits matter, log-probabilities, and nudging a token with logit bias.

In the last lesson, you saw the model's chances for the next token. But the model does not create those chances directly. First, it gives every token a raw score. These raw scores are called logits.

This lesson shows the real logits from the model in your browser, and how they turn into the chances you saw before.

The main idea: the model scores every token with a logit, and softmax turns those scores into chances. Only the gaps between the scores really matter.

A Real-World Example

Think about a talent show with three singers. The judges give each singer points: 8, 6, and 3.

Raw points are hard to compare between different shows. One show uses points out of 10, and another uses points out of 100. So the organisers turn the points into a "chance of winning the public vote": 80%, 18%, 2%.

• The points can be any number, even a negative one.
• The chances are always between 0% and 100%, and they add up to 100%.
• What matters is how far ahead each singer is, not the size of the raw points.

Logits work in the same way.

• The judges' points are the logits.
• Turning points into chances is called softmax.
• How far ahead each singer is is the gap between the logits.

In simple words: logits are raw scores. Softmax turns them into chances.

What You Will Learn

In this lesson, you will learn:

1. What a logit is, and where it comes from.
2. How softmax turns logits into chances, step by step.
3. Why adding the same number to every logit changes nothing.
4. Why bigger gaps between logits make the model more confident.
5. What log-probabilities are.
6. How logit bias can push a token up or down.

What Is a Logit?

At the end of the generation loop, after all the attention layers have done their work, the model has one final vector for the last token. It compares that vector with every token in its vocabulary and gives each one a score.

• A high logit means "this token fits well here".
• A low logit means "this token does not fit".
• Logits can be any size, positive or negative.
• There is one logit for every token in the vocabulary. For Qwen2.5, that is about 152,000 numbers at every single step.

See Real Logits

Press the button below. The Next token tab will show each top token with its logit and its chance.

Things to notice:

• The token with the highest logit always has the highest chance.
• A gap of just a few points between logits becomes a big gap in chances.
• The logits are not chances. They do not add up to anything special on their own.

From Logits to Chances: Softmax

You met softmax in the Attention lesson. Here it does the same job, but for the next token.

1. Take Math.exp() of every logit. This makes them all positive, and makes the big logits much bigger.
2. Add all those results together.
3. Divide each one by the total.

Softmax, step by step

const tokens = [" Paris", " the", " a", " located"];
const logits = [6.2, 3.1, 2.5, 1.0];

const exps = logits.map((l) => Math.exp(l));
const total = exps.reduce((a, b) => a + b, 0);

tokens.forEach((token, i) => {
  const chance = exps[i] / total;
  console.log(JSON.stringify(token).padEnd(11), "logit", logits[i], "->", (chance * 100).toFixed(1) + "%");
});

Expected output:

" Paris"    logit 6.2 -> 93.0%
" the"      logit 3.1 -> 4.2%
" a"        logit 2.5 -> 2.3%
" located"  logit 1 -> 0.5%

A logit of 6.2 compared to 3.1 is only about twice as big. But the chance is more than 20 times as big. Softmax turns small gaps into big differences.

Only the Gaps Matter

Here is a surprising fact. If you add the same number to every logit, the chances do not change at all.

Add 10 to every logit

function softmax(logits) {
  const exps = logits.map((l) => Math.exp(l));
  const total = exps.reduce((a, b) => a + b, 0);
  return exps.map((e) => (e / total * 100).toFixed(1) + "%");
}

const logits = [6.2, 3.1, 2.5, 1.0];
const shifted = logits.map((l) => l + 10);

console.log("Original:", softmax(logits).join("  "));
console.log("Plus 10: ", softmax(shifted).join("  "));

Expected output:

Original: 93.0%  4.2%  2.3%  0.5%
Plus 10:  93.0%  4.2%  2.3%  0.5%

The chances are exactly the same. Softmax only cares about how far apart the logits are. That is why a logit of 16 is not "better" than a logit of 6 on its own. It all depends on the other logits at the same step.

In simple words: a logit only means something when you compare it with the other logits.

Big Gaps, Sure Model

When the top logit is far ahead of the rest, the model is confident. When the logits are close to each other, the model is unsure.

Close logits vs one clear winner

function softmax(logits) {
  const exps = logits.map((l) => Math.exp(l));
  const total = exps.reduce((a, b) => a + b, 0);
  return exps.map((e) => (e / total * 100).toFixed(1) + "%");
}

console.log("Close:      ", softmax([3.0, 2.9, 2.8]).join("  "));
console.log("Clear winner:", softmax([9.0, 2.9, 2.8]).join("  "));

Expected output:

Close:       36.7%  33.2%  30.1%
Clear winner: 99.6%  0.2%  0.2%

In the Sampling chapter, you will learn how to change these gaps on purpose using a setting called temperature.

Log-Probabilities

Chances can get very small, like 0.0000003. Numbers that small are awkward to work with. So people often use the log-probability instead: the natural logarithm of the chance, which is Math.log(chance).

• A chance of 1 (100%) has a log-probability of 0.
• Smaller chances have more negative log-probabilities.
• You met this idea in Model Training: the loss is minus the log-probability of the right token.

Chances and their log-probabilities

for (const chance of [1, 0.82, 0.05, 0.001]) {
  console.log("chance", chance, "-> log-probability", Math.log(chance).toFixed(2));
}

Expected output:

chance 1 -> log-probability 0.00
chance 0.82 -> log-probability -0.20
chance 0.05 -> log-probability -3.00
chance 0.001 -> log-probability -6.91

Many AI services can return log-probabilities for each token they write. This is often called logprobs. It lets you see how sure the model was about each word of its answer.

Logit Bias

Because logits are just numbers, you can change them before softmax runs. Adding to a token's logit makes it more likely. Taking away from a token's logit makes it less likely. This is called logit bias.

Push one token up with logit bias

function softmax(logits) {
  const exps = logits.map((l) => Math.exp(l));
  const total = exps.reduce((a, b) => a + b, 0);
  return exps.map((e) => (e / total * 100).toFixed(1) + "%");
}

const tokens = [" Paris", " the", " a", " located"];
const logits = [6.2, 3.1, 2.5, 1.0];

const biased = [...logits];
biased[2] += 4;  // add 4 to the logit of " a"

console.log("Before:", softmax(logits).join("  "));
console.log("After: ", softmax(biased).join("  "));

Expected output:

Before: 93.0%  4.2%  2.3%  0.5%
After:  41.7%  1.9%  56.2%  0.2%

Adding 4 to one logit moved " a" from 2.3% all the way to the top. A very large negative bias, like −100, bans a token completely.

• Apps use logit bias to block words they never want to see.
• They also use it to favour a fixed set of answers, like "yes" and "no".
• WebLLM and many AI services accept a logit_bias setting that uses token IDs.

Common mistakes

Exercises

Each exercise ends with a value on the last line. Press Check to compare your answer with the expected one.

What to Remember

• A logit is the raw score the model gives each token.
• There is one logit for every token in the vocabulary, at every step.
• Softmax turns logits into chances that add up to 100%.
• Only the gaps between logits matter. Adding the same number to all of them changes nothing.
• A big gap means a confident model. Close logits mean an unsure one.
• A log-probability is the log of a chance. It is what logprobs returns.
• Logit bias pushes token IDs up or down before softmax.

Next: Generation Loop, where you watch the model repeat this step to write a full reply.