Embeddings

By the Syncfusion DevTutorials team · Last updated October 20, 2026

How a model turns each token into a list of numbers that carries meaning, how similarity between vectors is measured, and how real embeddings group similar text.

A token ID is just a number, such as 11 or 4,213. The number by itself has no meaning. Token 11 is not "smaller" than token 4,213 in any useful sense.

So the first thing a model does with each token ID is trade it for something much richer: a long list of numbers that describes what the token means. This list of numbers is called an embedding.

The main idea: an embedding turns a token into a point in space, and tokens with similar meanings end up close to each other.

A Real-World Example

Imagine a map of a city.

Every shop on the map has an address made of two numbers, like a latitude and a longitude. Shops of the same kind are often found near each other. All the cafés are on one street, close together. All the car garages sit together near the highway.

If you know two addresses, you can tell how close two shops are without ever going there.

Embeddings work the same way, like map addresses for words.

• A shop is a token or a small piece of text.
• Its address is the embedding, which is a list of numbers.
• Shops of the same kind that are nearby stand for words with similar meanings that are close together.
• Measuring the distance between addresses is how a computer compares the meanings of words.

A city map uses 2 numbers for each address. Real embeddings use hundreds or even thousands of numbers. The idea is still the same.

In simple words: an embedding is an address for meaning.

What You Will Learn

In this lesson, you will learn:

1. What an embedding is, and where it comes from.
2. Why similar words get similar numbers.
3. How to measure distance between two embeddings.
4. How cosine similarity compares directions.
5. How to find the nearest neighbours of a word.
6. How real sentence embeddings group text by meaning.

What Is an Embedding?

An embedding is a list of numbers that stands for a token or a piece of text. A list of numbers like this is also called a vector.

• Each number describes one hidden feature of the meaning.
• No person picks these features by hand. The model learns them by itself during training.
• Similar meanings end up with similar lists of numbers.

Below are four words with tiny two-number embeddings. You can think of the numbers as an x and a y position on a map.

Words as lists of numbers

const words = {
  cat:   [0.9, 0.8],
  dog:   [0.85, 0.75],
  car:   [0.1, 0.9],
  truck: [0.15, 0.95],
};

for (const word in words) {
  console.log(word.padEnd(6), words[word].join(", "));
}

Expected output:

cat    0.9, 0.8
dog    0.85, 0.75
car    0.1, 0.9
truck  0.15, 0.95

cat and dog have very similar numbers. car and truck also have very similar numbers. The animals and the vehicles are far apart from each other.

Where Embeddings Live in a Model

Inside the model, there is a large table called the embedding table. It has one row for every token in the vocabulary.

1. The tokenizer turns text into token IDs.
2. Each token ID picks its row from the embedding table.
3. That row, a list of numbers, is what the rest of the model works with.

Look up token IDs in the embedding table

const embeddingTable = [
  [0.0, 0.0, 0.0],   // row 0
  [0.9, 0.8, 0.1],   // row 1: "cat"
  [0.85, 0.75, 0.2], // row 2: "dog"
  [0.1, 0.2, 0.95],  // row 3: "car"
];

const tokenIds = [1, 3];
for (const id of tokenIds) {
  console.log("Token", id, "->", embeddingTable[id].join(", "));
}

Expected output:

Token 1 -> 0.9, 0.8, 0.1
Token 3 -> 0.1, 0.2, 0.95

A real model has more than 100,000 rows, and each row holds hundreds or even thousands of numbers. The numbers start out random. Training slowly moves them until similar tokens end up close together.

In simple words: the token ID is the row number, and the row itself is the embedding.

Measuring Closeness

To compare two embeddings, you need a way to measure how close they are. There are two common ways to do this.

Distance

The first way is the straight-line distance between two points, the way you would measure with a ruler on a map.

Straight-line distance

const cat = [0.9, 0.8];
const dog = [0.85, 0.75];
const car = [0.1, 0.9];

function distance(a, b) {
  let sum = 0;
  for (let i = 0; i < a.length; i++) {
    sum += (a[i] - b[i]) ** 2;
  }
  return Math.sqrt(sum);
}

console.log("cat to dog:", distance(cat, dog).toFixed(2));
console.log("cat to car:", distance(cat, car).toFixed(2));

Expected output:

cat to dog: 0.07
cat to car: 0.81

A small distance means the two meanings are close. cat is much closer to dog than to car.

Cosine Similarity

The second way is used most often with embeddings. It is called cosine similarity. It checks whether two vectors point in the same direction, and it does not care how long they are.

• 1 means they point the same way: very similar meaning.
• 0 means they are unrelated.
• Below 0 means they point in opposite directions.

Cosine similarity

function cosine(a, b) {
  let dot = 0, lengthA = 0, lengthB = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    lengthA += a[i] * a[i];
    lengthB += b[i] * b[i];
  }
  return dot / (Math.sqrt(lengthA) * Math.sqrt(lengthB));
}

const cat = [0.9, 0.7, 0.1];
const dog = [0.7, 0.8, 0.3];
const car = [0.1, 0.2, 0.95];

console.log("cat vs dog:", cosine(cat, dog).toFixed(2));
console.log("cat vs car:", cosine(cat, car).toFixed(2));

Expected output:

cat vs dog: 0.97
cat vs car: 0.29

cat and dog score 0.97, which is close to 1. cat and car score only 0.29.

In simple words: cosine similarity asks "do these two point the same way?"

Nearest Neighbours

With cosine similarity, you can find the words closest to any given word. These closest words are called its nearest neighbours.

Find the nearest neighbours

function cosine(a, b) {
  let dot = 0, lengthA = 0, lengthB = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    lengthA += a[i] * a[i];
    lengthB += b[i] * b[i];
  }
  return dot / (Math.sqrt(lengthA) * Math.sqrt(lengthB));
}

const vectors = {
  cat:    [0.9, 0.7, 0.1],
  dog:    [0.7, 0.8, 0.3],
  kitten: [0.95, 0.5, 0.1],
  car:    [0.1, 0.2, 0.95],
  bus:    [0.2, 0.1, 0.9],
};

const query = "kitten";
const scores = [];
for (const word in vectors) {
  if (word !== query) scores.push([word, cosine(vectors[query], vectors[word])]);
}
scores.sort((a, b) => b[1] - a[1]);

for (const [word, score] of scores) {
  console.log(word.padEnd(4), score.toFixed(2));
}

Expected output:

cat  0.98
dog  0.92
bus  0.33
car  0.28

kitten is closest to cat, and then to dog. The vehicles are far behind. This is the idea behind semantic search: finding text by meaning, not by matching the same words.

Directions Can Carry Meaning

Embeddings can do something surprising. Some directions in the space can carry meaning too. A well-known example: moving from man to woman is a similar step to moving from king to queen.

king - man + woman

const v = {
  king:  [0.9, 0.9, 0.1],
  man:   [0.5, 0.9, 0.1],
  woman: [0.5, 0.1, 0.1],
  queen: [0.9, 0.1, 0.1],
  apple: [0.1, 0.5, 0.9],
};

const result = v.king.map((x, i) => x - v.man[i] + v.woman[i]);
console.log("king - man + woman =", result.map((x) => x.toFixed(1)).join(", "));

let best = "";
let bestDistance = Infinity;
for (const word in v) {
  if (["king", "man", "woman"].includes(word)) continue;
  const d = Math.hypot(...v[word].map((x, i) => x - result[i]));
  if (d < bestDistance) {
    bestDistance = d;
    best = word;
  }
}
console.log("Closest word:", best);

Expected output:

king - man + woman = 0.9, 0.1, 0.1
Closest word: queen

These numbers were made up to show the idea clearly. Real embeddings also show this pattern, but not as neatly.

Try Real Embeddings

The lab below uses a real, small embedding model. It turns each sentence into a list of 384 numbers, and then compares each sentence with your question using cosine similarity.

Things to try:

• Look at the top result. Does it share words with your question, or only meaning?
• Change the question to "how much does it cost?" and compare again.
• Add a sentence in another language with the same meaning as your question.

This model makes one embedding for a whole sentence. That is different from the per-token embeddings inside an LLM. Both work on the same idea: similar meaning, similar numbers. Sentence embeddings like these power search and the RAG course later on.

Common mistakes

Exercises

Each exercise ends with a value on the last line. Press Check to compare it with the expected answer.

What to Remember

• An embedding is a list of numbers, a vector, that stands for meaning.
• Similar meanings get similar vectors that sit close together.
• Inside an LLM, each token ID picks a row from the embedding table.
• Cosine similarity compares directions: near 1 means close in meaning.
• Nearest neighbours are the vectors closest to a given one.
• Sentence embedding models make one vector per text, which powers search.
• Only compare embeddings that come from the same model.

Next: Attention, how each token looks at the other tokens to understand its context.