Model Training

By the Syncfusion DevTutorials team · Last updated October 20, 2026

How an LLM is made: pretraining on next-token guesses, loss and learning by small steps, fine-tuning on instruction examples, and learning from human preferences.

In the last two lessons, you met the parts inside an LLM: the embedding table, and the attention layers with their queries, keys, and values. All of these parts are made of numbers. A large model has billions of them.

Nobody types those numbers in. They are learned. This lesson explains how.

The main idea: a model learns by guessing the next token, checking how wrong it was, and nudging its numbers a tiny bit to do better next time. It does this billions of times.

A Real-World Example

Imagine learning to throw darts.

Your first throw misses the board. You see how far off it was, and you adjust your arm a little. The next throw is closer. You adjust again. After thousands of throws, you hit near the centre most of the time.

Nobody gave you a formula for the perfect throw. You got there by making many small corrections.

Training an LLM works the same way.

• A throw is one guess of the next token.
• How far you missed is called the loss.
• Adjusting your arm a little is changing the model's numbers by a tiny step.
• Thousands of throws are billions of guesses on huge amounts of text.

In simple words: the model learns by missing, measuring the miss, and correcting a little, over and over.

What You Will Learn

In this lesson, you will learn:

1. The three stages that turn random numbers into a chat assistant.
2. How pretraining makes practice questions from plain text.
3. What loss is, and how it is measured.
4. How small steps make the loss shrink.
5. How fine-tuning teaches a model to follow instructions.
6. How human preferences make a model more helpful.

The Three Stages

Most chat models are made in three stages.

1. Pretraining. The model reads a huge amount of text and learns to guess the next token. This is the longest and most expensive stage.
2. Instruction fine-tuning. The model studies example conversations, so it learns to answer like an assistant.
3. Preference tuning. People compare answers, and the model learns to give the kinds of answers people prefer.

The model you get after stage 1 is called a base model. After stages 2 and 3, it is called a chat model or instruct model.

Stage 1: Pretraining

Practice Questions From Plain Text

Pretraining needs no human labels. Any text can be turned into practice questions: hide the next word, and ask the model to guess it.

Turn a sentence into practice questions

const text = "the cat sat on the mat";
const words = text.split(" ");

for (let i = 1; i < words.length; i++) {
  const context = words.slice(0, i).join(" ");
  console.log(JSON.stringify(context), "->", words[i]);
}

Expected output:

"the" -> cat
"the cat" -> sat
"the cat sat" -> on
"the cat sat on" -> the
"the cat sat on the" -> mat

One short sentence gives five practice questions. Trillions of tokens of text give trillions of questions. That is why pretraining can use so much text.

Measuring the Miss: Loss

For each question, the model gives a chance to every possible next token. The loss measures how wrong the model was.

The loss most LLMs use is simple: take the chance the model gave to the right token, and calculate minus its logarithm, -Math.log(chance).

• If the model gave the right token a high chance, the loss is small.
• If it gave the right token a low chance, the loss is big.

Loss for different guesses

for (const chance of [0.9, 0.5, 0.1]) {
  const loss = -Math.log(chance);
  console.log("Chance for the right word:", chance, "-> loss:", loss.toFixed(2));
}

Expected output:

Chance for the right word: 0.9 -> loss: 0.11
Chance for the right word: 0.5 -> loss: 0.69
Chance for the right word: 0.1 -> loss: 2.30

A confident right guess costs almost nothing. A wrong guess costs a lot. Training tries to make the average loss as small as possible.

In simple words: loss is a score for "how surprised was the model by the right answer?"

Small Steps Downhill

After each batch of guesses, every number in the model is nudged a tiny bit in the direction that lowers the loss. This method is called gradient descent. The size of each nudge is the learning rate.

The next box trains a model with just one number, w. The model's job is to learn that the answer is always 2 times the input. It starts with a bad guess, w = 0.5.

Watch the loss shrink

const data = [[1, 2], [2, 4], [3, 6]];  // [input, right answer]
let w = 0.5;
const learningRate = 0.05;

for (let step = 1; step <= 6; step++) {
  let loss = 0;
  let slope = 0;
  for (const [x, y] of data) {
    const error = w * x - y;
    loss += error * error;
    slope += 2 * error * x;
  }
  loss /= data.length;
  slope /= data.length;
  w -= learningRate * slope;
  console.log("Step", step, "loss:", loss.toFixed(2), "w:", w.toFixed(2));
}

Expected output:

Step 1 loss: 10.50 w: 1.20
Step 2 loss: 2.99 w: 1.57
Step 3 loss: 0.85 w: 1.77
Step 4 loss: 0.24 w: 1.88
Step 5 loss: 0.07 w: 1.94
Step 6 loss: 0.02 w: 1.97

Every step, the loss shrinks and w moves closer to 2. A real LLM does the same thing with billions of numbers at once.

Try changing learningRate to 0.3 and run it again. With steps that are too big, the model overshoots, and the loss grows instead of shrinking.

What Pretraining Gives You

A base model is very good at continuing text. But it is not yet an assistant. Ask it a question, and it may just write more questions, because that is what often comes next in its text.

Stage 2: Instruction Fine-Tuning

To make a helpful assistant, the base model is trained some more on example conversations. Each example has a question and a good answer, written or checked by people. This is called instruction fine-tuning, or supervised fine-tuning (SFT).

The training is the same as before: guess the next token, measure the loss, take a small step. Only the data changes.

Fine-tuning examples are conversations

const examples = [
  {
    messages: [
      { role: "user", content: "Summarise: The meeting moved to Friday." },
      { role: "assistant", content: "The meeting is now on Friday." },
    ],
  },
  {
    messages: [
      { role: "user", content: "Translate 'thank you' to French." },
      { role: "assistant", content: "Merci." },
    ],
  },
];

console.log("Examples:", examples.length);
for (const example of examples) {
  const [question, answer] = example.messages;
  console.log("Q:", question.content, "| A:", answer.content);
}

Expected output:

Examples: 2
Q: Summarise: The meeting moved to Friday. | A: The meeting is now on Friday.
Q: Translate 'thank you' to French. | A: Merci.

Real fine-tuning sets have thousands to millions of examples like these. The model learns the chat format, when to stop, and how to answer instead of rambling.

Stage 3: Learning From Preferences

Many questions have more than one good answer. Some are clearer, safer, or more useful than others. To teach this, people are shown two answers and asked "which is better?".

People pick the better answer

const comparisons = [
  { a: "short and clear", b: "long and vague", picked: "a" },
  { a: "long and vague", b: "short and clear", picked: "b" },
  { a: "short and clear", b: "rude", picked: "a" },
];

const wins = {};
for (const c of comparisons) {
  const winner = c.picked === "a" ? c.a : c.b;
  wins[winner] = (wins[winner] || 0) + 1;
}

for (const style in wins) {
  console.log(style + ":", wins[style], "wins");
}

Expected output:

short and clear: 3 wins

From many such choices, the model is tuned to give more answers of the winning kind. One well-known way to do this is called RLHF, which stands for reinforcement learning from human feedback. Newer methods use the same kind of preference data in simpler ways.

After Training

When training ends, the numbers are frozen. This has some important results.

• The model does not learn from your chats while you use it.
• Its knowledge stops at the knowledge cutoff, the date its training text ends.
• To change the model, someone must train it again.

Common mistakes

Exercises

Each exercise ends with a value on the last line. Press Check to compare it with the expected answer.

What to Remember

• An LLM's numbers are learned, not written by hand.
• Pretraining turns plain text into next-token practice questions.
• Loss measures how wrong a guess was: -log(chance of the right token).
• Gradient descent nudges every number a little to lower the loss.
• The learning rate sets the step size. Too big, and training fails.
• Instruction fine-tuning on example conversations makes a chat model from a base model.
• Preference tuning, such as RLHF, teaches the model which answers people prefer.
• After training, the numbers are frozen. The model does not learn from your chats.

Next: Token Prediction, where you load a real model in your browser and watch it guess the next token.