Should You Fine-tune at All?

By Syncfusion®, Inc. · Last updated November 2, 2026

Four requests that all sound like fine-tuning. Only two need it. Learn to name the gap, climb the customization ladder from the cheapest rung, and count what a fine-tune costs to own.

Should You Fine-tune at All?

Your team runs Shop Helper, the chat helper in an online shop.

One morning, four requests land on your desk. Each one ends with the same idea: "Let's fine-tune a model."

Most teams say yes too fast. Then they spend weeks on the wrong fix.

Watch one team do exactly that.

The team was not lazy. They worked hard for six weeks. They just picked a tool that cannot do the job: the policy changes every month, and a fine-tune freezes it.

Simple lesson: fine-tuning is a powerful tool for a small set of problems. Find out which problem you have first.

A Real-World Example

Think about a new employee at a help desk.

If they do not know today's opening hours, you do not send them on a three-week course. You give them the page with the hours. They look it up.

If they know everything but answer customers rudely, a page will not help. You train them, with examples of good replies, until the new habit sticks.

A language model is the same. Missing facts are fixed by looking things up. Wrong habits are fixed by training.

Simple lesson: look up facts, train habits.

What You Will Learn

In this lesson, you will learn:

1. What fine-tuning changes inside a model, and what it does not.
2. The four kinds of gap: knowledge, behaviour, cost and eval.
3. The customization ladder, and why you start at the bottom.
4. What a fine-tune costs to own for a year, not just to build.
5. How a tiny model learns a new habit in your browser with LoRA.

TL;DR

Fine-tuning = training an existing model a little more, on your own examples, so it changes how it answers.

It is good at behaviour: a fixed format, a house tone, a narrow task done cheaply.

It is bad at facts that change. Those belong in RAG.

Always try the cheaper rungs first: a better prompt, examples in the prompt, a JSON schema, then RAG. Fine-tune only when the eval shows those stopped improving.

What Fine-tuning Changes

A language model is a very large list of numbers, called weights. They decide what it writes next.

Pre-training = how the model was first made. It read a huge amount of text and set its weights.

Fine-tuning = training it a bit more, on a small set of your examples. Some weights move. The model's habits change.

LoRA (Low-Rank Adaptation) = a cheap way to fine-tune. The original weights stay frozen. You train a small adapter, often about 1% of the size, that sits beside them.

Here is the key idea. Training on examples is good at teaching how to answer. It is unreliable at storing what is true, and it cannot update itself when the truth changes.

Simple lesson: fine-tuning teaches form, not facts.

Name the Gap First

Before you pick a tool, name the gap. There are four kinds.

• Knowledge gap: the model does not know a fact, or the fact changes.
• Behaviour gap: the model knows enough, but the format, tone or style is wrong.
• Cost or speed gap: the answers are fine, but too slow or too expensive at your volume.
• Eval gap: nobody can say what "better" means yet, so nothing can be measured.

Watch the four requests get sorted.

Only two of the four end with training a model. The behaviour gap may need a LoRA fine-tune. The cost gap may need distillation: a small model trained on a big model's answers.

The other two never needed training at all.

Simple lesson: "it should be better" is not a gap. Name which kind of better.

The Customization Ladder

Now you know the gap. Which tool do you use?

Think of the tools as a ladder. The bottom rungs are cheap and quick. Each step up costs more to build and more to keep running.

The rule is simple: stop at the first rung that meets your target.

Watch the team climb for the JSON ticket request. The target is 95% of tickets in the exact format and tone.

A better prompt gets 62%. Adding examples gets 79%.

Structured output gets 88%. You hand the model your JSON schema, and every reply now parses with every field. But the tone and some values, like the priority, are still wrong. A schema fixes the shape, not the content. Exhaust the Cheaper Rungs First shows how to set it up.

RAG gets 80%, almost no gain, because the problem was never missing facts.

The LoRA fine-tune gets 96%. That meets the target, so the team stops there.

Now look at the refund policy request. The LoRA fine-tune scored 88% in its first month. Next month, after the policy changed, it fell to 34%. RAG scored 93% and stayed high, because it reads the current page every time.

Simple lesson: climb one rung at a time, and measure at every rung.

Climb the Ladder in Code

The ladder is a loop. Go through the rungs from cheapest to most expensive. Stop at the first one that meets the target, this month and next month.

type Rung = "prompt" | "few-shot" | "schema" | "rag" | "lora" | "full";
type Result = { rung: Rung; score: number; nextMonth?: number };

// Measured results for one request, cheapest rung first.
// A rung nobody tried is simply not in the list.
const jsonTicket: Result[] = [
  { rung: "prompt", score: 62 },
  { rung: "few-shot", score: 79 },
  { rung: "schema", score: 88 },
  { rung: "rag", score: 80 },
  { rung: "lora", score: 96 },
  { rung: "full", score: 96 },
];
const refundPolicy: Result[] = [
  { rung: "prompt", score: 41 },
  { rung: "few-shot", score: 46 },
  { rung: "rag", score: 93, nextMonth: 92 },
  { rung: "lora", score: 88, nextMonth: 34 },
];

// Climb the ladder: stop at the first rung that meets the target,
// this month AND next month (if we measured next month).
function decide(results: Result[], target: number): string {
  for (const r of results) {
    const stillGood = r.nextMonth === undefined || r.nextMonth >= target;
    if (r.score >= target && stillGood) return `${r.rung} (${r.score}%)`;
    console.log(`  ${r.rung.padEnd(8)} ${r.score}% -> not enough, climb`);
  }
  return "nothing works yet: improve the eval or the data";
}

console.log("JSON ticket, target 95%");
console.log("  choose:", decide(jsonTicket, 95));
console.log("Refund policy, target 90%");
console.log("  choose:", decide(refundPolicy, 90));

// Measured results for one request, cheapest rung first.
// A rung nobody tried is simply not in the list.
var jsonTicket = new List<Result>
{
    new("prompt", 62),
    new("few-shot", 79),
    new("schema", 88),
    new("rag", 80),
    new("lora", 96),
    new("full", 96),
};
var refundPolicy = new List<Result>
{
    new("prompt", 41),
    new("few-shot", 46),
    new("rag", 93, NextMonth: 92),
    new("lora", 88, NextMonth: 34),
};

// Climb the ladder: stop at the first rung that meets the target,
// this month AND next month (if we measured next month).
string Decide(List<Result> results, int target)
{
    foreach (var r in results)
    {
        bool stillGood = r.NextMonth == null || r.NextMonth >= target;
        if (r.Score >= target && stillGood) return $"{r.Rung} ({r.Score}%)";
        Console.WriteLine($"  {r.Rung.PadRight(8)} {r.Score}% -> not enough, climb");
    }
    return "nothing works yet: improve the eval or the data";
}

Console.WriteLine("JSON ticket, target 95%");
Console.WriteLine($"  choose: {Decide(jsonTicket, 95)}");
Console.WriteLine("Refund policy, target 90%");
Console.WriteLine($"  choose: {Decide(refundPolicy, 90)}");

// Rung is one of: "prompt", "few-shot", "schema", "rag", "lora", "full"
record Result(string Rung, int Score, int? NextMonth = null);

Expected output:

JSON ticket, target 95%
  prompt   62% -> not enough, climb
  few-shot 79% -> not enough, climb
  schema   88% -> not enough, climb
  rag      80% -> not enough, climb
  choose: lora (96%)
Refund policy, target 90%
  prompt   41% -> not enough, climb
  few-shot 46% -> not enough, climb
  choose: rag (93%)

The full fine-tune also scored 96%. The loop never reaches it, because LoRA got there first for much less.

Try this: change the JSON ticket's lora score to 93. Predict what decide prints, then run it.

This loop trusts the list order. Real results come in any order, and some targets also cap cost and speed. In the Exercises you write pickRung, which handles both.

Your Fine-tune Is a Fork

Building a fine-tune is only the start. After that, you own it.

Fork = your own copy that splits off from the original and no longer gets its updates for free.

When the vendor ships a better base model, your fine-tune does not move with it. To get the upgrade, you prepare the data again, train again, and run the evals again.

Watch what each choice costs for one year.

Simple lesson: count the cost of owning it, not just the cost of building it.

Count the Yearly Cost

Here is the same count in code. It uses a simple model: an engineer-day costs $600, and a LoRA training run costs $150 on rented GPUs.

// What does each choice cost to OWN for one year?
const dayRate = 600;          // one engineer-day, in dollars
const gpuPerTraining = 150;   // one LoRA training run on rented GPUs

type Choice = { name: string; buildDays: number; trains: boolean; servingPerMonth: number };

const choices: Choice[] = [
  { name: "few-shot prompt", buildDays: 1, trains: false, servingPerMonth: 420 },
  { name: "LoRA fine-tune", buildDays: 15, trains: true, servingPerMonth: 260 },
];

for (const c of choices) {
  const build = c.buildDays * dayRate + (c.trains ? gpuPerTraining : 0);
  // A fine-tune is re-trained about 3 times a year (new base model, new data),
  // 4 days each, and its eval is re-run every month (1 day).
  const upkeep = c.trains ? 3 * (4 * dayRate + gpuPerTraining) + 12 * 1 * dayRate : 0;
  const serving = 12 * c.servingPerMonth;
  const total = build + upkeep + serving;
  console.log(`${c.name.padEnd(16)} build $${build}  upkeep $${upkeep}  serving $${serving}  total $${total}`);
}

// What does each choice cost to OWN for one year?
int dayRate = 600;          // one engineer-day, in dollars
int gpuPerTraining = 150;   // one LoRA training run on rented GPUs

var choices = new List<Choice>
{
    new("few-shot prompt", 1, false, 420),
    new("LoRA fine-tune", 15, true, 260),
};

foreach (var c in choices)
{
    int build = c.BuildDays * dayRate + (c.Trains ? gpuPerTraining : 0);
    // A fine-tune is re-trained about 3 times a year (new base model, new data),
    // 4 days each, and its eval is re-run every month (1 day).
    int upkeep = c.Trains ? 3 * (4 * dayRate + gpuPerTraining) + 12 * 1 * dayRate : 0;
    int serving = 12 * c.ServingPerMonth;
    int total = build + upkeep + serving;
    Console.WriteLine($"{c.Name.PadRight(16)} build ${build}  upkeep ${upkeep}  serving ${serving}  total ${total}");
}

record Choice(string Name, int BuildDays, bool Trains, int ServingPerMonth);

Expected output:

few-shot prompt  build $600  upkeep $0  serving $5040  total $5640
LoRA fine-tune   build $9150  upkeep $14850  serving $3120  total $27120

The few-shot prompt costs $5,640 a year. The LoRA fine-tune costs $27,120, nearly five times as much.

For the JSON ticket that is still the right call: few-shot never reached the 95% target. But now everyone can see the price before saying yes.

Try this: set the re-trains per year from 3 to 1. How much does the LoRA total drop?

The numbers are written into the loop here. In the Exercises you turn them into a yearCost(choice, model) function, so a lead can try other rates.

Watch a Model Learn a Habit

Enough theory. Let's fine-tune a real model.

The lab has a tiny language model. Its base was trained to reply to shop messages in plain sentences, like "sure, i can help with your refund."

Your job is to teach it a new behaviour: reply only with a fixed JSON format, like {"intent":"refund"}.

Press Start fine-tune and watch three things:

1. Training loss goes down. The model gets better at the training examples.
2. Validation loss goes down too. These are different wordings it never trained on. If this line falls, it learned the pattern, not just the lines.
3. The Fine-tuned column switches from sentences to JSON.

Loss = a number that says how wrong the model's guesses are. Lower is better.

Then type your own message, like "refund my kettle". Also look at "hello". It was not in the training data, yet the tuned model no longer says "hello!" back. It starts a JSON answer and gets lost. That is a first taste of forgetting: training one habit can change others. The Evaluate Before and After lesson measures it properly.

Simple lesson: fine-tuning changes habits, and sometimes more than the one you aimed at.

When Fine-tuning Is the Right Call

Fine-tuning earns its cost when:

• The format must be exact, every time, and prompting stalls below the target.
• The tone or style is yours, and long prompts to describe it cost too much.
• The task is narrow and huge in volume, so a small tuned model beats a big general one on cost.
• Prompts are getting very long with examples, and every call pays for them.

Common mistakes

Check Your Understanding

Exercises

What to Remember

• Fine-tuning trains a model a bit more, on your examples, to change how it answers.
• It teaches form, not facts. Facts that change belong in RAG.
• Name the gap first: knowledge, behaviour, cost or eval.
• Climb the ladder from the bottom and stop at the first rung that meets the target.
• A fine-tune is a fork. Count what it costs to own for a year.

Next: Exhaust the Cheaper Rungs First, how to push prompting and RAG as far as they go and prove they stopped improving.