# Should You Fine-tune at All?

Course: [Fine-tuning & Model Customization](https://www.devtutorials.ai/learn/fine-tuning) · Section: Decide and Prepare
Source: https://www.devtutorials.ai/learn/fine-tuning/decide-and-prepare/should-you-fine-tune

> Four requests that all sound like fine-tuning. Only two need it. Learn to name the gap, climb the customization ladder from the cheapest rung, and count what a fine-tune costs to own.

## Should You Fine-tune at All?

Your team runs **Shop Helper**, the chat helper in an online shop.

One morning, four requests land on your desk. Each one ends with the same idea: "Let's fine-tune a model."

Most teams say yes too fast. Then they spend weeks on the wrong fix.

Watch one team do exactly that.

The team was not lazy. They worked hard for six weeks. They just picked a tool that cannot do the job: the policy changes every month, and a fine-tune freezes it.

**Simple lesson:** fine-tuning is a powerful tool for a small set of problems. Find out which problem you have first.

### A Real-World Example

Think about a new employee at a help desk.

If they do not know today's opening hours, you do not send them on a three-week course. You give them the page with the hours. They look it up.

If they know everything but answer customers rudely, a page will not help. You train them, with examples of good replies, until the new habit sticks.

A language model is the same. Missing facts are fixed by **looking things up**. Wrong habits are fixed by **training**.

**Simple lesson:** look up facts, train habits.

## What You Will Learn

In this lesson, you will learn:

1. What **fine-tuning** changes inside a model, and what it does not.
2. The four kinds of gap: **knowledge**, **behaviour**, **cost** and **eval**.
3. The **customization ladder**, and why you start at the bottom.
4. What a fine-tune costs to **own** for a year, not just to build.
5. How a tiny model learns a new habit in your browser with **LoRA**.

**Before you start**

- You should know the basics of **Large Language Models** and **Prompt Engineering**.
- **RAG** and **AI Evals & Observability** help a lot. Every decision in this course is made with an eval.
- No machine learning background is needed. New words are explained when they appear.

**About the data in this course**

- Real fine-tunes take hours on rented GPUs. So the **training runs** in this course come from a **simulated run bank**: fixed results written to show what real runs show. You get the same numbers every time, and the exercises can be graded.
- The **Live fine-tune** tab is real. A tiny model (about 44,000 numbers) trains in your browser in a few seconds, so you can watch a LoRA adapter learn.
- You can also turn on a **real model** in the Fine-tune Lab's **Settings** tab and download it. It shows what prompting alone can do, next to the fine-tune. It never grades.

## TL;DR

**Fine-tuning** = training an existing model a little more, on your own examples, so it changes how it answers.

It is good at **behaviour**: a fixed format, a house tone, a narrow task done cheaply.

It is bad at **facts that change**. Those belong in RAG.

Always try the cheaper rungs first: a better prompt, examples in the prompt, a JSON schema, then RAG. Fine-tune only when the eval shows those stopped improving.

## What Fine-tuning Changes

A language model is a very large list of numbers, called **weights**. They decide what it writes next.

**Pre-training** = how the model was first made. It read a huge amount of text and set its weights.

**Fine-tuning** = training it a bit more, on a small set of your examples. Some weights move. The model's habits change.

**LoRA** (Low-Rank Adaptation) = a cheap way to fine-tune. The original weights stay frozen. You train a small **adapter**, often about 1% of the size, that sits beside them.

Here is the key idea. Training on examples is good at teaching **how** to answer. It is unreliable at storing **what** is true, and it cannot update itself when the truth changes.

**Simple lesson:** fine-tuning teaches form, not facts.

## Name the Gap First

Before you pick a tool, name the gap. There are four kinds.

- **Knowledge gap:** the model does not know a fact, or the fact changes.
- **Behaviour gap:** the model knows enough, but the format, tone or style is wrong.
- **Cost or speed gap:** the answers are fine, but too slow or too expensive at your volume.
- **Eval gap:** nobody can say what "better" means yet, so nothing can be measured.

Watch the four requests get sorted.

Only two of the four end with training a model. The behaviour gap may need a LoRA fine-tune. The cost gap may need **distillation**: a small model trained on a big model's answers.

The other two never needed training at all.

**Simple lesson:** "it should be better" is not a gap. Name which kind of better.

## The Customization Ladder

Now you know the gap. Which tool do you use?

Think of the tools as a ladder. The bottom rungs are cheap and quick. Each step up costs more to build and more to keep running.

The rule is simple: **stop at the first rung that meets your target.**

Watch the team climb for the JSON ticket request. The target is 95% of tickets in the exact format and tone.

A better prompt gets **62%**. Adding examples gets **79%**.

**Structured output** gets **88%**. You hand the model your JSON schema, and every reply now parses with every field. But the tone and some values, like the priority, are still wrong. A schema fixes the shape, not the content. **Exhaust the Cheaper Rungs First** shows how to set it up.

RAG gets **80%**, almost no gain, because the problem was never missing facts.

The LoRA fine-tune gets **96%**. That meets the target, so the team stops there.

Now look at the refund policy request. The LoRA fine-tune scored **88%** in its first month. Next month, after the policy changed, it fell to **34%**. RAG scored **93%** and stayed high, because it reads the current page every time.

**Simple lesson:** climb one rung at a time, and measure at every rung.

## Climb the Ladder in Code

The ladder is a loop. Go through the rungs from cheapest to most expensive. Stop at the first one that meets the target, this month and next month.

```ts runnable title="Pick the cheapest rung that meets the target"
type Rung = "prompt" | "few-shot" | "schema" | "rag" | "lora" | "full";
type Result = { rung: Rung; score: number; nextMonth?: number };

// Measured results for one request, cheapest rung first.
// A rung nobody tried is simply not in the list.
const jsonTicket: Result[] = [
  { rung: "prompt", score: 62 },
  { rung: "few-shot", score: 79 },
  { rung: "schema", score: 88 },
  { rung: "rag", score: 80 },
  { rung: "lora", score: 96 },
  { rung: "full", score: 96 },
];
const refundPolicy: Result[] = [
  { rung: "prompt", score: 41 },
  { rung: "few-shot", score: 46 },
  { rung: "rag", score: 93, nextMonth: 92 },
  { rung: "lora", score: 88, nextMonth: 34 },
];

// Climb the ladder: stop at the first rung that meets the target,
// this month AND next month (if we measured next month).
function decide(results: Result[], target: number): string {
  for (const r of results) {
    const stillGood = r.nextMonth === undefined || r.nextMonth >= target;
    if (r.score >= target && stillGood) return `${r.rung} (${r.score}%)`;
    console.log(`  ${r.rung.padEnd(8)} ${r.score}% -> not enough, climb`);
  }
  return "nothing works yet: improve the eval or the data";
}

console.log("JSON ticket, target 95%");
console.log("  choose:", decide(jsonTicket, 95));
console.log("Refund policy, target 90%");
console.log("  choose:", decide(refundPolicy, 90));
```

```csharp runnable title="Pick the cheapest rung that meets the target"
// Measured results for one request, cheapest rung first.
// A rung nobody tried is simply not in the list.
var jsonTicket = new List<Result>
{
    new("prompt", 62),
    new("few-shot", 79),
    new("schema", 88),
    new("rag", 80),
    new("lora", 96),
    new("full", 96),
};
var refundPolicy = new List<Result>
{
    new("prompt", 41),
    new("few-shot", 46),
    new("rag", 93, NextMonth: 92),
    new("lora", 88, NextMonth: 34),
};

// Climb the ladder: stop at the first rung that meets the target,
// this month AND next month (if we measured next month).
string Decide(List<Result> results, int target)
{
    foreach (var r in results)
    {
        bool stillGood = r.NextMonth == null || r.NextMonth >= target;
        if (r.Score >= target && stillGood) return $"{r.Rung} ({r.Score}%)";
        Console.WriteLine($"  {r.Rung.PadRight(8)} {r.Score}% -> not enough, climb");
    }
    return "nothing works yet: improve the eval or the data";
}

Console.WriteLine("JSON ticket, target 95%");
Console.WriteLine($"  choose: {Decide(jsonTicket, 95)}");
Console.WriteLine("Refund policy, target 90%");
Console.WriteLine($"  choose: {Decide(refundPolicy, 90)}");

// Rung is one of: "prompt", "few-shot", "schema", "rag", "lora", "full"
record Result(string Rung, int Score, int? NextMonth = null);
```

Expected output:

```text
JSON ticket, target 95%
  prompt   62% -> not enough, climb
  few-shot 79% -> not enough, climb
  schema   88% -> not enough, climb
  rag      80% -> not enough, climb
  choose: lora (96%)
Refund policy, target 90%
  prompt   41% -> not enough, climb
  few-shot 46% -> not enough, climb
  choose: rag (93%)
```

The full fine-tune also scored 96%. The loop never reaches it, because LoRA got there first for much less.

**Try this:** change the JSON ticket's `lora` score to `93`. Predict what `decide` prints, then run it.

This loop trusts the list order. Real results come in any order, and some targets also cap cost and speed. In the Exercises you write `pickRung`, which handles both.

## Your Fine-tune Is a Fork

Building a fine-tune is only the start. After that, you **own** it.

**Fork** = your own copy that splits off from the original and no longer gets its updates for free.

When the vendor ships a better base model, your fine-tune does not move with it. To get the upgrade, you prepare the data again, train again, and run the evals again.

Watch what each choice costs for one year.

**Simple lesson:** count the cost of owning it, not just the cost of building it.

## Count the Yearly Cost

Here is the same count in code. It uses a simple model: an engineer-day costs $600, and a LoRA training run costs $150 on rented GPUs.

```ts runnable title="What each choice costs to own for a year"
// What does each choice cost to OWN for one year?
const dayRate = 600;          // one engineer-day, in dollars
const gpuPerTraining = 150;   // one LoRA training run on rented GPUs

type Choice = { name: string; buildDays: number; trains: boolean; servingPerMonth: number };

const choices: Choice[] = [
  { name: "few-shot prompt", buildDays: 1, trains: false, servingPerMonth: 420 },
  { name: "LoRA fine-tune", buildDays: 15, trains: true, servingPerMonth: 260 },
];

for (const c of choices) {
  const build = c.buildDays * dayRate + (c.trains ? gpuPerTraining : 0);
  // A fine-tune is re-trained about 3 times a year (new base model, new data),
  // 4 days each, and its eval is re-run every month (1 day).
  const upkeep = c.trains ? 3 * (4 * dayRate + gpuPerTraining) + 12 * 1 * dayRate : 0;
  const serving = 12 * c.servingPerMonth;
  const total = build + upkeep + serving;
  console.log(`${c.name.padEnd(16)} build $${build}  upkeep $${upkeep}  serving $${serving}  total $${total}`);
}
```

```csharp runnable title="What each choice costs to own for a year"
// What does each choice cost to OWN for one year?
int dayRate = 600;          // one engineer-day, in dollars
int gpuPerTraining = 150;   // one LoRA training run on rented GPUs

var choices = new List<Choice>
{
    new("few-shot prompt", 1, false, 420),
    new("LoRA fine-tune", 15, true, 260),
};

foreach (var c in choices)
{
    int build = c.BuildDays * dayRate + (c.Trains ? gpuPerTraining : 0);
    // A fine-tune is re-trained about 3 times a year (new base model, new data),
    // 4 days each, and its eval is re-run every month (1 day).
    int upkeep = c.Trains ? 3 * (4 * dayRate + gpuPerTraining) + 12 * 1 * dayRate : 0;
    int serving = 12 * c.ServingPerMonth;
    int total = build + upkeep + serving;
    Console.WriteLine($"{c.Name.PadRight(16)} build ${build}  upkeep ${upkeep}  serving ${serving}  total ${total}");
}

record Choice(string Name, int BuildDays, bool Trains, int ServingPerMonth);
```

Expected output:

```text
few-shot prompt  build $600  upkeep $0  serving $5040  total $5640
LoRA fine-tune   build $9150  upkeep $14850  serving $3120  total $27120
```

The few-shot prompt costs **$5,640** a year. The LoRA fine-tune costs **$27,120**, nearly five times as much.

For the JSON ticket that is still the right call: few-shot never reached the 95% target. But now everyone can see the price before saying yes.

**Try this:** set the re-trains per year from `3` to `1`. How much does the LoRA total drop?

The numbers are written into the loop here. In the Exercises you turn them into a `yearCost(choice, model)` function, so a lead can try other rates.

- **Which gap is it?** If the answer is "the facts are wrong", send it to RAG.
- **What did the cheaper rungs score?** Ask for the prompt and few-shot numbers on the same eval.
- **What is the target, and how will we know?** No eval, no fine-tune.
- **Who owns it next year?** Someone must re-train it when the base model changes.
- **What does it cost to own?** Ask for the yearly number, not only the build cost.

## Watch a Model Learn a Habit

Enough theory. Let's fine-tune a real model.

The lab has a tiny language model. Its base was trained to reply to shop messages in plain sentences, like "sure, i can help with your refund."

Your job is to teach it a new **behaviour**: reply only with a fixed JSON format, like `{"intent":"refund"}`.

Press **Start fine-tune** and watch three things:

1. **Training loss** goes down. The model gets better at the training examples.
2. **Validation loss** goes down too. These are different wordings it never trained on. If this line falls, it learned the pattern, not just the lines.
3. The **Fine-tuned** column switches from sentences to JSON.

**Loss** = a number that says how wrong the model's guesses are. Lower is better.

Then type your own message, like "refund my kettle". Also look at "hello". It was not in the training data, yet the tuned model no longer says "hello!" back. It starts a JSON answer and gets lost. That is a first taste of **forgetting**: training one habit can change others. The Evaluate Before and After lesson measures it properly.

**Simple lesson:** fine-tuning changes habits, and sometimes more than the one you aimed at.

**Prompting can get you far.**

Before you fine-tune, try the same task with a real model and a few examples in the prompt. Turn on the real model in the Fine-tune Lab's **Settings** tab, download it, and press **Run** in the Live fine-tune tab.

If few-shot prompting already gives the exact format, you are done. No training needed.

## When Fine-tuning Is the Right Call

Fine-tuning earns its cost when:

- **The format must be exact**, every time, and prompting stalls below the target.
- **The tone or style is yours**, and long prompts to describe it cost too much.
- **The task is narrow and huge in volume**, so a small tuned model beats a big general one on cost.
- **Prompts are getting very long** with examples, and every call pays for them.

**Do's and Don'ts**

- **Do** write the eval before anything else.
- **Do** try prompt, few-shot, a schema and RAG first, and write their scores down.
- **Do** stop at the cheapest rung that meets the target.
- **Do** count the yearly cost of owning a fine-tune.
- **Don't** fine-tune to teach facts that change.
- **Don't** jump to a full fine-tune when LoRA has not been tried.
- **Don't** fine-tune for a goal like "be smarter" that nobody can measure.

## Common mistakes

| Mistake | Wrong idea | Correct idea | What goes wrong |
|---|---|---|---|
| Fine-tuning for facts | "Train it on our policy so it knows the policy." | Facts that change go in RAG. Fine-tune for behaviour. | The policy changes and the model keeps quoting the old one. |
| Skipping the cheap rungs | "Prompting is too simple to work." | Try prompt, few-shot and RAG first, and measure each. | Weeks of training for a result a prompt could reach. |
| No eval before training | "We will know it is better when we see it." | Write the eval and the target first. | Nobody can tell if the fine-tune helped or hurt. |
| Counting only the build cost | "Training costs $150, so it is cheap." | Count re-training, evals and serving for a year. | The real bill is many times the build cost. |
| Going straight to full fine-tuning | "Bigger training must be better." | Try LoRA first. Full fine-tuning is rarely needed. | Months and big GPUs for the same score. |

## Check Your Understanding

## Exercises

## What to Remember

- **Fine-tuning** trains a model a bit more, on your examples, to change how it answers.
- It teaches **form, not facts**. Facts that change belong in **RAG**.
- Name the gap first: **knowledge**, **behaviour**, **cost** or **eval**.
- Climb the **ladder** from the bottom and stop at the first rung that meets the target.
- A fine-tune is a **fork**. Count what it costs to **own** for a year.

Next: **Exhaust the Cheaper Rungs First**, how to push prompting and RAG as far as they go and prove they stopped improving.