Run a Real Fine-tune

By Syncfusion®, Inc. · Last updated October 28, 2026

Use the starter kit to go from a helpdesk export to a scored QLoRA adapter: config, data prep, template check, training with TRL, generation and a CI-ready score.

Run a Real Fine-tune

Everything so far ran in your browser. Now do it for real: a real base model, a real GPU, a real adapter.

The course has a starter kit: a small Python project that runs every step from the helpdesk export to a ship decision. Watch the steps first.

Three of the seven steps need no GPU at all. You can run them on the fixtures right now and see real output.

Simple lesson: a fine-tune is a short pipeline of scripts, not one magic command.

A Real-World Example

Think about cooking from a recipe card for the first time.

You read the whole card, lay out the ingredients, taste as you go, and write down what you changed. Next time, you can repeat it, or know exactly what to change.

A fine-tuning run is the same: a config (the recipe), prepared data (the ingredients), checks along the way (tasting), and run notes (what you changed).

Simple lesson: write the recipe down, so the next run is not a guess.

What You Will Learn

In this lesson, you will learn:

1. What is in the starter kit, and which steps need a GPU.
2. How to set up a run in config.yaml.
3. How batch size, gradient accumulation and steps fit together.
4. How to read a training log and keep the best checkpoint.
5. How to score base and fine-tune, and use the exit code as a gate.

TL;DR

python -m unittest test_kit.py                                    # no GPU
python prepare_data.py fixtures/tickets-export.json --out data    # no GPU
python check_template.py                                          # no GPU (downloads the tokenizer)
python generate.py --base-only --out preds/base.jsonl             # GPU
python train_qlora.py                                             # GPU
python generate.py --adapter runs/ticket-adapter-v1/best --out preds/tuned.jsonl   # GPU
python score.py --eval data/eval.jsonl --base preds/base.jsonl --tuned preds/tuned.jsonl   # no GPU

What Is in the Kit

The Config

One file holds every setting, so a run can be repeated and compared:

base_model: Qwen/Qwen2.5-1.5B-Instruct
max_seq_length: 512
load_in_4bit: true        # QLoRA
lora_r: 16
lora_alpha: 32
learning_rate: 0.0002
num_train_epochs: 3
per_device_train_batch_size: 4
gradient_accumulation_steps: 4
eval_steps: 50
temperature: 0            # for the eval: greedy, repeatable

Swap base_model for the one you chose in the Choosing the Method lesson, after checking its licence.

Batches, Accumulation and Steps

// How many optimiser steps, and how long, for one run?
const rows = 560;
const batch = 4;          // per_device_train_batch_size
const accumulation = 4;   // gradient_accumulation_steps
const epochs = 3;
const secondsPerStep = 13; // measure this from your first 20 steps

const effective = batch * accumulation;
const perEpoch = Math.ceil(rows / effective);
const total = perEpoch * epochs;
console.log(`effective batch: ${effective}`);
console.log(`steps: ${perEpoch} per epoch, ${total} in total`);
console.log(`time: about ${Math.round((total * secondsPerStep) / 60)} minutes`);

// How many optimiser steps, and how long, for one run?
int rows = 560;
int batch = 4;          // per_device_train_batch_size
int accumulation = 4;   // gradient_accumulation_steps
int epochs = 3;
int secondsPerStep = 13; // measure this from your first 20 steps

int effective = batch * accumulation;
int perEpoch = (int)Math.Ceiling((double)rows / effective);
int total = perEpoch * epochs;
Console.WriteLine($"effective batch: {effective}");
Console.WriteLine($"steps: {perEpoch} per epoch, {total} in total");
Console.WriteLine($"time: about {Math.Round(total * secondsPerStep / 60.0, MidpointRounding.AwayFromZero)} minutes");

Expected output:

effective batch: 16
steps: 35 per epoch, 105 in total
time: about 23 minutes

With 560 rows, an effective batch of 16 gives 35 steps per epoch and 105 steps in total. With eval_steps: 50, you would only get two validation checks, so lower it to about 20 for a run this short.

Read the Log

When training ends, train_qlora.py saves the best checkpoint to runs/ticket-adapter-v1/best and writes run_notes.json: the config, the best validation loss, the row counts and every library version. Commit it next to the dataset card.

Experiment tracking = a tool that records every run's settings and curves, so you can compare them later. Set report_to: wandb or mlflow in the config if your team uses one.

Score It

score.py needs no GPU. Run it on the kit's fixture predictions to see real output:

python score.py --eval fixtures/eval.jsonl --base fixtures/predictions-base.jsonl --tuned fixtures/predictions-tuned.jsonl

cases: 100
base exact-match: 76.0%
fine-tune exact-match: 96.0%
change: +20.0 points (95% interval +10.0 to +29.0)

fine-tune failures:
  extra text after the JSON: 1 (E-008)
  never stops (text after the JSON): 1 (E-042)
  template tokens in output: 1 (E-024)
  truncated or invalid JSON: 1 (E-067)

(per-intent table here)

SHIP RULE: PASS (task slice only: add your general and safety slices before release)

The exit code is 0 when the ship rule passes and 1 when it fails, so the same command can block a merge in CI. The When the Fine-tune Misbehaves lesson explains each failure type.

What It Costs

GPU time for a small LoRA is cheap. Count every run, not just the final one:

// What does an experiment cost? Count every run, not just the final one.
const gpuPerHour = 1.2;   // illustrative price for a rented 24 GB GPU; check current prices
const runMinutes = 23;
const runs = [
  { why: "first try", count: 1 },
  { why: "learning rate sweep (3 values)", count: 3 },
  { why: "fix a template bug, re-run", count: 1 },
  { why: "final run with best settings", count: 1 },
  { why: "re-trains over a year", count: 3 },
];

let total = 0;
for (const r of runs) {
  const cost = (r.count * runMinutes / 60) * gpuPerHour;
  total += cost;
  console.log(`${r.why.padEnd(32)} $${cost.toFixed(2)}`);
}
console.log(`${"GPU total for the year".padEnd(32)} $${total.toFixed(2)}`);
console.log("People time is the big cost: compare with the first lesson's ownership numbers.");

// What does an experiment cost? Count every run, not just the final one.
double gpuPerHour = 1.2;   // illustrative price for a rented 24 GB GPU; check current prices
int runMinutes = 23;
var runs = new List<Run>
{
    new("first try", 1),
    new("learning rate sweep (3 values)", 3),
    new("fix a template bug, re-run", 1),
    new("final run with best settings", 1),
    new("re-trains over a year", 3),
};

double total = 0;
foreach (var r in runs)
{
    double cost = (r.Count * runMinutes / 60.0) * gpuPerHour;
    total += cost;
    Console.WriteLine($"{r.Why.PadRight(32)} ${cost:F2}");
}
Console.WriteLine($"{"GPU total for the year".PadRight(32)} ${total:F2}");
Console.WriteLine("People time is the big cost: compare with the first lesson's ownership numbers.");

record Run(string Why, int Count);

Expected output:

first try                        $0.46
learning rate sweep (3 values)   $1.38
fix a template bug, re-run       $0.46
final run with best settings     $0.46
re-trains over a year            $1.38
GPU total for the year           $4.14
People time is the big cost: compare with the first lesson's ownership numbers.

Common mistakes

Check Your Understanding

Exercises

What to Remember

• The starter kit runs the whole pipeline; data and scoring need no GPU.
• Keep every setting in config.yaml, and commit run_notes.json.
• Effective batch = batch × accumulation; count steps to set eval_steps.
• Generate base and fine-tune with the same settings.
• score.py's exit code is your ship gate.

Next: Fine-tune with a Hosted API, the same pipeline on a provider's GPUs, driven from TypeScript or C#.