# Choosing the Method: LoRA, QLoRA, Full, Preference Tuning

Course: [Fine-tuning & Model Customization](https://www.devtutorials.ai/learn/fine-tuning) · Section: Train and Operate
Source: https://www.devtutorials.ai/learn/fine-tuning/train-and-operate/choosing-the-method

> Full fine-tuning, LoRA, QLoRA, DPO, GRPO-style training and distillation: what each changes, the data and GPU memory it needs, and sensible starting settings.

## Choosing the Method: LoRA, QLoRA, Full, Preference Tuning

The JSON ticket team has clean data and a plan. They open a fine-tuning tool, and it asks: "Full fine-tune, LoRA, or QLoRA? SFT, DPO, or something else?"

Each option changes different things, needs different data, and needs a different amount of GPU memory. Start with the memory, because it decides what you can run at all.

A full fine-tune of an 8B model needs about **136 GB**. LoRA needs about **26 GB**. QLoRA needs about **10 GB**, small enough for one consumer GPU.

**Simple lesson:** pick the lightest method that can do the job, then prove it with the eval.

### A Real-World Example

Think about changing how a shop looks.

A **full fine-tune** is rebuilding the shop. A **LoRA** adapter is new signs and shelves on top of the same building. **QLoRA** is the same new signs, but you packed the building's furniture more tightly to fit a smaller van.

**Preference tuning** is asking customers "which of these two window displays do you like more?" many times, and learning from their choices. **Distillation** is training a new, smaller shop assistant by having them watch your best one.

**Simple lesson:** most of the time, new signs and shelves are enough.

## What You Will Learn

In this lesson, you will learn:

1. What **full fine-tuning**, **LoRA** and **QLoRA** change, and what they need.
2. How to estimate **GPU memory** for each one.
3. When **preference tuning (DPO)** and **reinforcement with a checker (GRPO-style)** help.
4. When **distillation** is the right method.
5. Sensible **starting settings** in the common tools.

**Before you start**

- This is the first lesson of **Train and Operate**. It assumes the data from the Decide and Prepare lessons.
- Memory numbers here are **rules of thumb**, not exact. Real use depends on sequence length, batch size and the tool.
- Tool names and settings come from each tool's docs. Check them before you run anything; they change often.

## TL;DR

**SFT** (supervised fine-tuning) teaches from input and answer pairs. It is the first step for almost every fine-tune.

**LoRA** trains a small adapter on a frozen base. **QLoRA** does the same with the base stored in 4-bit, using much less memory.

**DPO** learns from pairs of better and worse answers, after SFT. **GRPO-style** training learns from a program that checks answers.

**Distillation** trains a small model on a big model's answers, to cut cost and speed things up.

## Memory: the First Filter

- Each weight is a number stored in memory. In **16-bit** form each weight takes 2 bytes. In **4-bit** form it takes half a byte.
- A model with 8 billion weights at 2 bytes is about 16 GB, just to hold it.
- Training needs more than holding: the optimiser keeps extra numbers for every weight it changes.

```ts runnable title="Rough GPU memory per method"
// Rough GPU memory to fine-tune a model, in GB.
// bytes per weight: full fine-tune ~16 (weights, gradients, optimiser state),
// LoRA ~2 (16-bit frozen base), QLoRA ~0.5 (4-bit frozen base).
const methods = [
  { name: "Full fine-tune", bytesPerWeight: 16, extraGb: 8 },
  { name: "LoRA", bytesPerWeight: 2, extraGb: 10 },
  { name: "QLoRA", bytesPerWeight: 0.5, extraGb: 6 },
];

for (const billions of [1, 8]) {
  for (const m of methods) {
    const gb = Math.round(billions * m.bytesPerWeight + m.extraGb);
    const fits = gb <= 24 ? "fits a 24 GB GPU" : gb <= 80 ? "needs an 80 GB GPU" : "needs several GPUs";
    console.log(`${billions}B ${m.name.padEnd(14)} ~${gb} GB  ${fits}`);
  }
}
```

```csharp runnable title="Rough GPU memory per method"
// Rough GPU memory to fine-tune a model, in GB.
// bytes per weight: full fine-tune ~16 (weights, gradients, optimiser state),
// LoRA ~2 (16-bit frozen base), QLoRA ~0.5 (4-bit frozen base).
var methods = new[]
{
    new { name = "Full fine-tune", bytesPerWeight = 16.0, extraGb = 8 },
    new { name = "LoRA", bytesPerWeight = 2.0, extraGb = 10 },
    new { name = "QLoRA", bytesPerWeight = 0.5, extraGb = 6 },
};

foreach (var billions in new[] { 1, 8 })
{
    foreach (var m in methods)
    {
        // AwayFromZero rounds 6.5 up to 7, like JavaScript's Math.round.
        var gb = Math.Round(billions * m.bytesPerWeight + m.extraGb, MidpointRounding.AwayFromZero);
        var fits = gb <= 24 ? "fits a 24 GB GPU" : gb <= 80 ? "needs an 80 GB GPU" : "needs several GPUs";
        Console.WriteLine($"{billions}B {m.name.PadRight(14)} ~{gb} GB  {fits}");
    }
}
```

Expected output:

```text
1B Full fine-tune ~24 GB  fits a 24 GB GPU
1B LoRA           ~12 GB  fits a 24 GB GPU
1B QLoRA          ~7 GB  fits a 24 GB GPU
8B Full fine-tune ~136 GB  needs several GPUs
8B LoRA           ~26 GB  needs an 80 GB GPU
8B QLoRA          ~10 GB  fits a 24 GB GPU
```

For a 1B model every method fits on one consumer GPU. For an 8B model only QLoRA does.

## Which Method for Which Job?

| Method | What it changes | Data it needs | Use it when |
|---|---|---|---|
| Full fine-tune | Every weight | Input and answer pairs | Rarely: LoRA was measured and fell short |
| LoRA | A small adapter | Input and answer pairs | The default for a behaviour gap |
| QLoRA | A small adapter, 4-bit base | Input and answer pairs | LoRA on a smaller GPU |
| DPO | Usually an adapter, after SFT | Better and worse answer pairs | Tone or style that is easier to compare than to write |
| GRPO-style | Usually an adapter, after SFT | Inputs and a checker program | Answers a program can verify |
| Distillation | A whole small model | Inputs labelled by a big teacher | Quality is fine, cost or speed is not |

**Preference pair** = two answers to the same input, marked "better" and "worse".

**Checker** = code that scores an answer, like "the JSON parses and the order number exists".

```ts runnable title="A first-pass method picker"
type Need = { gap: "format" | "taste" | "checkable" | "volume"; gpuGb: number };

// A first-pass method picker. Always confirm with an eval.
function pick(n: Need): string {
  if (n.gap === "volume") return "distillation (small student, big teacher)";
  if (n.gap === "taste") return "SFT first, then DPO on better/worse pairs";
  if (n.gap === "checkable") return "SFT first, then GRPO-style with a checker";
  return n.gpuGb >= 32 ? "LoRA" : "QLoRA";
}

const cases: Need[] = [
  { gap: "format", gpuGb: 24 },
  { gap: "format", gpuGb: 80 },
  { gap: "taste", gpuGb: 80 },
  { gap: "checkable", gpuGb: 80 },
  { gap: "volume", gpuGb: 80 },
];
for (const c of cases) console.log(`${c.gap.padEnd(9)} ${String(c.gpuGb).padStart(2)} GB -> ${pick(c)}`);
```

```csharp runnable title="A first-pass method picker"
// A first-pass method picker. Always confirm with an eval.
string Pick(Need n)
{
    if (n.Gap == "volume") return "distillation (small student, big teacher)";
    if (n.Gap == "taste") return "SFT first, then DPO on better/worse pairs";
    if (n.Gap == "checkable") return "SFT first, then GRPO-style with a checker";
    return n.GpuGb >= 32 ? "LoRA" : "QLoRA";
}

var cases = new List<Need>
{
    new("format", 24),
    new("format", 80),
    new("taste", 80),
    new("checkable", 80),
    new("volume", 80),
};
foreach (var c in cases) Console.WriteLine($"{c.Gap.PadRight(9)} {c.GpuGb,2} GB -> {Pick(c)}");

// Gap is one of: "format", "taste", "checkable", "volume"
record Need(string Gap, int GpuGb);
```

Expected output:

```text
format    24 GB -> QLoRA
format    80 GB -> LoRA
taste     80 GB -> SFT first, then DPO on better/worse pairs
checkable 80 GB -> SFT first, then GRPO-style with a checker
volume    80 GB -> distillation (small student, big teacher)
```

**SFT first, then the fancy methods.**

DPO and GRPO-style training improve a model that already does the task. They start from an SFT adapter. If SFT is not working, they will not rescue it: fix the data first.

## Choosing the Base Model

The method is only half the choice. You also pick the **base model** the adapter sits on.

| Check | Why it matters |
|---|---|
| **Licence** | Some licences limit commercial use, user numbers or training on outputs |
| **Size** | Bigger is usually better and always slower and dearer. Start small: 1-3B for format tasks, 7-8B if the eval says you need it |
| **Context length** | Your longest prompt plus answer must fit |
| **Tokenizer and languages** | A tokenizer that splits your language into many tokens is slower and weaker |
| **Instruct or base** | An instruction-tuned model already follows a chat template; usually the right start |
| **Where it can run** | Your GPUs, a serving tool you use, or a provider that hosts fine-tunes |
| **How long it will be supported** | A retired base forces a re-train (Serving and Maintaining lesson) |

Then run the **same eval** on two or three candidates with your best prompt, before any training. The smallest one that is close is often the right base.

Memory also grows with **sequence length** and **batch size**, not only model size:

```ts runnable title="Memory with sequence length and batch size"
// Memory also grows with sequence length and batch size (activations).
// Rough rule for a 7-8B model with gradient checkpointing: about
// 0.5 MB per token in the batch, on top of the weights.
const weightsGb = { lora: 16, qlora: 4 };   // 8B model: 16-bit vs 4-bit base
const mbPerToken = 0.5;

for (const seq of [512, 2048, 8192]) {
  for (const batch of [1, 4]) {
    const act = (seq * batch * mbPerToken) / 1024;
    console.log(`seq ${String(seq).padStart(4)} batch ${batch}: LoRA ~${Math.round(weightsGb.lora + 2 + act)} GB, QLoRA ~${Math.round(weightsGb.qlora + 2 + act)} GB`);
  }
}
```

```csharp runnable title="Memory with sequence length and batch size"
// Memory also grows with sequence length and batch size (activations).
// Rough rule for a 7-8B model with gradient checkpointing: about
// 0.5 MB per token in the batch, on top of the weights.
var weightsGb = new { lora = 16, qlora = 4 };   // 8B model: 16-bit vs 4-bit base
var mbPerToken = 0.5;

double JsRound(double x) => Math.Round(x, MidpointRounding.AwayFromZero);

foreach (var seq in new[] { 512, 2048, 8192 })
{
    foreach (var batch in new[] { 1, 4 })
    {
        var act = (seq * batch * mbPerToken) / 1024;
        Console.WriteLine($"seq {seq,4} batch {batch}: LoRA ~{JsRound(weightsGb.lora + 2 + act)} GB, QLoRA ~{JsRound(weightsGb.qlora + 2 + act)} GB");
    }
}
```

Expected output:

```text
seq  512 batch 1: LoRA ~18 GB, QLoRA ~6 GB
seq  512 batch 4: LoRA ~19 GB, QLoRA ~7 GB
seq 2048 batch 1: LoRA ~19 GB, QLoRA ~7 GB
seq 2048 batch 4: LoRA ~22 GB, QLoRA ~10 GB
seq 8192 batch 1: LoRA ~22 GB, QLoRA ~10 GB
seq 8192 batch 4: LoRA ~34 GB, QLoRA ~22 GB
```

## Starting Settings

The same starting point works in most tools:

- **LoRA rank 16**, alpha 32 (alpha is a scaling number, often twice the rank).
- **Learning rate 2e-4**, which is 0.0002.
- **3 epochs**, with a validation check every 50 steps.
- **Keep the best checkpoint**, not the last one.

Here are those settings in four common tools.

**Simple lesson:** start from the defaults, change one thing at a time, and let the validation line decide.

**Out of scope for this course**

- **Pre-training** a model from scratch: months of compute on huge datasets. Fine-tuning starts from a model someone else pre-trained.
- **Large-scale RLHF** with human raters and a reward model: DPO covers the common product case more simply.
- **Multi-node training** across many GPU machines: needed for full fine-tunes of big models, rarely for LoRA.

- **Why this method?** "LoRA, because the gap is behaviour" is a good answer. "Full, because it is best" is not.
- **What hardware, for how long?** A QLoRA run on one GPU for an hour is cheap. A full fine-tune on many GPUs is not.
- **Are we allowed to fine-tune this base model?** Check its licence before any work starts.
- **Which base models were compared,** on the same eval, before choosing?
- **For DPO:** where do the preference pairs come from, and who checks them?

**Do's and Don'ts**

- **Do** start with SFT and LoRA or QLoRA.
- **Do** estimate memory before you rent hardware.
- **Do** use the tool's defaults first, then change one setting at a time.
- **Do** check the base model's licence.
- **Don't** start with a full fine-tune.
- **Don't** reach for DPO or GRPO before SFT works.
- **Don't** copy settings from a blog without your own validation line.

## Common mistakes

| Mistake | Wrong idea | Correct idea | What goes wrong |
|---|---|---|---|
| Full fine-tune by default | "More weights changed, better result." | Start with LoRA. Go full only with evidence. | Weeks of GPU time for the same score. |
| Ignoring memory | "We will find a GPU." | Estimate memory per method first. | The run crashes with out-of-memory, or the bill surprises you. |
| DPO without SFT | "Preference tuning will teach the format." | SFT first. DPO polishes a model that already works. | The model never learns the basic task. |
| Copying settings blindly | "This learning rate worked for someone." | Use defaults, watch validation, adjust. | A run that looks fine on paper overfits or explodes. |
| No licence check | "It is open, so we can use it." | Read the base model's licence. | A shipped product you are not allowed to sell. |

## Check Your Understanding

## Exercises

## What to Remember

- **SFT** first, with **LoRA**, or **QLoRA** on a smaller GPU.
- Estimate **memory** before you rent anything.
- **DPO** learns from better/worse pairs; **GRPO-style** from a checker; both after SFT.
- **Distillation** trades a little quality for a lot of cost and speed.
- Start with **defaults**, keep the **best checkpoint**, and check the **licence**.

Next: **Chat Templates, Masking and Stop Tokens**, the formatting bugs that break real fine-tunes without showing up in the loss.