Train and Operate in Fine-tuning & Model Customization

Train and Operate is a chapter of the free Fine-tuning & Model Customization tutorial with 11 lessons: Choosing the Method: LoRA, QLoRA, Full, Preference Tuning, Chat Templates, Masking and Stop Tokens, Run a Real Fine-tune, Fine-tune with a Hosted API, Reading a Training Run, When the Fine-tune Misbehaves, Evaluate Before and After: Wins and Regressions, Distillation for Cost and Latency, Serving and Maintaining a Fine-tune, Ship It: Flags, Canaries and Rollbacks and Capstone: Decision Record and Dataset Kit. Plan on about 380 minutes. It has 39 exercises you run and check in your browser.

What each lesson covers:

  • Choosing the Method: LoRA, QLoRA, Full, Preference Tuning: Full fine-tuning, LoRA, QLoRA, DPO, GRPO-style training and distillation: what each changes, the data and GPU memory it needs, and sensible starting settings.
  • Chat Templates, Masking and Stop Tokens: The formatting bugs that break real fine-tunes: chat templates, loss masking, end-of-turn tokens and max sequence length, with real outputs from a tiny model.
  • Run a Real Fine-tune: Use the starter kit to go from a helpdesk export to a scored QLoRA adapter: config, data prep, template check, training with TRL, generation and a CI-ready score.
  • Fine-tune with a Hosted API: Upload training files, start a supervised fine-tuning job over a provider's REST API, poll it, read the loss from its events and call the new model ID, in TypeScript or C#.
  • Reading a Training Run: Diagnose six training runs from their loss curves: healthy, overfitting, stopped early, learning rate too high, and the run that looks too good. Pick the best checkpoint.
  • When the Fine-tune Misbehaves: Debug a fine-tune from its outputs: template tokens, run-on answers, cut-offs, prose and wrong values, each traced to a cause and a fix.
  • Evaluate Before and After: Wins and Regressions: Score the base model and the fine-tune on the same cases with 95% intervals, catch catastrophic forgetting and safety regressions, and fix them before you ship.
  • Distillation for Cost and Latency: Train a small student model on a big teacher's labels to cut cost and latency, check options against quality, cost and p95 targets, and route hard cases back.
  • Serving and Maintaining a Fine-tune: Serve LoRA adapters on one base model, version everything, re-train on measured triggers, upgrade the base model safely and keep a rollback ready.
  • Ship It: Flags, Canaries and Rollbacks: Route between the baseline and the fine-tune with a flag, run shadow and canary steps with an automatic rule, fall back on bad answers, and block unapproved releases in CI.
  • Capstone: Decision Record and Dataset Kit: Decide all four requests with evidence, check your decision record against the course rubric, and download the full pack: record, dataset kit, method, eval plan and re-training checklist.

Before you start, it helps to have finished Decide and Prepare, because these lessons build on it.

When you finish, continue with Reference.

Lessons in this section

  1. Choosing the Method: LoRA, QLoRA, Full, Preference Tuning

    Full fine-tuning, LoRA, QLoRA, DPO, GRPO-style training and distillation: what each changes, the data and GPU memory it needs, and sensible starting settings.

    • Say what full fine-tuning, LoRA and QLoRA change
    • Estimate GPU memory per method
    • Know when DPO and GRPO-style training help
    • Know when distillation fits
  2. Chat Templates, Masking and Stop Tokens

    The formatting bugs that break real fine-tunes: chat templates, loss masking, end-of-turn tokens and max sequence length, with real outputs from a tiny model.

    • Explain what a chat template is and why to use apply_chat_template
    • Train on the answer only with loss masking
    • Make sure every answer ends with an end-of-turn token
    • Measure token lengths before setting max_seq_length
  3. Run a Real Fine-tune

    Use the starter kit to go from a helpdesk export to a scored QLoRA adapter: config, data prep, template check, training with TRL, generation and a CI-ready score.

    • Know which steps need a GPU
    • Set up a run in one config file
    • Work out effective batch size and steps
    • Read a training log and keep the best checkpoint
  4. Fine-tune with a Hosted API

    Upload training files, start a supervised fine-tuning job over a provider's REST API, poll it, read the loss from its events and call the new model ID, in TypeScript or C#.

    • Run the four calls of a hosted fine-tune
    • Write a poller that waits and stops correctly
    • Read the loss from job events
    • Estimate trained tokens before upload
  5. Reading a Training Run

    Diagnose six training runs from their loss curves: healthy, overfitting, stopped early, learning rate too high, and the run that looks too good. Pick the best checkpoint.

    • Recognise the five common run shapes
    • Pick the best checkpoint
    • Check general skills and safety per checkpoint
    • Choose a fix for each problem
  6. When the Fine-tune Misbehaves

    Debug a fine-tune from its outputs: template tokens, run-on answers, cut-offs, prose and wrong values, each traced to a cause and a fix.

    • Recognise common output failures
    • Sort failures by type
    • Trace each failure to data, training format or serving
    • Write an output shape check
  7. Evaluate Before and After: Wins and Regressions

    Score the base model and the fine-tune on the same cases with 95% intervals, catch catastrophic forgetting and safety regressions, and fix them before you ship.

    • Compare base and fine-tune on the same cases
    • Read a change as win, regression or noise
    • Explain catastrophic forgetting
    • Fix forgetting by mixing old skills back in
  8. Distillation for Cost and Latency

    Train a small student model on a big teacher's labels to cut cost and latency, check options against quality, cost and p95 targets, and route hard cases back.

    • Explain teacher and student models
    • Check options against quality, cost and latency
    • Estimate teacher labelling cost
    • Choose the student size
  9. Serving and Maintaining a Fine-tune

    Serve LoRA adapters on one base model, version everything, re-train on measured triggers, upgrade the base model safely and keep a rollback ready.

    • Serve several adapters on one base model
    • Version base, adapter and dataset together
    • Re-train on measured triggers
    • Upgrade the base model with shadow and canary steps
  10. Ship It: Flags, Canaries and Rollbacks

    Route between the baseline and the fine-tune with a flag, run shadow and canary steps with an automatic rule, fall back on bad answers, and block unapproved releases in CI.

    • Route with off, shadow, canary and full modes
    • Pick canary customers with a stable hash
    • Fall back to the baseline on bad answers
    • Decide promote, hold or rollback from live numbers
  11. Capstone: Decision Record and Dataset Kit

    Decide all four requests with evidence, check your decision record against the course rubric, and download the full pack: record, dataset kit, method, eval plan and re-training checklist.

    • Decide each request from evidence
    • Check a decision record against a rubric
    • Build the full fine-tuning pack
    • Review a fine-tuning proposal as a lead