Fine-tuning & Model Customization tutorial

Decide when to fine-tune and when not to. Climb the customization ladder, train a LoRA adapter, read the loss curves, and own the result.

Start the course

Course contents

Decide and Prepare

  1. Should You Fine-tune at All? — Four requests that all sound like fine-tuning. Only two need it. Learn to name the gap, climb the customization ladder from the cheapest rung, and count what a fine-tune costs to own.
  2. Exhaust the Cheaper Rungs First — Push prompting step by step, prove a plateau on the same eval, count what few-shot examples cost, and learn when RAG or both tools are the answer.
  3. How Fine-tuning Works — Weights, loss, epochs, learning rate, validation sets, overfitting and LoRA, explained for developers with no machine learning background.
  4. Dataset Design — What one training example looks like in chat JSONL, why answers must follow one labelling guide, how to check coverage, and how to split the eval by customer.
  5. From Tickets to Training Data — Turn a real helpdesk export into training lines: check consent, convert HTML, cut quoted replies and signatures, redact personal data and split by customer.
  6. Data Quality: Duplicates, Leakage, Label Errors, Bias — Clean a 600-row export: exact and near-duplicates, eval leaks, wrong labels and one dominant customer. Write a near-duplicate check and see how leaks inflate the eval.
  7. Synthetic Data from a Stronger Model — Fill rare intents with tickets a teacher model writes, filter them five ways, choose a safe mix, and export the v1 dataset kit with a dataset card.

Train and Operate

  1. Choosing the Method: LoRA, QLoRA, Full, Preference Tuning — Full fine-tuning, LoRA, QLoRA, DPO, GRPO-style training and distillation: what each changes, the data and GPU memory it needs, and sensible starting settings.
  2. Chat Templates, Masking and Stop Tokens — The formatting bugs that break real fine-tunes: chat templates, loss masking, end-of-turn tokens and max sequence length, with real outputs from a tiny model.
  3. Run a Real Fine-tune — Use the starter kit to go from a helpdesk export to a scored QLoRA adapter: config, data prep, template check, training with TRL, generation and a CI-ready score.
  4. Fine-tune with a Hosted API — Upload training files, start a supervised fine-tuning job over a provider's REST API, poll it, read the loss from its events and call the new model ID, in TypeScript or C#.
  5. Reading a Training Run — Diagnose six training runs from their loss curves: healthy, overfitting, stopped early, learning rate too high, and the run that looks too good. Pick the best checkpoint.
  6. When the Fine-tune Misbehaves — Debug a fine-tune from its outputs: template tokens, run-on answers, cut-offs, prose and wrong values, each traced to a cause and a fix.
  7. Evaluate Before and After: Wins and Regressions — Score the base model and the fine-tune on the same cases with 95% intervals, catch catastrophic forgetting and safety regressions, and fix them before you ship.
  8. Distillation for Cost and Latency — Train a small student model on a big teacher's labels to cut cost and latency, check options against quality, cost and p95 targets, and route hard cases back.
  9. Serving and Maintaining a Fine-tune — Serve LoRA adapters on one base model, version everything, re-train on measured triggers, upgrade the base model safely and keep a rollback ready.
  10. Ship It: Flags, Canaries and Rollbacks — Route between the baseline and the fine-tune with a flag, run shadow and canary steps with an automatic rule, fall back on bad answers, and block unapproved releases in CI.
  11. Capstone: Decision Record and Dataset Kit — Decide all four requests with evidence, check your decision record against the course rubric, and download the full pack: record, dataset kit, method, eval plan and re-training checklist.

Reference

  1. Cheat Sheet — Every idea from Fine-tuning & Model Customization on one page: the gap types and ladder, training terms and curve shapes, data cleaning, methods and settings, before/after evals, distillation, maintenance and the decision record.