Serving and Maintaining a Fine-tune

By Syncfusion®, Inc. · Last updated October 29, 2026

Serve LoRA adapters on one base model, version everything, re-train on measured triggers, upgrade the base model safely and keep a rollback ready.

Serving and Maintaining a Fine-tune

The JSON ticket adapter ships in January. The team celebrates and moves on.

A year later, they look back. The adapter was re-trained three times, rolled back once, and moved to a new base model. Watch the year.

The monthly eval caught a drift in May (91%) and a bad release in October (89%). The alert fired in May and Oct, and both times the team had a plan, so neither became a long incident.

Simple lesson: shipping a fine-tune is the start of owning it.

A Real-World Example

Think about a car. Buying it is one cost. Fuel, servicing, new tyres and the yearly check are the rest.

A fine-tune is the same. Training is the purchase. Monitoring, re-training, base-model upgrades and rollbacks are the servicing. Skip them and it breaks down at the worst moment.

Simple lesson: plan the servicing before you buy.

What You Will Learn

In this lesson, you will learn:

1. How adapters are served: one base model, many small adapters.
2. Why you version the base model, the adapter and the dataset together.
3. What should trigger a re-train, and what should not.
4. How to upgrade the base model safely: shadow, canary, then all traffic.
5. Why you always keep a rollback ready.

TL;DR

Serve one base model with small adapters on top. Log the base, adapter and dataset versions with every response.

Re-train for measured reasons: the base model is retired, the eval stays under the alert line, the schema changes, or a safety problem appears.

Upgrade with board → shadow → canary → all traffic, and keep the previous version ready to roll back.

Serving Adapters

Multi-adapter serving = one base model loaded once, with several LoRA adapters applied per request. Many serving tools support it; check yours.

Version tag = a label like "base 2026-01 + adapter v3 + dataset v3" saved with every answer. Without it, nobody can say which model wrote a bad answer.

With version tags, a rollback is a one-line switch:

// Every answer carries a version tag, so you can trace it and roll back.
type Version = { base: string; adapter: string; dataset: string };
const live: Version = { base: "2026-01", adapter: "v3", dataset: "v3" };
const previous: Version = { base: "2025-06", adapter: "v2", dataset: "v2" };
const tag = (v: Version) => `base ${v.base} + adapter ${v.adapter} + dataset ${v.dataset}`;

let current = live;
console.log("serving:", tag(current));

const safetyScore = 84; // from the monthly board
if (safetyScore < 95) {
  current = previous;
  console.log("safety below 95%: rolled back");
}
console.log("serving:", tag(current));

// Every answer carries a version tag, so you can trace it and roll back.
var live = new Version("2026-01", "v3", "v3");
var previous = new Version("2025-06", "v2", "v2");
string Tag(Version v) => $"base {v.Base} + adapter {v.Adapter} + dataset {v.Dataset}";

var current = live;
Console.WriteLine($"serving: {Tag(current)}");

int safetyScore = 84; // from the monthly board
if (safetyScore < 95)
{
    current = previous;
    Console.WriteLine("safety below 95%: rolled back");
}
Console.WriteLine($"serving: {Tag(current)}");

record Version(string Base, string Adapter, string Dataset);

Expected output:

serving: base 2026-01 + adapter v3 + dataset v3
safety below 95%: rolled back
serving: base 2025-06 + adapter v2 + dataset v2

Merge or Load at Runtime?

The starter kit has both:

# Option A: base + adapter at runtime (vLLM, OpenAI-compatible API)
vllm serve Qwen/Qwen2.5-1.5B-Instruct --enable-lora \
  --lora-modules ticket-v1=runs/ticket-adapter-v1/best --max-lora-rank 16

# Option B: merge, then serve the merged model
python merge_adapter.py --adapter runs/ticket-adapter-v1/best --out merged/ticket-v1
vllm serve merged/ticket-v1

Whichever you pick, serve with the same chat template used in training, keep the JSON schema on (structured output), set temperature 0 for this task, and run the eval against the served endpoint once, not only against the training script. Flags change between versions: check vllm serve --help.

When to Re-train

type Month = { label: string; score: number; baseRetiring?: boolean; schemaChanged?: boolean };
const ALERT = 93;

const months: Month[] = [
  { label: "Mar", score: 95 },
  { label: "Apr", score: 93 },
  { label: "May", score: 91 },
  { label: "Jun", score: 90 },
  { label: "Jul", score: 96 },
  { label: "Aug", score: 96, baseRetiring: true },
  { label: "Sep", score: 95, schemaChanged: true },
];

months.forEach((m, i) => {
  const reasons: string[] = [];
  const prev = months[i - 1];
  if (m.score < ALERT && prev && prev.score < ALERT) reasons.push(`eval under ${ALERT}% two months running`);
  if (m.baseRetiring) reasons.push("base model being retired");
  if (m.schemaChanged) reasons.push("output schema changed");
  console.log(`${m.label} ${m.score}%  ${reasons.length ? "RE-TRAIN: " + reasons.join(", ") : "ok"}`);
});

const int ALERT = 93;

var months = new List<Month>
{
    new("Mar", 95),
    new("Apr", 93),
    new("May", 91),
    new("Jun", 90),
    new("Jul", 96),
    new("Aug", 96, BaseRetiring: true),
    new("Sep", 95, SchemaChanged: true),
};

for (int i = 0; i < months.Count; i++)
{
    var m = months[i];
    var reasons = new List<string>();
    var prev = i > 0 ? months[i - 1] : null;
    if (m.Score < ALERT && prev != null && prev.Score < ALERT) reasons.Add($"eval under {ALERT}% two months running");
    if (m.BaseRetiring) reasons.Add("base model being retired");
    if (m.SchemaChanged) reasons.Add("output schema changed");
    Console.WriteLine($"{m.Label} {m.Score}%  {(reasons.Count > 0 ? "RE-TRAIN: " + string.Join(", ", reasons) : "ok")}");
}

record Month(string Label, int Score, bool BaseRetiring = false, bool SchemaChanged = false);

Expected output:

Mar 95%  ok
Apr 93%  ok
May 91%  ok
Jun 90%  RE-TRAIN: eval under 93% two months running
Jul 96%  ok
Aug 96%  RE-TRAIN: base model being retired
Sep 95%  RE-TRAIN: output schema changed

Notice what is not a trigger: a calendar date, a bad demo, or someone's feeling. Those lead to re-training that changes things nobody measured.

Watch It Every Day

Re-training triggers only work if something checks them. The router writes one log line per request, so a small daily job can turn yesterday's lines into numbers:

• the fine-tune's fail rate per day,
• the most common failure type,
• an alert when the rate stays above the baseline plus a margin for two days in a row.

In the simulated logs, subscriptions launch on day 9. The fine-tune has never seen them, its failures jump to about 10%, and the alert fires on day 10. The failed messages become new eval cases, the schema gets a new intent, and the re-train starts from data, not from a feeling.

npm run monitor -- ../fixtures/logs-14-days.jsonl                          # TypeScript, in ts/
dotnet run --project TicketTuner.Cli -- monitor ../fixtures/logs-14-days.jsonl   # C#, in csharp/

Both exit with code 3 when the alert fires, so a scheduled job can page someone.

Upgrading the Base Model

Base models are retired. When yours is, the adapter must be re-trained on a new base. It is a new model, so it goes through every check again:

1. Re-run the cleaning and the leak check on the current data.
2. Train the adapter on the new base model.
3. Run the full before/after board: task, general skills, safety.
4. Shadow test: run it on live traffic without showing its answers.
5. Canary: send 5% of traffic to it and watch the alerts.
6. Send all traffic, and keep the old version ready.

Common mistakes

Check Your Understanding

Exercises

What to Remember

• Serve one base, many adapters, and log versions with every answer.
• Watch a monthly eval with an alert line.
• Re-train for measured triggers: retired base, drift, schema change, safety.
• Upgrade with board → shadow → canary → all.
• Keep a rollback ready.

Next: Ship It: Flags, Canaries and Rollbacks, how to put the fine-tune in front of customers a step at a time.