# Fine-tune with a Hosted API

Course: [Fine-tuning & Model Customization](https://www.devtutorials.ai/learn/fine-tuning) · Section: Train and Operate
Source: https://www.devtutorials.ai/learn/fine-tuning/train-and-operate/fine-tune-with-a-hosted-api

> Upload training files, start a supervised fine-tuning job over a provider's REST API, poll it, read the loss from its events and call the new model ID, in TypeScript or C#.

## Fine-tune with a Hosted API

The team has clean training files and a ship rule. They do not have a GPU, and nobody wants to run one.

Most model providers will train for you. You upload a file, start a job, and get back a new model name. Your app then calls that name like any other model.

The recorded job above trained on **1,512,000** tokens over **3** epochs and returned `ft:gpt-4o-mini-2024-07-18:example-org:tickets-v2:Tt9xQ2b1`.

**Simple lesson:** a hosted fine-tune is four API calls and a loop that waits. The hard parts are still the data and the eval.

### A Real-World Example

Think about sending a suit to a tailor instead of sewing it yourself.

You bring the measurements and the cloth. The tailor does the work in their shop, on their machines, and hands back the finished suit with a ticket number.

You still have to check the fit. And if the tailor keeps your cloth, you need to be happy with that.

**Simple lesson:** hosted training saves you the machines, not the measuring or the fitting.

## What You Will Learn

In this lesson, you will learn:

1. The **four calls** of a hosted fine-tune: upload, create, poll, use.
2. How to write a **poller** that waits the right amount and stops at the right time.
3. How to read the **loss from the job's events** and decide on the next run.
4. How to **estimate the training tokens** before you upload.
5. When hosted is the wrong choice, and what changes in your code if you switch.

**Before you start**

- **Hosted fine-tune** = the provider trains a copy of one of its models on your file, on its own GPUs, and serves it under a new model name.
- The calls follow OpenAI's fine-tuning REST API. Azure OpenAI uses the same shapes under its own endpoint and `api-version`. Other providers have similar steps with different names.
- Field names, supported base models and prices change. Check your provider's current API reference before you run this for real.

## TL;DR

1. `POST /v1/files` with `purpose=fine-tune`, once for train and once for validation.
2. `POST /v1/fine_tuning/jobs` with the base model, both file IDs and `method: { type: "supervised" }`.
3. `GET /v1/fine_tuning/jobs/{id}` until the status is `succeeded`, `failed` or `cancelled`.
4. Read `/events` for the loss. Write the model ID into your manifest. Run the eval.

## Build the Request from Config

Every setting comes from `ticket-tuner.config.json`, so a run can be repeated. Hyperparameters go under `method`. The older top-level `hyperparameters` field is deprecated.

```ts runnable title="Build the create-job request"
// Build the create-job request from your config, the way the repo's createJob() does.
const config = { baseModel: "gpt-4o-mini-2024-07-18", suffix: "tickets-v2", nEpochs: 3, seed: 42 };
const datasetSha256 = "9c1e0d7a52b4f6e8a1c3d5f7b9e0a2c4d6f8b1a3c5e7d9f0b2a4c6e8d0f1a3b5";

const body = {
  model: config.baseModel,
  training_file: "file-tt-train-0001",   // from POST /v1/files
  validation_file: "file-tt-val-0001",
  suffix: config.suffix,                 // becomes part of the model name
  seed: config.seed,                     // same seed, same shuffling
  metadata: { dataset_sha256: datasetSha256.slice(0, 16) },
  method: { type: "supervised", supervised: { hyperparameters: { n_epochs: config.nEpochs } } },
};
console.log("POST /v1/fine_tuning/jobs");
console.log(JSON.stringify(body, null, 2));
```

```csharp runnable title="Build the create-job request"
using System.Text.Encodings.Web;
using System.Text.Json;

// Build the create-job request from your config, the way the repo's CreateJobAsync() does.
var config = new { BaseModel = "gpt-4o-mini-2024-07-18", Suffix = "tickets-v2", NEpochs = 3, Seed = 42 };
var datasetSha256 = "9c1e0d7a52b4f6e8a1c3d5f7b9e0a2c4d6f8b1a3c5e7d9f0b2a4c6e8d0f1a3b5";

var body = new Dictionary<string, object>
{
    ["model"] = config.BaseModel,
    ["training_file"] = "file-tt-train-0001",   // from POST /v1/files
    ["validation_file"] = "file-tt-val-0001",
    ["suffix"] = config.Suffix,                 // becomes part of the model name
    ["seed"] = config.Seed,                     // same seed, same shuffling
    ["metadata"] = new { dataset_sha256 = datasetSha256[..16] },
    ["method"] = new { type = "supervised", supervised = new { hyperparameters = new { n_epochs = config.NEpochs } } },
};
Console.WriteLine("POST /v1/fine_tuning/jobs");
Console.WriteLine(JsonSerializer.Serialize(body, new JsonSerializerOptions { WriteIndented = true, Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping }));
```

Expected output:

```text
POST /v1/fine_tuning/jobs
{
  "model": "gpt-4o-mini-2024-07-18",
  "training_file": "file-tt-train-0001",
  "validation_file": "file-tt-val-0001",
  "suffix": "tickets-v2",
  "seed": 42,
  "metadata": {
    "dataset_sha256": "9c1e0d7a52b4f6e8"
  },
  "method": {
    "type": "supervised",
    "supervised": {
      "hyperparameters": {
        "n_epochs": 3
      }
    }
  }
}
```

The `metadata` ties the job to the exact training file. When someone asks which data made this model, the answer is on the job.

## Poll Until It Ends

A job goes through `validating_files`, `queued` and `running`, then ends. Your poller waits longer as the job gets longer, and stops for good when it ends.

```ts runnable title="Poll a job until it ends"
// Poll a job until it ends. These are the statuses the recorded job went through.
type Job = { status: string; fine_tuned_model: string | null };
const polls: Job[] = [
  { status: "validating_files", fine_tuned_model: null },
  { status: "queued", fine_tuned_model: null },
  { status: "running", fine_tuned_model: null },
  { status: "running", fine_tuned_model: null },
  { status: "succeeded", fine_tuned_model: "ft:gpt-4o-mini-2024-07-18:example-org:tickets-v2:Tt9xQ2b1" },
];

function nextAction(job: Job): { action: "wait" | "done" | "stop"; waitMs: number } {
  if (job.status === "validating_files") return { action: "wait", waitMs: 15000 };
  if (job.status === "queued") return { action: "wait", waitMs: 30000 };
  if (job.status === "running") return { action: "wait", waitMs: 60000 };
  if (job.status === "succeeded" && job.fine_tuned_model) return { action: "done", waitMs: 0 };
  return { action: "stop", waitMs: 0 };
}

let waited = 0;
for (const job of polls) {
  const next = nextAction(job);
  console.log(`${job.status.padEnd(17)} -> ${next.action}${next.waitMs ? ` (poll again in ${next.waitMs / 1000} s)` : ""}`);
  waited += next.waitMs;
  if (next.action !== "wait") {
    console.log(next.action === "done" ? `model: ${job.fine_tuned_model}` : "job ended without a model");
    break;
  }
}
console.log(`time spent waiting between polls: ${waited / 1000} s`);
```

```csharp runnable title="Poll a job until it ends"
// Poll a job until it ends. These are the statuses the recorded job went through.
var polls = new List<Job>
{
    new("validating_files", null),
    new("queued", null),
    new("running", null),
    new("running", null),
    new("succeeded", "ft:gpt-4o-mini-2024-07-18:example-org:tickets-v2:Tt9xQ2b1"),
};

(string Action, int WaitMs) NextAction(Job job)
{
    if (job.Status == "validating_files") return ("wait", 15000);
    if (job.Status == "queued") return ("wait", 30000);
    if (job.Status == "running") return ("wait", 60000);
    if (job.Status == "succeeded" && job.FineTunedModel != null) return ("done", 0);
    return ("stop", 0);
}

var waited = 0;
foreach (var job in polls)
{
    var next = NextAction(job);
    Console.WriteLine($"{job.Status,-17} -> {next.Action}{(next.WaitMs > 0 ? $" (poll again in {next.WaitMs / 1000} s)" : "")}");
    waited += next.WaitMs;
    if (next.Action != "wait")
    {
        Console.WriteLine(next.Action == "done" ? $"model: {job.FineTunedModel}" : "job ended without a model");
        break;
    }
}
Console.WriteLine($"time spent waiting between polls: {waited / 1000} s");

record Job(string Status, string? FineTunedModel);
```

Expected output:

```text
validating_files  -> wait (poll again in 15 s)
queued            -> wait (poll again in 30 s)
running           -> wait (poll again in 60 s)
running           -> wait (poll again in 60 s)
succeeded         -> done
model: ft:gpt-4o-mini-2024-07-18:example-org:tickets-v2:Tt9xQ2b1
time spent waiting between polls: 165 s
```

**Run the poller as a background job, not in a web request.**

Queue time depends on the provider and can be hours. Start the job from CI or a worker, store the job ID, and let a scheduled task finish the work: read the result, write the manifest, start the eval.

## Read the Loss from the Events

The provider does not hand you a chart. It reports training and validation loss as events, newest first.

```ts runnable title="Find the best point in the job's events"
// The events endpoint lists newest first. Sort by step, then find the best point.
const events = [
  { step: 105, train: 0.364, valid: 0.544 }, { step: 100, train: 0.369, valid: 0.527 },
  { step: 90, train: 0.377, valid: 0.497 }, { step: 80, train: 0.392, valid: 0.472 },
  { step: 70, train: 0.416, valid: 0.456 }, { step: 60, train: 0.455, valid: 0.495 },
  { step: 50, train: 0.515, valid: 0.555 }, { step: 40, train: 0.61, valid: 0.65 },
];

const curve = [...events].sort((a, b) => a.step - b.step);
let best = curve[0];
for (const p of curve) if (p.valid < best.valid) best = p;
for (const p of curve) console.log(`step ${String(p.step).padStart(3)}  train ${p.train.toFixed(3)}  valid ${p.valid.toFixed(3)}${p === best ? "  <- lowest" : ""}`);

const last = curve[curve.length - 1];
const totalSteps = 105, epochs = 3;
const bestEpoch = Math.ceil((best.step / totalSteps) * epochs);
console.log(`validation loss rose from ${best.valid} to ${last.valid} after step ${best.step}`);
console.log(`next run: n_epochs ${bestEpoch} instead of ${epochs}, then compare both on the eval set`);
```

```csharp runnable title="Find the best point in the job's events"
// The events endpoint lists newest first. Sort by step, then find the best point.
var events = new List<Point>
{
    new(105, 0.364, 0.544), new(100, 0.369, 0.527),
    new(90, 0.377, 0.497), new(80, 0.392, 0.472),
    new(70, 0.416, 0.456), new(60, 0.455, 0.495),
    new(50, 0.515, 0.555), new(40, 0.61, 0.65),
};

var curve = events.OrderBy(p => p.Step).ToList();
var best = curve[0];
foreach (var p in curve) if (p.Valid < best.Valid) best = p;
foreach (var p in curve) Console.WriteLine($"step {p.Step,3}  train {p.Train:F3}  valid {p.Valid:F3}{(p == best ? "  <- lowest" : "")}");

var last = curve[^1];
int totalSteps = 105, epochs = 3;
var bestEpoch = (int)Math.Ceiling((double)best.Step / totalSteps * epochs);
Console.WriteLine($"validation loss rose from {best.Valid} to {last.Valid} after step {best.Step}");
Console.WriteLine($"next run: n_epochs {bestEpoch} instead of {epochs}, then compare both on the eval set");

record Point(int Step, double Train, double Valid);
```

Expected output:

```text
step  40  train 0.610  valid 0.650
step  50  train 0.515  valid 0.555
step  60  train 0.455  valid 0.495
step  70  train 0.416  valid 0.456  <- lowest
step  80  train 0.392  valid 0.472
step  90  train 0.377  valid 0.497
step 100  train 0.369  valid 0.527
step 105  train 0.364  valid 0.544
validation loss rose from 0.456 to 0.544 after step 70
next run: n_epochs 2 instead of 3, then compare both on the eval set
```

Validation loss was lowest at **step 70** (**0.456**) and rose to **0.544** by the end. The last epoch made the model worse on data it had not seen. Most hosted APIs keep only the final model, so the fix is another run with fewer epochs, judged on the eval set.

- Some providers also keep a few **checkpoints** from earlier in the run and let you call them by name.
- If yours does, compare the best checkpoint with the final model on the eval set before re-running.
- If it does not, set `n_epochs` from what the events showed and run again.

## Estimate the Tokens Before You Upload

Hosted training is billed by tokens processed: the file's tokens multiplied by the epochs. Count first.

```ts runnable title="Estimate the trained tokens"
// Roughly how many tokens will the job train on? Count before you upload.
// Hosted providers bill training by tokens processed: the file's tokens × epochs.
const trainingLines = [
  { chars: 412 }, { chars: 388 }, { chars: 455 }, { chars: 401 }, { chars: 1960 },
];
const linesInFile = 560;            // wc -l train.jsonl
const epochs = 3;
const pricePerMillion = 3.0;        // illustrative only: use your provider's current price

const avgChars = trainingLines.reduce((s, l) => s + l.chars, 0) / trainingLines.length;
const tokensPerLine = Math.ceil(avgChars / 4);       // about 4 characters per token in English
const fileTokens = tokensPerLine * linesInFile;
const trained = fileTokens * epochs;
console.log(`about ${tokensPerLine} tokens per line, ${fileTokens.toLocaleString("en-US")} in the file`);
console.log(`× ${epochs} epochs = ${trained.toLocaleString("en-US")} trained tokens`);
console.log(`at $${pricePerMillion.toFixed(2)} per million: about $${((trained / 1e6) * pricePerMillion).toFixed(2)} per run`);
```

```csharp runnable title="Estimate the trained tokens"
// Roughly how many tokens will the job train on? Count before you upload.
// Hosted providers bill training by tokens processed: the file's tokens × epochs.
int[] lineChars = [412, 388, 455, 401, 1960];
var linesInFile = 560;            // wc -l train.jsonl
var epochs = 3;
var pricePerMillion = 3.0;        // illustrative only: use your provider's current price

var avgChars = lineChars.Average();
var tokensPerLine = (int)Math.Ceiling(avgChars / 4);   // about 4 characters per token in English
var fileTokens = tokensPerLine * linesInFile;
var trained = fileTokens * epochs;
Console.WriteLine($"about {tokensPerLine} tokens per line, {fileTokens:N0} in the file");
Console.WriteLine($"× {epochs} epochs = {trained:N0} trained tokens");
Console.WriteLine($"at ${pricePerMillion:F2} per million: about ${trained / 1e6 * pricePerMillion:F2} per run");
```

Expected output:

```text
about 181 tokens per line, 101,360 in the file
× 3 epochs = 304,080 trained tokens
at $3.00 per million: about $0.91 per run
```

GPU time is rarely the big cost. A failed job from a bad file wastes an hour of someone's day, which is why the data check runs before the upload.

## When a Job Fails

A failed job says why. The recorded one failed with `invalid_training_file`: line 37 did not end with an assistant message.

| What you see | Where to look | Fix |
|---|---|---|
| `invalid_training_file` | One line of the file | Run the data check, fix the line, upload again |
| Queued for a long time | Nothing is wrong | Keep polling, less often |
| HTTP 401 in CI only | The secret store | Add the key to CI secrets |
| Validation loss rises late | The epochs | One epoch fewer, compare on the eval |
| Model works in the playground, 404 in the app | The app's config | Use the exact model ID from the manifest |

## Other Providers

The four calls are the same idea everywhere. The names change.

| Provider | Start a job | What you call afterwards |
|---|---|---|
| OpenAI | `POST /v1/fine_tuning/jobs` | The `ft:` model ID |
| Azure OpenAI | The same jobs API on your Azure resource, with an `api-version` | A deployment of the new model, which you create |
| Amazon Bedrock | A model customization job, with the data in S3 | The custom model, through the throughput option your model supports |
| Google Vertex AI | A supervised tuning job for a Gemini model | The tuned model's endpoint |
| Self-hosted | The starter kit's `train_qlora.py` on your GPU | Your vLLM server, with the adapter loaded |

Check each provider's current documentation for field names, supported base models and prices. The repo's client code only changes in `apiBase`, the model name and the auth header.

**No API key? Measure the baseline anyway.**

The Fine-tune Lab runs a model in your browser (turn it on in Settings). Its Live eval runs the course's eval tickets through the repo's baseline prompt and scores them like score.py, so you know the number a hosted fine-tune has to beat before you pay for one.

## Hosted or Self-hosted?

| | Hosted | Self-hosted LoRA (the starter kit) |
|---|---|---|
| GPUs | The provider's | Yours, or rented |
| Where the training data goes | To the provider | Stays in your account |
| Base models | The provider's list | Any open model with a licence that fits |
| Serving | The provider's API | vLLM or similar, which you run |
| App code | Same client | Same client, different `apiBase` |

The client in the build-along repo works with both. Only the model name and the URL change.

- **May this data leave our systems?** Get the answer in writing. Redaction lowers the risk. It does not change who receives the data.
- **Who owns the API key,** and where does it live: a secret store, never a laptop or the repo.
- **What happens when the provider retires the base model.** The fine-tune goes with it. Put the date in the calendar.
- **Who signs off** the manifest before release.

**Do's and Don'ts**

- **Do** run the data check before every upload.
- **Do** set hyperparameters under `method`, from a committed config file.
- **Do** store the job ID, the model ID and the dataset hash together.
- **Do** read the events, not just the final status.
- **Don't** poll in a tight loop, or from a web request.
- **Don't** put the API key in code or in the notebook.
- **Don't** ship the model ID without running the eval.

## Common mistakes

| Mistake | Wrong idea | Correct idea | What goes wrong |
|---|---|---|---|
| Upload without checking | "The provider validates it." | Check first, it is faster. | The job fails after waiting in the queue. |
| Polling forever | "Keep asking until it is done." | Stop on succeeded, failed or cancelled. | A failed job is polled all night. |
| Status only | "It succeeded, ship it." | Read the events and run the eval. | An overfitted model goes live. |
| Key in code | "It is just for testing." | Use a secret store. | The key leaks with the repo. |
| Model ID typed by hand | "I copied it from the dashboard." | Read it from the manifest. | The app calls the wrong model. |

## Check Your Understanding

## Exercises

## What to Remember

- A hosted fine-tune is **upload, create, poll, use**.
- Settings come from **config**, epochs go under **method**, and **metadata** ties the job to the data.
- The poller **waits longer** as the job runs and **stops** when it ends.
- Read the **events**: the best point is often before the end.
- The app's client stays the same: **only the model name changes**.

Next: **Reading a Training Run**, how to tell a healthy run from a broken one by its curves.