Why apply the levers one at a time?

So you know what each lever gave and can undo one

What does the kit report hold?

Cost model, models, latency plan and quality report

Capstone: A Cost-and-Latency Kit for Your Feature

By Syncfusion®, Inc. · Last updated November 2, 2026

Take the expensive support agent, apply every lever from the course one at a time, check each against the quality gate and the latency SLO, and export a cost-and-latency kit with a ship or block decision, in TypeScript or C#.

What you will learn

Apply the course's levers in order and measure cost, first-text time and quality after each one
Build a kit report: cost model, model per step, latency plan and quality report
Make a ship or block decision from quality, latency and budget, and export the kit as JSON

Redesign the Expensive Agent

This is where the course comes together. The support agent from Lesson 2 still runs as it did on day one:

Qwen2.5 7B for every step
a long prompt with every help article
a safety judge the user waits for
the CRM summary on the real-time API

It costs too much, and the first text arrives after 8 seconds. Your job: bring it under the budget and the 4-second SLO, without losing quality, and write it all down in a kit.

Press Play, or use Next to go one step at a time. Each frame adds one lever, from one lesson of the course.

Simple lesson: one lever at a time, measured and gated after each.

A Real-World Example

A team applied six optimisations in one release. Cost fell, and quality fell too. Nobody could tell which change broke the answers, so they rolled back all six.

The next time, they shipped one lever a week, each with an eval. They kept five and dropped one.

Simple lesson: small, measured steps are easy to keep or undo.

What You Will Build

In this capstone, you will:

apply the levers one by one and record cost, latency and quality after each
build the kit report: cost model, models, latency plan and quality report
decide to ship or block, and export the kit

You will finish three files in the kit folder:

Waterfall (src/kit/waterfall.ts, or Kit/Waterfall.cs): the levers in order.
Report (src/kit/report.ts, or Kit/KitReport.cs): the kit.
Release (src/kit/release.ts, or Kit/Release.cs): ship or block.

Open the Code tab. The harness has the expensive ticket, each lever as a small function, the eval result after each lever, and the targets: a 2-point accuracy allowance, a 4-second SLO and a $1,200 monthly budget at a million requests.

Simple lesson: the kit is code, so you can re-run it on your own feature.

The Levers, One by One

Lever	Lesson	What it changes
token-diet	The Token Diet	58% fewer input tokens on the agent and the reply
prompt-cache	Prompt Architecture and Caching	80% of the agent prompt read from cache
judge-after-stream	Latency Engineering	the judge checks after the reply streams
route-small	Routing and Escalation	0.5B for classify and the judge, 1.5B for the agent, 22% escalate
batch-crm	Batch Work	the CRM summary goes to the batch API
tiny-everywhere	(too far)	0.5B for everything

Run the cell. Watch cost and first-text time fall at each step. Then add "tiny-everywhere" and watch accuracy drop.

Simple lesson: each lever builds on the last, and each one is measured.

The Kit Report

The kit is what you hand to your team. It has:

cost model: per request, per 1,000, per user a month, per month
models: which model runs each step
latency plan: time to first text against the SLO
quality report: accuracy before and after, and critical regressions

Run the cell. It shows the model per step after routing, and which steps run in the background.

The kit is a starting point for your own feature. Replace the baseline steps with your records, the levers with the ones you try, and the eval results with your golden set.

Simple lesson: one report with all three corners.

Ship or Block

The release decision is the quality gate from Lesson 1, plus latency and budget. Block when:

accuracy dropped more than the allowance
any critical case regressed
first text is over the SLO
the monthly cost is over budget

Cheap enough is not enough. With only the token diet, the cache and batch, the plan is under budget and still misses the SLO. Every blocker must be listed, so nobody fixes one and ships with another.

Run npm run kit (or dotnet run -- kit). Before: $2.7372 per 1,000 and 8,019 ms to first text. After: $0.4947 per 1,000 and 3,116 ms, which is 82% less. The release says SHIP, and the last line is the kit as JSON.

Try it on a real model. Run the long and the trimmed prompt on a real model, and compare real tokens and first-token time. Switch Model to Real in the lab's top bar. It runs an open model in your browser with WebLLM, and needs WebGPU. It is not graded.

Simple lesson: ship only when quality, latency and budget all pass.

Put It in Your App

Run the kit on your feature. Use your cost records for the baseline, and your golden set for each eval.
Ship one lever at a time. Measure in production after each, with the tags from Measure in Production.
Keep the kit in the repo. Re-run it when prices, traffic or models change, and in the design review of every new AI feature.
Add the guardrails. Budgets, step caps and alerts from Budgets, Limits and Cost Alerts go live with the plan.

Download the kit. After npm run kit runs, the terminal shows a Download kit.json button. Commit the file next to your feature's code.

For product managers: the kit is your one-page answer to "what does this feature cost, how fast is it, and is it still good". Review it before every launch and every price change.

Simple lesson: the kit lives with the code, and is re-run on every change.

Common mistakes

Mistake	Wrong idea	Correct idea	What goes wrong
All levers in one release	"Ship the whole plan."	One lever at a time.	You cannot tell what broke.
Skipping the eval per lever	"Each lever was tested in its lesson."	Re-run the golden set after each.	Levers interact.
Stopping at the first blocker	"Fix this and ship."	List every blocker.	A second problem ships.
Cost without latency	"We are under budget."	Check the SLO too.	Users still wait.
A one-off kit	"We did the analysis."	Keep it in the repo, re-run it.	It is out of date in a month.

Check Your Understanding

Exercises

Build It in the Project

The other files of this lesson's project are optional practice. Finish them to run the lesson's script on your own code.

What to Remember

Start from the measured baseline.
Apply one lever at a time, and measure all three corners after each.
The kit report holds the cost model, models, latency plan and quality report.
Ship only when quality, latency and budget all pass, and list every blocker.
Keep the kit in the repo, and re-run it on every change.

Next: The Cost-Latency-Quality Triangle. Go back to Lesson 1 and run the quality gate on your own feature's golden set.