Why does the capstone use a held-out set?

Settings tuned on the lessons' items might not work on new ones

The Cost axis fails. Where do you look first?

Lesson 2 image sizes and Lesson 13 photo uploads

Capstone: Ship the Claims Pipeline

By the Syncfusion DevTutorials team · Last updated October 27, 2026

Run every lesson's settings on a held-out set, pass five axes (accuracy, verification, cost, latency and privacy), and export the full pipeline.

What the Capstone Is

Capstone = the final project. You run the whole claims pipeline you built, with every lesson's settings, on items it has never seen.

Each lesson tested one stage on its own items. That is not enough. Settings can be tuned until they pass a small test, and then fail on the next receipt. This is why the capstone uses a held-out set: 4 receipts, 3 invoices, 6 extractions and 4 voice notes that no lesson used.

Press Play, or use Next to go one step at a time.

Frame 1: every stage, with the lesson that set it.
Frame 2: the scorecard: five axes, each with its target.

Simple lesson: a pipeline is ready when it passes on items it has never seen.

A Real-World Example

Think about a driving test.

You practise on roads you know, again and again.
The examiner takes you on roads you have never driven.
They score you on several things: control, safety, signals, speed.
Failing one is enough to fail the test, and they tell you which one.

The capstone works the same way:

Roads you know = each lesson's own items.
New roads = the held-out set.
Several scores = accuracy, verification, cost, latency and privacy.
Which one failed = the scorecard points you back to the lesson to fix.

Simple lesson: test on new items, on every axis, and fix the one that fails.

What You Will Learn

By the end of this lesson, you will be able to:

set targets before you test
run a held-out evaluation
read a five-axis scorecard
trace a failing axis back to its lesson
export the full pipeline

First: Five Questions

Before you run anything, answer five questions about your own app. The answers become PLAN.md in your export.

Step through both frames.

Frame 1: the five questions and the reference answers.
Frame 2: with no answers, even perfect settings do not pass.

Question	Why it matters
Which media will your app take in?	Decides which stages you need
Which fields are critical?	Decides what must be checked
What accuracy bar must automatic values meet?	Decides how much goes to review
How fast must a claim be processed?	Decides cloud or device, streaming or batch
What may leave the user's device?	Decides where each model runs

Targets chosen after you see the scores are not targets. If you pick the bar after the run, you will pick whatever passed. Decide first.

Five Axes

Axis	Target	Lesson
Accuracy	at least 95% on receipts, invoices and speech	2, 3, 7
Verification coverage	0 silent totals, at least 90% of wrong fields caught, at most 10% false alarms	5
Cost	at most 8¢ per claim	2, 13
Latency	at most 15 s per claim	3, 7, 13
Privacy	0 pieces of personal data uploaded	13

These are the defaults. Your answers change them:

Your answer	What changes
Accuracy bar 98% or 99%	The accuracy target rises to match
"100%: every value is reviewed"	Every wrong field must be caught; false alarms are allowed
"Under 1 minute" or slower	The latency target relaxes
Media: voice notes only, or receipts and invoices only	Only those parts count for accuracy
"Anything, under a data agreement"	Uploads are allowed, but still logged

The reference settings pass every axis: 100% accuracy, 4 of 4 wrong fields caught with 7% false alarms, about 1.5¢ and 8.1 s per claim, and no personal data uploaded.

Step through both frames.

A tiny speech model with no fixes: Accuracy fails.
The receipt photo sent to the cloud: Privacy fails.

One shortcut fails one axis. Do not lower a target to make it pass. Go back to the lesson that owns the setting, fix it there, and run the capstone again.

Easy way to remember: questions first, held-out set, five green rows, then export.

Lab Exercise: Run the Capstone

Your task: answer the five questions in the Capstone tab, then press Run the capstone.

Click the button and open the Capstone tab.

The questions: one answer each. The targets they set appear under the questions.
Follow one held-out claim: see the claim at each of the six stages, with your settings.
The scorecard: each axis, its value and its target.
To pass: all five axes green, with 0 silent errors on totals.

Export the Full Pipeline

Click the button and open the Export tab.

The table: every part of the pipeline, and whether it passes.
The zip: multimodal-pipeline.zip when everything is ready.

The export adds these files to your v1 pipeline from Lesson 6:

File	What it holds
`src/speech.ts`	Your speech settings and `normalizeNumbers()`
`src/turnTaking.ts`	Your endpointing and barge-in settings
`src/turnTaking.check.ts`	Your `shouldEndTurn()`
`src/imageStudio.ts`	Your image studio settings and `judge()`
`src/sampleFrames.ts`	Your `sampleFrames()` and video settings
`src/videoGeneration.ts`	Your video generation settings
`src/routeQuery.ts`	Your search routing and citation settings
`src/redact.ts`	Your `redact()` rules
`PLAN.md`	Your five answers and the targets

Run the eval harness in the zip before you connect a real model. Then swap the seed bank for your own provider, and run it again.

TL;DR

Test on a held-out set, not the lessons' own items.
Answer the five questions first; they set the targets.
Five axes: accuracy, verification, cost, latency, privacy.
All five must pass, with 0 silent totals.
A failing axis points you to its lesson.
Export, then run the harness with your real model.

Common mistakes

Mistake	Wrong idea	Correct idea	What goes wrong
Testing on the lessons' items	"It passed already."	Use the held-out set.	It fails on the first new receipt.
Setting targets after	"Let's see what we get."	Answer the questions first.	Any result looks good enough.
Lowering a target	"8¢ is close to 10¢."	Fix the setting in its lesson.	The pipeline ships over budget.
Watching one axis	"Accuracy is 100%."	All five must pass.	Personal data leaks while accuracy is perfect.
Skipping the harness	"The lab passed."	Run it with your real model.	The real model behaves differently.

Check Your Understanding

Exercises

What to Remember

Decide the targets first.
Test on items you have never seen.
Every axis must pass.
Fix a failing axis in its own lesson.
Export, then test again with your real model.