# Human Review and Your v1 Pipeline

Course: [Multimodal AI](https://www.devtutorials.ai/learn/multimodal-ai) · Section: Seeing and Reading
Source: https://www.devtutorials.ai/learn/multimodal-ai/seeing-and-reading/review-and-export

> Build the review screen: each doubtful field beside its source region, one click to accept or correct, and every correction saved as an Evals row. Then export your v1 document pipeline.

## What Human Review Is

**Human review** = a person checks the values the pipeline was not sure of, using the source, before they are used.

In this lesson, you will build the review screen, turn every correction into test data, and export your v1 document pipeline.

Lesson 5 sent the £1,860.00 total to review. The reviewer sees the value, a "check this" flag, and two buttons. Nothing else. They are busy, the number looks like a number, and they press **Accept**.

The check worked. The review screen did not.

Press **Play**, or use **Next** to go one step at a time.

- **Frame 1:** the review screen with no source: the reviewer has to guess.
- **Frame 2:** the source region beside the value: the mistake is obvious in a second.

**Simple lesson:** a reviewer who cannot see the source is guessing.

### A Real-World Example

Think about a teacher marking an exam.

- The answer sheet shows each answer **next to the question** it answers.
- The teacher marks it right or wrong, and writes the right answer next to a mistake.
- At the end of term, the school counts which questions most students got wrong, and teaches them again.

Review works the same way:

- **The question next to the answer** = the source region next to the value.
- **The teacher's correction** = the reviewer's fix.
- **Counting mistakes per question** = Evals rows: every correction saved so the pipeline can be tested on it.

**Simple lesson:** show the evidence, record the fix, and learn from it.

## What You Will Learn

By the end of this lesson, you will be able to:

- explain why reviewers need the source region beside each value
- build a review screen for routed fields
- save corrections as Evals dataset rows
- export a v1 pipeline with an eval harness
- run the harness before connecting a real model

This lesson finishes **Route** with a person in the loop, and packs Lessons 2 to 6 into your v1 pipeline.

## The Review Queue

With the Lesson 5 checks, a small set of fields goes to review. Most of them are really wrong, which is what keeps reviewers trusting the queue.

Step through both frames.

- **Frame 1:** the queue: what was flagged, and why.
- **Frame 2:** the parts of a good review screen.

| Part | Why it is there |
|---|---|
| Source region | The reviewer checks the value against the page, not against the model |
| The model's value and confidence | What to accept or correct |
| Failed checks | Why this field is here, and where to look |
| Accept or correct | One click, or one edit |
| An Evals row | So the fix is not lost |

The source region comes from the box the model gave in Lesson 5. Without boxes, a reviewer has to open the whole file and search.

**Easy way to remember:** evidence beside the claim, reason beside the evidence.

**Simple lesson:** the review screen is part of the pipeline, and it needs the same care.

## Every Correction Is Test Data

A reviewer's fix helps one claim. Saved the right way, it helps every future claim.

Step through all seven frames.

- **Frame 1:** a correction saved as a note: nobody can test against it.
- **Frame 2:** the same correction as an Evals row.
- **Frames 3 to 7:** from one correction to a better next version.

An Evals row has six parts:

| Field | Example |
|---|---|
| `item_id` | `x02` |
| `field` | `total` |
| `model_value` | `1860.00` |
| `corrected_value` | `186.00` |
| `source_box` | `x02#total` |
| `pipeline_version` | `claims-v1` |

**Keep personal data out of the dataset.** Store what a test needs: the item, the field, the values and where they came from. Do not copy customers' names, emails or card numbers into Evals rows.

The Evals & Observability course runs these rows every time you change the pipeline, so a mistake that was fixed once is caught if it ever comes back.

**Simple lesson:** a correction you cannot test against is a correction you will make again.

### Lab Exercise: Review the Queue

Your task: set up the review screen, then accept or correct every field in the queue against its source. Then press **Check**.

Click the button and open the **Review** tab.

- **Review screen:** turn on the source region, the failed checks, and Evals rows.
- **Each card:** the source region, the model's value, and a box for the right value.
- **Saved rows:** every decision, as it will be stored.
- **To pass:** every field shows its source region, and every correction is saved as an Evals row with the value the source shows.

## Your v1 Pipeline

Lessons 2 to 6 built a document pipeline, stage by stage. The **Export** tab packs it into a zip you can add to your own app.

Step through all four frames.

- **The stages:** `ingest.ts`, `perceive.ts`, `structure.ts` and `verify.ts`.
- **Review and cost:** the review queue component, the Evals row type and the cost model.
- **Evals:** a harness and 16 hand-checked items.
- **The README:** which parts passed, and a checklist for connecting a real model.

The export uses your Pipeline settings and the files you wrote in the Code tab. For any stage you did not write, it uses the lab's reference version.

The harness runs your `verify()` and `route()` on the hand-checked items, and fails if any wrong total would be accepted:

```ts
// evals/harness.ts (part)
for (const file of readdirSync(dir)) {
  const { input, truth } = JSON.parse(readFileSync(join(dir, file), "utf8"));
  const { decision } = check(input);
  // compare each field with the truth, and count silent errors
}
console.log({ silentTotals, catchRate, falseAlarmRate });
if (silentTotals > 0) process.exitCode = 1;
```

Click the button and open the **Export** tab.

- **The table:** which lessons' parts pass the lab's checks.
- **Download:** `multimodal-pipeline-v1.zip`.
- **Files in the zip:** the full list.

**The v1 pipeline does not call a model yet.** `perceive.ts` is a stub, so you can choose your provider. Connect it, then run the harness on 30 of your own items with hand-checked answers before you trust it.

**Simple lesson:** your v1 is the six stages, the review screen, and a test that fails on silent errors.

## TL;DR

1. **Human review** = a person checks doubtful values against the source before use.
2. **The source region** goes beside each value. Without it, review is guessing.
3. **Show why** each field is in the queue: the failed checks.
4. **Every correction** is saved as an Evals row: item, field, model value, corrected value, source box, version.
5. **No personal data** in Evals rows.
6. **The v1 export** = your stages, the review queue, a cost model and an eval harness.

## Common mistakes

| Mistake | Wrong idea | Correct idea | What goes wrong |
|---|---|---|---|
| Showing only the value | "The reviewer knows what to do." | Show the source region beside it. | Reviewers accept wrong values. |
| Saving corrections as notes | "We just need to fix this claim." | Save Evals rows. | The same mistake comes back, untested. |
| A huge queue | "Review everything to be safe." | Route only doubtful fields (Lesson 5). | Reviewers stop reading. |
| Copying personal data into Evals | "More context helps." | Keep only what a test needs. | A privacy problem in your test data. |
| Connecting a model before testing | "The lab already passed." | Run the harness on your own items first. | Silent errors in production. |

## Check Your Understanding

## Exercises

## What to Remember

- Review without the source is guessing.
- Show the source region, the value, and why it is in the queue.
- Save every correction as an Evals row.
- Keep personal data out of test data.
- Your v1 pipeline: six stages, a review screen, and a harness that fails on silent errors.
- Test on your own items before you trust a real model.

## Further Reading

- [People + AI Guidebook, by Google PAIR](https://pair.withgoogle.com/guidebook/): feedback, control and human review in AI products.
- [Web Workers API, on MDN](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API): how the lab runs your stage code safely in the browser.

Next: **Speech to Text**, the first lesson of Listening, Speaking and Generating, where you measure word error rate and fix product names a speech model keeps getting wrong.