# Multimodal AI Cheat Sheet

Course: [Multimodal AI](https://www.devtutorials.ai/learn/multimodal-ai) · Section: Reference
Source: https://www.devtutorials.ai/learn/multimodal-ai/reference/cheat-sheet

> Every rule from the course on one page: tool choice, image sizing, documents, verification, review, speech, voice agents, the image studio, video understanding and generation, search, privacy and every lab target.

## How to Use This Page

This page is for coming back to. Every lesson's main rule is here, as a table or a short snippet, with the lesson number. The last sections list every target from the lab, and a glossary.

Model names, sizes and prices change quickly. The numbers on this page come from the lab's seed bank and were checked when the course was written. Measure your own models before you rely on a number.

## The One-Line Version

**A model that can see is not a model that is right.** Pick the right tool, check what it returns, and know what each choice costs.

## Pick the Tool (Lesson 1)

| Job | Best tool |
|---|---|
| Clean printed text | OCR model |
| Counting or locating objects | Object detector |
| Messy, mixed or unusual content | Multimodal LLM |
| Audio to text | Speech-to-text model |

## Images and Tokens (Lesson 2)

| Rule | Why |
|---|---|
| Resize to the smallest size that still reads the digits | Image tokens cost money |
| Keep digits at least 10 px tall after resizing | Smaller, and decimal points vanish |
| Crop to the part you need | Fewer tokens, more detail |

## Documents (Lesson 3)

| Rule | Why |
|---|---|
| Split long documents into pages | Tables across pages stay complete |
| Give layout hints for merged cells | Values stay in the right column |
| Parse money and dates with rules, then a schema | "1.234,50 €" is 1234.50 |

## Screenshots and Charts (Lesson 4)

| Rule | Why |
|---|---|
| Crop small error codes before reading | They are too small at full size |
| Mark chart values as estimates | Pixels are not exact numbers |

## Verify and Review (Lessons 5 and 6)

| Check | Catches |
|---|---|
| Cross-check (lines add up) | Wrong totals |
| Format and range | Garbled or impossible dates |
| Grounding (re-read the box) | Misread names and digits |
| Confidence threshold | Values the model was unsure of |

Route any failed check to a person. Show the source region beside each value. Save every correction as an Evals row.

## Speech and Voice (Lessons 7 and 8)

| Rule | Target |
|---|---|
| Word error rate (WER) | at most 5% |
| Vocabulary hints | product names at least 95% right |
| Normalise numbers | every number as digits |
| Timestamps and speaker labels | a reviewer can jump to the moment and see who said it |
| Stream every step | first audio within 1.5 s |
| Semantic endpoint | 350 ms after a finished sentence, 1,100 ms after an unfinished one |
| Barge-in with a robust VAD | stop within 0.3 s; ignore coughs |
| Holding line during a tool call | no silence over 1.5 s |
| Hand-over | transfer to a person when asked, with a summary |

## Image Generation (Lesson 9)

| Step | Rule |
|---|---|
| Prompt | Name objects, colours and size; negative prompt for people, logos, watermarks |
| Reference | Attach the real product photo |
| Words | Leave space; add a text layer |
| Model | Choose by cost, time, size and spelling; upscale small on-device images |
| Check | Objects, colours, words (OCR), shape, safety |
| Edit | One flaw: inpaint, recolour, outpaint or text layer, then check again |
| Publish | A person approves; attach a C2PA manifest |

## Video (Lessons 10 and 11)

| Rule | Why |
|---|---|
| frames = seconds × fps | Each frame costs tokens |
| Two passes: free motion and sound detectors, then 4 fps near spikes | Finds 1 s events within 40 frames |
| Long clips: one key frame per minute, plus detectors | 15 minutes in 39 frames |
| Video-native models sample about 1 fps | Fine for short clips, costly for long ones |
| Generate from the product photo (image-to-video) | Keeps the product on-model |
| Check sampled frames: identity, words, physics, flicker | One good first frame is not enough |
| cost = max(seconds, 8) × price per second × takes | Work it out before you generate |
| Visible "AI-generated" mark + C2PA | Viewers trust video |

## Search (Lesson 12)

| Rule | Why |
|---|---|
| Words → OCR text index; looks → page-image index | Each holds what the other misses |
| Re-rank the top 10 | Better order for little cost |
| Cite a page for every fact | People can check the answer |

## Privacy and Cost (Lesson 13)

| Rule | Why |
|---|---|
| OCR and speech on the device | Photos and audio cannot be redacted with text rules |
| Send text, not the photo | Cheaper, and no image leaks |
| Redact card, email, phone and address | Nothing personal leaves the device |
| Blur faces and plates in photos you must send | The damage stays visible; the people do not |
| Keep 30 days; delete everywhere on request | Including both search indexes |

```ts
const PATTERNS: [RegExp, string][] = [
  [/\b(?:\d[ -]?){12,15}\d\b/g, "[CARD]"],
  [/[\w.+-]+@[\w-]+\.[\w.]+/g, "[EMAIL]"],
  [/(?:\+44\s?|\b0)7\d{3}\s?\d{6}\b/g, "[PHONE]"],
  [/\b\d{1,4} [A-Z][a-z]+ (?:Street|Road|Lane|Avenue|Close)\b/g, "[ADDRESS]"],
];
```

## Every Lab Target

| Lesson | Target |
|---|---|
| 1 | Best tool for at least 8 of 10 items; cheaper than always-LLM |
| 2 | All 12 totals read, at the lowest cost |
| 3 | At least 95% of key fields right; every multi-page table complete |
| 4 | Bug-report fields at least 90% right; every chart value flagged |
| 5 | 0 silent totals; at least 90% caught; at most 10% false alarms |
| 7 | WER at most 5%; names at least 95%; numbers as digits; timestamps; speaker labels |
| 8 | First audio within 1.5 s; no cut-offs; barge-in within 0.3 s; no stop for a cough; hand-over |
| 9 | A usable 1024 px image for all 6 briefs; no flawed image accepted; under $1.00; approved and labelled |
| 10 | All 6 moments within 2 s; at most 40 frames per clip |
| 11 | A clean take for all 3 shots; no flawed take accepted; under $12; marked and labelled |
| 12 | At least 80% recall on word and layout questions; citations on |
| 13 | 0 leaks; under 0.5¢ per claim; kept at most 30 days; deleted on request |
| 14 | Your five answers set the targets; all five axes pass on the held-out set |

## Glossary

| Term | Meaning |
|---|---|
| Multimodal | Takes in more than one kind of input: text, images, audio, video |
| OCR | Optical character recognition: reading text from an image |
| Image tokens | The pieces an image is cut into for the model; they cost money |
| Grounding | Checking a value against the place it came from |
| Silent error | A wrong value accepted with no warning |
| WER | Word error rate: word changes ÷ words said |
| Endpointing | Deciding the speaker has finished |
| Barge-in | The agent stops when the user talks |
| VAD | Voice activity detection: is someone speaking? |
| Provenance | A record of how content was made |
| Embedding | Numbers that stand for meaning, used for search |
| Recall | Right items found ÷ right items |
| Re-ranker | A slower model that reorders the top search results |
| Negative prompt | What an image model must not draw |
| Inpainting | Redrawing one area of an image |
| Image-to-video | Generating a clip that starts from a real picture |
| Identity drift | A generated product changing between frames |
| Diarization | Labelling who is speaking in a recording |
| C2PA | The open standard for provenance labels |
| PII | Personally identifiable information |
| Redaction | Replacing personal data with a tag |
| Held-out set | Test items never used for tuning |