# Video Generation

Course: [Multimodal AI](https://www.devtutorials.ai/learn/multimodal-ai) · Section: Listening, Speaking and Generating
Source: https://www.devtutorials.ai/learn/multimodal-ai/listening-and-generating/video-generation

> Generate short product videos that stay on-model: image-to-video from the product photo, several takes, frame checks for identity, words and motion, cost per second, and a visible AI label.

## What Video Generation Is

**Video generation** = a model that makes a short video clip from a prompt (**text-to-video**), or from a picture plus a prompt (**image-to-video**).

In this lesson, the claims app needs three short clips for its help pages: the KettleMax with steam rising, the Zephyr Pro fan spinning up, and a parcel box with the words "Free returns".

The first version types a prompt and uses the first take. The kettle in the clip is a different kettle. Halfway through, it even changes colour.

Press **Play**, or use **Next** to go one step at a time.

- **Frame 1:** text-to-video, three takes per shot. Most takes drift off the real product.
- **Frame 2:** image-to-video, starting from the real product photo. The product stays the same.

**Simple lesson:** a text prompt describes *a* kettle. Show the model *your* kettle.

### A Real-World Example

Think about a film crew shooting an advert.

- They bring the real product to the set, not a drawing of it.
- They shoot several takes of each shot.
- The director watches every take, frame by frame, for mistakes.
- The titles are added afterwards, in editing.
- The end card says it is an advert.

Video generation works the same way:

- **The real product** = image-to-video from the product photo.
- **Several takes** = generate 2 or 3 takes, from different seeds.
- **Watch every take** = frame checks: product identity, words, motion.
- **Titles in editing** = words as a text layer, not drawn by the model.
- **The end card** = a visible "AI-generated" mark and a C2PA label.

**Simple lesson:** anchor on a real image, shoot takes, check frames, label it.

## What You Will Learn

By the end of this lesson, you will be able to:

- choose between text-to-video and image-to-video
- spot the common flaws: identity drift, impossible motion, flicker and misspelt words
- check generated takes frame by frame
- work out the cost of a batch before you run it
- label generated video for viewers

## The Flaws to Look For

| Flaw | What you see | Check |
|---|---|---|
| Identity drift | The product changes shape or colour between frames | Compare sampled frames with the product photo (image embeddings) |
| Wrong words | "Fre Retunrs" | OCR on frames, or add the words as a text layer |
| Impossible physics | Steam falls, fan blades bend | Watch motion in sampled frames (a vision model, or a person) |
| Flicker | Brightness jumps between frames | Compare neighbouring frames |
| Wrong length | The model makes 8 s; you asked for 6 s | Trim to length |

**Checking a video means checking many images.** Sample frames from each take, as in Lesson 10, and check each one. The same image embeddings you use for search (Lesson 12) can tell whether a frame still shows your product.

Step through all three frames.

- **The row of checks:** product identity, words, then motion.
- **The score:** flawed takes still accepted, and shots with a clean take.

**Simple lesson:** each check catches its own kind of flaw.

## What It Costs

Video models are billed **per second of video**, and most make a fixed length (often 8 seconds) whether you need it or not.

Step through all three frames.

- **High quality, 3 takes:** far over budget.
- **Fast, 5 takes:** still over.
- **Fast, 3 takes:** within the $12 budget, and every shot has a clean take.

cost = 8 s (or the shot's length, if longer) × price per second × takes

| Model | Price per second | Wait per take |
|---|---|---|
| Hosted, high quality (e.g. Veo 3, Sora 2 Pro) | about $0.40 | about 1.5 minutes |
| Hosted, fast (e.g. Veo 3 Fast, Sora 2) | about $0.15 | about 30 seconds |

Prices here are illustrative. Check your provider's current prices before you plan a batch.

**Label generated video, visibly.** People trust video more than pictures. Put a visible "AI-generated" mark on the clip, and attach a C2PA manifest. Never generate a real person, or footage that looks like a real event, such as a "customer's" damaged parcel.

**Easy way to remember:** photo first, a few takes, check the frames, words on top, label it.

Click the button and look at the **Pipeline** tab.

- **The player:** press **Play take** to watch each take. The flaws really happen as it plays.
- **The table:** the real flaws, and what your checks found.

### Lab Exercise: A Clean, Labelled Take for Every Shot

Your task: set up the video stages, then press **Check**.

Click the button and open the **Pipeline** tab.

- **Ingest:** text only, or the product photo as the first frame.
- **Perceive:** the model, takes per shot, and the seed.
- **Structure:** a text layer for words, and trimming.
- **Verify:** identity, words and motion checks.
- **Use:** the visible mark and the C2PA label.
- **To pass:** a clean take for all 3 shots, no flawed take accepted, under $12, marked and labelled.

## Build It in TypeScript

Work out the cost before you press Generate.

```ts
export function costFor(seconds: number, perSec: number, takes: number): number {
  const billed = Math.max(seconds, 8);                 // the model always makes at least 8 s
  return Math.round(billed * perSec * takes * 100) / 100;
}
```

## TL;DR

1. **Text-to-video** invents the product; **image-to-video** starts from the real photo.
2. Generate **a few takes**; keep the first clean one.
3. Check **frames**: identity, words, physics, flicker, length.
4. Put words on as a **text layer**.
5. Video is billed **per second**, often 8 s minimum: work out the cost first.
6. Add a **visible mark** and a **C2PA label**.

## Common mistakes

| Mistake | Wrong idea | Correct idea | What goes wrong |
|---|---|---|---|
| Text-to-video for a real product | "The prompt names it." | Start from the product photo. | A different kettle, changing colour. |
| One take | "The first one looks fine." | Two or three takes, then check. | A flawed clip, or none at all. |
| Checking the first frame only | "It starts right." | Check sampled frames across the clip. | The product drifts halfway through. |
| Too many takes on the best model | "Quality first." | Work out the cost first. | A $29 batch for three help clips. |
| No visible label | "The metadata is enough." | A visible mark and a manifest. | Viewers think it is real footage. |

## Check Your Understanding

## Exercises

## What to Remember

- Show the model your real product.
- Shoot a few takes; keep a clean one.
- Check frames, not just the first one.
- Words go on as a text layer.
- Work out the cost first.
- Label it, visibly.

## Further Reading

- [C2PA specification, by the Coalition for Content Provenance and Authenticity](https://c2pa.org/specifications/specifications/2.1/index.html): provenance for video as well as images.
- [Content Credentials](https://contentcredentials.org/): how labels are shown to viewers.

Next: **Multimodal Search**, where the app finds the right page in an archive by what it says and by how it looks.