Document and Media Context

By the Syncfusion DevTutorials team · Last updated October 20, 2026

How PDFs, Word files, spreadsheets, slides, images, audio and video become labelled text a model can use, how to keep unclear parts marked as unclear, and a four-step framework for any file.

You have a big PDF file with many pages. How do you turn it into something the AI model can use? You can't just say "here's a PDF" to the model.

AI models only read text. So images, videos, audio files, and documents must be turned into text or labels the model understands.

A Real-World Example

Think about a restaurant manager.

• A customer sends a picture of a broken plate.
• The manager doesn't just look at the picture. She says: "I see this is the blue plate from the Premium line, broken at the rim. This is our problem, I'm 80% sure."
• She tags it: broken, high priority, our fault.
• Next week, she counts broken plates by tag, to see if there's a pattern.

The tags, the labels, and the confidence level are as important as the picture.

In simple words: pull out the data, mark where it came from, say how sure you are, and keep a record.

What You Will Learn

In this lesson, you will learn:

1. How to turn PDFs, Word files, spreadsheets, and presentations into plain text.
2. How to pull text and images out of pictures.
3. How to turn audio and video into text.
4. How to mark where data came from, and how sure it is.
5. The four steps: pull it out, label it, sum it up, check it.

Turning Documents Into Text

PDFs

A PDF is a picture of a page. To turn it into text that the model can read, you must:

1. Get every page.
2. Read the text or use image reading (OCR) to get the words.
3. Label which page it came from.
4. Mark if it has images or tables that need special work.

Label each page with its source

const pages = [
  { number: 1, text: "Welcome to Brightgrid.", hasImage: false },
  { number: 2, text: "Pricing: Free plan includes 5,000 rows.", hasImage: false },
  { number: 3, text: "[Diagram of features]", hasImage: true },
];

for (const p of pages) {
  console.log(`Page ${p.number}: ${p.hasImage ? "[has image] " : ""}${p.text.substring(0, 40)}...`);
}

Expected output:

Page 1: Welcome to Brightgrid.
Page 2: Pricing: Free plan includes 5,000 rows.
Page 3: [has image] [Diagram of features]...

Every page gets a label. The model later knows: "this came from page 2 of the PDF, not from the internet."

Word Documents and Spreadsheets

Word files are similar to PDFs. Spreadsheets are different: they have rows and columns.

For a spreadsheet:

• Pick the rows and columns you need.
• Turn them into a table or list.
• Say which sheet and row.

Turn a spreadsheet into a table the model sees

const sheet = [
  ["Customer", "Plan", "Monthly cost"],
  ["Alice", "Free", "$0"],
  ["Bob", "Pro", "$20"],
  ["Carol", "Team", "$50"],
];

for (const row of sheet) {
  console.log(row.join(" | "));
}

Expected output:

Customer | Plan | Monthly cost
Alice | Free | $0
Bob | Pro | $20
Carol | Team | $50

The model sees a clear table with labels. It knows the second row is Alice, and she is on the Free plan.

Presentations

A PowerPoint or Keynote is a set of slides. For each slide:

1. Get the text on the slide.
2. Note if there are charts, graphs, or images.
3. Say what slide number.

Reading Text and Images From Pictures

OCR: Reading Text From Images

OCR means "read text from a picture". A tool looks at an image and finds the words.

For example, a customer sends a picture of a receipt from a store. OCR pulls out the words: "Total: $45.99, Date: 2026-01-15."

Mark OCR confidence

const images = [
  { id: "img-1", text: "Total: $45.99", confidence: 0.95 },
  { id: "img-2", text: "Paid by Visa", confidence: 0.78 },
  { id: "img-3", text: "[blurry number]", confidence: 0.42 },
];

for (const i of images) {
  console.log(`${i.id}: "${i.text}" (${Math.round(i.confidence * 100)}% sure)`);
}

Expected output:

img-1: "Total: $45.99" (95% sure)
img-2: "Paid by Visa" (78% sure)
img-3: "[blurry number]" (42% sure)

When OCR is unsure (below 70%), mark it. The model then knows: "this number might be wrong."

Diagrams and Charts

When a picture shows a diagram or chart:

1. Try OCR to pull text.
2. If it's a chart, describe what it shows: "bar chart of sales by month, January highest."
3. If OCR fails, describe the picture by eye.

Handwriting and Receipts

Handwriting is hard for OCR. If a receipt has handwritten notes, mark them as "hard to read" and give the AI what it might say.

Turning Audio and Video Into Text

Audio Files

An audio file (MP3, WAV) must be turned into words. A transcription tool listens to the audio and writes down what it hears.

Mark a transcript with speaker and time

const transcript = [
  { time: "0:00", speaker: "Agent", text: "How can I help you?" },
  { time: "0:05", speaker: "Customer", text: "I want to cancel my plan." },
  { time: "0:12", speaker: "Agent", text: "I can help with that. May I ask why?" },
];

for (const t of transcript) {
  console.log(`[${t.time}] ${t.speaker}: ${t.text}`);
}

Expected output:

[0:00] Agent: How can I help you?
[0:05] Customer: I want to cancel my plan.
[0:12] Agent: May I ask why?

Now the model can read the conversation. It knows who spoke and when.

Video Files

A video is pictures plus audio. To turn it into text:

1. Get the audio and turn it into a transcript (like above).
2. Get the pictures that show important moments (key frames).
3. Describe what happens in each key frame.
4. Tie the description to the time in the video.

Build a video summary with time stamps

const keyFrames = [
  { time: "0:00", description: "Customer on screen, pointing at screen" },
  { time: "0:08", description: "Close-up of error message: 'Quota exceeded'" },
  { time: "0:15", description: "Agent appears on screen, nods" },
];

for (const k of keyFrames) {
  console.log(`[${k.time}] ${k.description}`);
}

Expected output:

[0:00] Customer on screen, pointing at screen
[0:08] Close-up of error message: 'Quota exceeded'
[0:15] Agent appears on screen, nods

The model now knows: at 8 seconds, an error message appeared.

Labeling Source and Confidence

Trust Levels

Mark every piece of data with a trust level:

• System: you made it, you trust it fully.
• Official: from your company's own sources.
• Community: from users or forums, be careful.
• Untrusted: from outside, could be wrong.

Confidence Scores

When a tool reads data (like OCR), it says how sure it is, from 0 to 1:

• 0.9 to 1.0: very sure
• 0.7 to 0.9: pretty sure
• 0.5 to 0.7: not very sure
• Below 0.5: don't trust it

Pull data with trust and confidence

const data = [
  { source: "official", confidence: 0.99, text: "Free plan: 5,000 rows" },
  { source: "ocr_receipt", confidence: 0.85, text: "Total: $99" },
  { source: "ocr_handwriting", confidence: 0.42, text: "Paid by ???" },
];

for (const d of data) {
  const sure = d.confidence > 0.7 ? "trust this" : "CHECK THIS";
  console.log(`${d.source}: "${d.text}" (${sure})`);
}

Expected output:

official: "Free plan: 5,000 rows" (trust this)
ocr_receipt: "Total: $99" (trust this)
ocr_handwriting: "Paid by ???" (CHECK THIS)

Provenance Labels

Provenance means "where it came from". Always label:

• The file name or page.
• The time, if important.
• What tool pulled it out (OCR, transcription, etc).

The Four-Step Process

Step 1: Extract

Pull the data out of the file. For a PDF, get each page. For an image, run OCR or describe it. For audio, run transcription. For video, get the key moments.

Step 2: Label

Tag every piece with:

• What file or page it came from.
• Its trust level.
• Its confidence score (if a tool pulled it).
• When it was created or updated.

Step 3: Summarize

For long documents, pull out the key points, not the whole thing. A 50-page handbook should become 10 key sections the question needs.

Step 4: Verify

Double-check:

• Did OCR read it right, or did it guess?
• Is it up to date?
• Is the label clear?
• Is the data good enough to use?

The four steps in order

const steps = [
  { step: "Extract", action: "Get pages, run OCR, transcribe audio, find key moments" },
  { step: "Label", action: "Mark file, source, confidence, date, trust" },
  { step: "Summarize", action: "Pull key points for this question" },
  { step: "Verify", action: "Check OCR, dates, labels, quality" },
];

for (const s of steps) {
  console.log(`${s.step}: ${s.action}`);
}

Expected output:

Extract: Get pages, run OCR, transcribe audio, find key moments
Label: Mark file, source, confidence, date, trust
Summarize: Pull key points for this question
Verify: Check OCR, dates, labels, quality

Multimodal Risks

OCR Mistakes

OCR reads pictures, but it can be wrong:

• A "0" might be read as "O".
• Handwriting is very hard.
• Small text can fail.
• Rotated text can be missed.

Always mark low confidence. When confidence is below 70%, tell the model: "this might be wrong."

Mixing Old and New

If you pull data from many files or pages:

• Mark the date of each one.
• If they don't match, tell the model they differ.
• Don't hide old data.

Format Confusion

If a picture shows a table, OCR might:

• Read it as plain text.
• Miss the rows and columns.
• Get the order wrong.

Run OCR, but also describe what kind of table it is, so the model knows.

Video vs. Audio

If you have a video:

• Get the audio transcript.
• Also describe what is on the screen.
• Link them by time.

Don't assume the audio alone tells the whole story. The customer might be showing something on the screen while they talk.

Common mistakes

Exercises

Each exercise ends with a value on the last line. Press Check to compare it with the expected answer.

What to Remember

• Extract: turn PDFs, images, audio, and video into text and labels.
• Label every piece: file, page, date, source, trust level, confidence score.
• Summarize long documents: pull key points, not everything.
• Verify data, especially OCR and transcription. Mark low confidence.
• Mark when data is old or mixed. The model then knows.

Next: Memory Management, how to keep facts from a long chat, and let them expire when they're no longer needed.