Speech to Text

By the Syncfusion DevTutorials team · Last updated October 27, 2026

Transcribe voice notes the app can trust: vocabulary hints for product names, number normalisation, word error rate, and timestamps.

What Speech to Text Is

Speech to text (also called ASR, automatic speech recognition) = a model that turns audio into written words.

In this lesson, you will add voice notes to the claims app. Customers record a short message about their claim, and the app turns it into text it can check.

Here is the problem. A customer says: "My Nimbus Mini stopped working." The model writes: "my nimbus many stopped working." The app searches for the product "nimbus many", finds nothing, and the claim stalls.

Press Play, or use Next to go one step at a time.

• Frame 1: what the customer said.
• Frame 2: what the model wrote. The product name is wrong.
• Frame 3: with a list of product names as hints, the model gets it right.

Simple lesson: a speech model only knows the words it has heard before. Your product names are new to it.

A Real-World Example

Think about a new call-centre worker taking notes on the phone.

• They do not know the product names yet, so they write what they hear.
• Someone gives them a list of product names. Now they spell them right.
• They write "£49", not "forty-nine pounds", so the form can add it up.
• They note the time in the call when the customer named the product.

The speech pipeline does the same:

• The product list = vocabulary hints (a prompt or word list for the model).
• "£49", not "forty-nine" = number normalisation.
• The time in the call = word timestamps.

Simple lesson: give the model your words, and turn its words into values your app can use.

What You Will Learn

By the end of this lesson, you will be able to:

• explain the common errors of speech models
• measure word error rate (WER)
• add vocabulary hints for product names
• normalise numbers before you store or compare them
• add word timestamps so a reviewer can jump to the right moment
• label who is speaking in a recorded call (diarization)

This lesson uses the same pipeline stages with audio: the speech model is Perceive, and normalisation is Structure.

Measuring It: Word Error Rate

Word error rate (WER) = the number of word changes needed to turn the transcript into what was really said, divided by the number of words said.

A change is a wrong word, a missing word, or an extra word.

Step through the frames.

• The two lines: what was said, and what the model wrote.
• The marks: each word change.
• The score: changes divided by words.

Simple lesson: measure WER, but also measure the words your app depends on.

Numbers: Words or Digits?

Speech models often write numbers as words. "Forty nine" will never match the 49 on a receipt.

Step through both frames.

• Frame 1: the price, as the model wrote it.
• Frame 2: after normalisation, the amount is 49 and can be checked.

Normalise = make text consistent before you compare or store it: lower case, no punctuation, number words turned into digits.

Simple lesson: apps need values, not words. Turn number words into digits.

Four Fixes, One at a Time

The lab has 12 voice notes, some in quiet rooms and some in noisy kitchens. Here is what each fix does.

Step through all five frames.

• Naive: the smallest model, nothing else. Every product name is wrong.
• Vocabulary hints: product names come right.
• Normalise numbers: every amount is digits.
• A bigger model: fewer errors in noisy clips.
• Word timestamps: a reviewer can jump to the moment.

Who Said It?

Some audio has more than one speaker: a recorded support call has the agent and the customer. A plain transcript runs them together, so nobody can tell who promised the refund.

Diarization = splitting a recording by speaker, and labelling each part (Agent, Customer). Some speech services do it for you; on the device, a small speaker model can do it next to Whisper.

Speech models also handle many languages and accents. Whisper base and small are multilingual; the tiny English model is not. Test with the accents and languages your customers really use.

Click the button and look at the Pipeline tab.

• The waveform: the voice note, with noise shown in grey.
• The transcript: errors are marked red.
• Settings: open the Settings tab to see which Whisper size your device can run.

Lab Exercise: Transcribe 12 Voice Notes

Your task: change the Perceive and Structure settings, then press Check.

Click the button and open the Pipeline tab.

• Model: tiny, base, small or cloud.
• Vocabulary hints, normalise numbers, timestamps, speaker labels: turn each on or off.
• To pass: WER at most 5%, product names at least 95% right, every number as digits, timestamps on, and speaker labels on the two calls.

Build It in TypeScript

normalizeNumbers() turns number words into digits.

const ONES = ["zero", "one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten", "eleven", "twelve",
  "thirteen", "fourteen", "fifteen", "sixteen", "seventeen", "eighteen", "nineteen"];
const TENS = ["", "", "twenty", "thirty", "forty", "fifty", "sixty", "seventy", "eighty", "ninety"];

export function normalizeNumbers(text: string): string {
  const words = text.toLowerCase().replace(/[^\w\s-]/g, "").split(/\s+/).filter(Boolean);
  const out: string[] = [];
  for (let i = 0; i < words.length; i++) {
    const tens = TENS.indexOf(words[i]), ones = ONES.indexOf(words[i]);
    if (tens >= 2) {                                   // "forty nine" → 49
      const next = ONES.indexOf(words[i + 1] ?? "");
      if (next > 0 && next < 10) { out.push(String(tens * 10 + next)); i++; } else out.push(String(tens * 10));
    } else if (ones >= 0) out.push(String(ones));
    else out.push(words[i]);
  }
  return out.join(" ");
}

Click the button and open the Code tab.

• The starter: only lower-cases the text.
• Run on the media bank: your function runs on every test sentence.

TL;DR

1. Speech to text turns audio into words. It does not know your product names.
2. WER = word changes ÷ words said. Aim for 5% or less.
3. Vocabulary hints fix product names.
4. Normalise numbers to digits before you store or compare.
5. A bigger model helps in noise, but costs more and is slower.
6. Timestamps let a reviewer jump to the right moment.
7. Speaker labels say who said what in a call.

Common mistakes

Check Your Understanding

Exercises

What to Remember

• A speech model writes what it hears, not what you meant.
• Give it your product names.
• Normalise numbers to digits.
• Measure WER and the words your app depends on.
• Keep timestamps for review.
• Label speakers in calls.

Further Reading

• Robust Speech Recognition via Large-Scale Weak Supervision (Whisper), by OpenAI: the paper behind the Whisper models.
• Transformers.js documentation, by Hugging Face: run Whisper in the browser.

Next: Voice Agents and Turn-Taking, where the app talks back, and must know when the customer has finished speaking.