How voice typing works

Audio formats and compression in voice typing

Digital audio, encoding and compression are three different things. How lossless and lossy compression work, why no algorithm can shrink every file, how speech codecs like Opus trade size against delay, and what that means for dictation.

MacWindowsiPhoneAndroid

In short

Raw speech audio is about 32 KB per second. Compression makes it smaller: lossless compression keeps every bit, lossy compression throws away detail people are unlikely to hear. Speech codecs like Opus are very efficient, but squeezing too hard can remove sounds a recognizer needs. Compression is about moving audio around; it has nothing to do with understanding words.

Teams

You said

the recording is too big to email so i'll share a link instead

SayVelio typed

The recording is too big to email, so I'll share a link instead.

Three words that are often mixed up

  • Digital audio is sound represented as numbers: samples taken many times a second (see how microphones capture speech).
  • Encoding is writing that information in a particular format, with agreed rules for how to read it back.
  • Compression is encoding that uses fewer bits than the plain representation.

Uncompressed audio, usually called PCM, stores every sample directly. A WAV file is typically PCM with a small header. Compressed formats use rules to describe the same sound with less data.

Lossless and lossy

Lossless compression, as in FLAC, lets you rebuild the original samples exactly. It works by finding structure: neighboring samples in speech are similar, so it's cheaper to store a prediction plus a small correction than every sample in full. Typical savings for audio are modest, often around half.

Lossy compression, as in MP3, AAC or Opus, deliberately throws away information that listeners are unlikely to notice and keeps the rest. Savings are much larger, often ten times or more, but the original can't be restored.

Why nothing can compress everything

There's a hard mathematical limit: no lossless method can make every possible file smaller. If it could, you could compress the output again and again until every file was one bit, and then two different files would have to share the same compressed form. Compression only works when data has patterns to exploit. Random-looking data, or data that's already compressed, barely shrinks. That's why zipping an MP3 does almost nothing.

Uncompressed speech compared with lossless and lossy compression: each step makes the data smaller
Approximate size of one minute of speech: uncompressed, lossless, and a speech codec

Speech codecs: built for voice

Voice has structure that general-purpose compression doesn't know about: the vocal tract shapes sound in predictable ways. Speech codecs use that. Opus, the open codec standardized by the IETF in 2012 and used in many calling apps and browsers, combines two approaches:

  • SILK, based on linear prediction, models how the vocal tract shapes sound. It's very efficient for speech.
  • CELT, based on a frequency transform (MDCT), handles music and wide-band sound well.
  • A hybrid mode uses both for different frequency ranges.

Opus works at bitrates from 6 to 510 kbit/s, compared with 256 kbit/s for raw 16 kHz speech.

Size, quality and delay pull against each other

For live audio, compression is a balance of three things:

If you chooseYou gainYou pay
Lower bitrateLess data, works on weak networksLost detail, especially consonants
Longer frames (up to 60 ms in Opus)Less overhead per secondMore delay; a lost packet loses more speech
Shorter frames (down to 2.5 ms)Lower delayMore overhead per second

A phone call needs low delay above all. A dictation that's recognized after you finish speaking cares more about keeping the details a recognizer needs, because those decide whether "fifteen" comes out as "fifty".

What this means for dictation

Compression is a transport decision, not part of recognition. A recognizer can receive uncompressed PCM, a compressed stream or another supported format, and no dictation app is required to use any particular codec. What matters for you:

  • Moderate compression is fine. Modern speech models handle well-encoded audio without trouble.
  • Aggressive compression costs accuracy. Very low bitrates blur exactly the quiet sounds that separate similar words.
  • Bluetooth headsets compress too. Many switch to a narrow, low-bitrate mode while the microphone is in use, which is often a bigger loss than anything the app does.

Questions

Does compression make dictation less accurate?

Light compression barely matters. Very strong compression, or a Bluetooth headset in its low-quality call mode, can remove details and make similar words harder to tell apart.

Is compression the same as encryption?

No. Compression makes data smaller; encryption makes it unreadable to anyone without the key. A recording can be compressed, encrypted, both or neither. Encryption is covered in privacy in voice typing.