Why AI Dubs Run Long: Speech Rate, Text Expansion, and the Timing Problem Nobody Talks About

You dub a 10-second English line into German, and the German audio comes back at 12 seconds. Now the speaker's mouth stops moving while the voice keeps talking, or the next line starts before this one ends. Most people blame the AI voice. The real cause sits one step earlier: the translated sentence simply takes longer to say.

Timing is the least discussed problem in AI dubbing, and it is often the one that makes a dub feel cheap. This guide explains why it happens, what the research says about speech rate across languages, and how a well-built dubbing pipeline keeps translated speech inside the original timing. (New to the basics? Start with how AI dubbing works, then come back.)

Languages don't take the same time to say the same thing

Some languages pack more meaning into each syllable, and speakers compensate by talking slower. That trade-off is the core finding of a well-known 2011 study in the journal Language by Pellegrino, Coupé and Marsico at the University of Lyon.

The researchers had native speakers read the same set of translated texts aloud, then measured two things: how many syllables per second each language uses, and how much information each syllable carries. Across the languages studied, syllable rates ranged from roughly 5 to 8 per second.

Japanese speakers move through syllables quickly because each one carries relatively little information. Mandarin speakers go slower because tone makes each syllable denser. A follow-up study in 2019 extended the analysis to 17 languages and found that the two effects nearly cancel out: information rate clusters around 39 bits per second, whatever the language.

Why this matters for dubbing: if information rate is roughly constant, then a faithful translation should take roughly the same time to say. In practice it often doesn't, because translation is not a clean transfer of information. Translators add articles, expand compound ideas, spell out what the source implied, and pick longer words when a short one would sound odd. The time budget is fixed by the video. The words are not.

The research also carries a caveat that matters to anyone dubbing real content: differences between individual speakers, regions and speaking styles are often larger than the average differences between languages. A fast-talking English YouTuber and a calm German narrator can break any rule of thumb.

Translated text usually grows, and short lines grow the most

English is a compact language, so text translated out of English is usually longer. The W3C's internationalization guide on text size republishes IBM's long-standing expansion guidelines for English into European languages:

Source: IBM guidelines, as cited by the W3C. These figures were written for software interfaces, not speech, so treat them as direction rather than a dubbing formula.

Two patterns carry over directly to dubbing:

• Short lines are the worst offenders. "Got it." "No way." "Let's go." These quick reactions have almost no room to absorb a longer translation, and dialogue-heavy videos are full of them.

• German, French, Italian, Spanish and Portuguese tend to run long from English. German also builds long compound words that are hard to shorten without losing meaning.

Character count is not the same as speaking time, though. Chinese and Japanese text can look shorter on the page and still take a similar time to say, because of the syllable-rate differences above. A good dubbing system measures in seconds, not characters.

Subtitles face the same expansion problem in a different form: longer lines mean less reading time per word. If you are deciding between captions and a full dub, our comparison of AI subtitle generators and AI video translators breaks down when each one makes sense, and the subtitle translator handles the captions route.

Four bad fixes you have probably heard

When a translated line runs long, a basic pipeline has only a few options, and each one leaves a fingerprint viewers notice.

1. Speed up the voice. The simplest fix: render the line, then play it faster until it fits. A little works. Too much and the speaker sounds rushed and breathless, which reads as nervous or robotic.

2. Time-stretch the audio. Compressing the waveform after synthesis keeps the length right but adds a metallic, warbly quality, especially on vowels.

3. Let it overflow. The line keeps going after the speaker's mouth closes, or it eats into the next line's slot. Over a long video, small overruns add up and the dub drifts out of sync.

4. Cut the end off. Some tools simply trim words to fit. The audio fits; the meaning doesn't.

All four treat timing as an audio problem to patch at the end. The better approach treats it as a translation problem to solve at the start.

How a well-built dubbing pipeline handles timing

The research community has a name for translation that respects a length budget: isometric translation. It became a formal benchmark at IWSLT 2022, the main academic conference on spoken language translation, with automatic dubbing as the motivating use case. Here is how the pieces fit together in a timing-aware pipeline.

1. Measure the time budget for every segment

Before translating anything, the pipeline transcribes the source and marks when each speaker starts and stops, including natural pauses. Each segment gets a budget in seconds, not characters. That budget is the constraint everything downstream has to respect. (For the full step-by-step pipeline, from speech recognition to synthesis, see our technical breakdown of AI video translation.)

2. Translate with length in mind

Instead of translating freely and fixing the length later, the translation step is told how much room it has. In the IWSLT 2022 shared task, a translation counted as length compliant if it stayed within plus or minus 10% of the source length. Research systems showed this is achievable without wrecking quality:

• AppTek's English to German system reached over 90% length compliance with small losses in translation quality.

• Huawei's submission reported meeting the length requirement for all three language pairs (English to German, French and Spanish).

A 2025 follow-up found that general-purpose large language models are not reliable at this on their own: simply asking them for a same-length translation often fails. Length control needs to be built into the pipeline, not left to a prompt.

3. Generate options, then pick the one that fits

A single translation rarely lands perfectly. Strong systems produce several candidates and choose the one that best balances meaning, natural phrasing and duration. "I'm heading out now" and "I'm off" carry the same meaning at very different lengths.

4. Use rate adjustment sparingly

Once the text roughly fits, a small change in speaking rate closes the last gap. Kept within natural limits, listeners don't notice it. Rate adjustment is a finishing tool, not the main fix.

5. Use the pauses

Real speech has gaps. A dub doesn't have to start and end on exactly the same frames as the original; it has to feel aligned. Shifting a line slightly into a neighboring pause, or redistributing silence across a sentence, often solves what aggressive speed-up cannot. Amazon researchers have published work on this kind of prosodic alignment for off-screen speech.

Exact timing is not the only goal

The largest study of professional human dubbing to date, by researchers at MIT and Amazon, analyzed nearly 320 hours of dubbed video from 54 professionally produced titles. Its finding should shape how anyone evaluates AI dubbing: professional dubbers prioritize vocal naturalness and translation quality over strict character-length matching and lip-sync, and treat timing as important but flexible.

In other words, a dub that is half a second off but sounds natural beats one that hits every frame and sounds rushed. The goal is speech that fits the scene, not speech that fits a stopwatch. (How AI stacks up against studio dubbing on cost and turnaround is a separate question, covered in AI video translation vs manual dubbing.)

Where timing still breaks

Even a timing-aware pipeline has hard cases. Knowing them helps you plan your edit instead of being surprised by the output.

• Rapid-fire dialogue. Two people trading one-word lines leave almost no slack. Short lines expand the most in translation, and there are no pauses to borrow from.

• Dense technical narration. Tutorials and explainers often run fast with few pauses. Shortening the translation risks dropping a step the viewer actually needs.

• Fixed on-screen events. If a line has to land exactly when a product appears or a cut happens, the timing budget becomes a hard wall rather than a soft target.

• Close-up talking heads. When the mouth is large on screen, viewers notice mismatches far more than in wide shots or voice-over, where timing can relax.

• Jokes and wordplay. A punchline that needs extra words to work in the target language may not fit the original beat, and trimming it can kill the joke.

Practical tips for creators

You can make any AI dub fit better by shaping the source and checking the right things.

1. Leave small pauses when you record. A beat between sentences gives the dub room to breathe. It costs you almost nothing in English and saves a lot in German or Spanish.

2. Avoid one-word filler lines when you can. "So." "Right." "Okay." are the lines most likely to come back too long or too rushed.

3. Check the fast sections first. When you review a dub, jump straight to the densest part of your video. If it holds up there, the rest usually will.

4. Listen for rushing, not just sync. A line that technically fits but sounds hurried is a timing failure too. Naturalness matters more than frame-perfect alignment.

5. Edit the transcript, not the audio. If one line keeps running long, shorten the translated text and regenerate. That beats speeding up or trimming the audio.

6. Match strictness to the shot. Be picky in close-ups, relaxed in voice-over and screen recordings.

7. On YouTube, control your own dub. YouTube's built-in auto-dubbing doesn't give you this kind of timing control. Uploading your own dubbed track does; see YouTube auto-dubbing vs uploading your own dub, or start from a link with the YouTube video translator.

Want to hear how your own video holds up in another language? Try AI dubbing on a short clip and check it against the points above. If voice quality is your bigger concern, our guide on how voice cloning works in AI dubbing covers that side of the problem. And since timing is only one piece of reaching a new audience, video localization explained walks through the rest.

References

1. Pellegrino, F., Coupé, C., & Marsico, E. (2011). A cross-language perspective on speech information rate. Language, 87(3), 539 to 558.

2. Coupé, C., Oh, Y. M., Dediu, D., & Pellegrino, F. (2019). Different languages, similar encoding efficiency. Science Advances, 5(9).

3. W3C Internationalization. Text size in translation (citing IBM, Guidelines to Design Global Solutions).

4. IWSLT 2022. Isometric Spoken Language Translation shared task.

5. Wilken, P., & Matusov, E. (2022). AppTek's Submission to the IWSLT 2022 Isometric Spoken Language Translation Task. Proceedings of IWSLT 2022.

6. Li, Z., et al. (2022). HW-TSC's Participation in the IWSLT 2022 Isometric Spoken Language Translation. Proceedings of IWSLT 2022.

7. Lakew, S. M., Virkar, Y., Mathur, P., & Federico, M. (2021). Isometric MT: Neural Machine Translation for Automatic Dubbing. arXiv:2112.08682.

8. Virkar, Y., Federico, M., Enyedi, R., & Barra-Chicote, R. (2022). Prosodic alignment for off-screen automatic dubbing. arXiv:2204.02530.

9. Prompting LLMs: Length Control for Isometric Machine Translation (2025). Proceedings of IWSLT 2025.

10. Brannon, W., Virkar, Y., & Thompson, B. (2023). Dubbing in Practice: A Large Scale Study of Human Localization With Insights for Automatic Dubbing. Transactions of the ACL, 11, 419 to 435.