How to translate a video into another language and lip-sync it

Last updated 2026-09-01 · 8 min read

What is the short version?

A subtitled version reaches the people who are willing to read. A dubbed version reaches considerably more of them. And a dubbed version where the mouth genuinely matches the new words stops reading as a dub at all, which is a different thing again from either.

1. Go to Vizard Agent and upload the video.
2. Say which language, and which accent if it matters.
3. Say explicitly that you want the lips synced.

What do you need before you start?

The video with clean speech in it. Vizard Agent clones your voice from that audio, so a recording where you are clearly audible produces a translated voice that still sounds like you rather than like a stranger reading your script.

• The video. With your original speech.
• The target language. And the region, if it matters.
• Any burnt-in text. Say if the old subtitles are baked in.
• The length. Whether it must stay the same.
• The face. Front-on works far better than profile.

What do you type into Vizard Agent?

Ask for the lips specifically, in those words. Vizard Agent will otherwise produce a voiceover in the new language laid over your unchanged mouth, which is a perfectly good deliverable in its own right and simply not the one you had in mind when you asked.

Prompt

Variants worth knowing:

• "Make my lips sync." Say it, or you get a voiceover.
• Mention burnt-in text. It has to be erased, not covered.
• Name the region. Spanish from Spain is not Spanish from Mexico.

What does Vizard Agent actually do?

Here is the order Vizard Agent worked in on a real translated short. About a third of the session goes on something the brief mentioned in passing: English subtitles were baked into the picture and had to come out.

1. Checks the file's details and transcribes the audio.
2. Extracts sample frames and surveys every shot in the video.
3. Reviews the voice options and extracts the audio for cloning.
4. Generates the translated narration in the cloned voice.
5. Measures the duration of every translated line.
6. Detects the burnt-in subtitles in the picture.
7. Adjusts the translated phrasing so each line fits its original slot.
8. Measures the real speech length of each line rather than trusting the file's duration.
9. Reviews the detected text boxes and locates the subtitle band over time.
10. Draws a heat map of where text sits across the whole video and reviews it.
11. Tracks the subtitle line word by word and measures its exact height.
12. Checks the doubtful positions and measures the title and the closing card.
13. Builds the mask that will erase the old text and chooses the replacement typeface.
14. Erases the old subtitles and applies the colour grade, then checks whether any survived.
15. Assembles the translated voice track and gets the timings for the new subtitles.

Step ten is the step that makes the erasure reliable. Subtitles move around — they wrap to two lines, they shift for a title card — so Vizard Agent maps where text actually appears over the whole runtime rather than assuming a fixed band.

What does the result look like?

The file this page is written from measured 1280x720 at 24fps and ran 46.29 seconds with AAC audio. It carries a Spanish voice cloned from the speaker's own, lips moving to the Spanish words, new subtitles, a colour grade — and no trace of the English text that was baked into the source.

Each translated line was fitted to the time its original took. That is what keeps the mouth honest: a Spanish sentence forty per cent longer than the English one cannot be lip-synced into the same gap.

When does this not work well?

Lip-sync is by some distance the most fragile part of this job, and it is also the most obvious thing in the video when it goes wrong. Vizard Agent tells you up front which shots will not carry it, rather than syncing them badly and leaving you to notice.

• Profile or moving shots. The sync works best front-on.
• Languages that run long. The phrasing has to be tightened.
• Fast speech. Little room to fit a longer translation.
• Text over faces. Erasing it damages the mouth region.
• Several speakers. Each one needs its own cloned voice.

How do you fix a result that came back wrong?

Say which line is wrong, or which shot. Vizard Agent keeps every cloned take, the measured line timings, the text heat map and the erasure mask, so a re-phrased line or an improved erasure at one timestamp does not disturb anything else in the video.

• "That line runs past the shot." Re-phrased shorter and re-synced.
• "I can still see the old text." Mask widened at that timestamp.
• "The accent is wrong." Re-cloned for the region you name.

How does Vizard Agent compare to doing it yourself?

By hand this is three separate specialists: a translator who writes to time, a dubbing session, and a compositor to paint out the old subtitles. Vizard Agent measures each line's real duration and maps the old text before touching either.

Common questions

Will it still sound like me? Yes. Vizard Agent clones your voice from the video's own audio.

Do the lips really match? On front-on shots, closely. Vizard Agent tells you which shots will not carry it.

What if the translation is longer? It gets tightened. Vizard Agent re-phrases so the line fits its original slot.

Can it erase my old subtitles? Yes. Vizard Agent maps where the text sits over time and masks it out.

Why not just cover them with a bar? Because a black bar across a cinematic shot is worse than the subtitle was. Erasing keeps the picture.

Can it add new subtitles? Yes, in the new language, timed to the new voice track.

Which accent will I get? Whichever you name. Vizard Agent treats regional variants as different requests.

Can it handle two speakers? Yes, but each needs a cloned voice. Vizard Agent separates them by speaker first.

Will the video get longer? Not if you ask it not to. Vizard Agent fits the translation to the original timings.

Does it keep the music and effects? Yes. Vizard Agent replaces the speech and leaves the rest of the mix underneath.

Is dubbing my own video honest? It is your words, your voice and your face. The line worth holding is not doing it to somebody else's video, or to make a person appear to say something they never said.

Can I get a subtitled version too? Yes. Vizard Agent produces both from the same translation.

Does it check the result? Yes. Vizard Agent reviews the frames to confirm the old text is gone and the sync holds.