How to separate the two colours of subtitle burnt into a video
Tell Vizard Agent that the video carries two kinds of burnt-in text in two colours and that you need them kept apart. It reads each colour in its own region of the frame, works out which one is dialogue and which is annotation, and returns a timed translation with the two tracks separated.
What do you type into Vizard Agent?
Name the colours explicitly. A request to "translate the subtitles" gets you one merged track, because to a transcriber every line of text on screen is a subtitle, and the distinction you care about is a visual one. If you know which colour means what, say so, and ask Vizard Agent to check it anyway.
Prompt
What does Vizard Agent actually do?
Almost all of the effort goes into reading small text accurately rather than translating it. Dense characters at subtitle size are a few dozen pixels tall, and read from the whole frame they come back as similar-looking characters that produce a fluent, confident and wrong translation. In the session behind this article it:
- Extracted frames from a short-form video with Chinese dialogue in white and green annotations above it, then a denser set so every subtitle change was captured.
- Opened a full-size frame to fix the position and colour of each line, then separated the two text regions and enlarged each one.
- Ran automatic recognition on both colours as an independent cross-check on its own reading of the enlarged frames.
- Compared several timestamps to see what the green text changed with, and checked it late in the video to confirm it was annotation rather than dialogue.
- Cross-checked with an analysis pass that separated the two by timestamp, then produced the timed translation with white and green kept apart.
- Repeated the whole process for the next video the user uploaded.
When does this not work well?
Some text cannot be separated by colour. If the two kinds share a colour, if the annotation sometimes appears in the dialogue position, or if heavy compression washes out thin coloured strokes, the separation has no reliable signal to work from. Text that changes colour for emphasis, overlapping layers and very small text on a busy background cause the same trouble.
Common questions
Does it need to be two colours? No. Vizard Agent can split by position too, if the two kinds sit consistently apart.
What if I do not read the language? That is the usual case. Vizard Agent gives you the original and the translation.
Can I get a subtitle file? Yes. Ask Vizard Agent for SRT and the timings come with it.