Since August we've been publishing short vertical videos about AI news: a presenter on camera, one story in thirty seconds, a practical takeaway at the end. The presenter is a synthetic avatar and the voice is synthetic too.
The videos are in Russian, and that turns out to matter far more than we expected. Most of what follows applies to any language that isn't English — the tooling is built and tested English-first, and everything else inherits the gaps.
Below is what we learned across the first seven videos. Not a tool review: specific failure modes, each of which cost us one ruined render.
A Russian voice will not read Latin script
This is the first wall anyone hits producing non-English technology content. Our synthesizer pronounces OpenAI as something close to "openy" — one mangled Russian word. Model names, product names, company names turn to mush in the voiceover, and in AI news those are half the story.
What worked was separating the two streams: what gets spoken and what gets displayed are different text.
The script sent to synthesis is written entirely in Cyrillic, phonetically — the Russian equivalent of writing "Oh-pen Ay Eye" instead of OpenAI. The on-screen subtitles then substitute the proper Latin spelling back in, through a lookup table that can also merge several spoken words into one written token.
Switching engines doesn't rescue you. We tested an alternative synthesizer: it handles isolated Latin words better, but turned OpenAI into a different nonsense word — worse, not better. No engine we had access to reads mixed-case abbreviations correctly in a Russian sentence.
Stress marks solve one problem and quietly create another
Russian has no written stress, so synthesizers guess — and get it wrong on names and loanwords. The standard fix is a combining acute accent (U+0301) on the stressed vowel.
It works, but not reliably and not for free.
First, it applies inconsistently: the same mark lands correctly on one word and is ignored on the next. Where it doesn't take, rewriting the phrase is cheaper than fighting the engine.
Second, and this one is not obvious: a stress mark can swallow the adjacent consonant. With the accent applied, one voice read Лу́на (Luna) as "una" — the initial L simply disappeared. A different voice read the identical string correctly.
The more robust trick turned out to be capitalizing the stressed vowel instead. Stress lands in the same place, and no consonant is harmed.
Test the word in context, never in isolation
Checking a word on its own proves nothing. One of our words was pronounced correctly in isolation and then, when it followed a question, got reduced into a different word entirely — the engine carried the interrogative intonation across the boundary.
Two things fixed it: hyphenating the word to force a syllable break, and lengthening the pause before it. A 0.4-second pause didn't help; 0.6 did. The shorter one doesn't give the previous phrase's intonation time to settle.
Related, and worth budgeting for: a requested 0.8-second pause produced about 0.66 seconds of actual silence. Ask for more than you need.
Continuous speech beats clean scene boundaries
It's tempting to split a video into many short scenes — it makes swapping visuals easy. But in most avatar tools each scene is synthesized separately and the audio is butt-joined, so speech cuts off abruptly at every boundary. It sounds cheap, and no amount of editing fixes it after the fact.
We keep the entire script in one presenter scene, place pauses with tags inside it, and get visual variety from overlays composited on top of the finished frame. A short trailing pause at the end of the scene stops the join from clipping the last sound.
Full-frame cards belong at the start and the end only
Our first version alternated text card, presenter, text card. It looks dead: the presenter keeps vanishing and the video becomes a narrated slide deck.
What works: full frame only for the first and last seconds, with data shown as semi-transparent cards over the presenter, who stays on screen throughout.
Those cards go below the face and above the subtitles — in a vertical frame that's a narrow band, and tall cards with a heading on top cover the presenter. And don't reuse one card three times; at a turn in the argument, a full-frame insert with short text works better, with the presenter returning between them.
Build subtitles from the script, not from speech recognition
The second temptation: run the finished video through speech recognition and get timed subtitles for free. Don't.
Recognition fails precisely on the words the whole exercise was about — it mishears the phonetic spellings, and it drops Latin script and numbers. Meanwhile the verbatim text already exists: it's the script. The right source is the subtitle sidecar the synthesizer itself produces, because it contains exactly what was spoken.
One rule we built into our subtitle generator: a period, question mark or exclamation point closes the group. Showing "finished sentence. Start of the next" on one line is wrong even when it fits the word limit.
We measured the monotony and couldn't fix it
Synthetic speech sounds flatter than human speech, and we wanted to fix that with settings. We measured pitch variation in semitones across identical text: the default engine came in at 3.47, an alternative at 3.35–3.37 with any settings applied, and a third voice at 1.89.
The unwelcome finding: parameters like "stability" and "expressiveness" are accepted and change nothing — the median pitch matched to the hertz. Exactly one lever remained: the text. Varying sentence length, using questions, using dashes. The engine responds to punctuation far more than to configuration.
Details that show up immediately
- Hook in the first second. The strongest number sits at second zero, with no introduction. Voice and image deliver the same fact two ways: "five times cheaper" in the ear, "−80 %" on screen.
- Loop it. First and last frame are the same asset, and the closing line repeats the opening one verbatim. The seam disappears and the video runs a second time.
- Pace. About 1.8 words per second including pauses. For thirty seconds that's 50–60 words, no more.
- Keep the bottom 22 % of the frame clear — that's where subtitles live.
- We removed the call to action from the video. Seven seconds of "subscribe" is nearly a fifth of the runtime, and it breaks the loop. The link lives in the description.
- Loudness. Raw synthesis output is always quiet — in our case −22 to −27 LUFS. Without normalization the video sounds noticeably duller than everything around it in the feed.
What transfers to other problems
The general rule we took away: separate what the model pronounces from what the viewer sees. Once those two streams are decoupled, most synthesis problems stop being presentation problems and become one lookup table.
And the second: every synthesis setting has to be verified by ear, in context. Quality metrics and isolated-word tests lie here — they're insensitive to exactly what a human hears.
Want a pipeline like this? We build content production automation — from generation through scheduled publishing — and integrate AI into products and processes. Get in touch.