XTTS v2: Clone a Voice From Six Seconds of Audio
Six seconds of someone talking, and XTTS v2 can say anything in their voice, in another language. Here's how Coqui's open voice cloning model works and how to get clean results.
Voice cloning used to mean a recording session. An hour in a booth, a script of carefully balanced sentences, then days of training before you heard a single generated line. That's fine for a flagship brand voice. It's absurd for a prototype, a game character you might cut next week, or a quick test of how your demo sounds in Turkish.
XTTS v2 takes the opposite route. Give it a short clip of someone talking, and it speaks any text you send in that voice, including in a different language from the one in the clip. No training step, no fine-tuning, no booth. The official model card puts the bar at a six-second audio clip. It's live on each::labs, and it's one of the most searched voice models in our catalog for good reason.
Here's what XTTS v2 actually is, how to get clones that don't sound like a phone call from 2004, and when it's the right voice model for the job.
Why Zero-Shot Voice Cloning Matters
Most text-to-speech gives you a menu. Pick a voice from a list of stock presenters, accept that your product now sounds like everyone else's. That works until you need a specific voice: your founder narrating the onboarding, a character with a recognizable rasp, a brand voice that has to stay the same across ten languages.
Zero-shot cloning removes the list. The model listens to a reference clip once, at generation time, and conditions every word on it. You don't train anything, so changing the voice is as easy as changing the reference file. That makes voice a variable in your pipeline instead of a fixed asset you commission.
XTTS v2 is the open model that made this practical for a lot of builders, and it's still a strong option when you want cloning from very little audio.
What XTTS v2 Actually Is
XTTS is a multilingual zero-shot text-to-speech model from Coqui. The research behind it was published as "XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model," accepted at Interspeech 2024. The paper describes a model that builds on the Tortoise architecture with changes for multilingual training, better voice cloning, and faster training and inference. Its parts are a VQ-VAE that turns audio into tokens, a GPT-2 style decoder of 443M parameters that predicts those tokens from text and the speaker conditioning, and a HiFi-GAN based decoder that turns them back into audio at 24 kHz.
The model card lists the headline features plainly: voice cloning from a six-second clip, cross-language voice cloning, emotion and style transfer by cloning, multilingual speech generation, and a 24 kHz sampling rate. Coqui's documentation adds streaming inference with under 200 ms latency when you run the model yourself.
Version 2 improved on the first XTTS in a few specific ways, again per the model card: two new languages (Hungarian and Korean), architectural improvements for speaker conditioning, support for multiple speaker references and interpolation between speakers, stability improvements, and better prosody and audio quality across the board.
One Voice, 16 Languages on each::labs
The part of XTTS v2 people find hard to believe until they hear it is cross-language cloning. Record six seconds of someone speaking English, send Spanish text, and you get Spanish in that person's voice. The timbre carries over; the language changes.
The XTTS v2 model card lists 17 languages. The each::labs endpoint exposes 16 of them: English, Spanish, French, German, Italian, Portuguese, Polish, Turkish, Russian, Dutch, Czech, Arabic, Chinese, Hungarian, Korean and Hindi. You pick one with the language field, which defaults to English.
The paper is a useful reality check on how the languages compare. Training data was heavily weighted toward English (around 14,500 of roughly 27,300 hours), and the smallest languages had far less. In practice, the big European languages are the safest bets, and it's worth listening to a few samples before you commit a less represented language to production.
How to Run XTTS v2 on each::labs
The each::labs endpoint keeps it simple. There are four inputs.
speaker (required): a URL to the reference audio of the voice you want to clone.
text: what the cloned voice should say.
language: one of the 16 language codes above, for example en, tr or de.
cleanup_voice: on by default; it refines the generated speech. Leave it on unless you're deliberately chasing a raw sound.
{
"speaker": "https://your-bucket/founder-intro-8s.wav",
"text": "Welcome to the app. Let's set up your first project together.",
"language": "en",
"cleanup_voice": true
}Because it's one more model behind the same each::labs API, you can drop it next to the rest of your audio and video stack, or chain it inside a flow: generate the script, voice it with XTTS v2, then sync it to a face with Sync 3.
Record a Reference Clip That Clones Well
The reference clip is the whole prompt. A great clip makes an average model sound good, and a bad clip makes a great model sound bad. A few habits make a big difference.
Give it more than the minimum. Six seconds is the floor the model card names, not a target to shave down to. The paper evaluated with clips of 3 to 8 seconds, so you don't need minutes of audio, but a full, clean sentence or two beats a clipped fragment that cuts off mid-word.
Record clean, not loud. One speaker, no music bed, no room echo, no compression artifacts from a video call. Background noise gets cloned along with the voice.
Match the energy you want back. The model card lists emotion and style transfer by cloning, which cuts both ways. A flat, tired reference gives you flat, tired output. If you want a warm, upbeat narrator, record the reference that way.
Keep a library of references. Since nothing is trained, a voice is just a file. Store a calm read and an energetic read of the same speaker and switch between them per script.
Write Text the Model Can Speak
XTTS v2 reads what you give it, so the script is half the result. Break long paragraphs into shorter generations; a sentence or a few sentences at a time keeps prosody stable and makes it easy to regenerate the one line that came out odd.
Write numbers, abbreviations and brand names the way they should be spoken. "Version two point five" is safer than "v2.5". Use punctuation as stage direction: commas for short pauses, full stops for breaths, a question mark when you want the rise. And keep each generation in one language. If a script mixes languages, split it and set language per segment.
When to Pick XTTS v2, and When Not To
XTTS v2 is at its best when you need a specific voice from very little audio, fast, in several languages. Prototypes, internal demos, research, accessibility tools, localization tests, game and narrative experiments: all good fits.
There's one thing to check before anything ships. XTTS v2 is released under the Coqui Public Model License, which grants use for non-commercial purposes. Coqui the company shut down in early 2024, and the open-source library is now maintained by the community. If your project is commercial, read the license on the model card carefully, or pick a voice model built for commercial work.
each::labs has several. Inworld TTS 1.5, MiniMax Speech 2.8 HD and Gemini 3.1 Flash TTS all run on the same API, so switching is a model id, not a rewrite. Our ranking of text to speech models by job compares them, and the full list lives on the text to voice models page. If you want to change a voice in existing recordings rather than generate new speech, voice conversion with RVC is the other route.
Frequently Asked Questions
How much audio does Coqui XTTS v2 need to clone a voice?
Very little. The model card says a six-second clip is enough, and the research paper evaluated with references of 3 to 8 seconds. Clean, single-speaker audio matters more than length.
Can XTTS v2 speak a different language than the reference clip?
Yes, that's its signature feature. Cross-language cloning keeps the speaker's timbre while generating speech in the language you set, so an English clip can produce Turkish, German or Korean in the same voice.
Which languages does XTTS v2 support on each::labs?
Sixteen: English, Spanish, French, German, Italian, Portuguese, Polish, Turkish, Russian, Dutch, Czech, Arabic, Chinese, Hungarian, Korean and Hindi. The model card lists Japanese as well, but it isn't one of the endpoint's language options.
Can I use XTTS v2 for commercial projects?
Check the license first. XTTS v2 ships under the Coqui Public Model License, which covers non-commercial use. For commercial voice work, Inworld TTS 1.5, MiniMax Speech 2.8 or Gemini 3.1 Flash TTS on each::labs are the safer picks.