Skip to main content
NexGPU

Speech synthesis model

Self-hosting XTTS-v2: 17-language voice cloning on one 24GB card

A 6-second reference clip, 1.87GB of weights, ~4GB peak VRAM. This is the lowest-friction open voice cloner there is — but to run it correctly you need to know about revisions, character caps and the licence.

Sign up freeView pricing

XTTS-v2 is still the fastest open voice-cloning model to get working. Hand it a 6-second reference clip and it reads any text in that voice, across languages — feed it a Chinese sample and it will carry that timbre into English. The core checkpoint, model.pth, is only 1.87GB; add the 211MB DVAE and the whole download is about 2.1GB. An RTX 3090 has room to spare. That accessibility is why the model sits at 3.7k likes and north of eight million monthly downloads on Hugging Face.

One thing has to be said up front: the weights have not moved since late 2023, Coqui the company has wound down, and the original coqui-ai/TTS repo is — in the words of the people who picked it up — "the original, unmaintained repository." What actually ships today is the Idiap Research Institute fork, published on PyPI as coqui-tts, which has tracked Python 3.14, PyTorch 2.10 and transformers 5. That is the package you install. Not pip install TTS, which will fail outright on torch.load with PyTorch 2.6 or newer.

This page walks through everything that bites people self-hosting XTTS: the language that differs between revisions v2.0.2 and v2.0.3, the hard 82-character cap on Chinese input, the 2–3x DeepSpeed speedup, the real VRAM floor for fine-tuning, and exactly what the non-commercial CPML licence blocks. NexGPU bills per second with no minimum — pick a 24GB card from 1,175 rentable nodes, prove it works, then decide whether to keep it running.

01 —

Which XTTS version? Pick the wrong revision and you lose a language

coqui-tts pulls weights from the Hugging Face main branch by default, and main does not carry the same language list as every revision branch.

VersionParametersVRAMContextNotes
XTTS-v2 (HF main, i.e. v2.0.3)GPT-2 style autoregressive backbone, 30 layers · 1024 dim · 16 heads, model.pth 1.87GB (fp32)~2.5GB loaded in fp32, single-stream inference peaks around 4GB402 text tokens / 605 audio tokens, 24kHz output17 languages — one more than v2.0.2, namely Hindi (hi). Uses a perceiver resampler for speaker conditioning, which is why v2 clones better than v1.x. This is what coqui-tts downloads when you do not pin a revision.
XTTS-v2 revision v2.0.2Same architecture and checkpoint size as v2.0.3Same, ~4GB peak402 text tokens / 605 audio tokens16 languages, no Hindi. A whole tier of downstream wrappers such as AllTalk pin this exact revision, so if you are reproducing someone else's results you must specify the branch — otherwise you are not listening to the same weights.
XTTS-v1.1 (revision v1.1.2)30-layer · 1024-dim GPT backbone, no perceiver resamplerSame order of magnitude as v2402 text tokens / 605 audio tokensOnly 14 languages — no Hungarian, Korean or Hindi — and noticeably weaker speaker similarity than v2. Touch it only when maintaining an older project that needs the exact old voice.
Community fine-tuned checkpointsGPT layers further trained on top of XTTS-v2Inference identical to stock; training is a separate budget, see the hardware tableSame as v2Hugging Face hosts seventy-plus fine-tunes of XTTS-v2, mostly single-voice or single-language specialisations. They inherit CPML too, so they are not a route around the licence.

02 —

Which GPU for XTTS? Match the card to the job

XTTS is a small model. The bottleneck is never VRAM capacity — it is autoregressive decode speed and how many streams you need at once.

  • Single-stream inference, running a persistent OpenAI-compatible TTS endpoint

    RTX 3090 24GB$0.193/GPU-hour

    1.87GB of weights plus a KV cache for at most 605 audio tokens does not even fill 4GB, and 24GB lets you keep conditioning latents for dozens of speakers resident on the card — at the lowest per-hour rate of any 24GB card we rent.

  • Batch narration, DeepSpeed enabled, holding sub-200ms time-to-first-chunk

    RTX 4090 24GB$0.540/GPU-hour

    Autoregressive decode lives on single-thread throughput and memory bandwidth; Ada is the fastest single card here, and the widely reported 2–3x DeepSpeed speedup shows up most clearly on it.

  • Fine-tuning your own voice (batch_size 4, 11-second clips, 10 epochs)

    RTX 5090 32GB$0.723/GPU-hour

    Community fine-tuning guides put the floor at 16GB VRAM on Linux; 32GB lets you push batch_size from 4 up to 8–12 instead of trading time for space with gradient accumulation.

  • Scaling out on one node so a single person can run a whole audiobook or podcast pipeline

    RTX A6000 48GB$0.817/GPU-hour

    Up to 14 GPUs per node, and 48GB each means several independent worker processes sit side by side without fighting for memory — scaling is just launching more containers, no inference code changes.

03 —

Self-hosting XTTS in four steps

From a bare machine to a voice-cloning endpoint your existing OpenAI SDK code can call.

  1. 01

    Install the runtime — and install the right package

    The PyPI package called TTS is abandoned; with PyTorch 2.6 or newer it throws UnpicklingError straight out of torch.load because weights_only now defaults to True. What you want is Idiap's coqui-tts (code under MPL-2.0, Python 3.10–3.14). Since 0.27.4 it no longer pulls PyTorch in by default, so the [cuda] extra is mandatory; Chinese segmentation via spacy_pkuseg and pypinyin arrives with [zh]. COQUI_TOS_AGREED skips the interactive CPML prompt, which otherwise stalls downloads in non-interactive environments.

    pip install "coqui-tts[cuda,zh,server]" && export COQUI_TOS_AGREED=1
  2. 02

    First synthesis, which also fetches the 2.1GB of weights

    The first call downloads model.pth (1.87GB), dvae.pth (211MB) and speakers_xtts.pth (7.75MB) into the cache directory returned by get_user_data_dir("tts"). Give it six seconds or more of clean speech as reference; input is resampled to 22.05kHz and output is always 24kHz mono. Note the language code is zh-cn, not zh.

    python -c 'from TTS.api import TTS; TTS("tts_models/multilingual/multi-dataset/xtts_v2").to("cuda").tts_to_file(text="Testing voice cloning quality", speaker_wav=["ref.wav"], language="en", file_path="out.wav")'
  3. 03

    Serve it, and call it with the OpenAI SDK

    The bundled tts-server listens on port 5002. Alongside the native /api/tts route it exposes /v1/audio/speech, an OpenAI-compatible endpoint returning wav, mp3, opus, aac, flac or pcm. Existing OpenAI TTS code only needs base_url pointed at http://<node>:5002/v1 with any placeholder api_key. For genuinely low-latency streaming, skip HTTP and drive model.inference_stream() with use_deepspeed=True instead. Check tts-server -h for the current GPU flag spelling.

    tts-server --model_name "tts_models/multilingual/multi-dataset/xtts_v2" --use_cuda true
  4. 04

    Fine-tune when zero-shot is not close enough

    There is an official Gradio fine-tuning demo defaulting to batch_size 4, grad_acumm 1, num_epochs 10 and an 11-second maximum clip length, served on port 5003. It handles transcription and segmentation, trains the GPT layers, then hands you a listening UI. Budget 18–20GB of scratch disk, most of it temporary. The official Colab notebook says a run takes up to about 40 minutes. When it finishes, point the server from the previous step at the new checkpoint directory.

    python -m TTS.demos.xtts_ft_demo.xtts_demo --port 5003 --batch_size 4 --grad_acumm 1 --num_epochs 10 --max_audio_length 11

Cloning a voice and producing a 10-hour audiobook: the actual bill

The official Colab notebook puts a fine-tuning run at up to roughly 40 minutes. Rent an RTX 3090 24GB at $0.193/GPU-hour for segmentation, training and listening tests and give the whole thing 3 hours: 3 × $0.193 = $0.579. With the voice settled, switch to an RTX 4090 24GB at $0.540/GPU-hour with DeepSpeed on for batch generation, another 3 hours: 3 × $0.540 = $1.62. XTTS outputs 24kHz 16-bit mono, which is 48,000 bytes per second, roughly 172.8MB per hour of audio; ten finished hours is about 1.73GB, and at the $0.0081/GB median egress rate that is 1.73 × $0.0081 ≈ $0.014. Add them up: $0.579 + $1.62 + $0.014 ≈ $2.21 of compute for an entire audiobook. Storage is separate — call it 10GB for weights plus dataset, so $0.414/GB-month × 10 = $4.14/month; compute billing stops the moment the instance stops, and storage stops when you destroy the volume. If you want a persistent service rather than a batch job: a 3090 running flat out for a month is $0.193 × 24 × 30 = $138.96, while weekdays 09:00–19:00 only comes to $0.193 × 10 × 22 = $42.46. Metered per second, priced per hour, no minimum, no setup fee, no quota request.

04 —

Frequently asked questions

How much VRAM does XTTS-v2 need to self-host? Is an 8GB card enough?

For a single stream, yes. model.pth is 1.87GB of fp32 weights; add the 211MB DVAE, the HiFi-GAN decoder and a KV cache for at most 605 audio tokens and peak inference VRAM lands in the 4GB range, which an 8GB card handles fine. Concurrency is what costs memory — every additional worker process holds another copy of the weights, so four or five streams on one card means 24GB. NexGPU's RTX 3090 24GB is $0.193/GPU-hour; rent one for an hour and measure your real concurrency instead of guessing locally.

Can I use XTTS commercially? What does the CPML licence actually restrict?

No. XTTS-v2 weights ship under the Coqui Public Model License 1.0.0, which grants non-commercial use only and explicitly treats "use of the model to train other models for commercial use" as not a non-commercial purpose; the model, any modification of it, and its outputs all carry the same terms downstream. Code and weights are separate licences: Idiap's coqui-tts code is MPL-2.0 and is commercially usable — the weights are what stop you. The commercial-licensing contact in CPML went nowhere once the original company wound down. For a commercial product, use Apache-2.0 Fun-CosyVoice3 or MIT-licensed Chatterbox instead — on NexGPU that is a different model name on the same GPU.

How good is XTTS at Chinese, and why do long sentences get truncated?

Chinese (zh-cn) has been supported since v1 and speaker similarity is solid, but the tokenizer hard-codes a per-language cap: 82 characters per chunk for Chinese, against 250 for English, 253 for German, 71 for Japanese and 95 for Korean. Exceed it and you get a "might cause truncated audio" warning and possibly real truncation. The fix is to split on punctuation yourself, synthesise sentence by sentence and concatenate — coqui-tts 0.27.3 and later return sentence-level timestamps, which makes stitching far easier. Chinese also depends on spacy_pkuseg and pypinyin, so do not skip the [zh] extra. Ten minutes on a per-second-billed RTX 3090 is enough to hit every one of these.

I installed TTS and now I get weights_only or transformers errors. How do I fix it?

Replace the package rather than patching around it. PyPI's TTS is the abandoned original: PyTorch flipped torch.load's weights_only default to True in 2.6, so loading an XTTS checkpoint raises UnpicklingError, and newer transformers releases break the GPT2InferenceModel inference path as well. Idiap's coqui-tts has kept pace — 0.27.3 adapted streaming to transformers 4.57, 0.27.4 added Python 3.14 and PyTorch 2.10, 0.27.5 fixed transformers 5. Uninstall TTS, install coqui-tts, done. If environment wrangling is not how you want to spend the afternoon, NexGPU ships 2,000+ prebuilt images with PyTorch ready on boot, leaving you a single pip command.

How much VRAM and how long does fine-tuning a custom voice take?

The official Gradio demo defaults to batch_size 4, grad_acumm 1, 10 epochs and an 11-second maximum clip. Community fine-tuning guides put the hardware floor at 16GB VRAM minimum on Linux and recommend 12GB or more on Windows, plus 18–20GB of scratch disk. The official Colab notebook says a run takes up to about 40 minutes. In practice an RTX 3090 24GB at $0.193/GPU-hour is plenty; if you want batch_size 8–12 and less waiting, take the RTX 5090 32GB at $0.723/GPU-hour. We bill per second, so a forty-minute training run costs you forty minutes, not a day.

The weights have not been updated in over two years. Is XTTS still worth using, and how does it compare to CosyVoice or Chatterbox?

It depends what you need. XTTS-v2's edge is a mature ecosystem, tutorials everywhere, cross-lingual cloning from a 6-second clip, and 17 languages in one checkpoint — still the quickest thing to get running for internal tooling or personal projects. But the weights are frozen and CPML rules out commercial use. For Chinese and regional accents, look at Fun-CosyVoice3-0.5B (Apache-2.0, 9 languages plus 18+ Chinese dialects, with an officially claimed 150ms minimum latency). For a permissive licence, look at Resemble AI's Chatterbox (MIT, 500M multilingual variant covering 23+ languages). Conveniently all three deploy the same way — small single-card models. NexGPU spans 51 countries and regions, 1,175 verified rentable nodes and 75 GPU models, so you can trial all three on the same RTX 4090 before committing, with bilingual human support on Telegram and no ticket queue.

More in Speech synthesis

Every model guide

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.

Sign up freeTalk to sales

Sign up to browse live network pricing. No payment method required.