Speech synthesis / TTS

Self-host VITS: 36.3M parameters, a 145MB checkpoint, one GPU end to end

The end-to-end TTS model that collapsed acoustic model and vocoder into a single network — and the common ancestor of MMS-TTS, VITS2, Bert-VITS2, MeloTTS and GPT-SoVITS. Picking the right fork matters more than picking the right card.

VITS · self-hosted

VITS stands for Variational Inference with adversarial learning for end-to-end Text-to-Speech (arXiv:2106.06103, by Jaehyeon Kim, Jungil Kong and Juhee Son at Kakao Enterprise). It packs a variational autoencoder, normalizing flows and a GAN discriminator into one network: during training a posterior encoder pulls latents straight from the linear spectrogram, and at inference you sample from the text prior, push it through the flow, and hand it to a HiFi-GAN-style decoder that emits the waveform. No mel-spectrogram middle stage, no separately trained vocoder. The original repo, jaywalnut310/vits, is MIT-licensed with 7.9k stars.

It is absurdly small for a model this widely deployed. The official kakao-enterprise/vits-ljs checkpoint on Hugging Face reports 36.3M parameters — 36.3M x 4 bytes is roughly 145MB of fp32 weights. The config fits on one screen: inter_channels and hidden_channels both 192, filter_channels 768, a 6-layer text encoder with 2 attention heads, and a HiFi-GAN decoder with upsample_rates [8, 8, 2, 2] off 512 initial channels. The paper measured 1480.15 kHz synthesis throughput on a V100 — about 67.12x real time — and 2005.03 kHz, or 90.93x, for the DDP variant. VRAM has never been the bottleneck for VITS inference. The one thing you do have to adapt to is that it is non-deterministic: the stochastic duration predictor plus the normalizing flow mean the same sentence comes out with different prosody on every sample, which is the one-to-many capability the paper set out to build, not a defect.

Training is where the card actually matters. ljs_base.json hardcodes batch_size 64 with fp16_run true, and the paper is explicit: mixed precision training on 4 NVIDIA V100 GPUs, batch size 64 per GPU, trained up to 800k steps. Coqui's official LJSpeech recipe is more conservative at batch_size 32 with mixed_precision. One more thing worth knowing before you provision anything: this family splintered badly after 2023. Bert-VITS2 stopped at v2.3, VITS-fast-fine-tuning was archived on 21 November 2025, so-vits-svc was archived back in November 2023, while MeloTTS and GPT-SoVITS are still shipping. Decide which branch you are running first.

01 —

What is actually runnable in the VITS family

Six branches off one architecture, with very different licences, sample rates and maintenance status

Version	Parameters	VRAM	Context	Notes
VITS original, ljs_base (jaywalnut310/vits, MIT)	36.3M	Inference ~145MB fp32 weights; training per paper on 4x V100 32GB, batch 64 per GPU, fp16	22.05kHz, single speaker	LJ Speech: 13,100 clips, roughly 24 hours from one speaker. text_cleaners is english_cleaners2 running espeak IPA phonemization, with a vocab_size of just 38 — an English-only starting point.
VITS multi-speaker, vctk_base (train_ms.py)	36.3M plus speaker embedding	Same class as single-speaker; VRAM is driven by batch_size and segment_size 8192	22.05kHz, 109 speakers	VCTK is ~44,000 clips, 44 hours, 109 native English speakers, downsampled to 22kHz. If you want multiple voices, start from this config rather than training 109 single-speaker models.
MMS-TTS (facebook/mms-tts-*, CC-BY-NC 4.0)	36.3M per language, one checkpoint each	Comfortable on any 16GB card; ~145MB of weights	16kHz	Meta trained a separate VITS checkpoint for each of 1,107 languages, loadable directly via VitsModel.from_pretrained. The licence is CC-BY-NC 4.0 — non-commercial only. Non-Roman scripts need uroman romanization first, and the checkpoints were trained on lowercase, unpunctuated text.
VITS2 (arXiv:2307.16430 / p0p4k/vits2_pytorch, MIT)	Same class as VITS; the changes are architectural	Comparable to VITS; slightly higher once the flow carries transformer blocks	22.05kHz	Kong et al.'s follow-up: an LSTM discriminator on the duration predictor for adversarial training, transformer blocks inside the normalizing flow, speaker embeddings pushed down into the text encoder, and noise-scaled monotonic alignment search. The community implementation shipped a 64k-step LJSpeech checkpoint and ONNX export.
Bert-VITS2 v2.3 / Extra-v2 / JP-Extra (fishaudio, AGPL-3.0)	VITS2 backbone plus a multilingual BERT front end	Noticeably higher than stock VITS — an extra BERT encoder in the graph. Plan on 24GB minimum, 48GB+ to be comfortable	High sample rate branch	v2.3 (19 December 2023) was the closing release: BF16 training, English G2P fixes, CLAP dropped in favour of plain BERT semantic fusion. Only Extra-v2 (Chinese-specialised G2P) and JP-Extra (Japanese, ~800 hours of monolingual data) followed. The authors have said they are not maintaining it and moved to Fish-Speech.
MeloTTS / GPT-SoVITS (the live VITS-lineage forks, both MIT)	MeloTTS descends from VITS/VITS2/Bert-VITS2; GPT-SoVITS is a GPT + SoVITS hybrid	MeloTTS claims CPU real-time inference; GPT-SoVITS reports RTF 0.028 on an RTX 4060 Ti and 0.014 on an RTX 4090	GPT-SoVITS v4 outputs 48kHz natively	For mixed Chinese-English reading, take MeloTTS (from MIT and MyShell.ai; its Chinese speaker handles code-switching). For voice cloning, take GPT-SoVITS: 5 seconds for zero-shot, 1 minute for few-shot, with v2Pro, v3 and v4 all in play across Chinese, English, Japanese, Korean and Cantonese.

02 —

Which card to rent

Pick for the job you are actually doing — do not buy 80GB for a 145MB checkpoint

Get inference working, stand up a TTS service, run batch synthesis
RTX 3090 24GB$0.193/GPU-hr
145MB of weights does not dent 24GB, so VRAM is irrelevant here; what decides throughput is decoder bandwidth against hourly price, and the 3090 wins that trade.
Single-GPU fine-tuning (Coqui LJSpeech recipe: batch_size 32, mixed_precision)
RTX 4090 24GB$0.540/GPU-hr
fp16 throughput at batch 32 is exactly what determines whether you wait a night or a week; the 4090 compresses an LJSpeech-scale fine-tune into one or two overnight runs.
Reproduce the paper setup: 4 GPUs, batch 64 each, fp16, out to 800k steps
Tesla V100 32GB x4$0.188/GPU-hr ($0.752/hr for four)
The paper used 4x V100 32GB with batch 64 per GPU, so that configuration is known to fit. The same card happens to be the cheapest thing in our catalogue.
Bert-VITS2-class forks with a BERT front end, large multi-speaker batches, high sample rates
A100 PCIE 80GB$0.824/GPU-hr
An extra BERT encoder plus longer segments will OOM a 24GB card outright; 80GB means you stop shrinking batch size and restarting the run.

03 —

From boot to first waveform

Including the two failure modes every VITS self-hoster hits

01
Boot a CUDA instance and confirm the GPU is visible
Pick an RTX 3090 or RTX 4090 at console.nexgpu.net and boot one of the 2,000+ prebuilt images with PyTorch already in place. Get in over SSH, Jupyter or the web terminal and sanity-check the environment first. Billing is metered per second, so if the card is not what you wanted you stop it — no minimum, no setup fee, no quota request.
```
nvidia-smi && python -c "import torch; print(torch.__version__, torch.cuda.get_device_name(0))"
```

Fastest path: three lines with the transformers VitsModel

If you never want to touch the original repo, pull the official checkpoint from Hugging Face. Two things to watch: the flow and the stochastic duration predictor are non-deterministic, so without set_seed you get a different take every run; and if you swap in a non-Roman-script MMS-TTS checkpoint you must romanize the text with uroman first.

pip install -U transformers torch scipy accelerate
python - <<'PY'
from transformers import VitsModel, AutoTokenizer, set_seed
import scipy.io.wavfile as wav
m = VitsModel.from_pretrained("kakao-enterprise/vits-ljs").cuda()
tk = AutoTokenizer.from_pretrained("kakao-enterprise/vits-ljs")
set_seed(555)
out = m(**tk("hello world", return_tensors="pt").to("cuda"))
wav.write("out.wav", m.config.sampling_rate, out.waveform[0].detach().cpu().numpy())
PY

03
To train you need the original repo: install espeak, then build monotonic_align by hand
These are the two potholes nobody warns you about. The text front end depends on system-level espeak, which pip cannot install for you. And monotonic alignment search is Cython — you must build the extension in-place after cloning or train.py dies on an ImportError before it reaches the first step. On a clean PyTorch image this takes two minutes; on a laptop it can eat an evening.
```
sudo apt-get install -y espeak && pip install -r requirements.txt && cd monotonic_align && mkdir -p monotonic_align && python setup.py build_ext --inplace
```
04
Start training, and watch the alignment plots in TensorBoard
Single speaker runs train.py, multi-speaker runs train_ms.py. ljs_base.json defaults to batch_size 64, fp16_run true, segment_size 8192, learning_rate 2e-4, epochs 20000, logging every 200 steps and evaluating every 1000. If you run out of memory, cut batch_size before you touch segment_size — segment_size also sets how much waveform the discriminator sees. For the first few tens of thousands of steps, do not judge by ear: check whether the attention plot has formed a clean diagonal. No diagonal means you are burning money.
```
python train.py -c configs/ljs_base.json -m ljs_base   # multi-speaker: python train_ms.py -c configs/vctk_base.json -m vctk_base
```

The actual arithmetic

Inference load test: an RTX 3090 24GB is $0.193/GPU-hr, so two hours of batch synthesis is 0.193 x 2 = $0.386. Single-GPU fine-tune: an RTX 4090 24GB is $0.540/GPU-hr, and 20 straight hours on Coqui's batch_size 32 recipe is 0.540 x 20 = $10.80; add 50GB of storage for corpus and checkpoints at the $0.414/GB-month median, which prorates to 0.414 x 50 / 30 = about $0.69 for the day, for roughly $11.49 all in. Want the paper configuration? Four Tesla V100 32GB at 0.188 x 4 = $0.752/hr, and 72 hours straight is 0.752 x 72 = $54.14 — the same four-GPU setup the authors ran out to 800k steps. Pulling the generated wavs back down costs the $0.0081/GB egress median, so 1GB of audio is about a cent. Everything is metered per second and priced per hour, with no minimum and no setup fee. Compute billing stops when the instance stops; storage keeps billing until you destroy it.

04 —

FAQ

How much VRAM does VITS inference need? Is a 24GB 3090 enough?

More than enough — it is overkill. The official kakao-enterprise/vits-ljs checkpoint reports 36.3M parameters, so fp32 weights are 36.3M x 4 bytes, roughly 145MB; add CUDA context and the activations for one sentence and you are still in the low single-gigabyte range. What actually shapes your experience is throughput: the paper measured 67.12x real time on a V100, 90.93x for the DDP variant. So choose on concurrency per dollar-hour. NexGPU's RTX 3090 24GB is $0.193/GPU-hr, metered per second, so you stop it the moment the load test finishes.

How many GPUs does training VITS need? Can I train from scratch on one card?

The paper's setup was mixed precision on 4 NVIDIA V100s with batch size 64 per GPU, trained up to 800k steps — that is where ljs_base.json's batch_size 64 and fp16_run true come from. One card absolutely works: Coqui's official LJSpeech recipe is batch_size 32 with mixed_precision, it just takes longer to converge. On NexGPU a Tesla V100 32GB is $0.188/GPU-hr, so you can reproduce the paper's four-card setup for $0.752/hr, with up to 14 GPUs available on a single node. If you want wall-clock speed instead, take the RTX 4090 24GB at $0.540/GPU-hr.

How do VITS, VITS2, Bert-VITS2, so-vits-svc and GPT-SoVITS relate? Which should I use?

VITS (arXiv:2106.06103) is the root. VITS2 (arXiv:2307.16430) is the same team's architectural refinement. Bert-VITS2 bolts a multilingual BERT onto the VITS2 backbone and has been unmaintained since v2.3 in December 2023. so-vits-svc does singing voice conversion, not TTS — the name is misleading — and was archived in November 2023. VITS-fast-fine-tuning was archived on 21 November 2025. For something you can ship today: MeloTTS (MIT) for mixed Chinese-English, GPT-SoVITS (MIT) for few-shot cloning at 5 seconds zero-shot or 1 minute few-shot. Whichever branch you pick, NexGPU's 2,000+ prebuilt images give you the matching PyTorch base so you can clone and go straight from boot.

Can I use VITS commercially? What is the licence?

It depends entirely on which weights you load, not on the architecture. The original jaywalnut310/vits repo and Kakao's vits-ljs and vits-vctk checkpoints are MIT, so commercial use is fine. Meta's 1,107 MMS-TTS language checkpoints are CC-BY-NC 4.0 — explicitly non-commercial. Bert-VITS2 is AGPL-3.0, which reaches into your server-side code. MeloTTS and GPT-SoVITS are MIT. The licensing call is yours to make, but keeping training on an instance you rent, with weights landing only in storage you control and can destroy on demand, is the part NexGPU handles.

Why does the original repo produce garbage when I feed it Chinese?

Because the original repo was never built for it. ljs_base.json sets text_cleaners to english_cleaners2, which runs espeak English IPA phonemization against a vocab_size of just 38 — Chinese characters get stripped or mapped to the wrong phonemes. The fix is a different G2P front end: use an MMS-TTS Chinese checkpoint (romanize with uroman first, and remember it was trained on lowercase unpunctuated text), or move to MeloTTS for native code-switching, or Bert-VITS2 Extra-v2 for specialised Chinese G2P. All of these carry heavy dependency trees, and iterating on a NexGPU RTX 3090 at $0.193/GPU-hr beats reinstalling CUDA and espeak on your laptop for the fourth time.

The same sentence comes out with different pacing every time — is the model broken?

No, that is the design. The stochastic duration predictor plus normalizing flows let one text have many valid readings; the paper calls this the one-to-many relationship. To reproduce a specific take, call set_seed. To tune how much it wanders, adjust noise_scale (default 0.667) and noise_scale_duration (default 0.8), and set pace with speaking_rate (default 1.0). Sweeping those parameters means dozens or hundreds of samples, which is exactly the shape of workload for a per-second-metered GPU you spin up and stop — and if you get stuck, our support is bilingual over Telegram with no ticket queue.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.