Zero-shot speech synthesis

F5-TTS self-hosted: 335.8M-parameter zero-shot voice cloning on a single 24GB card

arXiv:2410.06885, MIT code, CC-BY-NC weights. The fp32 checkpoint is 1.35GB and ten seconds of reference audio clones a voice — here is the VRAM, the NFE tuning, the 30-second ceiling and the fine-tuning batch math.

F5-TTS · self-hosted

F5-TTS comes out of the X-LANCE lab at Shanghai Jiao Tong University and is a fully non-autoregressive zero-shot TTS system: flow matching on a Diffusion Transformer, with the text simply padded with filler tokens to the length of the speech and denoised straight into audio. No duration model, no text encoder, no phoneme alignment. Against the E2 TTS design it reproduces, F5-TTS adds ConvNeXt V2 to refine the text representation so alignment stops being the bottleneck, then applies inference-time Sway Sampling (default coefficient -1.0) to lift quality and speed together. The current base checkpoint is F5-TTS v1 Base from March 2025, and the f5-tts Python package is now at 1.1.22.

The reason to self-host it is size. F5-TTS Base is 335.8M parameters — DiT with 22 layers, 16 heads, 1024/2048 dims, plus a 4-layer ConvNeXt V2 — and the model_1250000.safetensors on Hugging Face weighs exactly 1,348,435,761 bytes, i.e. 1.35GB in fp32. That footprint means one consumer card can serve inference, fine-tuning and several concurrent streams at once, and quality does not suffer for it: 1.56% WER and 0.76 SIM on Seed-TTS test-zh, better on both counts than its own English numbers (1.83% WER, 0.67 SIM). On speed, the paper reports RTF 0.15 at 16 NFE steps, and the official Triton + TensorRT-LLM stack hits 253ms average latency and RTF 0.0394 at concurrency 2 on a single L20.

The rough edges are specific, and worth knowing before you rent anything. A single generation is capped at 30 seconds total — and that budget includes the reference audio, so keep the reference under 12 seconds and pad it with a little trailing silence. Longer text goes through chunk_text (max_chars=135 by default) and is stitched with a 0.15-second cross-fade, which is where the audible breaths and prosody seams at chunk boundaries come from. Uppercase letters are read out letter by letter (that is how you get K.F.C.), so normal words must be lowercase. And the licence is the real trap: the code is MIT, but because training used the in-the-wild Emilia dataset, the released weights are CC-BY-NC — commercial use means retraining or fine-tuning on data you can actually license.

01 —

Which F5-TTS checkpoints exist

Official bases, the vocoder variant, and the community multilingual fine-tunes, with real parameter counts and on-disk sizes

Version	Parameters	VRAM	Context	Notes
F5-TTS v1 Base (model_1250000.safetensors)	335.8M	1.35GB fp32 weights / 8GB card is plenty for inference	24kHz output, ≤30s per generation incl. reference	The current default, released 2025-03-12, trained on Emilia 95K zh+en. It is the default value of --model in the CLI, uses the pinyin tokenizer and the Vocos vocoder. Start here.
F5-TTS Base (model_1200000.safetensors)	335.8M	1.35GB fp32 weights	24kHz output, ≤30s per generation	The original October 2024 base. Its one remaining advantage: it is the only line with a BigVGAN variant (F5TTS_Base_bigvgan/model_1250000.pt) — --vocoder_name bigvgan only works here. Fall back to it when you want the last few percent of audio fidelity.
E2 TTS Base	333.2M	~1.33GB fp32 weights	24kHz output, ≤30s per generation	Flat-UNet Transformer, 24 layers, 16 heads, 1024/4096 dims — the closest reproduction of the E2 TTS paper. Slow to converge and less robust; keep it as a control experiment, not a production voice.
F5-TTS v1 Small (configs/F5TTS_v1_Small.yaml)	158M	~0.63GB fp32 weights	24kHz output, char tokenizer	Config only — no official weights. dim 768, depth 18, heads 12, trained on LibriTTS_100_360_500 for 686 epochs. The starting point for training a lightweight single-language model from scratch, or when VRAM is genuinely tight.
Community multilingual fine-tunes	335.8M (Base line) / 158M (Small line)	Same as the corresponding base	24kHz per language	infer/SHARED.md already lists Finnish, French, German, Italian, Japanese, Latvian, Russian, Spanish (218 hours), Hindi (SPRINGLab/F5-Hindi-24KHz, Small line) and Arabic (silma-ai, Small line). The CLI loads them directly from hf:// links.

02 —

Which GPU to rent for F5-TTS

Four tiers — inference, online serving, fine-tuning, pretraining — priced at NexGPU list rates per GPU-hour

Batch offline synthesis / single-box inference
RTX 3090 24GB$0.193/GPU-hour
1.35GB of weights plus the Vocos vocoder barely dents 24GB, and Ampere gives you fp16/bf16 and flash attention — the cheapest modern card that runs F5-TTS properly.
Low-latency API / Triton + TensorRT-LLM serving
RTX 4090 24GB$0.540/GPU-hour
The official 253ms / RTF 0.0394 benchmark was measured on an L20; the 4090 is the closest same-generation Ada card you can rent, and it is the right home for 16-step real-time narration.
Fine-tuning a custom voice or a new language
RTX A6000 48GB$0.817/GPU-hour
The repo's own formula is batch_size_per_gpu = 38400 × (VRAM_GB − 5) / 75, so 48GB buys roughly 22016 frames versus 9728 on a 24GB card — steadier gradients and faster convergence.
Pretraining from scratch / large multilingual retrain
A100 SXM4 80GB$1.088/GPU-hour
The paper's exact recipe is 8× A100 80G at 38400 frames per GPU, 307,200 frames total, 1.2M updates — this is the only configuration that reproduces it one-for-one.

03 —

Getting F5-TTS running on NexGPU

Boot to first cloned voice in four steps; every command is straight from the upstream repo

01
Spin up a CUDA instance and install
Pick an RTX 3090 24GB or RTX 4090 24GB in the console, boot one of the 2,000+ prebuilt PyTorch images, and SSH in. F5-TTS needs Python ≥3.10 (upstream tests on 3.11); the pip package pulls torch, torchaudio, vocos, x_transformers and torchdiffeq. Weights download from Hugging Face on first run — 1.35GB, a matter of seconds.
```
conda create -n f5-tts python=3.11 -y && conda activate f5-tts && pip install f5-tts
```
02
Clone a voice from the CLI
Take a reference clip under 12 seconds with a little trailing silence. Fill in ref_text by hand — leaving it blank triggers ASR transcription, and a bad transcript drags the timbre off with it. Drop nfe_step from the default 32 to 16: the paper's RTF falls from 0.31 to 0.15 while WER moves only from 2.42% to 2.53%, which is close to a free 2× speedup. Keep text lowercase unless you actually want letters spelled out.
```
f5-tts_infer-cli --model F5TTS_v1_Base --ref_audio ref.wav --ref_text "Transcript of the reference clip." --gen_text "The text you want spoken." --nfe_step 16
```
03
Bring up Gradio or Docker for interactive work
f5-tts_infer-gradio gives you multi-voice, multi-style and voice chat (backed by Qwen2.5-3B-Instruct). Note that 1.1.21 is the release that patched a path-traversal issue in the finetune Gradio handlers and raised the gradio floor to ≥6.15.0 — stay current, and do not expose the fine-tuning UI on a public port; forward it over SSH instead. The official image comes up in one line.
```
docker run --rm -it --gpus=all -p 7860:7860 --mount 'type=volume,source=f5-tts,target=/root/.cache/huggingface/hub/' ghcr.io/swivid/f5-tts:main
```
04
Fine-tune your own voice, or serve it through Triton
Fine-tuning runs through accelerate, with batch_size_per_gpu sized off VRAM: about 9728 frames on a 24GB card, 22016 on 48GB. For a real reference point, the German community fine-tune used 8000 frames with bf16 mixed precision across 4.2M steps. Upstream warns explicitly that use_ema=True hurts early fine-tuned checkpoints — turn it off via the finetune Gradio option or load_model(..., use_ema=False). For production, switch to the Triton + TensorRT-LLM stack.
```
accelerate launch src/f5_tts/train/finetune_cli.py --exp_name F5TTS_v1_Base --dataset_name my_voice --batch_size_per_gpu 9728 --batch_size_type frame --learning_rate 1e-5 --finetune
```

What F5-TTS actually costs to run

Using the paper's own RTF of 0.15 at 16 NFE steps, one minute of finished audio costs about 9 seconds of GPU time. (1) Bulk voiceover: 1,000 clips of 15 seconds is 15,000 seconds, or 4.17 hours of audio; 4.17 × 0.15 = 0.63 GPU-hours. On an RTX 3090 24GB at $0.193/GPU-hour that is 0.63 × 0.193 = $0.12; on an RTX 4090 at $0.540 it is 0.63 × 0.540 = $0.34. (2) Fine-tuning a custom voice: an RTX A6000 48GB at $0.817/GPU-hour for 24 hours is $19.61; keeping 10 checkpoints at 1.35GB each is 13.5GB, and at $0.414/GB-month for three days that is 13.5 × 0.414 × 0.1 = $0.56; pulling the 1.35GB final weights home costs 1.35 × $0.0081 = $0.01 in egress — under $21 for the whole run. (3) Reproducing the paper's pretraining: 8 × A100 SXM4 80GB = 8 × $1.088 = $8.704/hour, and the paper's 1.2M updates took over a week, so 168 × $8.704 = $1,462 — which is exactly why almost everyone fine-tunes instead. Billing is metered per second and priced per hour, with no minimum, no setup fee and no quota request. Compute billing stops when the instance stops; storage keeps billing until you destroy it.

04 —

FAQ

How much VRAM does F5-TTS actually need? Is 8GB enough?

Yes. F5-TTS v1 Base is 335.8M parameters and the official safetensors file is 1.35GB in fp32; add the Vocos vocoder and the activations for a 30-second segment and an 8GB card handles inference without breaking a sweat. What genuinely wants more memory is fine-tuning and heavy concurrency. For headroom on a budget, NexGPU's RTX 3090 24GB is $0.193/GPU-hour — cheaper than the Tesla T4 16GB at $0.298 — billed per second, so you can stop it the moment you are done.

Why does long-form F5-TTS output have seams, or come out the wrong length?

Two causes. First, the hard 30-second ceiling that includes the reference audio: long text has to go through chunk_text (max_chars=135 by default), and the real limit shrinks dynamically with reference length — the longer your reference clip, the less text fits per chunk, so keep it under 12 seconds. Chunks are joined with cross_fade_duration=0.15, which is where the breath and prosody seams live. Second, release 1.1.22 (2026-07-23) is the one that fixed duration being multiplied N× across chunks, so older installs mangle long-form timing outright — run pip install -U f5-tts. On NexGPU, reinstalling on a clean image takes minutes.

Can I use F5-TTS commercially? How does the licence work?

Code and weights are licensed separately. The repository code is MIT and unrestricted. The released checkpoints are CC-BY-NC and explicitly non-commercial, because the Emilia training set was collected in the wild. The commercial path is to train or fine-tune from the F5TTS_v1_Base or F5TTS_v1_Small config on data you can license — which is precisely why teams rent capacity for this. A100 SXM4 80GB is $1.088/GPU-hour, with up to 14 GPUs per node and a maximum node VRAM of 2,152GB, enough for a serious retrain.

How good is F5-TTS at Chinese compared to English?

Chinese is its strongest suit. On Seed-TTS test-zh it scores 1.56% WER and 0.76 SIM, against 1.83% WER and 0.67 SIM on test-en — better intelligibility and better speaker similarity. Practically, it defaults to the pinyin tokenizer, swapped jieba for the faster rjieba for segmentation, and Emilia 95K is a bilingual zh+en corpus to begin with. Note that punctuation and spaces are treated as pause cues, so how you format the script directly shapes the prosody. An RTX 3090 at $0.193/GPU-hour will get you through a whole batch of scripts in a quarter of an hour.

What batch_size_per_gpu should I use when fine-tuning, and how many GPUs?

The repo's finetune_gradio spells out the formula: batch_size_per_gpu = 38400 × (VRAM_GB − 5) / 75. That gives roughly 9728 frames on a 24GB card, 22016 on 48GB, and exactly 38400 on 80GB — the value in the official F5TTS_v1_Base.yaml, matching 8× A100 80G for 307,200 frames total. As a real-world anchor, the German community fine-tune ran 8000 frames with bf16 for 4.2M steps across 40 epochs. For single-card fine-tuning we would put you on an RTX A6000 48GB at $0.817/GPU-hour; on a tighter budget, an RTX 4090 24GB at $0.540 with batch around 8000 and gradient accumulation.

Has F5-TTS been superseded? Is it still worth deploying?

Here is the honest state of it: the official base weights have not moved since F5-TTS v1 Base on 2025-03-12, so there is no newer base checkpoint. The code, however, is very much alive — 1.1.22 shipped to PyPI on 2026-07-23, and recent releases added MMDiT flash attention, a fused AdamW option, gradient checkpointing in MMDiT and a fix for speech-editing boundary artifacts. Its position today is clear: the smallest parameter count, the lowest fine-tuning barrier, and the widest set of community language weights — Finnish, French, German, Italian, Japanese, Latvian, Russian, Spanish, Hindi and Arabic all exist off the shelf. If you want to benchmark it against alternatives before committing, NexGPU carries 75 GPU models across 1,175 verified rentable nodes in 51 countries and regions, with 2,000+ prebuilt images including PyTorch, vLLM and Whisper ASR, and bilingual support on Telegram with no ticket queue. The console is at console.nexgpu.net.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.