Text-to-Speech Model

Self-host Fish Speech. 24GB is the hard line.

Q: How much VRAM does Fish Speech actually need to run locally?

The official install docs give one number for today's S2-Pro: GPU Memory 24GB (Inference). At ~5B parameters, bf16 weights alone approach 10GB, and the rest goes to the codec, KV and batching. The 4GB-inference / 8GB-fine-tuning figures you'll see quoted belong to the old 1.x branch — don't mix them up. Sized to that line, NexGPU has the RTX 4090 24GB at $0.540/GPU-hr and the RTX 5090 32GB at $0.723/GPU-hr, both ready at boot.

Q: Are Fish Speech, OpenAudio S1 and Fish Audio S2 the same thing?

Three stages of one lineage. fish-speech 1.x was the original open branch, the flagship then became OpenAudio S1 / S1-mini, and today it's Fish Audio S2 — with the code repository still at fishaudio/fish-speech throughout. The team states that OpenAudio is the research lab of Hanabi AI Inc. and Fish Audio is the product platform. All three generations can be swapped and compared on a single NexGPU instance, billed per second, stopped when the comparison is done.

Q: Can I use Fish Speech commercially?

Not out of the box. S2's code and weights ship under the FISH AUDIO RESEARCH LICENSE: free for research and non-commercial use, with commercial deployment requiring separate licensing from Fish Audio. The earlier fish-speech 1.5 and openaudio-s1-mini use CC-BY-NC-SA-4.0, which carries the same NonCommercial restriction. Settle licensing before you ship — but technical evaluation is unaffected, and per-second instances on NexGPU are built for exactly that phase.

Q: Can I run Fish Speech on Windows?

S2's stated system requirement is Linux or WSL; compilation features aren't fully supported on native Windows or macOS. Only the older 1.x branch officially supports Windows, plus macOS inference through MPS. Rather than fighting CUDA passthrough in WSL and portaudio build errors, open a Linux instance on NexGPU and reach it over SSH, Jupyter or the web terminal — the environment matches the official docs line for line.

Q: Will a Tesla T4, V100 or P40 run Fish Speech?

For the 1.x branch, yes. For S2-Pro, no. Two reasons: the T4 has only 16GB and never reaches the 24GB floor, and Turing and Volta lack bf16 entirely — which is why the project ships a --half flag specifically for GPUs without bf16 support. If you want cheap and 24GB, NexGPU's RTX 3090 24GB is $0.193/GPU-hr, below the Tesla T4's $0.298 with 8GB more memory.

Q: What should I watch out for when fine-tuning Fish Speech?

The flow is: organise .wav files with matching .lab transcripts per speaker, extract semantic tokens with extract_vq.py, pack them into protobuf with build_dataset.py, then train with LoRA using the official r_8_alpha_16 config. The trap is documented but easy to miss: do not fine-tune a model that has already been trained with reinforcement learning — the docs warn it shifts the model distribution and degrades quality. Give fine-tuning an RTX A6000 48GB on NexGPU at $0.817/GPU-hr, which holds weights, optimiser state and dataset together.

The official install docs give exactly one number for S2-Pro: GPU Memory 24GB (Inference). An RTX 4090 24GB on NexGPU is $0.540/GPU-hour, metered per second, stopped the moment you're done.

Fish Speech · self-hosted

Most Fish Speech tutorials you'll find are still on 1.4 or 1.5. The project has been renamed twice since then: the repo is still fishaudio/fish-speech, but the flagship became OpenAudio S1 / S1-mini and is now Fish Audio S2, with weights published at fishaudio/s2-pro on HuggingFace. The team's own framing is that OpenAudio is the research lab of Hanabi AI Inc. and Fish Audio is the product platform — three names, one lineage. Follow an old guide looking for 1.5 checkpoint paths and you stall at step one.

S2-Pro's architecture is worth stating precisely: it's a Dual-AR design where a 4B slow AR predicts the primary semantic codebook along the time axis and a 400M fast AR fills in the remaining 9 residual codebooks per frame, roughly 5B parameters in total. The backbone is Qwen3; the codec is a ModifiedDAC RVQ with 10 codebooks at about 21 Hz. Training ran to over 10 million hours across 80+ languages, with Japanese, English and Chinese as Tier 1. The team also open-sourced an SGLang-based inference engine measured at RTF 0.195 with sub-100ms time-to-first-audio on a single H200; the technical report is arXiv:2603.08823.

So the barrier to self-hosting Fish Speech was never the algorithm — it's VRAM. A 16GB card does not clear the official 24GB line, and buying a 4090 just to try voice cloning once doesn't pencil out. Rent one by the second instead: install per the official docs, pull weights to the official paths, and if the quality justifies a long-term commitment you'll know within the hour. If it doesn't, stop the instance and compute billing stops with it.

01 —

Fish Speech generations and what each one costs in VRAM

From 1.4 to S2-Pro: two renames and one architecture rewrite on the same line.

Version	Parameters	VRAM	Context	Notes
Fish Audio S2-Pro	4B slow AR + 400M fast AR (~5B total)	bf16 inference: 24GB minimum per official docs	RVQ, 10 codebooks / ~21 Hz	The current flagship. Instruction-following control via natural-language descriptions, 15,000+ inline emotion and prosody tags, native multi-speaker multi-turn generation, and an SGLang streaming engine benchmarked at RTF 0.195 and <100ms first audio on one H200. Released under the FISH AUDIO RESEARCH LICENSE.
OpenAudio S1	4B	bf16 weights ~8GB (4B × 2 bytes); comfortable on a 24GB card	13 languages	The previous flagship, trained on 2M+ hours with online RLHF. English eval: WER 0.008, CER 0.004, and a #1 finish in human subjective evaluation on HuggingFace TTS-Arena-V2.
OpenAudio S1-mini	0.5B	Distilled from S1; bf16 weights ~1GB. No separate official deployment figure — the 1.x branch's 4GB floor is the sane estimate	13 languages	Weights are on HuggingFace under CC-BY-NC-SA-4.0. English WER 0.011, CER 0.005. The best value tier if you want a usable self-hosted TTS API on a cheap card.
Fish Speech v1.5 / v1.5.1	0.5B class	Officially 4GB (inference) / 8GB (fine-tuning)	Multilingual	The last stable tag of the old main line, and the branch that officially supports Windows plus macOS inference via MPS. Licensed CC-BY-NC-SA-4.0. Pin here if you need to reproduce an older project's results.
Fish Speech v1.4	0.5B class	Same as the 1.x branch, 4GB inference	8 languages (EN, ZH, DE, JA, FR, ES, KO, AR)	700k hours of training; the matching paper is arXiv:2411.01156. The overwhelming majority of "Fish Speech deployment" blog posts are describing this generation, and none of their commands or directory layouts match S2.

02 —

Pick the card by what you're actually doing

Live NexGPU rates, metered per second, no minimum and no setup fee.

Single-GPU S2-Pro inference and voice-cloning trials
RTX 4090 24GB$0.540/GPU-hr
Sits exactly on the official 24GB line — bf16 weights plus the codec fit, making it the cheapest honest way to evaluate S2-Pro.
Always-on API service and long-form batch synthesis
RTX 5090 32GB$0.723/GPU-hr
The extra 8GB absorbs the memory cost of --compile and the KV of concurrent requests, so you aren't hand-tuning batch size at peak.
LoRA fine-tuning and multi-stream production
RTX A6000 48GB$0.817/GPU-hr
48GB holds the ~5B weights, the LoRA optimiser state and a batch of preprocessed .npy files at once, so you skip gradient-accumulation gymnastics.
Running S1-mini or the 1.5 branch for cheap evaluation
RTX 3090 24GB$0.193/GPU-hr
A 0.5B-class model needs only 4GB, and this is the cheapest 24GB card on the network — leave it running demos all day without flinching.

03 —

From a blank instance to your first synthesised clip

Four steps, every path and flag taken straight from the official docs.

01
Spin up a Linux instance and build a Python 3.12 environment
The official install page states the system requirement as Linux or WSL on Python 3.12, with CUDA wheels offered for cu126, cu128 and cu129. NexGPU's prebuilt PyTorch image boots ready; add the audio-side system dependencies, then let uv resolve everything in one pass.
```
apt install portaudio19-dev libsox-dev ffmpeg && uv sync --python 3.12 --extra cu129
```
02
Pull the S2-Pro weights
Weights live at fishaudio/s2-pro on HuggingFace and include both the text-to-semantic backbone and codec.pth. Land them under checkpoints/s2-pro — every command afterwards resolves against that relative path.
```
hf download fishaudio/s2-pro --local-dir checkpoints/s2-pro
```
03
Run the three-stage pipeline: extract tokens, generate semantics, decode audio
First DAC extracts VQ tokens from a 10-30 second reference clip, then text2semantic takes your target text plus the reference transcript and emits semantic codes, then codes_0.npy goes back through DAC to become a wav. Add --compile for speed; add --half on Turing or Volta cards that lack bf16 support.
```
python fish_speech/models/text2semantic/inference.py --text "Text you want to synthesise" --prompt-text "Transcript of your reference audio" --prompt-tokens "fake.npy" --compile
```
04
Bring up the WebUI or the HTTP API
Once the pipeline works you shouldn't be typing three commands per clip. The Gradio WebUI is for auditioning voices by hand; api_server is for wiring into your own product. Docker users can run docker compose --profile webui up, prefixed with COMPILE=1 for the compiled path. Expose the port from your NexGPU instance over an SSH tunnel or a public mapping.
```
python tools/api_server.py --listen 0.0.0.0:8888 --compile
```

What this actually costs to stand up

At current NexGPU rates. Rent an RTX 4090 24GB at $0.540/GPU-hr: roughly 20 minutes to install and pull weights, another 20 to get the three-stage pipeline running — 40 minutes total, or 0.667 hours × $0.540 = $0.36 for a working deployment. Run it two hours a day for batch synthesis and a week is 14 × $0.540 = $7.56. For LoRA fine-tuning, move to an RTX A6000 48GB: $0.817/GPU-hr × 6 hours = $4.90. S2-Pro is ~5B parameters, so bf16 weights run close to 10GB; keeping those plus a dataset resident at the median $0.414/GB-month is 10GB × $0.414 = $4.14/month, and exporting 5GB of finished audio costs 5 × $0.0081 = $0.04 at median egress. Compute is metered per second and stops when the instance stops; storage keeps billing until you destroy it. End to end, plus a month of keeping it around, still single-digit dollars.

04 —

Frequently asked questions

How much VRAM does Fish Speech actually need to run locally?

The official install docs give one number for today's S2-Pro: GPU Memory 24GB (Inference). At ~5B parameters, bf16 weights alone approach 10GB, and the rest goes to the codec, KV and batching. The 4GB-inference / 8GB-fine-tuning figures you'll see quoted belong to the old 1.x branch — don't mix them up. Sized to that line, NexGPU has the RTX 4090 24GB at $0.540/GPU-hr and the RTX 5090 32GB at $0.723/GPU-hr, both ready at boot.

Are Fish Speech, OpenAudio S1 and Fish Audio S2 the same thing?

Three stages of one lineage. fish-speech 1.x was the original open branch, the flagship then became OpenAudio S1 / S1-mini, and today it's Fish Audio S2 — with the code repository still at fishaudio/fish-speech throughout. The team states that OpenAudio is the research lab of Hanabi AI Inc. and Fish Audio is the product platform. All three generations can be swapped and compared on a single NexGPU instance, billed per second, stopped when the comparison is done.

Can I use Fish Speech commercially?

Not out of the box. S2's code and weights ship under the FISH AUDIO RESEARCH LICENSE: free for research and non-commercial use, with commercial deployment requiring separate licensing from Fish Audio. The earlier fish-speech 1.5 and openaudio-s1-mini use CC-BY-NC-SA-4.0, which carries the same NonCommercial restriction. Settle licensing before you ship — but technical evaluation is unaffected, and per-second instances on NexGPU are built for exactly that phase.

Can I run Fish Speech on Windows?

S2's stated system requirement is Linux or WSL; compilation features aren't fully supported on native Windows or macOS. Only the older 1.x branch officially supports Windows, plus macOS inference through MPS. Rather than fighting CUDA passthrough in WSL and portaudio build errors, open a Linux instance on NexGPU and reach it over SSH, Jupyter or the web terminal — the environment matches the official docs line for line.

Will a Tesla T4, V100 or P40 run Fish Speech?

For the 1.x branch, yes. For S2-Pro, no. Two reasons: the T4 has only 16GB and never reaches the 24GB floor, and Turing and Volta lack bf16 entirely — which is why the project ships a --half flag specifically for GPUs without bf16 support. If you want cheap and 24GB, NexGPU's RTX 3090 24GB is $0.193/GPU-hr, below the Tesla T4's $0.298 with 8GB more memory.

What should I watch out for when fine-tuning Fish Speech?

The flow is: organise .wav files with matching .lab transcripts per speaker, extract semantic tokens with extract_vq.py, pack them into protobuf with build_dataset.py, then train with LoRA using the official r_8_alpha_16 config. The trap is documented but easy to miss: do not fine-tune a model that has already been trained with reinforcement learning — the docs warn it shifts the model distribution and degrades quality. Give fine-tuning an RTX A6000 48GB on NexGPU at $0.817/GPU-hr, which holds weights, optimiser state and dataset together.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.