Voice cloning / voice conversion

Self-hosting RVC: do the VRAM math, then pick the card

Short version: inference fits in 4GB, training follows the WebUI's own default of batch = VRAM_GB / 2, and 10 to 50 minutes of clean vocals is enough for a usable voice. NexGPU starts at $0.193/GPU-hr on an RTX 3090 24GB, metered per second, billed to zero the moment you stop.

RVC · self-hosted

RVC stands for Retrieval-based Voice Conversion. The repo lives under the RVC-Project organisation on GitHub, MIT licensed, 38k stars. It is not TTS — you cannot type text and get speech. You bring a source recording and RVC swaps its timbre for a target voice, while pitch, phrasing and timing stay with the source. The trick is the word retrieval: a top-1 lookup replaces the source's HuBERT features with the nearest features from your training set, which is what kills timbre leakage and why RVC sounds more like the target than bare VITS on the same amount of data. The base model was trained on roughly 50 hours of the open VCTK corpus, so all you are doing is fine-tuning on ten-odd minutes.

The project moved meaningfully in 2026. Release 2.3.260718 swapped the vocal/instrumental separation backend from UVR5 to PyMSS (pymss==2.0.14), added FCPE pitch extraction, and put CUDA Graph on the realtime path — the maintainers measured up to 4.7x faster algorithmic inference latency on a 4090D. Feature extraction, training and index building can now all be interrupted and resumed, inference auto-detects GPU mode and precision, and picking a model auto-matches its .index path. The stack turned over too: Python 3.12 x64, torch 2.7.1+cu128 or +cu118, and a requirements file literally named requirments_cu128_py312.txt (yes, the e is missing upstream). The layout changed as well — the entry point is webui.py and the training scripts moved to a top-level train/ directory, so any 2023 tutorial telling you to run infer-web.py will simply fail.

On RVCv3: the README has long promised a base model retrained with more parameters and more data, but no weights have shipped. Do not wait for it. Applio (IAHispano), the most popular community fork, has publicly moved to maintenance mode — security patches and dependency bumps only. That makes the mainline repo the sane choice for a private deployment today, and its bottleneck was never the code. RMVPE F0 extraction wants a GPU, training wants VRAM, and because the dependency pinned is faiss-cpu, index building hammers your CPU rather than your card. On NexGPU all of that is metered per second and stops billing when the instance stops.

01 —

Which RVC variant you actually pick

Not a parameter-count ladder — the choices are sample rate, feature dimension, and whether pitch is modelled at all

Version	Parameters	VRAM	Context	Notes
RVC v2 40k (with f0)	f0G40k.pth 73.1MB + f0D40k.pth 143MB	Inference <4GB / training from 6GB; a 24GB card defaults to batch 12	40kHz output, features taken at 16kHz via HuBERT	The default almost everyone should use. Singing, speech, covers — and virtually every community voice checkpoint you will find is this tier.
RVC v2 48k (with f0)	f0G48k.pth 75.5MB + f0D48k.pth 143MB	One notch heavier than 40k; 24GB recommended	48kHz output	Use it when the render goes straight into a film mix or a 48kHz master. For dubbing and short-form video it buys you nothing audible and costs real VRAM and time.
RVC v2 32k (with f0)	f0G32k.pth 74MB + f0D32k.pth 143MB	Cheapest of the three; trainable on a 16GB card	32kHz output	For telephone-grade source, old recordings, or when VRAM is genuinely tight. If the input cannot support 40k, forcing 40k just upsamples the noise with it.
RVC v2 no-pitch (no-f0)	G40k.pth 73MB + D40k.pth 143MB	Similar to the f0 build, minus the F0 extraction pass	40kHz output, fundamental frequency not modelled	Speech-only work where you want the timbre swapped but not the pitch tracked. Never use it for singing — without F0 modelling everything comes out off-key.
RVC v1 (the pretrained/ folder)	40k and 48k tiers	Slightly lighter than v2	256-dim HuBERT features	Kept purely to load the v1 voice checkpoints that circulated in 2023. Do not start new work here — the feature dimension differs, so v1 and v2 weights are not interchangeable.
Companion weights: HuBERT + RMVPE	hubert_base 190MB, rmvpe.pt 181MB (rmvpe.onnx 362MB)	Under 1GB resident, combined	Both operate at 16kHz	Not optional — nothing runs without them. Note the current README wants the transformers-format hubert_base/ directory (config.json + pytorch_model.bin), not the old single-file fairseq checkpoint.

02 —

Pick the card for your job

RVC's default batch size is literally VRAM in GB divided by two, which makes this an unusually easy decision

First voice, 10-30 minute dataset, v2 40k
RTX 3090 24GB$0.193/GPU-hr
24GB puts the WebUI's default batch at 12 and keeps you on the fp16 6G config path (x_pad=3, x_query=10, x_center=60, x_max=65) — the best value on the whole price list for RVC.
Long dataset, 48k output, pushing to 200 epochs
RTX 4090 24GB$0.540/GPU-hr
Ada's fp16 throughput turns a 200-epoch run from an overnight job into a few hours, and RMVPE F0 extraction speeds up noticeably too. On a deadline, the extra spend buys same-day delivery.
Realtime pipeline work and CUDA Graph validation
RTX 5090 32GB$0.723/GPU-hr
The CUDA Graph prewarm and run_cuda_graph paths added in 2.3.260718 pay off most on the newest architecture, and 32GB lets PyMSS separation and inference coexist without fighting over memory.
Dozens of voices in one sitting, or a shared team box
RTX A6000 48GB$0.817/GPU-hr
48GB maps to a default batch of 24, or several experiment names training in parallel without contention. Nodes take up to 14 GPUs, with a 2,152GB ceiling on node VRAM.

03 —

Getting RVC running on NexGPU

Python 3.12 plus torch 2.7.1, four steps — most of the wall clock is downloading 1.3GB of pretrained weights

01
Launch an instance, clone the repo, create a Python 3.12 venv
Boot one of the prebuilt PyTorch images and confirm Python is 3.12 x64. That is the only version upstream supports; 3.10 and 3.11 will stall in dependency resolution.
```
git clone https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI && cd Retrieval-based-Voice-Conversion-WebUI && python3.12 -m venv .venv && source .venv/bin/activate
```
02
Install in two stages: torch first, then the requirments file
torch and torchaudio are deliberately excluded from the requirements file and must be installed separately. Use cu128 for RTX 50-series, cu118 for 40-series and older. The filename really is missing an e — don't correct it.
```
python -m pip install torch==2.7.1+cu128 torchaudio==2.7.1+cu128 --index-url https://download.pytorch.org/whl/cu128 && python -m pip install -r requirments_cu128_py312.txt
```
03
Pull the pretrained assets into assets/
The twelve files in pretrained_v2 total about 1.3GB, plus hubert_base at 190MB and rmvpe.pt at 181MB. If you only need 40k, filtering out the 32k and 48k checkpoints halves the download.
```
hf download lj1995/VoiceConversionWebUI --local-dir assets --include "hubert_base/*" "rmvpe/rmvpe.pt" "pretrained_v2/*"
```
04
Start the WebUI and reach it through an SSH tunnel
webui.py listens on 7865 by default and now auto-selects a free port if that one is taken. Do not expose Gradio directly — the pin is gradio 3.14.x, and share links from that era do not belong on the open internet. Tunnel it to localhost instead.
```
python webui.py   # in a second local terminal: ssh -L 7865:localhost:7865 root@<instance-ip>
```

What one voice actually costs

Take a clean 30-minute dataset, v2 40k, 200 epochs. On an RTX 3090 24GB at $0.193/GPU-hr: budget 0.4h for setup plus the 1.3GB weight download, 0.3h for preprocessing, RMVPE F0 extraction and feature extraction, 2.5h for training, and 0.3h to build the faiss index and audition checkpoints — 3.5 GPU-hours total, so 3.5 x $0.193 = $0.68. Move to an RTX 4090 24GB at $0.540/GPU-hr and the training leg drops to roughly 1.1h, about 1.9 GPU-hours: 1.9 x $0.540 = $1.03. Better than twice as fast for $0.35 more. Storage is the line item to watch: dataset, intermediate wavs and per-epoch checkpoints reach 40GB easily, and at $0.414/GB-month that is 40 x $0.414 = $16.56 a month — storage keeps billing after compute stops, right up until you destroy the volume. The right move is to pull the final .pth and .index off the box and destroy it. Those two files land around 250MB, so at $0.0081/GB egress: 0.25 x $0.0081 ≈ $0.002. Two tenths of a cent.

04 —

Frequently asked questions

How much VRAM does RVC need? Will a 4GB card work?

For inference, yes — configs/config.py has a dedicated branch for 4GB and under that drops x_pad to 1, x_query to 5, x_center to 30 and x_max to 32. Training is another matter: the official FAQ says outright that cards at 4GB and below (the 3GB 1060, assorted 2GB cards) can simply be written off. The WebUI's default batch size is max(1, VRAM_GB // 2), so 24GB gives you 12 while 6GB gives you 3. Rather than wrestling OOM locally, take an RTX 3090 24GB at $0.193/GPU-hr, metered per second.

How much audio do I need, and how many epochs?

Upstream recommends 10 to 50 minutes. A tight, characterful 5 to 10 minutes works too; under 1 to 2 minutes is discouraged, and under 1 minute nobody has reported success. Epochs depend on source quality: 20 to 30 is plenty for noisy material, since pushing higher only drags the base model down to your recording's level, while clean and lengthy datasets happily take 200. The WebUI ships with total_epoch at 20 and a checkpoint every 5. Preparing the data is your job; making it finish fast is ours — 3090 through A6000, your pick.

Is RVC still maintained? Did RVCv3 ship? What about Applio?

The mainline repo is very much alive. The 2026 release, 2.3.260718, is a substantial one: PyMSS replaces UVR5, FCPE joins the pitch extractors, CUDA Graph lands on the realtime path (up to 4.7x measured on a 4090D), and training and feature extraction became interruptible and resumable. RVCv3 is still only a README promise of a bigger base model on more data — no public weights. The Applio fork has announced it is no longer updating frequently, just security patches and dependency bumps. So mainline is the deployment target, and on NexGPU the PyTorch environment it needs is already in the 2,000+ prebuilt images.

v1 or v2? And how do I choose between 40k, 48k and 32k?

Use v2, and 40k unless you have a reason. v1 runs on 256-dim HuBERT features and exists only to load 2023-era checkpoints; the two generations' weights are not interchangeable. Take 48k when you are delivering a 48kHz master, 32k when the source is mediocre or VRAM is tight. The one hard rule: never change sample rate mid-training. FAQ Q18 is explicit — start a new experiment name instead, or you will hit a tensor dimension mismatch. Want to try all three at once? Spin up three instances in parallel. Per-second billing, no minimum, no setup fee.

Why is RVC so slow on my Tesla P40 or GTX 1660?

It is not superstition, it is in the code. configs/config.py forces fp32 whenever sm_version == 6.1 (Pascal — the Tesla P40 sits exactly there) or the GPU name matches 16\d{2} at sm 7.5 (the GTX 16 series). You never get the half-precision speedup at all. We do list a Tesla P40 at $0.214/GPU-hr, but route around it for RVC: an RTX 3090 24GB is $0.193/GPU-hr — cheaper — and gives you fp16 the whole way.

What do I need for realtime voice changing, and how low does latency go?

realtime_gui.py defaults to a 0.25s block time, 0.05s crossfade and 2.5s of extra inference; algorithmic delay computes as block + crossfade + 0.01, about 310ms out of the box. The README reports 170ms end to end already achieved, and 90ms with ASIO input/output devices. Note that RVCRealtimeVST is a C++17 VST2/VST3 plugin for Windows 10/11 x64 only — that part runs on your own machine. What belongs in the cloud is training and batch inference: train the voice and validate the CUDA Graph path on an RTX 5090 32GB at $0.723/GPU-hr, then bring the .pth and .index home to your DAW. Questions go straight to us on Telegram, English or Chinese, no ticket queue.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.