Speech Recognition · ASR

Self-host Faster-Whisper: large-v3 runs in 4,525MB of VRAM

SYSTRAN rewrote OpenAI Whisper on top of CTranslate2 — same accuracy, up to four times faster, less memory. Here is exactly what faster-whisper 1.2.1 costs you in VRAM, which card to rent, and the two things that break every self-hosted install.

Faster-Whisper · self-hosted

faster-whisper is not another Whisper wrapper. It is SYSTRAN's full reimplementation of OpenAI Whisper on the CTranslate2 inference engine, and the project's own claim is that it is "up to 4 times faster than openai/whisper for the same accuracy while using less memory." The repo is github.com/SYSTRAN/faster-whisper, MIT licensed, and the current stable release on PyPI is 1.2.1. You do not even need FFmpeg on the host — audio decoding goes through PyAV, which ships the FFmpeg libraries inside the wheel.

What made it the default choice again is BatchedInferencePipeline, added in 1.1.0: swap one class and the official benchmark's 13-minute audio file drops from 1m03s to 17s. Since then 1.1.1 fixed the VAD memory blowup that caused OOM errors, 1.2.0 added distil-large-v3.5 support plus silence removal inside batched transcription, and 1.2.1 upgraded Silero-VAD to V6 and stopped <|nocaptions|> tokens leaking out of the batched pipeline. If you got here after openai-whisper turned out too slow, this is the right migration.

The VRAM story is almost counterintuitively cheap. The 1550M-parameter large-v3 weight file is only 3.09GB, and SYSTRAN measured a 4,525MB fp16 peak and a 2,926MB int8 peak on a single RTX 3070 Ti 8GB. A 24GB consumer card is not "just barely enough" — it is enough for three or four concurrent workers with room to spare. What actually blocks self-hosting is never capacity; it is the CUDA 12 + cuDNN 9 dependency chain, and VAD parameters left at defaults producing dropped or repeated speech. Both are covered below.

01 —

Which models run, and what each one costs in VRAM

faster-whisper loads CTranslate2 weights directly. Weight sizes below are actual model.bin byte counts on Hugging Face; peak VRAM figures are from the official benchmark

Version	Parameters	VRAM	Context	Notes
large-v3 (Systran/faster-whisper-large-v3)	1550M	fp16 weights 3.09GB / measured peak 4,525MB; int8 peak 2,926MB; at batch_size=8, fp16 6,090MB and int8 4,500MB	30s window · 448 tokens · 100 languages	The multilingual accuracy ceiling and the default for long-form transcription. 1.18M downloads a month on Hugging Face; nearly every downstream project pulls this checkpoint by default.
large-v3-turbo (deepdml/faster-whisper-large-v3-turbo-ct2)	809M	fp16 weights 1.62GB, roughly half of large-v3's 3.09GB	30s window · 448 tokens · 99 languages	Decoder pruned from 32 layers to 4, encoder untouched. OpenAI's own framing: "way faster, at the expense of a minor quality degradation." Pick it when latency matters and you still need multilingual coverage. Natively supported since faster-whisper 1.1.0.
distil-large-v3.5 (distil-whisper/distil-large-v3.5-ct2)	756M	fp16 weights 1.51GB	30s window · English only	Added in faster-whisper 1.2.0. Around 1.46x the relative real-time factor of large-v3-turbo, with 7.08% out-of-distribution short-form WER and 11.39% long-form. Best value for English-only pipelines, and usable as a draft model for speculative decoding against large-v3. No non-English support.
large-v2 (Systran/faster-whisper-large-v2)	1550M	Same footprint as large-v3: 4,525MB fp16 peak, 2,926MB int8 peak — this is the model the official benchmark was measured on	30s window · 99 languages	Do not skip it. The repo's "Whisper-v3 worse than v2" issue has collected over two dozen comments; on some languages and accents v2 is still the steadier model. Benchmark both on your own audio before committing — it costs minutes of GPU time.
medium / small / base / tiny	769M / 244M / 74M / 39M	fp16 weights 1.53GB / 0.48GB / ~0.15GB / ~0.08GB	30s window · 99 languages	medium is already good enough for plenty of production workloads and far faster. small and below suit keyword spotting, language-ID prefiltering, or a coarse first pass that routes only suspect segments to large-v3.

02 —

Which GPU to rent

Sized against CTranslate2's compute_type support and the measured VRAM peaks — no padding, no under-provisioning

large-v3 fp16, batch transcription of long-form audio, cost first
RTX 3090 24GB$0.193/GPU-hour
fp16 at batch_size=8 peaks at 6,090MB, so 24GB holds three to four workers at once; Ampere compute capability 8.6 supports fp16, bf16 and int8 natively, so CTranslate2 never falls back to a slower type.
Live captioning and meeting transcription with large-v3-turbo or distil-large-v3.5, latency first
RTX 4090 24GB$0.540/GPU-hour
Turbo weights are only 1.62GB, so the bottleneck is clocks and memory bandwidth rather than capacity — the 4090 gives the lowest per-request latency of any 24GB card here.
int8 quantized batch jobs at the lowest cost per audio hour
Tesla T4 16GB$0.298/GPU-hour
The int8 peak is 2,926MB, so 16GB is comfortable, and Turing's compute capability 7.5 has native INT8 tensor cores that map straight onto CTranslate2's int8_float16 path.
Transcription plus forced alignment plus diarization in one pipeline (WhisperX, whisper-diarization)
RTX A6000 48GB$0.817/GPU-hour
48GB keeps large-v3, the wav2vec2 alignment model and pyannote's diarization model resident on the same card, removing load/unload churn and doubling the concurrency you can sustain.

03 —

Four steps to a running service

From a cold NexGPU instance to an OpenAI-compatible transcription endpoint in under ten minutes

01
Launch a CUDA 12 instance and install faster-whisper
Pick a PyTorch or Ubuntu CLI prebuilt image in the console — there is also a ready-made Whisper ASR image among the 2,000+ available. SSH in and pip install. Python 3.9 or newer is the only requirement, and you do not need to install FFmpeg separately because PyAV bundles it.
```
pip install faster-whisper==1.2.1
```
02
Supply cuBLAS and cuDNN 9
This is the one step that reliably breaks self-hosted faster-whisper. Recent CTranslate2 releases only support CUDA 12 with cuDNN 9. On CUDA 12 with cuDNN 8, pin ctranslate2 to 4.4.0; on CUDA 11 with cuDNN 8, pin it to 3.24.0. If you containerise, base the image on nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu22.04 and skip the problem entirely. Note that LD_LIBRARY_PATH must be set before Python starts.
```
pip install nvidia-cublas-cu12 nvidia-cudnn-cu12==9.*
export LD_LIBRARY_PATH=`python3 -c 'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__))'`
```
03
Run your first file through the batched pipeline
Do not judge throughput from plain model.transcribe. BatchedInferencePipeline is a drop-in replacement added in 1.1.0 and is roughly four times faster. One thing to internalise: segments is a generator, so transcription does not start until you iterate it — the classic first-run confusion of "it returned instantly and did nothing" is exactly this.
```
from faster_whisper import WhisperModel, BatchedInferencePipeline

model = WhisperModel("large-v3", device="cuda", compute_type="float16")
batched = BatchedInferencePipeline(model=model)
segments, info = batched.transcribe("audio.mp3", batch_size=16, vad_filter=True)
for s in segments:
    print("[%.2fs -> %.2fs] %s" % (s.start, s.end, s.text))
```
04
Expose an OpenAI-compatible transcription endpoint
speaches, listed in the project's own community integrations and MIT licensed, uses faster-whisper as its backend and serves an OpenAI-compatible API with SSE streaming as audio is transcribed. Existing OpenAI SDK code needs nothing changed but the base_url. For true streaming use WhisperLive or Whisper-Streaming; for speaker diarization use WhisperX.
```
git clone https://github.com/speaches-ai/speaches && cd speaches
docker compose --file compose.cuda.yaml up --detach
```

The real number: what 1,000 hours of audio costs

The official benchmark is refreshingly concrete: 13 minutes of audio, large-v2, beam_size=5, batch_size=8, fp16, finished in 17 seconds on a single RTX 3070 Ti 8GB — about 45.9x real time (780 ÷ 17 ≈ 45.9). NexGPU's RTX 3090 24GB is the larger GA102 die from the same generation, so treating 45.9x as a conservative floor is fair. At $0.193/GPU-hour: one GPU-hour covers roughly 45.9 hours of audio, so 1,000 hours of audio needs about 1000 ÷ 45.9 ≈ 21.8 GPU-hours, and 21.8 × $0.193 ≈ $4.21 — about 0.42 US cents per audio hour. Move the same batch to an RTX 4090 24GB at $0.540/GPU-hour and even at that deliberately pessimistic 45.9x it is 21.8 × $0.540 ≈ $11.77, and the 4090 will in practice be faster, so that is a ceiling rather than a floor. Keeping the model cache on persistent storage? large-v3's model.bin is 3.09GB, which at the $0.414/GB-month median works out to about $1.28/month — or skip it and re-pull from Hugging Face on each boot, which takes a few minutes. Billing is metered per second and priced per hour, so a 17-second job is charged as 17 seconds: no minimum, no setup fee, no quota request. Compute billing stops when the instance stops; only storage keeps accruing until you destroy it.

04 —

FAQ

How much VRAM does faster-whisper actually need for large-v3? Is 8GB enough?

It is — the official benchmark was run on an RTX 3070 Ti 8GB: 4,525MB peak for fp16 sequential decoding and 2,926MB for int8. Only batch_size=8 in fp16 pushes it to 6,090MB, which gets tight on 8GB. But VRAM is not the place to economise: 8GB cards are slow, and NexGPU's RTX 3090 24GB is $0.193/GPU-hour, where the extra 16GB buys you three or four concurrent workers and a lower cost per audio hour.

large-v3, large-v3-turbo or distil-large-v3.5 — which should I deploy?

If you need any language other than English, the choice is between large-v3 and large-v3-turbo; distil-large-v3.5 is English-only. Take large-v3 (1550M, 100 languages) for peak accuracy, turbo (809M, decoder cut from 32 layers to 4) for latency. Also benchmark large-v2 honestly — discussion about v3 regressing on certain languages has never stopped in the issue tracker. Spin up one RTX 4090 on NexGPU, run all three over the same audio, and the comparison costs less than lunch.

How do I fix "Could not locate cudnn_ops64_9.dll" or a missing libcudnn_ops.so.9?

This is the single most common class of issue in the repo, and it has one root cause: CTranslate2 4.5.0 and later require cuDNN 9, which requires CUDA 12.3 or newer. The pairing is PyTorch cu121 with ctranslate2 ≤ 4.4.0, and PyTorch cu124 or 2.4.0+ with ctranslate2 ≥ 4.5.0. On Linux, after pip installing nvidia-cudnn-cu12 you must export LD_LIBRARY_PATH before launching Python; on Windows, Purfview's whisper-standalone-win ships the libraries prepackaged. Pick a CUDA 12 prebuilt image on NexGPU and none of this ever comes up.

My output loops, hallucinates, or drops whole passages. What do I change?

Start with vad_filter=True — Silero-VAD was upgraded to V6 in 1.2.1 and strips silences longer than 2 seconds by default, which is where most "inventing text over silence" comes from. For repetition loops, set condition_on_previous_text=False. If you are on BatchedInferencePipeline and seeing mid-file language switches or missing sections, there are open quality-regression issues: pin the language argument instead of letting it auto-detect, and keep a sequential run as a control. This kind of tuning takes dozens of iterations, and NexGPU bills per second — stop the instance and compute billing stops with it.

How many concurrent streams fit on one GPU, and when do I need multiple cards?

Saturate one card first. int8 at batch_size=8 peaks at 4,500MB, so a 24GB card fits five in theory and three to four with sane headroom, and CTranslate2's num_workers gives you in-card parallelism on top. When you genuinely need to scale out, NexGPU offers up to 14 GPUs per node and a maximum node VRAM of 2,152GB, across 2,498 GPUs on 1,175 verified rentable nodes in 51 countries and regions — no quota request in the way.

Can I just run this on CPU, since faster-whisper supports int8 on CPU too?

For small models and short clips, yes. But in the official CPU benchmark the small model at int8 with batch_size=8 needs 51 seconds for that same 13-minute file, while the GPU run does large-v2 in 17 seconds — a model seven times larger, three times faster. The dividing line is volume: a few dozen recordings a day, CPU is fine; hundreds or thousands of audio hours, and CPU costs you more in power and waiting than renting does. NexGPU's Tesla V100 32GB is $0.188/GPU-hour, with SSH, Jupyter, a web terminal, a REST API and a CLI all available, and bilingual support on Telegram with no ticket queue.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.