Skip to main content
NexGPU

Speech Recognition

Self-host Whisper. One 24GB card handles the whole transcription pipeline.

From a 574MB q5_0 quant to a 10GB large-v3, Whisper is one of the rare models small enough for any card and accurate enough for production. Rent a GPU and keep the audio inside your network.

Sign up freeView pricing

Whisper is OpenAI's automatic speech recognition model, open-sourced in 2022 and still deployed everywhere. It is a plain encoder-decoder Transformer: audio is sliced into 30-second windows, converted to a log-Mel spectrogram for the encoder, and the decoder autoregressively emits timestamped text. The newest open weights are still large-v3-turbo from October 2024 — OpenAI has not released a newer open Whisper since, and the last openai-whisper pip release is v20250625. So "which version is current" has effectively converged for Whisper; the real iteration is happening in the inference runtimes around it.

Know the two workhorse checkpoints. large-v3 has 1550M parameters, uses 128 Mel bins (large-v2 used 80), was trained on 1 million hours of weakly labelled plus 4 million hours of pseudo-labelled audio, covers 99 languages, and OpenAI reports a 10%–20% error reduction over large-v2. large-v3-turbo is a pruned-and-finetuned 809M version whose decoder layers were cut from 32 to 4; the official table lists it at ~8x relative speed for a minor accuracy loss. Critically, it was never trained for translation — pass --task translate and it will still return the source language. That is the single most common selection mistake.

There are two sets of VRAM numbers for Whisper and you need both. The official repo quotes the reference implementation: ~1GB for tiny/base, ~2GB small, ~5GB medium, ~10GB large, ~6GB turbo. But almost everyone shipping to production runs faster-whisper (CTranslate2) or whisper.cpp (GGML), where the measured footprint is far lower — faster-whisper's own benchmark shows large-v2 at 4525MB in fp16 and 2926MB in int8. So the honest answer to "how much VRAM does Whisper need" depends on your runtime, and the table below lists both.

01 —

Every Whisper variant and what it actually costs in VRAM

Official reference-implementation requirements, real CTranslate2 and GGML weight sizes, and who should use which

VersionParametersVRAMContextNotes
whisper-large-v31550MOfficial ~10GB / faster-whisper fp16 measured 4.5GB / GGML fp16 3.1GB / q5_0 1.08GB30-second audio window, 448-token decoder limitThe accuracy baseline. 128 Mel bins, 99 languages, includes a Cantonese token. Use it when you need top transcription quality or multilingual translation.
whisper-large-v3-turbo809MOfficial ~6GB / GGML fp16 1.62GB / q5_0 just 574MB30-second audio window, 448-token decoder limitDecoder pruned 32 layers to 4 then finetuned; ~8x relative speed, RTFx around 200 on the model card. The default choice for pure transcription — but it will not translate.
whisper-medium / medium.en769MOfficial ~5GB / GGML fp16 1.53GB30-second audio window, 448-token decoder limitLargely superseded by turbo. Still worth it on an 8GB legacy card, or when an English-only checkpoint is needed to hold hallucination rates down.
whisper-small / small.en244MOfficial ~2GB / GGML fp16 488MB30-second audio window, 448-token decoder limitThe sweet spot for low-latency streaming. A T4 can host several concurrent streams, and whisper.cpp gets close to realtime on CPU alone.
distil-large-v3.5756M (English-only, MIT)Same class as turbo, roughly 1.6GB of fp16 weights30-second audio window, 448-token decoder limitDistilled from large-v3 on 98k hours, about 1.5x faster than turbo. The better trick: use it as the draft model for speculative decoding with large-v3 for ~2x speedup with byte-identical output.

02 —

Which GPU to rent for Whisper

Metered per second, compute billing stops the moment the instance stops, no quota request

  • Single-card evaluation and small-to-mid offline batches (turbo fp16 or q5_0)

    RTX 3090 24GB$0.193/GPU-hour

    Turbo weights are only 1.6GB, so 24GB leaves room to push batch size to 16–32 and still keep an alignment model resident; Ampere gives you SDPA and FlashAttention-2. Best price-performance on the whole fleet for Whisper.

  • Production pipeline: large-v3 fp16 at batch 16 with WhisperX alignment and pyannote diarization co-resident

    RTX 4090 24GB$0.540/GPU-hour

    All three models still fit inside 24GB, and Ada's decode throughput is well ahead of the 3090 — lowest unit cost once your queue is thousands of audio-hours a day.

  • Many concurrent workers, speculative-decoding model pairs, or multi-hour long-form queues

    A100 PCIE 80GB$0.824/GPU-hour

    80GB holds large-v3 and distil-large-v3.5 together for speculative decoding, or six to eight workers saturating one card. It is $0.007 more than the 48GB RTX A6000 at $0.817 for an extra 32GB.

  • Absolute cheapest fp16 offline batch runs

    Tesla V100 32GB$0.188/GPU-hour

    Cheapest card we rent and it carries 32GB; the fp16 paths in faster-whisper and whisper.cpp are rock solid on Volta. Just know that Volta has no bf16 and no FlashAttention-2, so the newer transformers optimisation paths are off the table.

03 —

Four steps to a running Whisper service

From bare instance to an OpenAI-compatible transcription endpoint

  1. 01

    Launch an instance on a PyTorch image and install faster-whisper

    NexGPU ships 2,000+ prebuilt images — boot the PyTorch or Whisper ASR image and the CUDA stack is already in place. Note that recent ctranslate2 builds only support CUDA 12 and cuDNN 9; nine out of ten "cannot find libcudnn_ops_infer.so.8" errors are a version mismatch, and a prebuilt image sidesteps the whole class.

    pip install -U faster-whisper==1.2.1
  2. 02

    Run a turbo baseline to confirm throughput and footprint

    The first run pulls the CTranslate2 weights from Hugging Face automatically. Start with compute_type float16; if VRAM is tight, switch to int8_float16 — faster-whisper's own benchmark drops large-v2 from 4525MB to 2926MB in int8 and gets faster doing it. Turning on VAD noticeably cuts hallucinations across silent stretches.

    python -c "from faster_whisper import WhisperModel; m=WhisperModel('turbo', device='cuda', compute_type='float16'); segs,info=m.transcribe('audio.wav', beam_size=5, vad_filter=True); print(info.language); [print(f'[{s.start:.2f}->{s.end:.2f}] {s.text}') for s in segs]"
  3. 03

    Add WhisperX for word-level timestamps and speaker labels

    Whisper natively emits token-level timestamps at 0.02s precision, so real word boundaries require forced alignment. WhisperX aligns with wav2vec2 and diarizes with pyannote's speaker-diarization-community-1, reaching roughly 70x realtime on large-v2 with batched inference while staying under 8GB at beam_size=5. The diarization model is gated — accept the agreement on Hugging Face and pass a token.

    whisperx audio.wav --model large-v3 --compute_type float16 --batch_size 16 --diarize --hf_token $HF_TOKEN --output_format srt
  4. 04

    Serve it: vLLM exposes an OpenAI-compatible transcription API

    vLLM's OpenAI-compatible server supports /v1/audio/transcriptions and /v1/audio/translations for ASR models, so existing clients can point the openai SDK at your instance with no application changes. Use the NexGPU vLLM image to skip the build, then reach the endpoint over an SSH tunnel or through the REST API.

    vllm serve openai/whisper-large-v3-turbo --port 8000

What 1,000 hours of audio actually costs

Start from a sourced speed anchor. faster-whisper's published benchmark: an RTX 3070 Ti 8GB on CUDA 12.4 transcribes 13 minutes of audio with large-v2 in fp16 at batch_size=8 in 17 seconds — roughly 46x realtime. large-v3-turbo has one-eighth the decoder depth of large, so estimating 60x realtime on an RTX 4090 is conservative. Take a 1,000-hour archive of meeting recordings: 1,000 / 60 = 16.7 GPU-hours, plus about 0.3 hours to pull weights and warm up, call it 17 hours. 17 x $0.540 = $9.18, or $0.0092 per audio-hour. Want it cheaper? Move to the RTX 3090 24GB. The same job at a conservative 40x realtime is 25 GPU-hours: 25 x $0.193 = $4.83. A thousand hours of recordings for under five dollars. Storage is separate: faster-whisper's large-v3 weights are 3.09GB and turbo is 1.62GB, and even with the wav2vec2 alignment model and pyannote diarization on top, a 10GB volume is plenty — 10 x $0.414 = $4.14/month. Storage keeps billing until the volume is destroyed, while compute billing stops the second the instance does, so stop the instance when the batch finishes and keep the volume for next time instead of re-downloading weights. Egress is a rounding error: transcripts are plain text. Even pulling all 1,000 hours of 16kHz mono WAV (about 115GB) back down costs 115 x $0.0081 = $0.93.

04 —

FAQ

How much VRAM does Whisper really need? Is an 8GB card enough?

It depends entirely on the runtime. The official reference implementation asks for ~1GB (tiny/base), ~2GB (small), ~5GB (medium), ~10GB (large) and ~6GB (turbo). Switch to faster-whisper and large-v2 measures 4525MB in fp16 and 2926MB in int8; whisper.cpp's q5_0 turbo weights are only 574MB. An 8GB card runs turbo comfortably and can even run large-v3 — you just cannot push batch size. What actually consumes VRAM is batched inference plus any alignment and diarization models sharing the card. Since an RTX 3090 24GB on NexGPU is $0.193/GPU-hour, laying the whole pipeline out on 24GB beats squeezing batches onto 8GB.

large-v3 or large-v3-turbo — is turbo simply better?

For pure transcription, take turbo: 809M parameters, decoder pruned from 32 layers to 4, about 8x relative speed with only minor accuracy loss. Two exceptions demand large-v3. First, translation — turbo was never trained for the translate task and returns the source language regardless of the flag. Second, low-resource languages and noisy audio, where the full 1550M model is markedly more stable. Want both? Pair large-v3 with distil-large-v3.5 for speculative decoding: roughly 2x faster with output identical to large-v3. That pair sits comfortably on a NexGPU A100 PCIE 80GB at $0.824/GPU-hour.

Whisper hallucinates, loops the same phrase, and invents captions over silence. How do I fix it?

This is Whisper's best-known failure mode, rooted in autoregressive decoding over fixed 30-second windows — hit silence or music and it keeps writing. Three layers of defence. First, run VAD in front of it; faster-whisper's vad_filter=True uses Silero-VAD V6 and handles most of it. Second, use the official quality fallbacks: compression_ratio_threshold=1.35 catches repetition, logprob_threshold=-1.0 catches low confidence, and passing temperature=[0.0, 0.2, 0.4, 0.6, 0.8, 1.0] retries a discarded segment at rising temperature. Third, disable condition_on_previous_text on long-form audio so one derailed segment cannot poison everything after it. Separately, always pass attention_mask for batched inference in transformers or you get subtle silent bugs.

Can Whisper identify speakers or give accurate word-level timestamps?

Neither, and this is the most common misconception. Whisper has no speaker diarization capability at all, and it emits token-level timestamps (0.02s default precision) whose word boundaries frequently drift. The standard fix is WhisperX: wav2vec2 forced alignment for genuine word-level timestamps, then pyannote's speaker-diarization-community-1 for diarization — the full stack still runs large-v2 inside 8GB. Note the pyannote model is gated, so accept the user agreement on Hugging Face and configure a token. The entire chain fits on one NexGPU RTX 4090 24GB at $0.540/GPU-hour.

Is Whisper still the right choice, or have Parakeet and Voxtral overtaken it?

Whisper's open weights do stop at large-v3-turbo from October 2024, but it remains the open ASR model with 99-language coverage, the deepest ecosystem and the most complete quantisation tooling. The alternatives trade something away: NVIDIA's parakeet-tdt-0.6b-v3 is only 600M parameters with an RTFx of 3,332 and a 6.34% average WER on the Open ASR leaderboard, but covers just 25 European languages and ships under CC-BY-4.0; Mistral's Voxtral-Mini-3B-2507 is Apache-2.0, swallows 30 minutes of audio at once and adds audio Q&A, but wants about 9.5GB in bf16. For Chinese, Japanese, Korean and Cantonese, Whisper is still the safe default. If you want a real comparison, spin up several per-second-billed instances on NexGPU and benchmark them on your own dataset — more reliable than any leaderboard.

What licence are the Whisper weights under, and does my audio have to leave my network?

The official repo README states plainly that both the code and the model weights are released under the MIT License; the Hugging Face card for large-v3-turbo is tagged MIT while the large-v3 card's metadata is tagged Apache-2.0 — both permissive, both fine commercially. As for keeping audio in-house, that is the whole point of self-hosting: calling a cloud transcription API means every recording is uploaded to a third party, which rarely clears compliance for medical, legal or call-centre audio. Self-host on NexGPU and the weights live on your instance, the audio lives on your volume, and compute billing stops the moment you stop the instance. We have 1,175 verified rentable nodes across 51 countries and regions, so you can pick a region that satisfies your data-residency rules, connect via SSH, Jupyter, web terminal, REST API or CLI, and reach bilingual support on Telegram with no ticket queue.

More in Speech recognition

Every model guide

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.

Sign up freeTalk to sales

Sign up to browse live network pricing. No payment method required.