Speech Recognition · NVIDIA NeMo Speech

Self-hosting NeMo speech recognition: from a 2GB Parakeet load to a 24-minute A100 transcript

NVIDIA's speech stack has split out of the monolithic NeMo repo, and the live line is NeMo Speech v3.0.0. This page covers what Parakeet, Canary and Nemotron ASR actually need in VRAM, which cards to rent, and the traps you will hit self-hosting them.

NeMo · self-hosted

First, the naming. NeMo is not a model — it is NVIDIA's speech framework, Apache 2.0, and the weights you actually download are called Parakeet, Canary and Nemotron ASR. It is also not Mistral NeMo, which is a 12B text LLM from Mistral and NVIDIA; same name, different thing entirely. More importantly, the repo itself split: v2.7.3 was the final pre-split release, after which NVIDIA/NeMo pivoted to audio, speech and multimodal LLMs, and the line you follow now is NeMo Speech v3.0.0 (shipping in the 26.07.00 NGC container). The environment bar moved with it: Python 3.12 or above, PyTorch 2.7 or above, with the actively-tested stack sitting on Python 3.13, PyTorch 2.12 and CUDA 13.2. Follow a two-year-old tutorial and you stall on step one.

Second, the thing people get wrong when picking a card: NeMo ASR memory scales with audio length, not parameter count. The model cards are blunt about it — parakeet-tdt-0.6b-v3 needs "at least 2GB RAM for model to load. The bigger the RAM, the larger audio input it supports", and canary-1b-v2 needs "at least 6GB". That sounds like any old card will do, until you read the next line: Parakeet v3 handles up to 24 minutes of audio with full attention on an A100 80GB, and only reaches 3 hours once you switch to local attention. Which is exactly why the single most common complaint in the Hugging Face discussions is CUDA out of memory on long audio, with nothing to do with model size. Size the card against your longest recording.

Third, if you need Chinese: the two flagships do not do it. Both parakeet-tdt-0.6b-v3 and canary-1b-v2 cover 25 European languages, and Chinese is not on the list. That leaves three routes — use nemotron-3.5-asr-streaming-0.6b, which spans 40 language-locales with zh-CN sitting in the second "broad-coverage" tier rather than the top "transcription-ready" tier, so it works but do not expect English-grade accuracy; fine-tune with NeMo on your own data, which is clearly viable given NVIDIA itself ships single-language builds like parakeet-tdt_ctc-0.6b-ja and parakeet-ctc-0.6b-Vietnamese; or pick a different model. That middle route is a rental workload by definition. On NexGPU an RTX 3090 24GB is $0.193/GPU-hour, an A10 24GB is $0.414, an A100 SXM4 80GB is $1.088, all metered per second — running your longest audio and hardest dataset once costs a fraction of buying the wrong card.

01 —

The current NeMo speech line-up and what it needs

Minimum load memory, audio-length boundaries and licence, straight from the official model cards.

Version	Parameters	VRAM	Context	Notes
nvidia/parakeet-tdt-0.6b-v3	600M (FastConformer-TDT)	2GB minimum to load; 24-minute full-attention audio needs an A100 80GB	24 minutes full attention / 3 hours local attention	The batch-transcription workhorse. 25 European languages with automatic language detection, 6.34% average WER and 3,332.74 RTFx on the Open ASR Leaderboard, punctuation, capitalisation and word- plus segment-level timestamps built in. CC-BY-4.0.
nvidia/canary-1b-v2	978M (32-layer FastConformer encoder + 8-layer Transformer decoder)	6GB minimum to load	Dynamic chunking with 1-second overlap kicks in above 40 seconds, at batch_size=1	Transcription and translation in one model: X-to-English and English-to-X. 7.27% WER on the MLS 6-language benchmark, 749 RTFx. The card only validates Ampere, Hopper and Blackwell — do not put it on Volta. CC-BY-4.0.
nvidia/canary-qwen-2.5b	2.5B (SALM: FastConformer encoder + Transformer decoder)	No official minimum published; 2.5B parameters puts bf16 weights around 5GB, so give it 16GB or more in practice	Trained on audio up to 40s and sequences up to 1024 tokens	The English accuracy ceiling: 5.63% mean WER on the Open ASR Leaderboard, at the cost of RTFx dropping to 418. Runs in both ASR mode and LLM mode, requires NeMo 2.5.0 or higher. CC-BY-4.0.
nvidia/parakeet-unified-en-0.6b	600M (Unified-FastConformer-RNNT, 24 encoder layers)	Same class as Parakeet v3 — budget headroom by longest audio, not by parameter count	One checkpoint for offline and streaming, at 2.08s / 1.12s / 0.56s / 0.40s / 0.32s / 0.24s / 0.16s / 0.08s	Two modes from one set of weights for English apps, minimum latency 160ms, with encoder, predictor and joint parameters shared across both. LibriSpeech test-clean 1.63%, test-other 3.11%. NVIDIA Open Model License.
nvidia/nemotron-3.5-asr-streaming-0.6b	600M (FastConformer-CacheAware-RNNT with Prompt)	No official minimum published; same 0.6B class, with throughput measured on a single H100	Chunk size continuously tunable from 80ms to 1,120ms	The multilingual streaming answer: 40 language-locales in three tiers — 19 transcription-ready (including ja-JP, ko-KR, ar-AR), 13 broad-coverage (zh-CN lands here), 8 adaptation-ready. OpenMDW-1.1.
nvidia/parakeet_realtime_eou_120m-v1	120M (FastConformer-RNNT, 17 encoder layers, attention context [70,1])	The smallest of the family — memory is not the constraint; Volta, Ampere, Hopper and Blackwell supported	80ms–160ms end-to-end, 16kHz mono input, 160ms minimum duration	Built for voice agents: it transcribes and emits an <EOU> token to signal the user has finished. EOU detection latency p50 160ms, p90 280ms, p95 320ms, 9.30% average WER. Needs NeMo 2.5.3 or higher. NVIDIA Open Model License.

02 —

Picking a card for NeMo ASR by workload

Size VRAM against your longest recording, not your parameter count — every tier maps to a card you can actually rent on NexGPU.

Parakeet TDT 0.6B v3 batch transcription with audio split into segments
RTX 3090 24GB$0.193/GPU-hour
The 0.6B weights officially load in 2GB, which leaves 22GB for batch size and audio length. The 3090 is Ampere, which is on Parakeet's supported-architecture list, and it is the cheapest 24GB card we rent. If you want a machine NVIDIA tested by name, the Tesla T4 16GB at $0.298/GPU-hour is on the card's own test list.
Canary-1B-v2 for transcription plus speech translation
A10 24GB$0.414/GPU-hour
It needs 6GB just to load, and the model card only validates Ampere, Hopper and Blackwell — Volta is not on it. The A10 is one of the machines NVIDIA lists as tested, and 24GB leaves plenty of headroom for the dynamic chunking that engages above 40 seconds.
Long-form audio at native full attention, 24 minutes in one pass
A100 PCIE 80GB$0.824/GPU-hour
That 24-minute full-attention figure in the model card was measured on an A100 80GB. Switching to local attention gets you to 3 hours, but if you want the result without slicing files or changing the attention config, this is the entry point.
Chinese or multilingual streaming, voice agent always on
RTX 4090 24GB$0.540/GPU-hour
Lovelace is on the supported list for nemotron-3.5-asr-streaming-0.6b, whose 40 language-locales include zh-CN. Streaming is a latency workload, not a memory one — per-request compute is small, so one 4090 carries many concurrent 80ms–1,120ms sessions.

03 —

Four steps to a running NeMo

From boot to first transcript, plus the one line you must add before going live.

01
Spin up an instance and check versions first
NeMo Speech v3.0.0 wants Python 3.12 or above and PyTorch 2.7 or above; the stack NVIDIA continuously tests is Python 3.13, PyTorch 2.12 and CUDA 13.2. Pick a PyTorch image from NexGPU's 2,000+ prebuilt images and skip an evening of CUDA setup. Get in over SSH or the web terminal and confirm — a version mismatch only surfaces when you load a .nemo checkpoint, by which point you have already paid for half an hour.
```
nvidia-smi && python -c "import torch; print(torch.__version__, torch.version.cuda)"
```
02
Install only the asr extra
ASR does not need the whole framework, and the extra shown here is exactly what the model cards specify. The first from_pretrained call pulls the .nemo checkpoint from Hugging Face into a local cache. Mind where that lands: NexGPU storage runs at a $0.414/GB-month median and keeps billing after the instance stops, so put weights and datasets on a volume you actually intend to keep.
```
pip install -U "nemo_toolkit[asr]"
```
03
Load the model and run your first file
Parakeet is one line to load and one to transcribe, with punctuation, casing and timestamps included. Canary additionally wants source_lang and target_lang — those two arguments are the only switch between transcribing and translating, and forgetting them is the single most common source of confusion in the discussions. You can also go the transformers route: Parakeet now has native AutoModelForTDT, AutoModelForRNNT and AutoModelForCTC support.
```
python -c "import nemo.collections.asr as nemo_asr; m = nemo_asr.models.ASRModel.from_pretrained('nvidia/parakeet-tdt-0.6b-v3'); print(m.transcribe(['audio.wav'])[0].text)"
```
04
Switch to local attention before long audio OOMs on you
Long-form CUDA out of memory is the top reported problem for these models. Parakeet v3 stops at 24 minutes with full attention on an A100 80GB, and reaches 3 hours with local attention. Add this line before you go live — it is far cheaper than tuning batch size one notch at a time after the fact. Once the pipeline holds, decide whether it is worth moving to an A100 or H100.
```
asr_model.change_attention_model(self_attention_model="rel_pos_local_attn", att_context_size=[256, 256])
```

What a NeMo run actually costs

Start with the common case: Parakeet TDT 0.6B v3 batch transcription on an RTX 3090 24GB at $0.193/GPU-hour, so an overnight 8-hour pass is 0.193 x 8 = $1.544. Want Canary-1B-v2 for transcription plus translation? An A10 24GB at $0.414/GPU-hour over the same 8 hours is 0.414 x 8 = $3.312. For native full attention on long audio, an A100 PCIE 80GB at $0.824/GPU-hour running a full day is 0.824 x 24 = $19.776. Keeping a streaming voice agent always on for a month on an RTX 4090 24GB works out to 0.540 x 24 x 30 = $388.80 — well under the cost of buying one 4090 and a machine to hold it, and compute billing stops the moment you stop the instance. Fine-tuning your own ASR on local-language data on an A100 SXM4 80GB at $1.088/GPU-hour is 1.088 x 12 = $13.056 for a 12-hour single-card epoch; four cards for 3 hours is 1.088 x 4 x 3 = $13.056, the same money for a quarter of the wall-clock time. Storage is separate: a 0.6B .nemo checkpoint plus a few tens of GB of audio, call it 50GB, is 50 x 0.414 = $20.70/month, and storage keeps billing after the instance stops, so destroy what you no longer need. Everything is metered per second and priced per hour, with no minimum, no setup fee and no quota request.

04 —

FAQ

Is NeMo the same thing as Mistral NeMo? Which one am I downloading?

Different things. Mistral NeMo is a 12B text LLM built by Mistral with NVIDIA. The NeMo on this page is NVIDIA's speech framework, NeMo Speech, under Apache 2.0, and the weights you actually pull are named Parakeet, Canary and Nemotron ASR. The repo also split once: v2.7.3 was the last pre-split release, after which NVIDIA/NeMo turned to audio, speech and multimodal LLMs, and the line to follow is NeMo Speech v3.0.0. If you are unsure, do not buy a card — rent an RTX 3090 24GB on NexGPU at $0.193/GPU-hour and you will know inside an hour.

Do Parakeet and Canary support Chinese? Can NeMo do Chinese speech recognition?

Neither flagship does. parakeet-tdt-0.6b-v3 and canary-1b-v2 both cover 25 European languages, and Chinese is not among them. Three routes remain: use nemotron-3.5-asr-streaming-0.6b, which spans 40 language-locales with zh-CN in the second broad-coverage tier rather than the top transcription-ready tier, so it works but do not benchmark it against English; fine-tune with NeMo on your own data, which NVIDIA has clearly demonstrated with single-language builds such as parakeet-tdt_ctc-0.6b-ja and parakeet-ctc-0.6b-Vietnamese; or use a different model family. The middle route is exactly what rented GPUs are for — an A100 SXM4 80GB on NexGPU is $1.088/GPU-hour, metered per second, so the cost of finding out is a few hours of card time.

Parakeet throws CUDA out of memory on long audio — is my card too small?

Usually the audio is too long rather than the model too large: NeMo ASR memory tracks duration, not parameter count. The model card states the boundary plainly — 24 minutes with full attention on an A100 80GB, up to 3 hours once you set self_attention_model="rel_pos_local_attn" with att_context_size=[256, 256]. So there are only three fixes: chunk the file, switch to local attention, or rent more VRAM. NexGPU's A100 PCIE 80GB is $0.824/GPU-hour — running your longest recording end to end for one hour beats endlessly guessing at batch sizes on a small card.

The transcript has words from other languages mixed in, or whole sentences missing. Is the model broken?

Not broken — these are behaviours the community reports repeatedly. Three keep coming up: without a language tag the model hallucinates words from other languages, it occasionally drops whole sentences, and digits sometimes come out as letters. The mitigations are to specify the language explicitly, chunk long audio on semantic boundaries, and apply rule-based number normalisation in post-processing. If you want the English accuracy ceiling instead, canary-qwen-2.5b sits at 5.63% mean WER on the Open ASR Leaderboard, with RTFx falling from Parakeet v3's 3,332.74 to 418. Both are worth running over the same audio once; NexGPU meters per second, so the comparison costs a few dollars.

For a real-time voice agent, which NeMo model do I pick and how big a card?

Depends what you need. For one checkpoint that does offline and streaming, take parakeet-unified-en-0.6b: 160ms minimum latency, with settings stepping from 2.08s down to 0.08s. For multilingual streaming, take nemotron-3.5-asr-streaming-0.6b with chunks tunable from 80ms to 1,120ms. To detect that the user has stopped talking, take parakeet_realtime_eou_120m-v1, which emits an <EOU> token while transcribing at p50 160ms, p90 280ms and p95 320ms detection latency in only 120M parameters. For overlapping speakers there is multitalker-parakeet-streaming-0.6b-v1, which injects speaker kernels into the encoder but needs one model instance per speaker and degrades noticeably past four speakers. Streaming is a latency workload, not a memory one, so an RTX 4090 24GB at $0.540/GPU-hour on NexGPU is enough to keep one online — and support is bilingual over Telegram with no ticket queue.

Can I use these commercially? How do the licences break down?

Two layers, and the weights are not uniform. The NeMo Speech framework itself is Apache 2.0. On the weights: parakeet-tdt-0.6b-v3, canary-1b-v2 and canary-qwen-2.5b are CC-BY-4.0, so attribution is enough for commercial use; parakeet-unified-en-0.6b, multitalker-parakeet-streaming-0.6b-v1 and parakeet_realtime_eou_120m-v1 are under the NVIDIA Open Model License; nemotron-3.5-asr-streaming-0.6b is OpenMDW-1.1. Read that column before you commit to a model, not after you ship. Where you run it does not change the licence — NexGPU spans 51 countries and regions, 1,175 verified rentable nodes and 2,498 GPUs, and your models and audio stay inside your own instance. The console is at console.nexgpu.net.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.