Speech Recognition Toolchain

Self-hosting Kaldi, where VRAM was never the wall

Chain acoustic models weigh tens of megabytes — a 24GB card is overkill. What actually picks your GPU is how many concurrent streams the CUDA decoder holds, and whether configure's gencode list recognises your silicon.

Kaldi · self-hosted

Kaldi is the reference implementation of WFST speech recognition. It landed on GitHub in 2015 and today carries 15.4k stars, 5.3k forks, and an Apache-2.0 licence. It also has a quirk no other project shares: there is not a single release tag in the repository. The version number lives in a plain text file at src/.version, and its entire contents are two characters — 5.5. Daniel Povey contributed close to 4,000 commits and then moved on; the last push to master was September 2025. It is not archived, and downstream projects like Vosk (15k stars, still shipping) still run directly on it.

So anyone searching for how to self-host Kaldi has to answer a prior question: which Kaldi? Mainline kaldi-asr/kaldi is a C++ WFST toolkit — GMM forced alignment, HCLG decoding graphs, lattice post-processing, and a batched CUDA decoder contributed by NVIDIA. Next-gen Kaldi is an entirely separate stack: k2 supplies differentiable FSA/FST operations, lhotse handles data, icefall holds the training recipes, and sherpa-onnx does cross-platform inference — its latest release, v1.13.6, shipped on 18 August 2026. Two lines, one name, zero shared code.

As for VRAM, none of the LLM arithmetic transfers. A chain model is 17 TDNN-F layers at dim 1536 with a 160-wide bottleneck; on disk it is tens of megabytes, and 8GB is already comfortable for training. The real resource walls are elsewhere. Feature extraction and alignment fan out across CPU cores. Building HCLG and dumping egs eats RAM and disk. Only batched GPU decoding genuinely consumes VRAM — and upstream's cudadecoder/README is blunt about the scale: roughly 200 concurrent decodes on a 16GB card, with memory growing linearly in both control threads and batch size. Size the card by concurrency, not by parameter count.

01 —

Which Kaldi are you actually installing?

The mainline C++ toolkit, the Next-gen training stack, and the cross-platform runtime each bill VRAM differently.

Version	Parameters	VRAM	Context	Notes
Mainline Kaldi (kaldi-asr/kaldi, src/.version = 5.5)	chain TDNN-F: 17 layers × dim 1536 / bottleneck 160	8GB is already ample for training; CUDA decoding ≈ 200 concurrent per 16GB	Full-utterance WFST decoding — no context window as such	No release tags at all on GitHub; the version string lives only in src/.version, and the last push was 2025-09-22. If you need WFST graphs, GMM alignment, lattice post-processing, or a byte-faithful reproduction of an s5 recipe, this is the only option.
k2 (k2-fsa/k2)	Differentiable FSA/FST operator library — no weights of its own	Negligible by itself; memory tracks PyTorch and --max-duration	—	The compute kernel behind Next-gen Kaldi, Apache-2.0, still updated as of July 2026. Prebuilt wheels come from k2-fsa.github.io/k2/cuda.html, with cuda-cn.html mirrors for China, and now ship for ROCm 7.2 with PyTorch 2.12.1.
icefall Zipformer-small	23.3M (LibriSpeech) / 30.17M (AISHELL-1)	fp16, 2 × 32GB, --max-duration 1500 (EN) / 1200 (ZH)	Non-streaming full utterance; add --causal 1 for 320ms-chunk streaming	LibriSpeech test-clean 2.42 / test-other 5.73; AISHELL-1 dev 4.67 / test 4.97. The official RESULTS.md numbers were produced on two 32G V100s, which makes this the cheapest honest starting point on the whole line.
icefall Zipformer (normal)	65.55M (LibriSpeech) / 73.41M (AISHELL-1)	fp16, 4 GPUs at --max-duration 1000, 50 epochs	Non-streaming; the streaming variant is 66.11M at 320ms chunks	LibriSpeech test-clean 2.21 / test-other 4.79; the 320ms streaming build lands at test-clean 2.79 / test-other 7.36. This is the tier most teams ship as-is.
icefall Zipformer-large / CR-CTC	148.4M (transducer) / 147.0M (CR-CTC) / 157.29M (AISHELL-1 large)	The CR-CTC recipe is 2 × 80GB A100 at --max-duration 1400	Non-streaming	CR-CTC pushes LibriSpeech test-clean to 2.02 and test-other to 4.35, the best figures on this line. The 80GB cards are not decorative — --max-duration 1400 will not fit on 32GB.
sherpa-onnx v1.13.6	Inference runtime — ships no weights	Runs on CPU; GPU builds target CUDA 12 + cuDNN 9, or CUDA 13 + cuDNN 9 + onnxruntime 1.27.1	Whatever the loaded model provides	Released 2026-08-18, Apache-2.0. One C++ core feeds Zipformer, Paraformer, SenseVoice, Whisper, Moonshine, TeleSpeech and Parakeet, and runs on Linux, Windows, macOS, Android, iOS, HarmonyOS, Raspberry Pi, RISC-V and several NPU families.

02 —

Pick the card for the job, not for the parameter count

Kaldi's memory profile has nothing to do with an LLM's — drop the 'params × 2 bytes' reflex here.

Compile Kaldi with CUDA and train chain / nnet3 models
RTX 3090 24GB$0.193/GPU-hr
sm_86 sits inside the default gencode list that Kaldi's configure emits for CUDA 12.x, so it builds with zero flag surgery; the chain model itself is tens of megabytes, leaving 24GB wide open at num-chunk-per-minibatch 64.
Batched offline transcription with batched-wav-nnet3-cuda2
Tesla T4 16GB$0.298/GPU-hr
Upstream's cudadecoder/README calibrates on exactly this size — about 200 concurrent decodes on a 16GB GPU, at 4 control threads × batch 50 — and sm_75 is in the default gencode list too.
High-concurrency transcription that runs the stock --num-channels 600
RTX A6000 48GB$0.817/GPU-hr
The README states memory scales linearly with control threads and batch size, so 48GB is roughly where the shipped defaults of --max-batch-size 400 / --num-channels 600 fit without dialling anything back.
Reproducing icefall Zipformer training or fine-tuning on your own data
Tesla V100 32GB$0.188/GPU-hr
The small (23.3M) and CR-CTC medium (66.2M) entries in icefall's RESULTS.md were trained on two 32G V100s, so the --world-size 2 command copies over verbatim; only the CR-CTC large run at --max-duration 1400 genuinely needs an A100 PCIE 80GB ($0.824/GPU-hr).

03 —

Four steps to a working Kaldi on NexGPU

From a stock Ubuntu CLI image to a GPU decoder emitting lattices.

01
Boot an instance, satisfy the dependencies, build with CUDA
Launch the Ubuntu CLI or PyTorch image, SSH in, and run check_dependencies.sh first — it will name g++ ≥ 4.8.3, gfortran, zlib, sox, libtool and automake, and you install whatever it flags. The tools/ makefile then builds OpenFst 1.8.4 and sph2pipe for you. One trap worth heading off: CUDA 12.x requires GCC below 12.3, so the g++-13 that ships with Ubuntu 24.04 gets rejected outright by configure — install g++-12 and point CXX at it. The in-repo CMake path exists but upstream itself warns it 'may not be well tested and some features are missing', so take the tools/ + src/ route.
```
cd kaldi/tools && extras/check_dependencies.sh && make -j$(nproc) && cd ../src && ./configure --shared --use-cuda=yes --cudatk-dir=/usr/local/cuda && make -j$(nproc) depend && make -j$(nproc)
```
02
Build the decoding graph — the number-one chain-model trap
mkgraph.sh defaults --self-loop-scale to 0.1, which is correct for legacy GMM and nnet3 models and wrong for chain models, which need 1.0. The script literally prints "WARNING: chain models need '--self-loop-scale 1.0'", but it scrolls past in a wall of FST output and people miss it, then spend a day wondering why their WER is inexplicably bad. This step costs CPU and RAM, not VRAM: across the LG → CLG → HCLGa → HCLG composition, determinizing a large n-gram is the memory peak of the entire pipeline.
```
utils/mkgraph.sh --self-loop-scale 1.0 data/lang_test_tgsmall exp/chain/tdnn_1d_sp exp/chain/tdnn_1d_sp/graph_tgsmall
```
03
Start batched GPU decoding — but lower the defaults first
batched-wav-nnet3-cuda2 ships with --max-batch-size 400 and --num-channels 600, while the upstream README only claims about 200 concurrent decodes on a 16GB card. Run the defaults on a T4 and you will run out of memory. Start at 4 control threads × batch 50, confirm it is stable, then climb. A second default is just as counter-intuitive: --cuda-use-tensor-cores is false out of the box — the source comment describes it as enabling FP16 tensor math and recommends it for inference — so unless you set it explicitly, the tensor cores on your T4 or A100 sit idle. And keep the README's own line in mind: a single utterance is not enough work to saturate any NVIDIA GPU. Concurrency is the whole point.
```
batched-wav-nnet3-cuda2 --cuda-use-tensor-cores=true --max-batch-size=50 --num-channels=200 --cuda-decoder-copy-threads=2 --frame-subsampling-factor=3 --acoustic-scale=1.0 final.mdl HCLG.fst scp:wav.scp ark:lat.ark
```
04
For state-of-the-art accuracy, cross over to Next-gen Kaldi
Mainline Kaldi's chain model scores 7.48% CER on AISHELL-1. On the same test set, icefall's Zipformer-small gets 4.97%, large 4.49%, and the CR-CTC tier 3.98%. No amount of hyperparameter tuning closes a gap that size. k2 publishes mirrored wheel indices for users behind slow links, the training command copies straight out of RESULTS.md, and once trained you export to ONNX and hand it to sherpa-onnx — the same artefact then runs on Android, iOS and HarmonyOS.
```
pip install k2 -f https://k2-fsa.github.io/k2/cuda.html && ./zipformer/train.py --world-size 2 --num-epochs 60 --start-epoch 1 --use-fp16 1 --exp-dir zipformer/exp --max-duration 1200
```

The arithmetic, shown

Training first. icefall's published AISHELL-1 Zipformer-small recipe is --world-size 2 --num-epochs 60 --max-duration 1200 on two 32GB V100s. NexGPU's Tesla V100 32GB is $0.188/GPU-hr, so two cards are $0.376/hr: a 48-hour run is $0.376 × 48 = $18.05, and 72 hours is $0.376 × 72 = $27.07. If you want to push --max-duration higher and buy back wall-clock, two A100 PCIE 80GB at $0.824/GPU-hr is $1.648/hr, and the same 48 hours costs $79.10. Now decoding. Mainline Kaldi's batched-wav-nnet3-cuda2 on a single Tesla T4 16GB ($0.298/hr), held at the ~200 concurrent decodes upstream calibrates against, costs $0.298 × 24 = $7.15 for a full day. Move to an RTX A6000 48GB ($0.817/hr) and run --num-channels at its stock 600, and a day is $19.61 — three times the streams for well under three times the price, so cost per concurrent stream actually falls. Finally, data on disk. Call features plus the egs dump 200GB: storage is $0.414/GB-month × 200 = $82.80/month, or $2.76/day. One rule matters here — compute billing stops the moment the instance stops, but storage keeps billing until it is destroyed, so delete the egs when the run finishes. Egress is $0.0081/GB, which makes pulling a 400MB model and its lattices back down 0.4 × $0.0081 ≈ $0.003. No minimum term, no setup fee, no quota request.

04 —

Frequently Asked Questions

Is Kaldi still maintained? Is it worth using in 2026?

The last push to mainline was September 2025, the repository has never cut a release tag, and the version string lives only in src/.version, where it reads 5.5. Daniel Povey contributed nearly 4,000 commits and then redirected his effort into Next-gen Kaldi. That said, it is not archived, and production projects like Vosk still run on it today. The test is simple: if you need WFST decoding graphs, GMM forced alignment, lattice post-processing, or a faithful reproduction of a published s5 recipe, use mainline; if you only need accuracy, use icefall. On NexGPU both live on the same machine and the same image, billed per second, so evaluating both costs a few dollars.

How much VRAM does Kaldi actually need?

Training barely registers — a chain TDNN-F is 17 layers at dim 1536 with a 160 bottleneck, the model file is tens of megabytes, and an RTX 3090 24GB at $0.193/GPU-hr is more than enough. Batched CUDA decoding is where VRAM matters: upstream's cudadecoder/README calibrates at roughly 200 concurrent decodes on a 16GB card, with memory rising linearly in control threads and batch size. So the right question is not 'how much VRAM does Kaldi need' but 'how many concurrent streams do I need'. T4 16GB and A6000 48GB are both in stock on NexGPU — start on the T4, measure your real concurrency, then decide whether you need the bigger card.

Can I build Kaldi on an RTX 5090?

Yes, but not out of the box. Kaldi's configure emits a default gencode list for CUDA 12.x that runs sm_50 through sm_90 — it stops at Hopper, and Blackwell's sm_120 is not in it. You will need to pass --cuda-arch="-gencode arch=compute_120,code=sm_120" yourself, on a CUDA release that knows Blackwell, while still keeping GCC below 12.3. If you just want it running today, NexGPU's RTX 3090 (sm_86, $0.193/GPU-hr) and A100 PCIE 80GB (sm_80, $0.824/GPU-hr) are inside the default list and build straight after configure. We stock the RTX 5090 32GB at $0.723/GPU-hr too — just budget the compile time.

What accuracy does Kaldi reach on Mandarin AISHELL, and how far behind Zipformer is it?

Kaldi's own egs/aishell/s5 RESULTS file spells it out: mono 36.41%, tri5a 12.12%, nnet3 tdnn_sp 8.65%, chain tdnn_1a_sp 7.48% CER. On the same AISHELL-1 test set, icefall reports Zipformer-small (30.17M) at 4.97%, normal (73.41M) at 4.67%, large (157.29M) at 4.49%, and CR-CTC medium (66.2M) at 3.98%. That is close to half the error rate. For a Mandarin system going to production, the numbers have already made the argument. Two Tesla V100 32GB on NexGPU at $0.188/GPU-hr reproduce icefall's recipe exactly as published.

My batched-wav-nnet3-cuda2 run has low GPU utilisation, or it OOMs. What now?

Two usual causes. First, the defaults are aggressive: --max-batch-size 400 and --num-channels 600 exceed what a 16GB card can hold, so drop to batch 50 and channels 200 and climb once it is stable. Second, tensor cores are off by default: --cuda-use-tensor-cores is false, and you must set it to true for FP16 tensor math to engage during inference. Then there is the README's own framing — one utterance cannot saturate any NVIDIA GPU, so concurrency is what fills the card. On NexGPU you can move between T4, A6000 and A100 without a quota request or a minimum rental term, so testing all three costs about the price of lunch.

Can I install k2 and icefall from a slow or restricted network?

Yes. k2 publishes mirrored wheel indices alongside the primary ones — swap k2-fsa.github.io/k2/cuda.html for cuda-cn.html, cpu.html for cpu-cn.html, and rocm.html for rocm-cn.html (now current through ROCm 7.2 with PyTorch 2.12.1). The genuinely slow part is never the packages; it is pulling LibriSpeech or AISHELL and running lhotse feature extraction. NexGPU has 1,175 verified rentable nodes across 51 countries and regions, so you can launch in whichever region sits closest to your data source, pull the corpus, then stop the instance — compute billing stops immediately while storage persists for the next run.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.