Speech recognition

Self-hosting Paraformer: 220M params, one RTX 3090 is overkill

Single-step non-autoregressive decoding gets Paraformer to 120x realtime on a GPU, and the whole chain — ASR plus VAD, punctuation and diarization — is under 520M parameters. Here are the versions, the VRAM, the commands and the bill.

Paraformer · self-hosted

Paraformer comes out of Alibaba DAMO Academy (now the FunAudioLLM team) and an INTERSPEECH 2022 paper by Zhifu Gao, Shiliang Zhang, Ian McLoughlin and Zhijie Yan. It attacks the obvious weakness of autoregressive transformers — emitting one token at a time is slow. A CIF (continuous integrate-and-fire) predictor first estimates how many tokens this audio should produce, a glancing language model sampler injects semantic embeddings back in to strengthen context modelling, and a single-step non-autoregressive decoder resolves the whole utterance in one pass. The paper evaluates on AISHELL-1, AISHELL-2 and a 20,000-hour industrial corpus, reporting more than 10x inference speedup over autoregressive baselines.

By 2026 Paraformer is no longer the headline act in the FunASR model zoo. Above it sit LLM-style recognisers — Fun-ASR-Nano (800M), Qwen3-ASR (1.7B), GLM-ASR-Nano (1.5B) — and beside it SenseVoiceSmall (234M), which also emits emotion and audio-event tags. But the official model-selection guide is blunt about it: if your scenario is pure Mandarin and you need character-level timestamps or hotword customisation, pick Paraformer. It is the mature non-autoregressive path — 220M params, Apache-2.0 weights on Hugging Face, 60,000+ hours of training audio, no engineering surprises. In production, boring is a feature.

VRAM was never Paraformer's constraint — throughput and pipeline plumbing are. Because it is non-autoregressive with no LLM decoder, FunASR's own vLLM guide states plainly that the vLLM acceleration path does not support Paraformer (vLLM serves only architectures with an LLM decoder, such as Fun-ASR-Nano and GLM-ASR-Nano). So renting an H100 buys you no continuous-batching dividend. The right move is cheap cards, many processes, and batch_size_s cranked up. On NexGPU an RTX 3090 24GB is $0.193/GPU-hour, billed per second with no minimum — transcribing ten thousand hours of backlog audio can cost less than lunch.

01 —

Which Paraformer variants exist, and what each one needs

Taken from the FunASR v1.4.3 model registry — stop copying model IDs from 2024 blog posts

Version	Parameters	VRAM	Context	Notes
paraformer-zh (actually resolves to SeACo-Paraformer-large)	220M	~0.9GB fp32 weights / ~0.45GB fp16; budget 4GB for single-card inference	Segmented by fsmn-vad, default max segment 60s (max_single_segment_time=60000)	Passing model="paraformer-zh" to FunASR actually fetches iic/speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch — the ICASSP 2024 SeACo-Paraformer with its hotword branch. For the plain model, pass model="paraformer".
paraformer (Paraformer-large, vocab8404)	220M	~0.9GB fp32 weights; runs in 2GB	Also relies on VAD segmentation for long audio	The original single-step NAR large model: Mandarin with code-switched English, character-level timestamps, no hotword branch. Easiest entry point for ONNX export.
paraformer-zh-streaming (...vocab8404-online)	220M	~2GB resident per stream, scaling linearly with concurrent sessions	chunk_size=[0, 10, 5], i.e. 600ms output granularity; encoder_chunk_look_back=4, decoder_chunk_look_back=1	This is the one for live dictation. chunk_stride = chunk_size[1] * 960 — your feed stride must match it or the output drifts.
paraformer-en (vad-punc, vocab10020)	220M	~2GB (VAD and punctuation are baked into the pipeline)	VAD + PUNC built in, long audio works out of the box	English offline transcription. Note that FunASR only fixed this alias's resolution on Hugging Face in v1.4.1 (2026/08/04); older versions pull the wrong weights.
Companion chain: fsmn-vad + ct-punc + cam++	0.4M + 290M + 7.2M ≈ 298M	~1.2GB fp32 weights combined, co-resident with the ASR model	VAD segments, punctuation restores sentence boundaries, cam++ labels speakers	The full stack is ~518M params, ~2.1GB of fp32 weights. ct-punc is larger than the ASR model itself — people who only size the ASR model miss half the footprint.
Newer siblings: SenseVoiceSmall / Fun-ASR-Nano-2512	234M / 800M	~0.9GB / ~3.2GB fp32	SenseVoice covers zh/en/ja/yue/ko; Fun-ASR-Nano covers zh/en/ja plus 7 Chinese dialects and 26 regional accents	If you need multilingual coverage, emotion and audio-event tags, or dialect and accent robustness, Paraformer is not the right pick — go straight to these. Fun-ASR-Nano also runs on vLLM, benchmarked at RTFx 340.

02 —

Which GPU to rent for Paraformer

A 220M model does not need a flagship card — spend the money on card count and throughput instead

Single card running the full paraformer-zh pipeline for batch transcription and quality checks
RTX 3090 24GB$0.193/GPU-hr
The whole chain is ~2.1GB of fp32 weights, so 24GB is barely a tenth used — and the 3090 is the cheapest modern CUDA card on NexGPU, which decides it.
Always-on streaming dictation service with paraformer-zh-streaming and concurrent sessions
A10 24GB$0.414/GPU-hr
A datacentre card built for 24/7 duty. Each stream costs about 2GB, so 24GB comfortably holds ten-plus concurrent sessions without thermal throttling under sustained load.
Bulk-transcribing a backlog of thousands of hours with batch_size_s maxed and multiple worker processes
RTX 4090 24GB$0.540/GPU-hr
Paraformer cannot use vLLM, so throughput comes purely from raw per-card speed and parallelism — the 4090's shorter wall-clock usually beats the 3090's lower hourly rate here.
Fine-tuning SeACo-Paraformer on domain hotwords with train_ds.py across multiple GPUs
RTX A6000 48GB$0.817/GPU-hr
Training holds weights, gradients, optimiser state and activations at once; 48GB lets you raise the batch instead of fiddling with accumulation steps. Nodes take up to 14 GPUs, so torchrun scales in-box.

03 —

Four steps to a running Paraformer on NexGPU

From an empty instance to an OpenAI-compatible transcription endpoint

01
Spin up an instance and install FunASR
Pick a PyTorch prebuilt image, start an RTX 3090, SSH in and install funasr. FunASR requires Python 3.8-3.13 and PyTorch >= 1.11.0; the prebuilt image already ships a CUDA-matched torch, so do not reinstall it on top and clobber the build.
```
pip3 install -U funasr modelscope && python -c "import torch; print(torch.cuda.is_available())"
```
02
Bring up the full offline pipeline in one call
AutoModel chains ASR, VAD, punctuation and diarization together. vad_kwargs caps each segment at 60 seconds, and batch_size_s batches dynamically by total audio duration (not by file count) — pushing it past 300 is where long-audio batch jobs get their gains. device accepts cuda:0, cpu, mps or xpu.
```
from funasr import AutoModel
model = AutoModel(model="paraformer-zh", vad_model="fsmn-vad", vad_kwargs={"max_single_segment_time": 60000}, punc_model="ct-punc", spk_model="cam++", device="cuda:0")
print(model.generate(input="meeting.wav", batch_size_s=300)[0]["text"])
```
03
Expose it: OpenAI-compatible endpoint or streaming WebSocket
funasr-server stands up a POST /v1/audio/transcriptions endpoint on port 8000, so existing Whisper API code only needs a new base_url. For live captions switch to paraformer-zh-streaming: chunk_size=[0, 10, 5] gives 600ms granularity, your feed stride must equal chunk_size[1] * 960, and audio must be 16kHz mono PCM16 — format mismatches are the single most common failure in the realtime WebSocket path.
```
pip install fastapi uvicorn python-multipart && funasr-server --model paraformer-zh --device cuda
```
04
Fine-tune for hotwords, or export ONNX to push inference onto CPU
If jargon, names and product terms come back wrong, fine-tune SeACo-Paraformer's hotword branch with train_ds.py under torchrun. If the final serving target is high-concurrency CPU, export to ONNX and serve via funasr-onnx, using the GPU only for training and validation — that split is the cheapest way to run this model.
```
torchrun --nnodes 1 --nproc_per_node 2 funasr/bin/train_ds.py ++model=paraformer-zh ++train_data_set_list=train.jsonl ++train_conf.max_epoch=20
```

What it costs to transcribe a thousand hours

Real numbers. Say you have 1,000 hours of Mandarin meeting recordings to transcribe — 3,600,000 seconds of audio. The funasr/paraformer-zh model card on Hugging Face states 120x realtime on GPU, so 3,600,000 / 120 = 30,000 seconds ≈ 8.33 hours of wall clock. On an RTX 3090 24GB at $0.193/GPU-hour that is 8.33 x $0.193 ≈ $1.61. Want it faster? An RTX 4090 24GB at $0.540/GPU-hour, estimated conservatively at 200x realtime, gives 3,600,000 / 200 = 18,000 seconds = 5 hours, so 5 x $0.540 = $2.70 — 40% faster for 60% more money; your call. The real trick is parallelism: run 8 x RTX 3090 at once and wall clock drops to 8.33 / 8 ≈ 1.04 hours while total GPU-hours stay at 8.33, so the cost is still $1.61. Per-second billing means parallelism is free. Add the edges: 300GB of source audio parked for a day is $0.414/GB-month / 30 x 300 ≈ $4.14; pulling back ~200MB of JSONL transcripts is 0.2 x $0.0081 ≈ $0.002. Under $6 for the whole run — and compute billing stops the moment you stop the instance, storage the moment you destroy the volume. No minimum, no setup fee, no quota request.

04 —

FAQ

How much VRAM does Paraformer actually need to self-host?

Paraformer-large and paraformer-zh are both 220M parameters — roughly 0.9GB of fp32 weights, 0.45GB in fp16. What people miss is the rest of the chain: ct-punc is 290M (larger than the ASR model itself), cam++ is 7.2M, fsmn-vad is 0.4M, totalling ~518M params and ~2.1GB of fp32 weights. Add headroom for activations and dynamic batching and a 4GB budget is comfortable for single-card inference; leave 8GB if you push batch_size_s past 300. Practically, any card with 16GB or more is overkill — which is why the RTX 3090 24GB at $0.193/GPU-hour on NexGPU is the most economical home for this model.

Are model="paraformer-zh" and model="paraformer" the same model?

No, and this is the most common FunASR trap. Open funasr/download/name_maps_from_hub.py and you will see: paraformer maps to iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch, the original Paraformer-large; paraformer-zh maps to iic/speech_seaco_paraformer_large_asr_nat-zh-cn-16k-common-vocab8404-pytorch, which is SeACo-Paraformer with the hotword branch. Both are 220M and both use vocab8404, but the weights and behaviour differ — do not conflate them in a baseline comparison. Want to run both and diff the output? Per-second billing on a single 3090 makes that a sub-$0.20 experiment.

Can Paraformer be accelerated with vLLM? Is an H100 worth renting?

No, and no. FunASR's vLLM guide is explicit: the vLLM path supports only architectures with an LLM decoder — Fun-ASR-Nano (Qwen3-0.6B encoder), GLM-ASR-Nano and the LLMASR family. Paraformer is non-autoregressive with no LLM component and is listed as unsupported; SenseVoice likewise. So renting an H100 SXM 80GB at $3.582/GPU-hour buys no continuous-batching benefit, and a 220M model will never saturate it. The correct move is cheap cards and many processes: eight RTX 3090s running in parallel on NexGPU cost less than half of one H100 hour.

In 2026, should I pick Paraformer, SenseVoice or Fun-ASR-Nano for Chinese ASR?

Depends what you need. Pure Mandarin with character-level timestamps and hotword customisation: Paraformer remains the mature path the official model-selection guide recommends. Multilingual coverage (zh/en/ja/yue/ko) plus emotion and audio-event tags: SenseVoiceSmall at 234M. Dialects, accents, or LLM-style error correction: Fun-ASR-Nano-2512 at 800M, which additionally runs on vLLM at RTFx 340. FunASR's own advice is to ignore leaderboards and benchmark 20-50 representative clips of your real audio. NexGPU ships 2,000+ prebuilt images, so installing all three on one machine and comparing usually costs about a dollar in compute.

How low can Paraformer streaming latency go, and how do I set chunk_size?

The reference configuration is chunk_size = [0, 10, 5], which corresponds to 600ms output granularity, paired with encoder_chunk_look_back = 4 and decoder_chunk_look_back = 1. When feeding audio, chunk_stride must equal chunk_size[1] * 960, and input must be 16kHz mono PCM16 — malformed audio format is the number one cause of realtime WebSocket failures. Shrinking chunk_size[1] cuts latency at the cost of accuracy and more calls. For an always-on streaming service use a datacentre card such as the A10 24GB at $0.414/GPU-hour: about 2GB per stream means one card handles ten-plus concurrent sessions.

Can I use Paraformer commercially? What is the licence?

The FunASR toolkit source is MIT. Pretrained weights follow their individual model cards — funasr/paraformer-zh on Hugging Face is Apache-2.0 with 60,000+ hours of training data, while some industrial pretrained models on ModelScope fall under the repository's model licence agreement. Check the terms for the exact model ID you ship; do not generalise one line across the whole family. On the technical side, FunASR moves fast: v1.4.0 added parameter validation that catches typos like vda_model, v1.4.1 fixed the paraformer-en alias resolution, v1.4.3 added a Silero VAD adapter. Pin your version and record it. On NexGPU you can stand up one instance per version to regression-test, stop them when done, and pay only for the seconds you used.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.