Speech Recognition · ASR

Self-hosting SenseVoice: 936MB of weights, one 3090 saturates it

SenseVoiceSmall is a 234M-parameter non-autoregressive model that transcribes 10 seconds of audio in 70 milliseconds — 15× faster than Whisper-Large. What actually stops you is not VRAM. It's the 30-second input cap, the missing word-level timestamps, and the <|zh|><|NEUTRAL|><|Speech|> tags in the raw output.

SenseVoice · self-hosted

SenseVoice comes from FunAudioLLM (Alibaba's Tongyi speech team), described in arXiv:2407.04051, with code at github.com/FunAudioLLM/SenseVoice and weights on both ModelScope (iic/SenseVoiceSmall) and Hugging Face (FunAudioLLM/SenseVoiceSmall). It covers Mandarin, Cantonese, English, Japanese and Korean, and a single forward pass returns four things at once: the transcript, a language ID, an emotion label (HAPPY / SAD / ANGRY / NEUTRAL / FEARFUL / DISGUSTED / SURPRISED), and an audio event tag (BGM / Speech / Applause / Laughter / Cry / Sneeze / Breath / Cough). Because the architecture is non-autoregressive, there is no beam search — and no Whisper-style hallucination loop repeating the same sentence thirty times at the end of a clip.

The size is the counterintuitive part. Hugging Face tags it 0.2B; model.pt is 936MB in fp32, which works out to roughly 234 million parameters. Whisper large-v3, for comparison, is 1.55B and about 3GB. Converted to GGUF, the f16 build is 470MB and q8 is only 254MB — and on the project's own 184-clip Mandarin benchmark f16 scores 8.01% CER while q8 scores 7.99%, so quantization costs essentially nothing. Which makes the answer to "how much VRAM does SenseVoice need" pretty anticlimactic: any modern GPU is overkill. The question you should actually be asking is about concurrency and throughput.

Three things will genuinely trip you up. First, direct inference is hard-capped at 30 seconds of input; anything longer needs fsmn-vad (0.4M params) in front of it, with max_single_segment_time set to 30000 in vad_kwargs. Second, SenseVoiceSmall does not produce word-level timestamps; since funasr 1.3.29 sentence_info restores VAD segment boundaries, but for finer alignment you bolt on the fa-zh (38M) timestamp predictor. Third, every result must go through rich_transcription_postprocess before it is human-readable — otherwise you get a raw string prefixed with angle-bracket tags. Licensing splits too: the repo source is MIT, but the weights ship separately under the FunASR model open-source agreement (commercial use permitted under its terms). And note that the same team's 2512 batch introduced Fun-ASR-Nano-2512 (0.8B, Apache 2.0) — SenseVoice has not been retired, the two lines run in parallel.

01 —

SenseVoice variants and what each one costs in VRAM

Only one downloadable checkpoint is officially released — everything else is a quantization or export of it

Version	Parameters	VRAM	Context	Notes
SenseVoiceSmall (PyTorch, iic/SenseVoiceSmall)	~234M (tagged 0.2B on HF)	936MB fp32 weights · under 2GB for a single stream	≤30s per segment; unlimited with fsmn-vad	The flagship checkpoint. Loads straight into funasr AutoModel with language='auto', use_itn and batch_size_s dynamic batching. Chain ct-punc (1.1G) for punctuation and CAM++ for speaker turns.
SenseVoiceSmall ONNX (funasr-onnx, quantize=True)	~234M	~240MB after INT8 export · runs on CPU or GPU	≤30s per segment, batch_size 10+	Use this when you need to drop the PyTorch dependency and embed ASR into an existing C++/Go service. Requires funasr-onnx >= 0.4.0; funasr-torch >= 0.1.1 gives you the LibTorch route instead.
SenseVoiceSmall GGUF f16	~234M	470MB · ~23× realtime on 8 CPU threads	Pair with fsmn-vad.gguf for long audio	The recommended GGUF precision, 8.01% CER on the 184-clip Mandarin benchmark. Runs as the single llama-funasr-sensevoice binary — no Python runtime on the box at all.
SenseVoiceSmall GGUF q8	~234M	~254MB · ~27× realtime on 8 CPU threads	Pair with fsmn-vad.gguf for long audio	Half the size and it actually scores 7.99% CER. The obvious pick for edge boxes and Pi-class hardware. Vocabulary is embedded, so no separate bpe file to ship.
SenseVoice-Large (paper only)	Not published	N/A	50+ languages, high-accuracy ASR	The encoder-decoder variant described in arXiv:2407.04051 covering 50+ languages. It has never appeared in the open model zoo — only Small is downloadable. For broad multilingual coverage, look at the row below instead.
Fun-ASR-Nano-2512 (same team, successor line)	0.8B	~1.6GB in bf16	Mandarin across 7 dialect groups and 26 accents, plus English and Japanese	Apache 2.0, seq2seq, loads directly via transformers AutoModelForSpeechSeq2Seq, and accepts prompt and keywords for hotword biasing. The sibling Fun-ASR-MLT-Nano-2512 covers 31 languages.

02 —

Which GPU to rent for SenseVoice, by scenario

A 234M model does not eat VRAM — it eats concurrency and throughput, so the selection logic is nothing like an LLM's

Getting the demo running, playing with webui.py and the emotion/event tags
RTX 3090 24GB$0.193/GPU-hr
The weights are under 1GB; 24GB holds SenseVoice plus fsmn-vad, ct-punc and CAM++ with more than half the card free — and this is the cheapest Ampere card per hour on the list.
An always-on ASR endpoint serving dozens of concurrent streams
A10 24GB$0.414/GPU-hr
A datacenter part with passive cooling and ECC, built for a fastapi run --port 50000 process that stays up for months rather than a consumer card pinned at 100% indefinitely.
Backfilling an archive of recordings with batch_size_s wide open
RTX 4090 24GB$0.540/GPU-hr
Ada's fp16 throughput is well ahead of the 3090, and bulk transcription is pure compute — the extra spend converts almost proportionally into a shorter wall clock.
Fine-tuning on your own accents and domain vocabulary via FunASR's finetune.sh
RTX 5090 32GB$0.723/GPU-hr
Full fine-tuning of 234M params plus Adam optimizer state is under 4GB; 32GB lets you push batch size and segment length and converge on one card without touching data parallelism.

03 —

From cold instance to live ASR endpoint in four steps

Boot a PyTorch prebuilt image, one pip line, and the weights pull themselves from ModelScope

01
Spin up an instance and install funasr
Pick an RTX 3090 24GB in the NexGPU console, choose a PyTorch image, and connect over SSH or Jupyter. SenseVoice's dependency tree is light — no CUDA kernels to compile, it just runs. If you want speaker diarization (spk_model='cam++'), install funasr from source instead: pip install git+https://github.com/modelscope/FunASR.git.
```
pip install -U "funasr>=1.3.29"
```
02
Wire up fsmn-vad and run your first long file
Direct inference only accepts clips under 30 seconds, so attach the VAD on day one rather than discovering this in production. Set max_single_segment_time to 30000 ms, then let merge_vad stitch fragments back toward merge_length_s before they hit the model. Always pass the returned text through rich_transcription_postprocess, or you will be reading a raw string that starts with <|zh|><|NEUTRAL|><|Speech|><|withitn|>.
```
python -c "from funasr import AutoModel; m=AutoModel(model='iic/SenseVoiceSmall', trust_remote_code=True, vad_model='fsmn-vad', vad_kwargs={'max_single_segment_time':30000}, device='cuda:0'); print(m.generate(input='audio.mp3', language='auto', use_itn=True, batch_size_s=60, merge_vad=True, merge_length_s=15)[0]['text'])"
```
03
Bring up the HTTP service and expose the port
The repo ships a FastAPI entrypoint and a Dockerfile. Point SENSEVOICE_DEVICE at cuda:0 to run on the GPU; leave it unset and it falls back to CPU. The container route is docker run --gpus all -p 50000:50000 sensevoice. Map port 50000 in the console and your frontend or backend can call it directly.
```
export SENSEVOICE_DEVICE=cuda:0 && fastapi run --port 50000
```
04
Scale up: quantized export or fine-tuning
To shed dependencies, export ONNX with quantize=True for INT8 and push batch_size past 10. To adapt it to your own accents and jargon, head to the FunASR repo, build the training jsonl with sensevoice2jsonl (five files: scp, text, text_language, emo, event), then run bash finetune.sh. Stop the instance when you're done and compute billing stops with it.
```
python -c "from funasr_onnx import SenseVoiceSmall; m=SenseVoiceSmall('iic/SenseVoiceSmall', batch_size=10, quantize=True); print(m(['audio.mp3'], language='auto', use_itn=True))"
```

The real math: transcribing 1,000 hours of audio

The published figure is 70 milliseconds for 10 seconds of audio — roughly 143× realtime. 1,000 hours is 3,600,000 seconds, so raw inference is about 3,600,000 ÷ 143 ≈ 25,200 seconds, or 7 GPU-hours. Pad that by 3× for audio decoding, fsmn-vad segmentation and writing results to disk and call it 21 GPU-hours: on an RTX 3090 24GB that is 21 × $0.193 = $4.05. On an RTX 4090 24GB you finish far sooner, say 10 GPU-hours, at 10 × $0.540 = $5.40. The totals are nearly identical — the only real difference is how soon you have the transcripts. In other words, a thousand hours of audio, transcribed with emotion and audio-event labels attached, costs single-digit dollars in compute. Storage is the line item that will surprise you: 1,000 hours of 16kHz mono WAV is about 115GB, which at $0.414/GB-month is $47.6 per month — ten times the compute. Convert to 64kbps MP3 first and it drops to roughly 29GB ($11.9/month), or destroy the instance once transcription finishes and keep only the text. NexGPU meters compute per second and prices it per hour: stop the instance and it stops costing. Storage keeps billing until you destroy it. Egress on the transcripts themselves runs $0.0081/GB median, so a few dozen megabytes of plain text rounds to nothing. No minimum, no setup fee, no quota request.

04 —

FAQ

How much VRAM does SenseVoice actually need? Is a 24GB card enough?

Wildly enough. SenseVoiceSmall's model.pt is 936MB in fp32 — about 234 million parameters — and a single inference stream stays under 2GB of VRAM. Even with fsmn-vad, ct-punc and CAM++ all resident, 24GB is barely dented. What actually grows memory is concurrency once you raise batch_size_s, not the model itself. So don't rent an A100 for this: an RTX 3090 24GB on NexGPU is $0.193/GPU-hr, metered per second, and you can stop it the moment your test finishes.

SenseVoice or Whisper large-v3 — which should I pick?

For Mandarin and Cantonese, SenseVoice: it beats Whisper on AISHELL-1, AISHELL-2 and WenetSpeech in the project's own benchmarks, runs 5× faster than Whisper-Small and 15× faster than Whisper-Large thanks to the non-autoregressive design, and hands you emotion and audio-event labels for free. If you need dozens of languages or genuinely require word-level timestamps, Whisper still fits better. The size gap is stark: 234M versus 1.55B. Want to benchmark both on your own audio? Spin up two RTX 4090 24GB instances on NexGPU at $0.540/GPU-hr each, run them in parallel, and you'll have an answer by the afternoon.

Can SenseVoiceSmall output word-level timestamps?

No — this is the single most common complaint. The non-autoregressive architecture produces no word-level alignment. Since funasr 1.3.29, sentence_info returns VAD segment boundaries, which is fine for splitting subtitle lines but nowhere near karaoke-style per-word highlighting. For real alignment, attach the fa-zh (38M) timestamp predictor, or switch to paraformer-zh (220M), which ships timestamps natively. All of these together are still under 1GB, so a single NexGPU card holds the whole stack without breaking a sweat.

What happens if my audio is longer than 30 seconds? How do I transcribe an hour-long meeting?

Direct inference is hard-capped at 30 seconds and results degrade past it. The correct pattern is to pass vad_model='fsmn-vad' at load time with vad_kwargs={'max_single_segment_time': 30000}, letting the VAD split on silence, then set merge_vad=True and merge_length_s=15 to stitch fragments back into ~15-second chunks. fsmn-vad is only 0.4M parameters, so it costs neither meaningful VRAM nor time. At 143× realtime, an hour-long meeting finishes in roughly 25 seconds on a NexGPU RTX 3090 — about $0.0013.

Can I use SenseVoiceSmall commercially? How does the licensing work?

The repository source is MIT. The weights are released separately under whatever each model card states — the official SenseVoiceSmall card points at the FunASR model open-source agreement, which permits commercial use provided you comply with its terms. Watch out for third-party repackages (the various GGUF and ONNX conversions), which may carry different terms; check each model card before shipping. And if your compliance rules say the audio never leaves your control, self-hosting is exactly the point: rent a card on NexGPU, keep the data inside your own instance, and stop billing the moment you shut it down.

Is SenseVoice still the right choice, or should I move to Fun-ASR?

Both lines are alive; neither replaced the other. SenseVoice is still actively maintained — funasr 1.3.27 added detected-language metadata to responses, 1.3.29 restored VAD segment timestamps, and a GGUF/llama.cpp single-binary runtime has since landed. Its edge is the 234M footprint, non-autoregressive speed, and getting emotion plus audio events out of one forward pass. The same team's Fun-ASR-Nano-2512 is 0.8B, Apache 2.0, seq2seq, covers 7 Mandarin dialect groups and 26 accents, supports keyword biasing, and has a 31-language MLT variant. The choice comes down to throughput versus dialect and multilingual coverage. Both bill per second on NexGPU — start two RTX 4090 24GB instances, run each against your own audio, and let the numbers decide.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.