Vision-language model

Self-host MiniCPM-V: an 8.7B VLM in 5.03GB

The Q4_K_M weights for MiniCPM-V 4.5 are 5.03GB, and the entire 4.6 release is 1.3B parameters. What actually stops people is never VRAM — it is the transformers version, the mmproj file they forgot to download, and torchcodec fighting their CUDA build.

MiniCPM-V · self-hosted

MiniCPM-V is the open-source vision-language family maintained by OpenBMB (ModelBest together with Tsinghua's THUNLP) at github.com/OpenBMB/MiniCPM-V, with weights and code released under Apache-2.0 and cleared for commercial use. What sets it apart from most VLMs is how small it is. The current workhorse, MiniCPM-V 4.5, holds 8,695,895,280 parameters — 17.39GB in bf16, so full precision fits on a single 24GB card. The newest release, MiniCPM-V 4.6, goes further down to 1,300,428,016 parameters and 2.60GB of bf16 weights, tagged by OpenBMB as an On-Device Model with shipped adaptation code for iOS, Android and HarmonyOS.

The family now runs on two tracks. MiniCPM-V is vision-only: 4.5 pairs a Qwen3-8B language backbone with a SigLIP2-400M vision tower and a 3D-Resampler that compresses video tokens 96x, scoring 77.0 average on OpenCompass and 73.5 on Video-MME. Inside its 40960-token window, query_num is 64 and max_slice_nums is 9, so one high-resolution image costs roughly 640 visual tokens after slicing. Version 4.6 swaps in a Qwen3.5-0.8B backbone, stretches the context to 262144, and adds mixed 4x/16x visual token compression you toggle with downsample_mode. The second track, MiniCPM-o, is fully omnimodal: o 4.5 carries 9,371,787,666 parameters, bolting a Whisper-medium audio encoder and a CosyVoice2-lineage TTS module onto the same Qwen3-8B core for full-duplex live speech and vision.

So "how much VRAM does MiniCPM-V need" has no single answer — it depends which track you are on. For on-device OCR and image understanding, 4.6's Q4_K_M file is only 529MB, while its mmproj vision tower is 1.11GB, larger than the language model itself. That surprises almost everyone on their first deploy. For image reasoning quality in GPT-4o territory, 4.5 in bf16 is the right call. For a real-time avatar that listens and talks back, o 4.5 is documented at 19.0GB in bf16 and 11.0GB at int4, with 28GB or more recommended for full-duplex streaming. Every tier and its matching card is laid out below.

01 —

Every MiniCPM-V release and what it actually costs in VRAM

Parameter counts come from the HuggingFace safetensors index; GGUF sizes are the real file sizes in the official quantisation repos.

Version	Parameters	VRAM	Context	Notes
MiniCPM-V 4.6	1.3B (1,300,428,016)	bf16 weights 2.60GB, ~4GB in practice; Q4_K_M 0.53GB + mmproj 1.11GB	262K (max_position_embeddings 262144)	The on-device flagship. Qwen3.5-0.8B plus SigLIP2-400M with a native 980px vision tower. Now upstreamed into transformers (model_type minicpmv4_6), so with transformers[torch]>=5.7.0 you no longer need trust_remote_code. Supports mixed 4x/16x visual token compression; video is sampled at 1 FPS up to 128 seconds, then uniformly.
MiniCPM-V 4.5	8.7B (8,695,895,280)	bf16 weights 17.39GB; Q4_K_M 5.03GB + mmproj 1.10GB; Q8_0 8.71GB; F16 GGUF 16.38GB	40K (max_position_embeddings 40960)	The general-purpose workhorse. Qwen3-8B plus SigLIP2-400M, with a 3D-Resampler giving 96x video compression; 77.0 on OpenCompass and 73.5 on Video-MME. Ships a hybrid reasoning switch, enable_thinking, off by default. Still a custom_code model, so loading requires trust_remote_code=True, and vLLM must be 0.10.2 or newer.
MiniCPM-o 4.5	9.4B (9,371,787,666)	bf16 19.0GB; int4 11.0GB; full GGUF stack ~8.85GB; full-duplex streaming wants 28GB or more	40K (max_position_embeddings 40960)	The omnimodal release. Qwen3-8B plus SigLip2 at 980px, a Whisper-medium audio encoder and a minicpmtts speech synthesiser. Handles images up to 1.8 megapixels and video up to 10 FPS, averages 77.6 on OpenCompass, and decodes at 154.3 tok/s in bf16 or 212.3 tok/s at int4. If you only need vision, pass init_audio=False and init_tts=False to reclaim several GB.
MiniCPM-V 4.0	4B class	Q4_K_M 2.19GB + mmproj 0.96GB; Q8_0 3.83GB	—	The previous mid-weight tier, sitting between 4.6's on-device brief and 4.5's do-everything role. If you are pinned at 6-8GB of VRAM and find 4.6 too light, it is still a fair compromise, but new projects should start on 4.6 or 4.5.
MiniCPM-V 2.6 / MiniCPM-Llama3-V 2.5	8B / 8.5B	roughly 16-17GB of bf16 weights	—	Archived generations. Most tutorials still floating around cover these two, and their chat template, image slicing parameters and loading calls all differ from the 4.x line — copying an old blog post is the single most common reason a first deploy fails. If a guide mentions MiniCPM-Llama3-V-2_5, skip it.

02 —

Which GPU to rent for MiniCPM-V

Sized from weights plus KV cache plus image-slice headroom — no inflated claims, and no paying for a config that will not run.

MiniCPM-V 4.6 at full precision, or 4.5 as Q4_K_M GGUF, for batch OCR, image tagging and offline evals
RTX 3090 24GB$0.193/GPU-hr
4.5's Q4_K_M plus mmproj is 6.13GB of weights, leaving 24GB free for large batches and long context — and this is the cheapest 24GB card on our network.
MiniCPM-V 4.5 in bf16 on a single card, when you will not accept quantisation loss
RTX 5090 32GB$0.723/GPU-hr
17.39GB of weights on a 24GB card leaves almost nothing for a 40K window; 32GB gives the KV cache and nine-way image slicing room to breathe.
Reproducing the official Cookbook video config: vLLM 0.10.2 with max-model-len 8192
RTX 4090 24GB x2$0.540/GPU-hr ($1.080/hr for the pair)
The Cookbook documents this exact setup on 2x4090 with gpu_memory_utilization at 0.8 and both context and max-num-batched-tokens at 8192; our nodes take up to 14 GPUs each, so you can mirror it directly.
MiniCPM-o 4.5 full-duplex live speech and vision, or vLLM serving real concurrency
RTX A6000 48GB$0.817/GPU-hr
OpenBMB recommends 28GB or more for full-duplex streaming, and on top of 19GB of weights you are also loading the audio encoder, TTS and concurrent KV — 48GB is the first genuinely comfortable size. Scale further on an A100 PCIE 80GB at $0.824/GPU-hr.

03 —

Four steps to a running MiniCPM-V

From boot to your first OpenAI-compatible request is usually under twenty minutes, most of it spent pulling weights.

01
Boot an instance with CUDA ready
Pick a card at console.nexgpu.net and start from a prebuilt PyTorch or vLLM image so you skip driver and CUDA setup entirely. Once it is up you can reach it over SSH, Jupyter, the web terminal, the REST API or the CLI. Billing starts the second the instance starts, and compute billing stops the moment you stop it.
```
ssh root@<your-node>.nexgpu.net -p <port>
```
02
Pull the weights — and do not forget mmproj
On the GGUF path the language model and the vision tower are two separate files. MiniCPM-V 4.6 makes this especially counterintuitive: the Q4_K_M language model is 529MB while mmproj-model-f16.gguf is 1.11GB. Download only the language model and your model quietly degrades to text-only and cannot see images at all.
```
hf download openbmb/MiniCPM-V-4_5-gguf MiniCPM-V-4_5-Q4_K_M.gguf mmproj-model-f16.gguf --local-dir ./mcpv45
```
03
Serve it through vLLM with an OpenAI-compatible API
MiniCPM-V 4.5 is a custom_code model, so --trust-remote-code is mandatory and vLLM must be at least 0.10.2; video work also needs vllm[video]. Declare your multi-image and multi-video ceilings with --limit-mm-per-prompt or those requests get rejected outright. For 4.6, swap the model name and drop --trust-remote-code.
```
vllm serve openbmb/MiniCPM-V-4_5 --trust-remote-code --dtype auto --max-model-len 8192 --gpu-memory-utilization 0.9 --limit-mm-per-prompt '{"image":5,"video":2}' --port 8000
```
04
Or go llama.cpp and push the whole GGUF onto the GPU
llama.cpp has supported MiniCPM-V 4.5 since build b6282. The --mmproj argument is the vision tower you downloaded in the previous step, and -ngl 99 offloads every layer to the GPU. 6.13GB of weights runs very comfortably on a 24GB card, leaving the rest of the VRAM for context.
```
llama-server -m ./mcpv45/MiniCPM-V-4_5-Q4_K_M.gguf --mmproj ./mcpv45/mmproj-model-f16.gguf -ngl 99 -c 8192 --port 8080
```

What a full evaluation run actually costs

Here is the real arithmetic. Take an RTX 5090 32GB at $0.723/GPU-hr running MiniCPM-V 4.5 in bf16 for document extraction: about 0.4 hours to pull 17.39GB of weights and install vLLM 0.10.2, then 6 hours of batch inference — 6.4 hours total, so compute is $0.723 x 6.4 = $4.63. Attach a 40GB volume for weights and outputs and keep it 3 days: $0.414 / 30 x 3 x 40 = $1.66. Egress 2GB of results: $0.0081 x 2 = $0.02. That is $6.31 for a complete evaluation run. Flip the approach and run the Q4_K_M build instead, and the same 6.4 hours on an RTX 3090 24GB costs $0.193 x 6.4 = $1.24 — less than a coffee. Billing is metered per second and priced per hour, with no minimum, no setup fee and no quota request to file. Compute billing stops the second you stop the instance; only storage keeps accruing until you destroy it.

04 —

FAQ

How much VRAM do I actually need to run MiniCPM-V 4.5 locally?

Full-precision bf16 weights are 17.39GB (8,695,895,280 parameters at 2 bytes each). Add the KV cache for a 40K window plus image-slice overhead and a 24GB card will run it but stay tight, while 32GB is comfortable. On the Q4_K_M GGUF path it is 5.03GB for the language model plus 1.10GB for mmproj — 6.13GB total, so 8GB is enough and 16GB or more lets you run real concurrency. On NexGPU that maps to an RTX 3090 24GB at $0.193/GPU-hr for the quantised build or an RTX 5090 32GB at $0.723/GPU-hr for full precision, both billed per second so compute stops when you do.

MiniCPM-V 4.6 is only 1.3B — is renting a GPU even worth it?

A single request will genuinely run on CPU or even a phone; OpenBMB ships adaptation code for iOS, Android and HarmonyOS. But the moment you go batch — tens of thousands of receipts through OCR, tagging an image library, running a full visual eval over private data — CPU throughput becomes the bottleneck, and that 1.11GB vision tower is itself compute-hungry. A few hours on an RTX 3090 24GB at $0.193/GPU-hr will finish the job for far less than waiting three days on CPU.

What is the difference between MiniCPM-V and MiniCPM-o, and which should I pick?

MiniCPM-V only sees — images, multi-image, video, OCR. MiniCPM-o adds a Whisper-medium audio encoder and a TTS module on the same Qwen3-8B backbone, so it hears and speaks, including full-duplex live conversation. The cost is VRAM: o 4.5 is documented at 19.0GB in bf16 and 11.0GB at int4, with 28GB or more recommended for full-duplex streaming. Vision-only work should stay on V, where you can also load with init_audio=False and init_tts=False. For a live avatar, take o 4.5 on an RTX A6000 48GB at $0.817/GPU-hr, which fits the audio stack, TTS and concurrent KV with room to spare.

Why does the model fail to load when I follow a tutorial?

Four recurring traps. First, MiniCPM-V 4.5 and MiniCPM-o 4.5 are custom_code models requiring trust_remote_code=True or --trust-remote-code, while 4.6 has been upstreamed and instead demands transformers[torch]>=5.7.0 — a 4.x transformers simply will not recognise the minicpmv4_6 model_type. Second, a missing mmproj file on the GGUF path leaves the model blind. Third, torchcodec clashes with certain CUDA builds, so switch to PyAV or pin your CUDA version. Fourth, do not use eager for attn_implementation — use sdpa or flash_attention_2. NexGPU ships 2,000+ prebuilt images with PyTorch and vLLM already configured, which removes most of this class of problem.

Is a 40960-token context enough for long video?

More than you would expect, because the 3D-Resampler in MiniCPM-V 4.5 compresses video tokens 96x. A few official parameters matter in practice: set max_slice_nums to 1 for video (the default of 9 is for slicing high-resolution stills), sample at 3-5 FPS, and use MAX_NUM_FRAMES=180 with MAX_NUM_PACKING=3. If you want a large context with several concurrent video streams, the official Cookbook's own tested configuration is 2x4090 with gpu_memory_utilization 0.8 and 8192 context — our RTX 4090 24GB is $0.540/GPU-hr, so $1.080/hr reproduces it exactly, and a single node can take up to 14 GPUs if you need to go further.

Can I use MiniCPM-V commercially, and is self-hosting safe for my data?

Yes. MiniCPM-V 4.5, 4.6 and MiniCPM-o 4.5 release both weights and code under Apache-2.0, so commercial use needs no extra licence; OpenBMB only asks that you optionally fill in a registration questionnaire, and disclaims liability for generated content. On the data side, the whole point of self-hosting is that your images and video never leave your own environment — on NexGPU you get a dedicated instance, and with 1,175 verified nodes across 51 countries and regions you can pick a region that fits your compliance requirements. There is no quota request to file, and bilingual support is on Telegram with no ticket queue.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.