Skip to main content
NexGPU

Vision-language model

Self-hosting DeepSeek-VL: 6.7GB to 55GB, three sizes, three very different cards

DeepSeek-VL2 is the version to deploy for general visual understanding on this line; the DeepSeek-OCR series is the document branch that split off from it. Here is the real weight footprint of every variant, the VRAM floor the maintainers actually state, and which NexGPU card each one wants.

Sign up freeView pricing

Most people searching for "DeepSeek-VL local deployment" still picture the 1.3B / 7B dense model. The line has moved twice since then. The first DeepSeek-VL used a hybrid vision encoder — SigLIP-L plus SAM-B — on a fixed 1024×1024 input. DeepSeek-VL2 swapped in a DeepSeekMoE backbone (the config reports DeepseekV2ForCausalLM, 72 routed experts, top-6 activation) with SigLIP-SO400M-384 and dynamic tiling, shipping tiny / small / full in one go. After that, DeepSeek's multimodal releases moved to documents: DeepSeek-OCR and DeepSeek-OCR-2. For general visual understanding, DeepSeek-VL2 is what you deploy.

The VRAM story here is genuinely counterintuitive. The bf16 weights of deepseek-vl2-small are 32.3GB, yet the official README states plainly that the sample script needs 80GB of GPU memory. The gap is entirely image tokens: dynamic tiling shreds a high-resolution image into a pile of 384×384 tiles (the config lists 23 candidate resolutions, up to 3456×384) and every tile's activations sit resident through prefill. The repo's answer is incremental_prefilling with chunk_size=512, which fits small into 40GB at the cost of a much slower first token. Sizing a card by weight footprint is the single most common way to get this model wrong.

The other thing to know up front: the quantization ecosystem is thin. There is no GGUF of DeepSeek-VL2 on Hugging Face, and the llama.cpp support request sat under a stale label and went nowhere, so Ollama and LM Studio are not options. Nearly all the low-bit weights that exist live under mlx-community (4-bit small is 9.3GB) and run only on Apple Silicon. On NVIDIA you either run bf16 honestly, or carry bitsandbytes load_in_4bit / a community AWQ yourself. Which means there is less room to save VRAM than you'd hope — renting a card that is genuinely large enough beats fighting the quantizers.

01 —

Every DeepSeek-VL variant and what it costs in VRAM

Weight sizes read from each repo's safetensors metadata; VRAM floors quoted from the official READMEs

VersionParametersVRAMContextNotes
DeepSeek-VL227.5B-A4.5B (MoE)bf16 weights 55.0GB / official script >80GB4096The flagship: a DeepSeekMoE-27B backbone with MLA attention. The README says small already needs 80GB and the full model "even larger" — without chunked prefill, even an A100 80GB is tight.
DeepSeek-VL2-Small16.1B-A2.8B (MoE)bf16 weights 32.3GB / 80GB naive, 40GB with chunked prefill4096Where most self-hosted evaluation and fine-tuning lands. A 40GB card works via chunk_size=512 but crawls; an 80GB card is the comfortable choice.
DeepSeek-VL2-Tiny3.37B-A1.0B (MoE)bf16 weights 6.74GB4096The only size that sits comfortably on a 24GB consumer card. Good for wiring up the pipeline, tuning prompt templates, debugging LoRA. mlx-community publishes 3/4/6/8-bit, but Mac only.
DeepSeek-VL-7B-Chat (first generation)7.3B (dense)fp16 weights 14.7GB4096Gen one: SigLIP-L + SAM-B hybrid encoder on a fixed 1024×1024 input. A 1.3B-chat sibling also exists (1.98B params, 3.95GB fp16). Worth deploying only to reproduce older results.
DeepSeek-OCR3.3B (DeepSeek3B-MoE-A570M decoder)bf16 weights 6.71GBTiny 64 / Small 100 / Base 256 / Large 400 vision tokensStart of the document branch. DeepEncoder = SAM-base + 16× convolutional compressor + CLIP-large. Decoding precision is roughly 97% under 10× compression and falls to about 60% at 20×. MIT for both code and weights.
DeepSeek-OCR-23.4Bbf16 weights 6.78GB(0-6)×144 + 256 vision tokensThe newest generation on this line, architecture id DeepseekOCR2ForCausalLM, built around Visual Causal Flow. 76.3% overall on olmOCR-bench, 82% on arXiv math OCR, and the licence loosens to Apache-2.0.

02 —

Which NexGPU card to rent

Matched to size and workload; prices are list rate per GPU per hour

  • Wire up deepseek-vl2-tiny, sanity-check prompts and grounding output

    RTX 3090 24GB$0.193/GPU-hr

    6.74GB of weights plus dynamic-tiling image tokens leave plenty of headroom in 24GB, and Ampere runs the torch==2.0.1 the repo pins — no CUDA version fight before you start.

  • DeepSeek-OCR / OCR-2 batch PDF-to-Markdown pipeline

    RTX 4090 24GB$0.540/GPU-hr

    A 6.7GB model fits easily and Ada handles the official torch 2.6.0 + flash-attn 2.7.3 stack; on vLLM remember the model card's advice to turn prefix caching off.

  • Full-precision deepseek-vl2-small evaluation or LoRA fine-tuning

    A100 PCIE 80GB$0.824/GPU-hr

    The README states this size needs 80GB for the stock script. RTX A6000 48GB is $0.817 — for $0.007 more you get another 32GB, so there is no reason to wrestle chunked prefill on a 48GB card.

  • Full 27.5B with interleaved multi-image sessions or concurrent serving

    A100 SXM4 80GB$1.088/GPU-hr

    55GB of weights plus prefill activations makes 80GB the floor; SXM4 bandwidth and NVLink keep MoE expert routing and multi-GPU tensor parallel from choking.

03 —

Four steps to a running DeepSeek-VL2

From instance to OpenAI-compatible endpoint, with the known traps flagged

  1. 01

    Start the instance, then check torch against your GPU architecture

    Pick a card that matches your size and boot one of the 2,000+ prebuilt images — PyTorch or vLLM — then come in over SSH, Jupyter or the web terminal. First job is the torch check: DeepSeek-VL2's requirements.txt pins torch==2.0.1 and transformers==4.38.2, which is fine on Ampere (3090 / A100) but impossible on an RTX 5090, where Blackwell sm_120 needs torch 2.7 or newer. On that card you drop both pins deliberately.

    nvidia-smi && python -c "import torch; print(torch.__version__, torch.cuda.get_device_capability(0))"
  2. 02

    Pull the source and the weights

    The official inference code is deepseek-ai/DeepSeek-VL2 and pip install -e . does the job. Watch for attrdict in the requirements: on Python 3.10+ it blows up on how it imports from collections, so install attrdict3 instead — this bites on both generations. Then pull the weights; small is 32.3GB, so check the instance's disk allocation before you start.

    git clone https://github.com/deepseek-ai/DeepSeek-VL2.git && cd DeepSeek-VL2 && pip install -e . && pip install attrdict3 && hf download deepseek-ai/deepseek-vl2-small --local-dir /workspace/deepseek-vl2-small
  3. 03

    Short on VRAM? Turn on chunked prefill

    If all you have is a 40GB-class card, use the repo's own incremental_prefilling to feed image embeddings in chunks with chunk_size=512. It is the only method the maintainers document for fitting deepseek-vl2-small into 40GB, and first-token latency climbs noticeably. For throughput work, don't — take the 80GB card instead.

    inputs_embeds, past_key_values = vl_gpt.incremental_prefilling(input_ids=prepare_inputs.input_ids, images=prepare_inputs.images, images_seq_mask=prepare_inputs.images_seq_mask, images_spatial_crop=prepare_inputs.images_spatial_crop, attention_mask=prepare_inputs.attention_mask, chunk_size=512)
  4. 04

    Move to vLLM for an OpenAI-compatible endpoint

    The README itself calls the bundled demo an implementation "without any deployment optimizations" and points at vLLM, SGLang or LMDeploy for production. vLLM already ships DeepseekVLV2ForCausalLM with LoRA and pipeline-parallel support; max-model-len tops out at 4096 because that is the model's own ceiling. For DeepSeek-OCR / OCR-2, add --no-enable-prefix-caching and --mm-processor-cache-gb 0 as the model card advises.

    vllm serve deepseek-ai/deepseek-vl2-small --trust-remote-code --max-model-len 4096

What this actually costs

Run 50,000 PDF pages through DeepSeek-OCR. The paper reports over 200,000 pages per day (about 2,500 tokens/s) on a single A100-40G, so 50,000 pages is roughly a quarter of a day — call it 6 hours. NexGPU's A100 PCIE 80GB is $0.824/GPU-hr: 6 × $0.824 = $4.94. Export 3GB of results at the $0.0081/GB median egress rate and that's another $0.02, so $4.96 converts the whole corpus to Markdown. For a deepseek-vl2-small bf16 evaluation instead: 32.3GB of weights load in a few minutes on the same card, and a 1,000-image benchmark at 2 hours is 2 × $0.824 = $1.65. If you are just wiring up the pipeline on tiny, RTX 3090 24GB is $0.193/GPU-hr — an afternoon of 4 hours is $0.77. Billing is metered per second and priced per hour, with no minimum, no setup fee and no quota request; compute stops billing the moment the instance stops, and only storage you keep continues at the $0.414/GB-month median until you destroy it.

04 —

FAQ

What is the difference between DeepSeek-VL and DeepSeek-VL2, and which should I use now?

The first DeepSeek-VL shipped as 1.3B and 7B dense models with a SigLIP-L plus SAM-B hybrid encoder on a fixed 1024×1024 input. DeepSeek-VL2 moved to a DeepSeekMoE architecture with MLA attention, swapped the vision side to SigLIP-SO400M-384 with dynamic tiling (23 candidate resolutions in the config, up to 3456×384) and added the <|grounding|> and <|ref|> localisation tokens. Start any new project on VL2; gen one only matters if you're reproducing older paper results. Both have ready-made PyTorch images on NexGPU, and a single RTX 3090 24GB at $0.193/GPU-hr is enough to sit tiny and the 7B side by side.

deepseek-vl2-small is only 32.3GB of weights — why does the repo say 80GB of VRAM?

Because what eats memory is not the weights, it's the one-shot prefill over image tokens. Dynamic tiling cuts a high-resolution image into multiple 384×384 tiles, each becomes its own stretch of the sequence after SigLIP, and all those attention activations are resident simultaneously during prefill — so the peak sits far above the weight footprint. The official README states outright that the sample script needs 80GB for small. The workaround is the repo's incremental_prefilling with chunk_size=512, which fits it into 40GB at the cost of a visibly slower first token. On NexGPU you can simply not have this problem: A100 PCIE 80GB is $0.824/GPU-hr against RTX A6000 48GB at $0.817 — $0.007 buys 32GB more, so just run the stock script on the 80GB card.

Is there a GGUF of DeepSeek-VL2? Can I run it in Ollama or llama.cpp?

No. The llama.cpp support request for deepseek-vl2 sat under a stale label and went nowhere, and there is still not a single GGUF of it on Hugging Face. The quantizations that exist come in two flavours: mlx-community 3/4/6/8-bit, which is Apple Silicon only (4-bit small is 9.3GB), and a scattering of community AWQ and bitsandbytes 4-bit builds. On NVIDIA, your path to less VRAM is load_in_4bit or a community AWQ — not GGUF. Given how thin that ecosystem is, renting a card that is large enough in the first place usually beats the fight: NexGPU runs from RTX 3090 24GB at $0.193 to H200 141GB at $6.660, all metered per second, all stoppable the moment you're done.

DeepSeek-VL2 only has a 4096 context — is that enough for interleaved multi-image chat?

max_position_embeddings is 4096 across all three sizes, which is short by today's standards, and the catch is that image tokens spend from the same 4096. After dynamic tiling one high-resolution image can easily consume a thousand-plus tokens, so the number of interleaved image turns you can actually hold is limited. The practical moves are to lower the tiling tier, summarise older images into text, or feed only two images at a time. If your workload is long documents rather than long conversations, the DeepSeek-OCR branch is the better fit. To find out how many turns your own prompts survive, spin up an RTX 4090 24GB on NexGPU at $0.540/GPU-hr and measure it on tiny.

Can DeepSeek-VL2 be used commercially? How does the licensing work?

The code repository is MIT and the model weights fall under the DeepSeek Model License, which the maintainers state supports commercial use. The thing to note is that the code licence and the weight licence are two separate documents — compliance review has to read both. The OCR branch is actually more permissive: DeepSeek-OCR weights are MIT and DeepSeek-OCR-2 is Apache-2.0, so if your legal team is uneasy about custom licences, that line is close to uncontroversial. Keeping the weights on a machine you rent, with data never leaving the instance, is the most direct way around this class of concern — NexGPU spans 51 countries and regions across 1,175 verified rentable nodes, so you can pick a region that matches your data rules.

If I only need PDF to Markdown, should I pick DeepSeek-VL2 or DeepSeek-OCR?

The OCR branch, without hesitation. DeepSeek-OCR is 3.3B with 6.71GB of bf16 weights, and the paper reports over 200,000 pages a day from a single A100-40G. On OmniDocBench it beats GOT-OCR2.0 — which spends 256 tokens per page — using 100 vision tokens, and beats MinerU2.0's 6,000+ tokens per page using under 800. DeepSeek-OCR-2 shifts the default tier to (0-6)×768×768 plus 1×1024×1024, i.e. (0-6)×144 + 256 vision tokens, scores 76.3% overall on olmOCR-bench and 82% on arXiv math OCR, and ships under Apache-2.0. A 6.7GB model saturates a pipeline on a single RTX 4090 24GB at $0.540/GPU-hr on NexGPU, with bilingual support over Telegram and no ticket queue when something breaks.

More in Multimodal vision models

Every model guide

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.

Sign up freeTalk to sales

Sign up to browse live network pricing. No payment method required.