Text LLM

Self-hosting DeepSeek: 284B to 1.57T, and what it really costs in VRAM

From the 3B OCR-2 to the 1.57T V4-Pro, DeepSeek is now a whole family. This page lays out the real weight sizes, the cards that can actually hold them, and the traps you hit on the way.

DeepSeek · self-hosted

First, correct an outdated premise: DeepSeek's mainline is no longer the R1 and V3 that most tutorials still describe — it is V4. V4 launched in April 2026, DeepSeek-V4-Flash-0731 landed on July 31, and DeepSeek-V4-Pro-0813 replaced the April preview as the official flagship on August 13. Both sizes are MoE: Flash is 284B total with 13B active per token; Pro is 1.57T total with 48B active. All of it is MIT licensed, with weights downloadable directly from the deepseek-ai org on Hugging Face. And to answer a very common search — R2 has never shipped, and DeepSeek has never published a date for it. The reasoning capability was folded into the mainline instead: V4 carries low / high / max reasoning-effort levels, so there is no separate reasoning model to wait for.

Architecturally V4 is a different animal from V3.2, and that difference drives your VRAM math. V4 uses hybrid sparse attention: every layer runs a sliding window over the last 128 raw tokens, then stacks one compression path on top — either 4:1 top-k sparse (C4) or 128:1 dense compression (C128). That is what pushes context to 1M with up to 384K output tokens. It also replaces the residual stream with mHC (manifold-constrained hyper-connections), a per-token mixture over hc_mult parallel branches. The efficiency gain is concrete: at 1M context, V4-Pro needs only 27% of the per-token inference FLOPs and 10% of the KV cache of V3.2. Pro specifically runs 61 layers, hidden size 7168, 384 routed experts plus 1 shared expert per MoE layer with 6 active per token, and ships native FP4 expert weights.

Now the question you actually came with: how big a card do you need to rent. Flash's native FP8 weights are about 148.66 GiB, the Q4 GGUF is 155GB, and even the most aggressive UD-IQ1_S is 82.5GB. Pro's Q4 is 850GB. So for "can a 24GB GPU run DeepSeek", the answer is a firm no for V4 and a yes for DeepSeek-OCR-2 (3B) and DeepSeek-R1-Distill-Qwen-32B (~18-20GB at Q4_K_M). Below, each realistic path gets its own configuration — no optimistic rounding.

01 —

DeepSeek variants: parameters and real weight sizes

Sizes taken from the official weight files and Unsloth's measured quants, not estimated

Version	Parameters	VRAM	Context	Notes
DeepSeek-V4-Pro-0813	1.57T total / 48B active	GGUF UD-Q4_K_XL 850GB, UD-Q8_K_XL 873GB	1M (384K max output)	The current flagship; the August 13 release supersedes the April preview, with agentic capability as the headline upgrade. Self-hosting starts at terabyte-scale VRAM — this is a rack-level job.
DeepSeek-V4-Flash-0731	284B total / 13B active	Native FP8 weights ~148.66 GiB; GGUF UD-Q4_K_XL 155GB, UD-IQ3_XXS 104GB, UD-IQ1_S 82.5GB	1M (384K max output)	The realistic tier for self-hosting. 13B active means decode compute is far lighter than the 284B headline suggests — your real constraint is simply fitting it.
DeepSeek-V4-Flash-DSpark	284B / 13B active + speculative decoding head	Budget ~10GB on top of your chosen quant; the drafter itself is 10.9GB at Q8_0, 11.3GB at BF16	1M	The same checkpoint with an MTP speculative-decoding module bolted on, worth up to roughly 2x faster decode at the cost of extra VRAM. Turn it on for production; skip it while you are still evaluating.
DeepSeek-V4-Flash-Vision-Exp	Multimodal experimental build on top of Flash	API-only today; no matching open weights published	1M	If you need vision input and hard privacy at the same time, this tier is currently a dead end — do not plan around it until weights are released.
DeepSeek-OCR-2	3B	~7GB in BF16, comfortable on any single card	Dynamic resolution (0-6)x768x768 + 1x1024x1024	Released January 27, using DeepEncoder V2 and Visual Causal Flow, scoring 76.3 overall on olmOCR-bench under Apache 2.0. Notably better on Chinese layouts than the previous generation. V4 is a general LLM and does not replace it.
DeepSeek-R1-Distill-Qwen-32B	32B dense	~18-20GB at Q4_K_M	128K (per config)	An older model, but still the only DeepSeek-family reasoning model that fits a single 24GB card. Leaving ~4GB for KV cache fills a 24GB card exactly — fine for local evaluation and offline batch work.

02 —

Pick the card by what you are actually running

If the VRAM does not add up it simply will not start — no "just barely fits" advice here

Run DeepSeek-OCR-2, or R1-Distill-Qwen-32B at Q4, for single-GPU local evaluation
RTX 4090 24GB$0.540/GPU-hr
OCR-2's 3B in BF16 takes only ~7GB, and the 32B distill at Q4_K_M lands at 18-20GB with room left for KV cache; on a tighter budget the RTX 3090 24GB does the same job at $0.193/GPU-hr.
Run V4-Flash's UD-Q4_K_XL (155GB) under llama.cpp for full-model evaluation
A100 PCIE 80GB x4$0.824/GPU-hr
Four cards give 320GB, comfortably holding 155GB of weights plus context, at $3.296/hr total — the cheapest way to get the complete DeepSeek-V4-Flash running.
Serve V4-Flash's native FP8 weights on vLLM or SGLang for throughput
H200 141GB x2$6.660/GPU-hr
FP8 tensor cores only exist from Hopper onward, and two H200s give 282GB for the 148.66 GiB checkpoint; paired with DSpark speculative decoding, throughput can match the hosted API.
Fully self-host V4-Pro at Q4 quantization, 850GB of weights
H200 141GB x8$6.660/GPU-hr
Eight cards at 1,128GB is what it takes to hold 850GB of weights plus KV cache for long context. Our nodes go up to 14 GPUs and 2,152GB of VRAM, so this tier is available.

03 —

Getting DeepSeek running on NexGPU

Using V4-Flash as the example, on two paths: GGUF to evaluate fast, FP8 to go to production

01
Start an instance on a prebuilt vLLM image
At console.nexgpu.net, pick a node with 4x A100 PCIE 80GB or 2x H200 and select vLLM or PyTorch from the 2,000+ prebuilt images, which saves you the CUDA, driver, and FlashInfer version alignment. SSH, Jupyter, web terminal, REST API, and CLI are all available. Billing starts at boot and is metered per second.
```
ssh root@<your-node> -p <port>
```
02
Pull the weights
For the GGUF path, grab Unsloth's dynamic quants; for FP8, pull the official repo directly. Do not put 155GB on the system disk — mount a data volume for it. Compute billing stops when the instance stops while the weights stay put, so you do not re-download next session.
```
hf download unsloth/DeepSeek-V4-Flash-0731-GGUF --include "*UD-Q4_K_XL*" --local-dir /workspace/v4-flash
```
03
Launch the server
To evaluate, llama.cpp is a one-liner: llama serve -hf unsloth/DeepSeek-V4-Flash-0731-GGUF:UD-Q4_K_XL. For production take the vLLM FP8 path, minding three things: V4 has its own tokenizer and tool-call parser that must be named explicitly, block-size should be 256, and KV cache should be fp8. If you want Think Max, max-model-len must be at least 393216 or output gets truncated.
```
vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 --tensor-parallel-size 2 --kv-cache-dtype fp8 --block-size 256 --trust-remote-code --tokenizer-mode deepseek_v4 --tool-call-parser deepseek_v4 --gpu-memory-utilization 0.92 --max-model-len 393216
```
04
Verify and wire it in
The server exposes an OpenAI-compatible API, so code that already calls the hosted endpoint switches over with a single base_url change. Once you have confirmed all three reasoning-effort levels behave as expected, decide whether to attach the DSpark drafter and trade ~10GB of VRAM for decode speed.
```
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"deepseek-ai/DeepSeek-V4-Flash-0731","messages":[{"role":"user","content":"hello"}]}'
```

What a full V4-Flash evaluation actually costs

Take 4x A100 PCIE 80GB running UD-Q4_K_XL: $0.824 x 4 = $3.296/hr, so eight hours of sustained load is $26.37. Keeping the 155GB of weights for those eight hours works out to 155 x 0.414 / 730 x 8 = about $0.70 at the $0.414/GB-month storage rate. Exporting 20GB of logs and eval results afterwards costs 20 x $0.0081 = $0.16 in egress. That is roughly $27.23 to fully characterize DeepSeek-V4-Flash. The same eight hours on the 2x H200 FP8 production configuration would be $6.660 x 2 x 8 = $106.56 — the premium buys throughput and a production-ready path, and you can judge whether that trade is worth it. Billing is metered per second and priced per hour, with no minimum, no setup fee, and no quota request. Compute billing stops when the instance stops; storage keeps accruing until you destroy it.

04 —

FAQ

How much VRAM does DeepSeek actually need to run locally?

It depends entirely on which tier. DeepSeek-V4-Flash is ~148.66 GiB in native FP8, 155GB as a Q4 GGUF, and 82.5GB even at the most aggressive UD-IQ1_S; V4-Pro at Q4 is 850GB. If you want single-GPU, you drop back to DeepSeek-OCR-2 (3B, ~7GB in BF16) or R1-Distill-Qwen-32B (18-20GB at Q4_K_M). NexGPU spans the RTX 3090 24GB at $0.193/GPU-hr up to the H200 141GB, with up to 14 GPUs and 2,152GB of VRAM per node, so all four tiers can be provisioned in one place.

Can a 24GB RTX 4090 run DeepSeek-V4?

No. The smallest V4-Flash quant is still 82.5GB, an order of magnitude past 24GB, and offloading to system RAM makes it too slow to be useful. The realistic DeepSeek options on a 24GB card are R1-Distill-Qwen-32B at Q4_K_M (18-20GB, leaving ~4GB for KV cache) and DeepSeek-OCR-2. Both run out of the box on NexGPU's RTX 4090 24GB at $0.540/GPU-hr, or the RTX 3090 24GB at $0.193/GPU-hr if you want it cheaper.

Can an A100 consume DeepSeek-V4's FP8 weights directly?

No. The A100 is Ampere and has no FP8 tensor cores — hardware FP8 support starts with Hopper (H100/H200). On A100 you either dequantize back to bf16, which balloons the 284B Flash past 500GB, or you take the llama.cpp GGUF route, which is the correct way to run V4-Flash on A100: four A100 PCIE 80GB give 320GB at $0.824/GPU-hr. If you genuinely need FP8 in production, NexGPU's H100 SXM 80GB at $3.582/GPU-hr and H200 141GB at $6.660/GPU-hr are available on demand.

When is DeepSeek R2 coming out, and should I wait for it?

Do not wait. R2 has never been officially released and no timeline has ever been given, so the R2 specs circulating online are essentially speculation. DeepSeek folded reasoning into the mainline instead: V4 ships low / high / max reasoning-effort levels, and you dial it to max for deep reasoning (remembering to set max-model-len to 393216, or output gets truncated). To try it today, spin up a prebuilt vLLM image on NexGPU, pay per second, and stop it when you are done.

Are DeepSeek V3.2 and R1 still worth deploying?

It depends on the job. V3.2 is 671B total / 37B active with DSA sparse attention, and V4 has clearly moved past it on efficiency — at 1M context, V4-Pro needs 27% of V3.2's per-token FLOPs and 10% of its KV cache, so a new project has no reason to start on V3.2. The same goes for R1 proper, though its 32B distill remains the only DeepSeek-family reasoning model that runs on a single 24GB card and still earns its place in offline batch work. Both paths are available on NexGPU; the only difference is whether you pick an RTX 4090 or an H200.

Is DeepSeek's licence usable commercially, and what should I watch out for when self-hosting?

Licensing is clean: V4-Pro and V4-Flash are both MIT and DeepSeek-OCR-2 is Apache 2.0, so there is no extra commercial licensing cost. The traps are on the engineering side — V4 uses its own encoding scripts rather than a Jinja chat template, so vLLM needs tokenizer-mode and tool-call-parser named explicitly; NVFP4 expert weights do not run on deep_gemm_mega_moe (that kernel is FP8-only) and fall back to the default MoE backend with a performance cost; and tensor parallelism lays out best on power-of-two GPU counts. Failing fast is cheap on NexGPU: per-second billing, no minimum, no setup fee, and bilingual engineering support over Telegram with no ticket queue.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.