Open-weight text LLM

Self-hosting Mistral: do the VRAM math before you pick a card

Ministral 3 8B at Q4_K_M is 5.2GB. Mistral Small 4's FP8 weights are 119GB. Mistral Large 3 wants a full eight-GPU node. Same brand, two orders of magnitude apart — here is every tier, with the machine that matches it.

Mistral · self-hosted

Mistral's open-weight lineup is far bigger than the single 7B that made its name in 2023, and almost all of it is Apache 2.0. The Mistral 3 launch on 2 December 2025 dropped Mistral Large 3 (675B total / 41B active sparse MoE plus a 2.5B vision encoder) alongside Ministral 3 in 3B, 8B and 14B — each size shipping Base, Instruct and Reasoning weights. A week later Devstral 2 (123B) and Devstral Small 2 (24B) filled in agentic coding. Mistral Small 4, released 16 March 2026, is the current general-purpose workhorse. Everything in the current line runs a 256K context and takes images natively.

The trap when you self-host is the naming. Mistral Small is no longer that 24B single-card model. Mistral Small 4 is a 119B-total MoE with 6.5B active (128 experts, 4 routed per token); the official floor is two H200-class GPUs, and 24GB of VRAM will not hold even its Q2 quant. The old Mistral Small 3.2 24B-2506 and Magistral Small 2509 were both deprecated on 30 April 2026, and magistral-small-latest now resolves to Mistral Small 4. Pixtral 12B is retired, its vision capability folded into Small 4 and Medium 3.5. If you want single-card, the models to look at now are Ministral 3 14B and Devstral Small 2 24B. The second trap is numeric format: Mistral ships a lot of weights as FP8 E4M3, and Ampere-and-older cards — A100, V100, RTX 3090 — have no native FP8, so you fall back to bf16 and double the footprint.

NexGPU covers both ends. The cheapest RTX 3090 24GB is $0.193/GPU-hour, plenty for Ministral 3 14B at Q4_K_M with a long-context KV cache on top. At the heavy end a single node goes up to 14 GPUs and 2,152GB of VRAM, so Mistral Large 3's full FP8 weights land in one TP=8 launch with no cross-node NCCL. The network spans 51 countries and regions, 1,175 verified rentable nodes, 2,498 GPUs and 75 GPU models, with 2,000+ prebuilt images including vLLM and PyTorch, reachable over SSH, Jupyter, web terminal, REST API or CLI. Metered per second, priced per hour, no minimum, no setup fee, no quota request.

01 —

Every Mistral open weight worth deploying, and what it costs in VRAM

Current releases plus the ones that are deprecated, so you do not build on a dead endpoint.

Version	Parameters	VRAM	Context	Notes
Mistral-Small-4-119B-2603	119B total / 6.5B active (128 experts, 4 routed)	bf16 ~238GB / FP8 ~119GB / NVFP4 ~72GB / GGUF UD-Q4_K_M 73.8GB / UD-Q3_K_M 54.4GB / UD-Q2_K_XL 40.2GB	256K	Released 16 March 2026 under Apache 2.0. Collapses what used to be three separate lines — Magistral reasoning, Pixtral vision, Devstral coding — into one checkpoint, with a per-request reasoning_effort that accepts only none or high. Official floor is 2xH200 / 2xB200 / 2xMI300X.
Mistral-Large-3-675B-Instruct-2512	675B total / 41B active + 2.5B vision encoder	FP8 weights ~675GB (official recommendation: one 8xH200 node); NVFP4 brings it down to a single 8xH100 or 8xA100 node; a BF16 release also exists	256K	The largest open-weight MoE from a major lab, Apache 2.0, vLLM with --tensor-parallel-size 8. Mistral is candid that it is not tuned for pure reasoning and loses to vision-first models on vision-only work.
Ministral 3 (3B / 8B / 14B, all -2512)	The 8B is an 8.4B language model plus a 0.4B vision encoder; every size ships Base, Instruct and Reasoning weights	8B: Q4_K_M 5.2GB / Q5_K_M 6.06GB / Q8_0 9.03GB / BF16 17GB. 14B: Q4_K_M 8.24GB / Q8_0 14.4GB / BF16 27GB. 8B in FP8 is ~12GB; 14B in FP8 fits a 24GB card	256K	Apache 2.0, 40+ languages, native image input. The 14B Reasoning variant hits 85% on AIME '25. This is the sweet spot for single-card self-hosting; Mistral recommends dropping temperature to 0.15 or even 0.1 for the Instruct weights.
Devstral-Small-2-24B-Instruct-2512	24B dense	FP8 ~25GB (the primary distributed format) / bf16 ~48GB	256K	Apache 2.0 coding agent: 68.0% on SWE-bench Verified, 55.7% on SWE-bench Multilingual, 22.5% on Terminal-Bench 2. Shares the Ministral 3 architecture, has vision, and pairs with Mistral Vibe CLI or OpenHands.
Devstral-2 (123B) and Mistral-Medium-3.5-128B	Devstral 2 is 123B dense; Medium 3.5 is 128B dense	Devstral 2 fits in about 128GB; Medium 3.5's reference launch uses --tensor-parallel-size 8	256K	These two are Modified MIT, not Apache 2.0 — free commercial use under $20M monthly revenue, negotiated above it. Devstral 2 scores 72.2% on SWE-bench Verified, and Medium 3.5 has since absorbed Devstral 2's coding role.
Deprecated / retired: Mistral 7B v0.3, Mixtral, Mistral Small 3.2 24B-2506, Magistral Small 2509, Pixtral 12B	7B / 8x7B / 24B / 12B	Mistral 7B at Q4 fits a 6GB card; Mistral Small 3.2 24B needs ~55GB in bf16/fp16, ~18.9GB at Q4_K_M	32K for Mistral 7B, 128K for Mistral Small 3.2	Mistral Small 3.2 and Magistral Small 2509 were both deprecated on 2026-04-30, with the docs pointing new integrations at Mistral Small 4; Magistral Small 2509 retired on 2026-07-31. The weights are still on Hugging Face and nothing stops you running them, but do not baseline a new project on them. Mistral 7B only still earns its place on 6-8GB legacy cards.

02 —

Pick the card that actually fits: live NexGPU inventory

Matched on real VRAM — we will not send a 675B job to a 24GB card.

Ministral 3 8B / 14B at Q4_K_M for local evals, RAG and batch extraction
RTX 3090 24GB$0.193/GPU-hour
The 8B Q4_K_M is 5.2GB and the 14B is 8.24GB, leaving most of the 24GB for a long-context KV cache — the cheapest card on the network that does real work.
Devstral Small 2 24B on the official FP8 weights as a coding agent, or Ministral 3 14B in bf16
RTX 5090 32GB$0.723/GPU-hour
FP8 weights land around 25GB — just past a 24GB card, wasteful on a 48GB one — and the 5090 is Blackwell, so FP8 and even NVFP4 run natively instead of falling back to bf16 and doubling the footprint.
Mistral Small 4 119B on the official FP8 checkpoint, served with vLLM in production
H100 SXM 80GB x2$3.582/GPU-hour (two cards: $7.164/hour)
FP8 weights are ~119GB and TP=2 gives you 160GB, with Hopper eating FP8 natively; Mistral warns that a full 256K context OOMs, so drop --max-model-len to 131072 — or move to 2xH200 141GB ($6.660/GPU-hour) if you need the whole window.
Mistral Large 3 675B, full FP8 weights on one node
H200 141GB x8$6.660/GPU-hour (eight cards: $53.28/hour)
The official recommendation is exactly one 8xH200 node for the FP8 weights, and a NexGPU node goes to 14 GPUs and 2,152GB, so TP=8 comes up in a single launch with no cross-node fabric to debug.

03 —

Four steps to a running Mistral

vLLM shown here; llama.cpp, Ollama and LM Studio are the easier path if you are on GGUF.

01
Boot an instance with vLLM
Pick a card from the table above at console.nexgpu.net and choose a prebuilt vLLM or PyTorch image — both are in the 2,000+ image library. Reach it over SSH, Jupyter or the web terminal. Use vLLM 0.20.0 or newer: anything older lacks --reasoning-parser mistral and --tool-call-parser mistral, and Mistral's own fallback there is the mistralllm/vllm-ms4 image.
```
pip install -U "vllm>=0.20.0" mistral_common
```
02
Pull the weights
Everything lives under the mistralai org on Hugging Face. The Apache 2.0 checkpoints — Small 4, Large 3, Ministral 3, Devstral Small 2, Leanstral 1.5, Shieldstral 1.0 — download with no gate to accept. If VRAM is tight, pull the NVFP4 or GGUF repo rather than defaulting to bf16.
```
hf download mistralai/Devstral-Small-2-24B-Instruct-2512 --local-dir ./devstral-small-2
```
03
Serve it with vLLM
Mistral-format checkpoints need the tokenizer and config format declared explicitly, otherwise vLLM falls back to the HF chat template and tool calls stop parsing. Anything agentic needs --enable-auto-tool-choice, and reasoning-capable models like Small 4 need --reasoning-parser mistral, with requests carrying reasoning_effort set to none or high — no other value is accepted.
```
vllm serve mistralai/Mistral-Small-4-119B-2603 --max-model-len 131072 --tensor-parallel-size 2 --tool-call-parser mistral --enable-auto-tool-choice --reasoning-parser mistral --max-num-batched-tokens 16384 --max-num-seqs 128 --gpu-memory-utilization 0.8
```
04
Verify and wire it in
vLLM exposes an OpenAI-compatible endpoint, so pointing base_url at it swaps out your existing API calls. On the GGUF path a single llama.cpp line brings up a local server with a web UI — a cheap way to sanity-check quality on an inexpensive card before you commit to the big machine.
```
llama serve -hf unsloth/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_XL --jinja -ngl 99 --ctx-size 32768 --temp 0.15
```

What this actually costs

Take a real case: a five-person backend team runs Devstral Small 2 24B in FP8 for code review and patch generation on an RTX 5090 32GB. Six hours a day, twenty working days, is 120 hours: 120 x $0.723 = $86.76. Keep 60GB of weights and image cache resident and that is 60 x $0.414 = $24.84/month. Roughly $111.60/month for one box serving the whole team. Now the heavy end: Mistral Small 4 119B on two H100 SXM 80GB is 2 x $3.582 = $7.164/hour, so a 40-minute batch eval costs 0.667 x $7.164 = about $4.78. Billing is metered per second and priced per hour — compute stops the second the instance stops, and only storage keeps accruing until you destroy it. Against Mistral's own API pricing for Small 4 at $0.15 per million input tokens and $0.60 per million output, you can compute your own crossover from your real throughput; for data that cannot leave your jurisdiction, sustained high concurrency, or anything that fine-tunes the weights, self-hosting wins nearly every time. No minimum, no setup fee, no quota request.

04 —

FAQ

How much VRAM do I actually need to run Mistral locally?

Depends which Mistral. Ministral 3 8B is 5.2GB at Q4_K_M and 17GB in BF16; the 14B is 8.24GB at Q4_K_M and 27GB in BF16. Devstral Small 2 24B is ~25GB in FP8 and ~48GB in bf16. Mistral Small 4 119B is ~72GB in NVFP4, ~119GB in FP8, ~238GB in bf16. Mistral Large 3 675B in FP8 approaches 675GB and needs a full eight-GPU node. NexGPU spans that whole range, from RTX 3090 24GB at $0.193/GPU-hour to 8xH200 141GB, billed by the second — prove out a quant on a cheap card first, then move up.

Can I run Mistral Small 4 on a single 24GB RTX 4090?

No, and not close. Small 4 is a 119B-total MoE: only 6.5B activate per token, but every expert weight has to sit in VRAM. Even the most aggressive GGUF, UD-Q2_K_XL, is 40.2GB, and UD-IQ1_M is still 32.3GB at quality you would not ship. If you need single-card, change models — Ministral 3 14B and Devstral Small 2 24B were designed for it. For Small 4 proper, 2x H100 SXM 80GB ($3.582/GPU-hour) or 2x H200 141GB ($6.660/GPU-hour) on NexGPU is the honest configuration.

Are Mistral Small 3.2 and Magistral still usable?

Both Mistral Small 3.2 24B-2506 and Magistral Small 2509 were marked deprecated on 30 April 2026, with the docs telling new integrations to use Mistral Small 4; Magistral Small 2509 retired on 31 July 2026 and magistral-small-latest now resolves to Mistral Small 4. Pixtral 12B is retired too, its vision folded into Small 4 and Medium 3.5. The open weights are still on Hugging Face, so self-hosting is unaffected — one A100 PCIE 80GB on NexGPU at $0.824/GPU-hour runs Small 3.2 in bf16 comfortably — but do not baseline anything new on them.

Will an A100 or V100 run Mistral's FP8 weights?

Not natively. The A100 is Ampere (sm80) and the Tesla V100 is older still; neither has FP8 tensor cores. Mistral has treated FP8 E4M3 as a primary distribution format since Ministral 3, so on those cards you dequantize back to bf16, double the footprint and lose the speedup. For real FP8 you want Ada or newer: RTX 4090 24GB ($0.540/GPU-hour), RTX 5090 32GB ($0.723, Blackwell, so NVFP4 works too), H100 SXM 80GB ($3.582), H200 141GB ($6.660). The A100 80GB (PCIE $0.824 / SXM4 $1.088) is the better fit for bf16 or GPTQ/AWQ INT4 weights. All of these sit in the same NexGPU console — switching is a new instance, not a new vendor.

vLLM keeps erroring on Mistral, or tool calls never parse. What is wrong?

Four usual suspects. Tokenizer: Mistral-format checkpoints need --tokenizer_mode mistral --config_format mistral --load_format mistral, otherwise vLLM reads the HF chat template, and Mistral's template lives in chat_template.json rather than tokenizer_config.json, which breaks tool calling. Version: --reasoning-parser mistral and --tool-call-parser mistral require vLLM 0.20.0 or newer. Context: Small 4 can OOM at the full 256K even on 2xH200, so drop --max-model-len to 131072 or 65536. Parameters: reasoning_effort accepts only none or high and rejects anything else. NexGPU's prebuilt vLLM images already ship a current version so you can serve straight from boot, and if you get stuck, support is bilingual over Telegram with no ticket queue.

What GPU should I rent to fine-tune Ministral 3 or Devstral Small 2?

LoRA or QLoRA on Ministral 3 8B fits a single RTX 4090 24GB ($0.540/GPU-hour), and Unsloth publishes ready-made bnb-4bit and FP8 weights for it. For 14B LoRA, move to RTX A6000 48GB ($0.817/GPU-hour); for full fine-tunes or bf16 training on Devstral Small 2 24B, use A100 SXM4 80GB ($1.088/GPU-hour). Keep datasets and checkpoints on persistent storage at a $0.414/GB-month median with $0.0081/GB median egress; when the run ends and you stop the instance, compute billing stops immediately and only storage continues until you destroy it. Need multi-GPU FSDP? A NexGPU node goes up to 14 GPUs and 2,152GB of VRAM, so it stays inside one node.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.