Text LLM · Small reasoning models

Run Phi-4 on your own card — 14B of model, a 9GB floor

Microsoft's Phi line has been betting on one idea: high-quality synthetic data lets small models punch above their weight. By Phi-4-reasoning-vision-15B, that bet is hard to argue with. MIT-licensed throughout, weights freely downloadable, commercial use unrestricted.

Phi · self-hosted

Phi earns its own self-hosting page because it is one of the very few open model families small enough for consumer hardware yet genuinely production-grade at reasoning. Phi-4 is a 14B dense decoder-only model, weights released 12 December 2024, trained on 9.8T tokens with a heavy emphasis on filtered and synthesised textbook-quality data — a recipe that let it outperform contemporaneous 70B models on maths and code reasoning.

The head of the line today is Phi-4-reasoning-vision-15B, released 4 March 2026: 15B parameters, a mid-fusion architecture pairing the Phi-4-reasoning language backbone with a SigLIP-2 vision encoder, 16K context, up to 3,600 visual tokens at dynamic resolution. Its most interesting property is that it decides for itself whether to think — roughly 80% of training data pushes direct answers for perception work (captioning, OCR, grounding), roughly 20% invokes `<think>` chain-of-thought for maths and science. Microsoft admits the boundary is imprecise, and you can force either mode with explicit tokens. One thing worth stating plainly: there is no Phi-5. The "Phi-5 deployment guides" circulating online are extrapolations from a roadmap.

For self-hosters the real draw is the cost structure. Batch extraction, classification and structured rewriting over private corpora is work Phi-4-mini handles at 3.8B parameters with a 128K context and only 7.6GB of bf16 weights; Phi-4 quantised to Q4_K_M is 9.05GB, comfortable on a single RTX 3090. The blanket MIT licence matters too — weights, fine-tuning code and benchmark logs are all yours, and no volume threshold triggers a sales conversation.

01 —

The Phi family and what each one costs in VRAM

Parameter counts, context lengths, licences and dates below all come from the official Hugging Face model cards.

Version	Parameters	VRAM	Context	Notes
Phi-4-reasoning-vision-15B	15B	bf16 weights ~30GB / plan for 40GB+ in practice	16K	Current flagship. Multimodal reasoning with a SigLIP-2 encoder at dynamic resolution, up to 3,600 visual tokens. Strong at maths/science reasoning and UI understanding (GUI agents, on-screen element grounding). Microsoft validated it on A6000, A100, H100 and B200; they recommend serving bf16 on vLLM ≥ 0.15.2.
Phi-4-reasoning-plus	14B	bf16 weights ~28GB / 48GB+ for 32K reasoning chains	32K	Adds a reinforcement-learning pass on top of Phi-4-reasoning's supervised fine-tuning. More accurate, but outputs run roughly 50% longer with latency to match. Set max_new_tokens up to 32768 for hard problems. The default pick for text-only reasoning.
phi-4	14B	bf16 ~28GB / Q4_K_M 9.05GB / Q5_K_M 10.60GB / Q8_0 15.58GB	16K	The foundation of the line, 9.8T training tokens, MIT. Best-supported GGUF ecosystem of the family: Q4_K_M is the community default, and IQ4_XS squeezes to 7.94GB at acceptable loss. Cheapest option when you need general capability without long reasoning chains.
Phi-4-mini-instruct	3.8B	bf16 ~7.6GB / ~2.5GB at Q4	128K	200K vocabulary, grouped-query attention, native function calling, 22 languages including Chinese — the most multilingual member of the family. The 128K context suits long-document extraction and batch pipelines. Runs on older cards (V100 and up) with eager attention.
Phi-4-mini-flash-reasoning	3.8B	bf16 ~7.6GB	64K	The SambaY decoder-hybrid-decoder architecture from Microsoft and Stanford: Mamba state-space layers plus sliding-window attention plus Gated Memory Units. At 2K prompt and 32K generation it delivers roughly 10× the vLLM decoding throughput of Phi-4-mini-reasoning, with latency growing near-linearly in generated tokens. Requires flash attention.
Phi-4-multimodal-instruct	5.6B	bf16 ~11.2GB	128K	Text, vision and audio in one model via speech-lora and vision-lora adapters; vLLM inference supports LoRA rank up to 320. Audio covers 8 languages including English, Chinese, German, French, Italian, Japanese, Spanish and Portuguese. The one to pick for combined transcription-and-understanding pipelines.

02 —

Pick a card by scenario — real NexGPU rates

VRAM sized as weights plus KV cache plus activations, with no optimistic assumptions.

Phi-4 Q4_K_M on one card for evaluation and prototyping
RTX 3090 24GB$0.193/GPU-hr
Q4_K_M occupies 9.05GB, leaving enough of the 24GB for a full 16K context plus concurrency — and this is the cheapest 24GB card on the platform.
Phi-4-mini or mini-flash-reasoning as a resident service using the full 128K context
A10 24GB$0.414/GPU-hr
3.8B in bf16 is only 7.6GB of weights, so all remaining 16GB goes to the 128K KV cache; a datacentre card holds up better than a consumer one under continuous load.
Phi-4-reasoning-plus at full bf16 precision with 32K reasoning chains
A100 PCIE 80GB$0.824/GPU-hr
28GB of weights plus a 32K-chain KV cache makes 48GB tight, and the A100 80GB costs just $0.007 more than the A6000 48GB for 32GB more memory — there is no reason to choose otherwise.
Phi-4-reasoning-vision-15B serving multimodal traffic with concurrency
RTX A6000 48GB$0.817/GPU-hr
Microsoft validated this exact card; 30GB of weights plus the 3,600-visual-token overhead fits in 48GB on one GPU. Step up to the A100 SXM4 80GB ($1.088/GPU-hr) for heavier concurrency.

03 —

Four steps to a running Phi-4 endpoint

From boot to an OpenAI-compatible API, typically about fifteen minutes.

01
Boot an instance with vLLM
Pick a card from the table above in the NexGPU console and select a prebuilt vLLM or PyTorch image — CUDA and drivers are already in place across 2,000+ images, so there is no environment to assemble. Get in over SSH or the web terminal. Mind the version floor: Phi-4-reasoning-vision-15B needs transformers ≥ 4.57.1 and vLLM ≥ 0.15.2, so upgrade older images first.
```
pip install -U "vllm>=0.15.2" "transformers>=4.57.1"
```
02
Pull weights and start an OpenAI-compatible server
MIT licence — no access request, no form. vLLM fetches from Hugging Face automatically. Serve Phi-4-reasoning-plus for text reasoning, or reasoning-vision-15B for multimodal. Set memory utilisation to 0.90 and leave the rest for CUDA context.
```
vllm serve microsoft/Phi-4-reasoning-plus --max-model-len 32768 --gpu-memory-utilization 0.90 --dtype bfloat16
```
03
Call it with the sampling parameters Microsoft actually recommends
The Phi-4-reasoning models are unusually sensitive to sampling settings and the system prompt — do not leave defaults in place. Microsoft specifies temperature=0.8, top_k=50, top_p=0.95, and a system prompt beginning "You are Phi, a language model trained by Microsoft" that asks for a `<think> {reasoning} </think> {solution}` structure. Skip it and the thinking block may never appear. Raise max_tokens to 32768 for hard problems.
```
curl http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"microsoft/Phi-4-reasoning-plus","messages":[{"role":"system","content":"You are Phi, a language model trained by Microsoft to help users. Format your response as <think> {reasoning} </think> {solution}."},{"role":"user","content":"Prove that the square root of 2 is irrational."}],"temperature":0.8,"top_p":0.95,"max_tokens":32768}'
```
04
Go quantised when you want it cheaper
For evaluation work, or anywhere the last few points of precision do not matter, reach for GGUF: Q4_K_M at 9.05GB is the accepted community default and leaves ample headroom on an RTX 3090. One trap to know about — vLLM's Phi-4 GGUF support has been broken for a long time, with load failures reproduced across v0.7.3 through v0.8.3. Run GGUF through llama.cpp or Ollama instead of forcing it into vLLM.
```
ollama run phi4
```

What this actually costs

Say you are wiring Phi-4-reasoning-plus into an internal knowledge base for reasoning Q&A, evaluating first and shipping second. Phase one: run Q4_K_M on an RTX 3090 24GB to validate the pipeline and tune prompts — $0.193/GPU-hr × 8 hours = $1.544, a full working day for under two dollars. Phase two: move to an A100 PCIE 80GB for a proper bf16 evaluation — $0.824/GPU-hr × 6 hours = $4.944. Both phases together: $6.488. In production, an always-on A100 PCIE 80GB runs 720 hours a month = $593.28; if traffic only exists during business hours, per-second metering means stopping the instance stops the charge — 22 working days × 10 hours × $0.824 = $181.28. No minimum, no setup fee, no quota request. Storage bills separately at a $0.414/GB-month median, so keeping Phi-4's 28GB of bf16 weights parked costs about $11.6 a month — and note that storage keeps billing after compute stops, so destroy volumes you no longer need.

04 —

Frequently asked questions

How much VRAM does Phi-4 actually need? Is a 24GB card enough?

It depends on precision. Q4_K_M is 9.05GB, Q5_K_M is 10.60GB, Q8_0 is 15.58GB — a 24GB card handles all three comfortably. But bf16 weights are 28GB, which will not fit on a single 24GB card no matter what optimisation anyone promises you. For the quantised path, take an RTX 3090 24GB on NexGPU at $0.193/GPU-hr; for full precision go straight to an A100 PCIE 80GB at $0.824/GPU-hr. An hour on either settles which route suits you.

Has Phi-5 shipped? Which version should I be using?

It has not. Microsoft has never released a Phi-5, and the "Phi-5 deployment guides" you will find are speculation from a roadmap — their parameter and VRAM figures are not trustworthy. The newest model in the line is Phi-4-reasoning-vision-15B, released 4 March 2026. Use Phi-4-reasoning-plus for text reasoning, phi-4 for general work, Phi-4-mini for lightweight tasks. To compare them side by side, per-second billing on NexGPU makes running several instance types in parallel genuinely affordable.

I get a rope_scaling short_factor length error on startup. How do I fix it?

This is the classic Phi-4 trap. The validation requires the length of rope_scaling["short_factor"] to equal exactly rotary_ndims // 2, where rotary_ndims depends on partial_rotary_factor (0.75 for Phi-4); older framework versions compute it wrong and raise a ValueError. Upgrading to transformers ≥ 4.49.0 and vLLM ≥ 0.7.3 resolves it. A separate issue: vLLM 0.11.1's CLI parser does not recognise --rope-scaling, and the only workaround today is downgrading to 0.11.0. On NexGPU you can just relaunch from a prebuilt image pinned to a known-good version rather than fighting a broken environment.

How good is Phi-4 at non-English work?

It varies by model. Phi-4-mini-instruct covers 22 languages through its 200K vocabulary and is genuinely usable beyond English; Phi-4-multimodal-instruct's audio side handles 8 languages. But Microsoft states plainly that Phi-4-reasoning and reasoning-vision were trained primarily on English and degrade noticeably elsewhere — test before you commit to them for non-English reasoning. Half an hour on a NexGPU RTX 3090 running the quantised build costs just over $0.10 and gives you a real answer.

Why is Phi-4-reasoning-plus so much slower than Phi-4-reasoning?

Because plus adds a reinforcement-learning stage on top of supervised fine-tuning, and the accuracy gain comes with roughly 50% longer outputs and proportionally higher latency. That is model behaviour, not a deployment misconfiguration. If your workload is latency-sensitive and indifferent to the last few accuracy points, plain Phi-4-reasoning is the better trade. For maximum throughput look at Phi-4-mini-flash-reasoning, whose SambaY architecture decodes roughly 10× faster than Phi-4-mini-reasoning at 32K generation lengths. All of these fit side by side on a NexGPU A10 24GB at $0.414/GPU-hr.

What should I run Phi-4-reasoning-vision-15B on, and where can I rent it?

Microsoft validated it on A6000, A100, H100 and B200, and recommends bf16 on vLLM. The 15B weights are about 30GB, and with up to 3,600 visual tokens of overhead a single RTX A6000 48GB is sufficient — $0.817/GPU-hr on NexGPU. For high-concurrency serving, move to an A100 SXM4 80GB at $1.088/GPU-hr. We operate 1,175 verified rentable nodes and 2,498 GPUs across 51 countries and regions, up to 14 GPUs per node and 2,152GB max node VRAM, with SSH, Jupyter, web terminal, REST API and CLI all open, plus bilingual support on Telegram and no ticket queue.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.