Text and multimodal LLM

Self-hosting InternLM, from 5GB to eight A100s

The Shusheng line has moved from text-only InternLM3 to the multimodal scientific Intern-S2 family. Here is what each version actually costs in VRAM, which card runs it, and the traps you will hit self-hosting.

InternLM · self-hosted

Most people searching for how to run InternLM locally are still anchored on InternLM2.5 or InternLM3. The line did not stop — it stopped being called InternLM4. Shanghai AI Laboratory moved the flagship track under the Intern-S name: Intern-S1 (241B, a 235B Qwen3 MoE language tower plus a 6B InternViT vision tower), Intern-S1-Pro (1T total parameters, 512 experts, 8 activated per token for 22B active), and through 2026 the Intern-S2-Preview series and Intern-S2-Mobius. Code and weights across the whole family are Apache-2.0.

VRAM here spans three orders of magnitude, and picking the wrong card is pure waste. At the light end, the official GGUF build of InternLM3-8B-Instruct is roughly 5GB at q4_k_m and about 8GB of VRAM loaded in 4-bit — a 16GB T4 has room to spare. Above that, Intern-S1-mini is 9B (a dense Qwen3-8B plus 0.3B InternViT) at roughly 18GB of bf16 weights. Above that, Intern-S2-Preview-35B is a 36B-total / 3B-active MoE across 256 experts: bf16 wants one H200 141GB or two H100 80GB, while FP8 fits a single H100. Intern-S1 at 241B in bf16 is a hard 8x80GB requirement, and Intern-S1-Pro starts at two 8-GPU H200 nodes.

That spread is exactly why renting wins: one model family, from a single 3090 to a full rack of A100s, and you never buy hardware sized for the largest variant. NexGPU runs 1,175 verified rentable nodes across 51 countries and regions — 2,498 GPUs, 75 GPU models, up to 14 GPUs per node and 2,152GB max node VRAM. PyTorch and vLLM are ready to go among 2,000+ prebuilt images, metered per second and priced per hour, with no minimum and no quota request.

01 —

Current InternLM / Intern-S versions and what they cost in VRAM

From an 8B single card to a 1T two-node job — pick the tier you actually need

Version	Parameters	VRAM	Context	Notes
Intern-S2-Preview-397B	397B	8x H100 80GB or 8x H200, TP8	256K text / 64K multimodal	The 2026 flagship for scientific and long-horizon agent work. The official command needs --mm-encoder-tp-mode data; long-context serving reaches 1,010K.
Intern-S2-Preview-35B	36B total / 3B active, 256 experts	bf16 on one H200 141GB or 2x H100 80GB; FP8 on a single H100	262K native, up to 512K with YaRN	Continued pretraining from Qwen3.5, covering text, vision and time series. The best value tier, and MTP speculative decoding gives it a real speedup.
Intern-S2-Mobius	35B (FP8 variant also published)	~70GB of bf16 weights; LMDeploy --tp 1 needs an 80GB-class single card	128K	The Mobius-v0 architecture pulls knowledge into a globally shared Memory and lets multiple Reasoners iteratively query and refine it. The technical report claims close to 4x end-to-end inference speedup with markedly shorter reasoning traces.
Intern-S1 / Intern-S1-FP8	241B (235B MoE language tower + 6B InternViT)	bf16 on 8x A100 80GB, 8x H100/H800 or 4x H200; FP8 on 4x H100/H800 or 2x H200	Not separately published — set it with --max-model-len	Pretrained on 5T multimodal tokens, of which more than 2.5T are scientific-domain. This is the tier for the full scientific multimodal capability.
Intern-S1-mini / Intern-S1-mini-FP8	9B (Qwen3-8B + 0.3B InternViT)	~18GB of bf16 weights; official table says one 80GB-class GPU. The FP8 build is H800/H100/H200 only	Not separately published — set it with --max-model-len	The only genuinely single-GPU multimodal member of the family. Requires transformers>=4.55.2, vLLM>=0.10.1, LMDeploy>=0.9.2.post1.
InternLM3-8B-Instruct	8B dense	~16GB bf16; official GGUF q4_k_m ~5GB, roughly 8GB VRAM loaded in 4-bit	128K (measured on RULER across 4-128K)	The end of the text-only track and the cheapest way into the family. Trained on just 4T tokens, with a deep thinking mode that runs chains of thought up to 8,192 tokens.

02 —

Which card to rent

Matched to the version you are actually running; prices are NexGPU list rates per GPU-hour

InternLM3-8B quantised inference and prompt iteration
RTX 3090 24GB$0.193/GPU-hr
Both the ~5GB q4_k_m build and the ~16GB bf16 weights fit inside 24GB, and this is the cheapest 24GB tier we rent — a full day of prompt work costs under two dollars.
Intern-S1-mini 9B multimodal bf16 on one card
A100 PCIE 80GB$0.824/GPU-hr
Leaves plenty of headroom past the 18GB of weights for InternViT's high-resolution tiling and the KV cache, matching the single 80GB-class GPU in the official hardware table.
Intern-S2-Preview-35B in FP8 with long context
H100 SXM 80GB$3.582/GPU-hr
FP8 weights need Hopper — A100 is explicitly excluded — and a single H100 carries the 36B/3B-active MoE together with its 262K context.
Intern-S1 241B bf16 full-precision serving, TP8
A100 SXM4 80GB x8$1.088/GPU-hr ($8.704/hr for eight)
8x80GB is the official bf16 floor, and SXM4's NVLink is what makes eight-way tensor parallelism work; up to 14 GPUs per node means 640GB sits inside a single node.

03 —

Four steps to a running Shusheng endpoint

Shown with Intern-S1-mini; for Intern-S1 the only change is adding --tensor-parallel-size 8

01
Boot the instance and pin your versions
Start from the vLLM or PyTorch prebuilt image. The Intern-S line is fussy about versions: anything below transformers 4.55.2 fails to load outright, and Intern-S2-Preview needs a vLLM build that knows InternS2PreviewForConditionalGeneration — the stable wheel does not recognise that architecture.
```
pip install -U "vllm>=0.10.1" "transformers>=4.55.2" "lmdeploy>=0.9.2.post1"
```
02
Pull the weights
Size the volume first: 9B in bf16 is about 18GB, 241B in bf16 is about 482GB. Storage runs at a $0.414/GB-month median, so destroy the volume when a large run finishes — compute billing stops the second the instance stops, storage does not.
```
hf download internlm/Intern-S1-mini --local-dir /workspace/Intern-S1-mini
```
03
Serve it, and get the parsers right
This is where most deployments break. The Intern-S1 family uses --reasoning-parser intern-s1 and --tool-call-parser intern-s1, while Intern-S2-Preview sits on a Qwen3.5 base and needs --reasoning-parser qwen3 with --tool-call-parser qwen3_coder. Get it wrong and tool calls come back silently as plain text. Serving Intern-S1 through SGLang also needs --grammar-backend none.
```
lmdeploy serve api_server internlm/Intern-S1-mini --reasoning-parser intern-s1 --tool-call-parser intern-s1
```
04
Turn thinking off and cap the context
Thinking mode is on by default across the Intern-S line, so every reply opens with a long chain of thought and doubles both latency and token spend. Pass enable_thinking=False through the chat template or the request. Native context at full length will also OOM: the official Intern-S1-Pro guide pins --max-model-len 65536, and Intern-S2-Preview is best started at 65K-131K before climbing.
```
curl http://localhost:23333/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"internlm/Intern-S1-mini","messages":[{"role":"user","content":"Introduce yourself"}],"chat_template_kwargs":{"enable_thinking":false}}'
```

The arithmetic, shown

Two real bills. First: iterating prompts against InternLM3-8B at q4_k_m on an RTX 3090 24GB at $0.193/GPU-hr — ten straight hours is 10 x $0.193 = $1.93, cheaper than a coffee. Second: full bf16 inference on Intern-S1 241B, where the official floor is eight 80GB cards. On A100 SXM4 80GB that is 8 x $1.088 = $8.704/hr, so a three-hour benchmark run costs 3 x $8.704 = $26.11. The same eight cards in H100 SXM 80GB come to 8 x $3.582 = $28.656/hr, or $85.97 for those three hours — same bf16 weights, roughly 70% saved on A100 SXM4. The line people forget is storage: Intern-S1's bf16 weights are about 482GB (241B x 2 bytes), and at the $0.414/GB-month median that is roughly $199 a month just sitting there. Compute billing stops the moment the instance stops; storage keeps running until the volume is destroyed. So either tear the volume down after the benchmark, or keep only the FP8 weights — half the footprint, and it drops the deployment from eight cards to four H100s or two H200s.

04 —

FAQ

Is InternLM dead? Why can't I find InternLM4?

It is not dead, it was renamed. The text-only track ends at InternLM3-8B-Instruct from January 2025, after which Shanghai AI Laboratory moved the flagship line to the multimodal scientific Intern-S family: Intern-S1 (241B), Intern-S1-Pro (1T total / 22B activated / 512 experts), and through 2026 the Intern-S2-Preview series and Intern-S2-Mobius — all Apache-2.0. If you want to work through the generations in order and compare them, per-second billing on NexGPU is the cheapest way to do it: stop the instance and compute charges stop with it.

How much VRAM does Intern-S1 need, and can it run on one GPU?

Not on one GPU. The official hardware table for 241B in bf16 lists 8x A100 80GB, 8x H100, 8x H800 or 4x H200; the FP8 build compresses that to 4x H100/H800 or 2x H200. NexGPU's A100 SXM4 80GB is $1.088/GPU-hr, so eight cards is $8.704/hr — and with up to 14 GPUs per node and 2,152GB max node VRAM, eight-way tensor parallelism stays inside a single node over NVLink.

Will the FP8 weights for Intern-S1 or Intern-S2 run on an A100?

No. The official hardware tables list FP8 support only for H800, H100 and H200, and explicitly exclude A100 — Ampere has no FP8 tensor cores. For FP8 you need Hopper: NexGPU H100 SXM 80GB at $3.582/GPU-hr or H200 141GB at $6.660/GPU-hr. If budget is the constraint, stay on bf16 with A100 PCIE 80GB at $0.824/GPU-hr — twice the weight footprint, but more than four times cheaper per card.

Why does every response start with a long block of reasoning, and how do I stop it?

Thinking mode ships on by default across the Intern-S line. Pass enable_thinking=False into tokenizer.apply_chat_template, or send chat_template_kwargs with the OpenAI-compatible request. Three related traps: SGLang needs --grammar-backend none for Intern-S1; the reasoning parser is intern-s1 on S1 but qwen3 on S2-Preview; and Intern-S2-Preview needs a vLLM build carrying its architecture. All of it takes a few minutes to shake out on a NexGPU vLLM prebuilt image.

Is 24GB of VRAM enough for InternLM3-8B?

Comfortably. The bf16 weights are about 16GB, which leaves a 24GB card room for a decent context window, and the official GGUF q4_k_m build is roughly 5GB — around 8GB of VRAM loaded in 4-bit, so even a 16GB T4 has headroom. NexGPU's RTX 3090 24GB at $0.193/GPU-hr is the cheapest route; for quantised inference only, Tesla T4 16GB at $0.298/GPU-hr works, and RTX 4090 24GB at $0.540/GPU-hr buys faster decoding.

Can I use InternLM and Intern-S commercially, and what tooling comes with them?

Yes — code and weights are Apache-2.0. The supporting stack is open source too: LMDeploy for compression, deployment and serving, XTuner for fine-tuning (single-card QLoRA on a 20B-class model fits in 24GB), and OpenCompass for evaluation. On NexGPU, PyTorch and vLLM are ready among 2,000+ prebuilt images, access is via SSH, Jupyter, web terminal, REST API or CLI, and there is no minimum, no setup fee and no quota request. Support is bilingual over Telegram with no ticket queue.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.