Skip to main content
NexGPU

Local inference runtime

Self-hosting LM Studio: how much VRAM, and how to keep it running

It stopped being just a desktop chat window. The 0.4 line split the engine out into a headless daemon called llmster, so it runs fine on a machine with no display — the only question left is whether your card is big enough.

Sign up freeView pricing

Most people still picture LM Studio as a GUI you double-click on a laptop. The 0.4 line changed that. The engine now ships separately as llmster, a headless daemon you can install on a Linux box, a cloud GPU, or a Colab runtime. Underneath, llama.cpp moved to 2.0.0, which brings continuous batching along with two new load-time options — Max Concurrent Predictions and Unified KV Cache — so parallel requests are actually batched instead of queued one behind the other. There is also a native stateful endpoint, /api/v1/chat, which hands back a response_id you pass as previous_response_id on the next turn, so you stop re-sending the whole transcript every request.

The software is rarely what stops you. VRAM is. The official requirements are deliberately modest: on Windows, at least 4GB of dedicated VRAM and 16GB of system RAM, with AVX2 required on the CPU; macOS 14.0 or newer on Apple Silicon only; Ubuntu 20.04 or newer on Linux. That floor only guarantees the app launches. Whether it runs well comes down to the size of the GGUF on disk — gpt-oss-20b in MXFP4 is 12.1GB, Qwen3-32B at Q4_K_M is 19.76GB, Llama-3.3-70B at Q4_K_M is 42.52GB, and gpt-oss-120b in MXFP4 is 63.4GB across two shards. KV cache sits on top of all of that. When it does not fit, llama.cpp quietly leaves the overflow layers on the CPU, throughput drops by an order of magnitude, and nothing in the UI tells you why it feels slow.

Which turns the whole thing into a purchasing question: is one evaluation run worth putting a 48GB or 80GB card in your machine? On NexGPU you rent one by the second instead — RTX 3090 24GB at $0.193/GPU-hr, RTX A6000 48GB at $0.817/GPU-hr, A100 PCIE 80GB at $0.824/GPU-hr, with no minimum, no setup fee and no quota request. Install llmster, tunnel port 1234 back over SSH, and every OpenAI client you already point at localhost keeps working untouched.

01 —

The four pieces of LM Studio 0.4

The desktop app is only one of them, and it is not the one you deploy

VersionParametersVRAMContextNotes
LM Studio 0.4.21 (desktop)Electron GUI, macOS / Windows / Linux4GB dedicated VRAM recommended as a floorSet at load time, capped by the modelSplit View, Developer Mode, and chat export to PDF and Markdown. The Linux build is an AppImage and needs a graphical session — which is exactly why it is the wrong thing to install on a cloud GPU.
llmster (headless daemon)Server-resident, no GUI at allNo UI overhead; the card is entirely the model'sSupports Unified KV Cache allocationThe piece 0.4 carved out. One install.sh line, then lms daemon up, then a systemd unit to keep it alive across reboots. This is what a rented GPU box actually wants.
lms CLI (MIT licensed)Open source at lmstudio-ai/lms, built on lmstudio.jsUses no VRAM itself--context-length sets it directlylms get / load / unload / ps / server / log stream / runtime / daemon / link. All of it scriptable, which means all of it works in CI.
llama.cpp 2.0.0 runtime (GGUF)CUDA / Vulkan / ROCm / CPU backendsGGUF file size is roughly your VRAM floor; KV cache is extraContinuous batching, Max Concurrent Predictions tunableThe path you take on rented NVIDIA hardware. Multi-GPU Priority Order allocation and the limit-to-dedicated-VRAM toggle are CUDA-only for now; AMD support is still being filled in.
MLX runtimeApple Silicon onlyUnified memory; irrelevant to discrete GPUsFollows the modelM-series Macs exclusively. It does nothing on an NVIDIA card, so pull the GGUF build rather than the MLX build — this is the single most common mistake people make on their first cloud deploy.

02 —

Picking a card by weight size: four honest tiers

It only counts as running if the weights and the KV cache both fit

  • Single card running gpt-oss-20b (12.1GB MXFP4) for local evaluation

    RTX 3090 24GB$0.193/GPU-hr

    With 12.1GB fully offloaded there is still a dozen GB left for KV cache, so 32K context holds comfortably — and this is the cheapest VRAM per gigabyte on the entire price list.

  • Qwen3-32B Q4_K_M (19.76GB) with long context and concurrency

    RTX 5090 32GB$0.723/GPU-hr

    Squeezing 19.76GB into a 24GB card leaves nothing for KV cache; 32GB is what lets you raise context and Max Concurrent Predictions at the same time.

  • Llama-3.3-70B Q4_K_M (42.52GB) fully offloaded on one card

    RTX A6000 48GB$0.817/GPU-hr

    48GB catches 42.52GB of weights with room to spare, so you skip multi-GPU layer splitting and never risk spilling back into system RAM.

  • gpt-oss-120b (63.4GB MXFP4 across two shards) on a single card

    A100 PCIE 80GB$0.824/GPU-hr

    Only $0.007/hr more than the 48GB A6000, and the one option at this price point that swallows 63.4GB of weights while still leaving headroom for KV cache.

03 —

Turning a rented GPU into an LM Studio endpoint

Four steps, all command line, no display required

  1. 01

    Start the instance and tunnel port 1234 home

    Spin up an Ubuntu CLI instance with an NVIDIA card in the NexGPU console and pass -L when you SSH in. The LM Studio server listens on localhost:1234 by default, and tunnelling it back is far safer than opening the port to the internet. Your local base_url still says localhost, so nothing in your code changes.

    ssh -L 1234:localhost:1234 root@<your-node>.nexgpu.net
  2. 02

    Install llmster and bring the daemon up

    Do not install the desktop build — it is an Electron app that needs a graphical session and will not start on a headless box. llmster is the GUI-free engine that 0.4 split out. One script, and the binary lands at ~/.lmstudio/bin/lms; run lms --help to confirm before going further.

    curl -fsSL https://lmstudio.ai/install.sh | bash && ~/.lmstudio/bin/lms daemon up
  3. 03

    Pull the model and offload every layer

    --gpu=max tells llama.cpp to push all layers onto the card. This is the step that decides your throughput: a single layer left on the CPU costs you an order of magnitude, and nothing errors out to warn you. Check the offload ratio with lms ps rather than trusting how it feels.

    lms get openai/gpt-oss-20b && lms load openai/gpt-oss-20b --gpu=max --context-length=32768
  4. 04

    Start the server and point your existing client at it

    Port 1234 serves three surfaces at once: the OpenAI-compatible /v1/chat/completions, /v1/completions, /v1/embeddings, /v1/responses and /v1/models; an Anthropic-style Messages API; and the native stateful /api/v1/chat, which takes an Authorization: Bearer token and threads conversations via previous_response_id. For boot persistence write a systemd unit with Type=oneshot and RemainAfterExit=yes, chaining lms daemon up then lms load --yes in ExecStartPre and lms daemon down in ExecStop.

    lms server start && curl http://localhost:1234/v1/models

Two real invoices

Case one: evaluating gpt-oss-20b on an RTX 3090 24GB. The 12.1GB of weights offload fully into 24GB, and compute bills at $0.193/GPU-hr. Six hours a day for a week is 30 hours, so 30 x $0.193 = $5.79. Keeping the model on disk for a month costs 12.1 x $0.414 = $5.01. Pulling down 2GB of evaluation logs is 2 x $0.0081 = $0.02. Total roughly $10.82 — two orders of magnitude below buying a 24GB card. Case two: gpt-oss-120b on an A100 PCIE 80GB, where the price list hides a trap. The RTX A6000 48GB is $0.817/GPU-hr and the A100 PCIE 80GB is $0.824/GPU-hr, a difference of $0.007 an hour — but 48GB will not hold 63.4GB of weights, so that seven-tenths of a cent buys whether it runs at all, not how fast. Ten hours is 10 x $0.824 = $8.24. Storage is the line worth watching: parking 63.4GB for a month is 63.4 x $0.414 = $26.25, more than triple the compute, so destroying the volume and running lms get again next time is usually the cheaper habit. Compute billing stops the moment the instance stops; storage keeps billing until the volume is destroyed. Track them separately.

04 —

FAQ

How much VRAM does LM Studio actually need?

The official floor is 4GB of dedicated VRAM and 16GB of RAM on Windows, but that only guarantees the app opens. What decides the outcome is GGUF weight size plus KV cache: 12.1GB for gpt-oss-20b in MXFP4, 19.76GB for Qwen3-32B Q4_K_M, 42.52GB for Llama-3.3-70B Q4_K_M, 63.4GB for gpt-oss-120b. Map those upward and on NexGPU they land on an RTX 3090 24GB ($0.193/GPU-hr), an RTX 5090 32GB, an RTX A6000 48GB and an A100 PCIE 80GB ($0.824/GPU-hr) respectively. Billing is per second, so guessing wrong just means restarting on a bigger card instead of returning hardware.

Can LM Studio run on a server with no graphical interface?

Yes — that is precisely what the 0.4 line was restructured for. llmster is the GUI-free daemon: install it with curl -fsSL https://lmstudio.ai/install.sh | bash, bring it up with lms daemon up, start serving with lms server start, then write a systemd unit to keep it resident. The documented template uses Type=oneshot with RemainAfterExit=yes, chaining lms daemon up and lms load --yes in ExecStartPre. NexGPU's Ubuntu CLI image is ready on boot, so you SSH in and paste that config straight through — no X server, no AppImage wrangling.

Is LM Studio free for commercial use, and is it open source?

It is free for both personal and workplace use — teams no longer need a separate licence or a procurement cycle to run it at work. Organisations wanting more get Teams and Enterprise plans with SSO, model gating and access controls. The desktop app itself is not open source, but the lms CLI is MIT licensed at lmstudio-ai/lms, and both SDKs — lmstudio-js for TypeScript and lmstudio-python for Python — are on GitHub. So dropping it into an internal service or a CI pipeline carries no licensing baggage, and on NexGPU a single 3090 at $0.193/GPU-hr is enough to host that internal endpoint.

How does LM Studio compare to Ollama and vLLM?

They aim at different targets. Ollama and LM Studio both ride llama.cpp and GGUF; LM Studio adds the GUI, MCP host support, the built-in Bionic agent and Agent Skills (SKILL.md) support, and since moving to llama.cpp 2.0.0 it does continuous batching rather than pure queueing. For genuinely high-concurrency production traffic, vLLM's PagedAttention is still the better tool. The cheapest way to decide is to run both on the same card — NexGPU's 2,000+ prebuilt images include vLLM and PyTorch, so switching stacks costs you a new instance and a few GPU-hours.

What is the safe way to reach LM Studio running on a remote GPU?

The native REST endpoint /api/v1/chat authenticates with an Authorization: Bearer token, but the server still binds localhost:1234 by default, and an SSH tunnel (ssh -L 1234:localhost:1234) remains the sound choice — do not shortcut this by exposing 1234 publicly. The other route is LM Link: after lms link enable, a model loaded on the remote machine can be called as though it were local, with lms link set-preferred-device choosing which box answers. Every NexGPU instance comes with its own SSH credentials, so once the tunnel is up you treat it as a local endpoint.

The model loaded fine — why is generation still slow?

Almost always incomplete offload. When VRAM runs short llama.cpp leaves the overflow layers on the CPU, costing an order of magnitude in throughput without raising an error — check the offload ratio with lms ps and add --gpu=max at load time. Next, context length multiplied by concurrency drives KV cache usage; turning on Unified KV Cache stops preallocation being rigidly partitioned per request, and Flash Attention is already on by default on CUDA. Beyond that, speculative decoding with a same-family, matching-tokenizer draft model is documented at 1.5x to 3x. If it is still tight after all that, the card is simply too small — and moving from 24GB to 48GB or 80GB on NexGPU is one new instance, billed per second, with compute charges stopping the moment you stop it.

More in Local runtimes

Every model guide

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.

Sign up freeTalk to sales

Sign up to browse live network pricing. No payment method required.