Coding agent framework

Self-hosting OpenHands: where the VRAM actually goes

OpenHands runs on 4GB of RAM and no GPU. Every gigabyte of VRAM is spent on the model you hand it. This page keeps the two bills separate.

OpenHands · self-hosted

OpenHands is an MIT-licensed open-source coding agent that changed most of its signage this year. The repo moved from All-Hands-AI/OpenHands to github.com/OpenHands/OpenHands, the docs 308-redirect from docs.all-hands.dev to docs.openhands.dev, and the community moved from Discord to Slack. The flagship in the main repo is now Agent Canvas — a self-hosted control center for coding agents — shipping as npm package @openhands/agent-canvas and Docker image ghcr.io/openhands/agent-canvas, both at 1.15.0. The CLI line is the openhands package on PyPI at 1.16.0, pinned to Python 3.12. The Agent Server that does the actual work comes from OpenHands/software-agent-sdk, image tag ghcr.io/openhands/agent-server:1.26.0-python. Four repos, three version numbers — memorise those and your first install goes a lot faster.

Now the question everyone asks wrong. The official host requirement is one sentence: a modern processor and a minimum of 4GB RAM. OpenHands does not run inference. It orchestrates, dispatches through LiteLLM, and executes commands inside a Docker sandbox (default image nikolaik/python-nodejs:python3.12-nodejs22). So "how much VRAM does OpenHands need" is aimed at the wrong target. The real question is which model you plan to put behind it, and what that model costs in VRAM.

The docs now name Qwen3.6-35B-A3B as the first local model to try: Apache-2.0, a 35B-total / 3B-active MoE routing 8 of 256 experts plus 1 shared, 40 layers where every fourth is full attention and the rest run Gated DeltaNet linear attention, 262,144 tokens of native context extensible to 1,010,000 with YaRN, and 73.4 on SWE-bench Verified. The stated hardware bar is "at least 24GB of VRAM for quantized variants, or multiple GPUs for full precision." Below, that sentence gets turned into configurations you can actually rent.

01 —

Qwen3.6-35B-A3B weight variants and what they cost in VRAM

OpenHands itself (CLI 1.16.0 / Agent Canvas 1.15.0 / agent-server 1.26.0) uses no VRAM. This table is what decides which card you rent. Sizes are actual file sizes from the Hugging Face repos.

Version	Parameters	VRAM	Context	Notes
Qwen/Qwen3.6-35B-A3B (bf16 safetensors)	35B total / 3B active	bf16 71.9GB (67.0GiB)	262K native, 1.01M via YaRN	The official checkpoint. Two-way tensor parallelism minimum on vLLM/SGLang — the OpenHands docs use --tensor-parallel-size 2; Qwen's own examples use --tp-size 8.
Qwen/Qwen3.6-35B-A3B-FP8	35B-A3B	FP8 37.5GB (34.9GiB)	262K	Official FP8 quantisation, fits one 48GB card. Native FP8 math needs Ada or Hopper; on Ampere vLLM falls back to Marlin dequant kernels — it runs, but you do not get the speedup.
unsloth UD-Q8_0 GGUF	35B-A3B	Q8_0 36.9GB (34.4GiB)	262K (KV on top)	The near-lossless tier for llama.cpp / Ollama / LM Studio backends. Comfortable on a single 48GB card with no tensor-parallel setup to debug.
unsloth UD-Q4_K_M GGUF	35B-A3B	Q4_K_M 22.1GB (20.6GiB)	32K (22000 is the official floor)	The realistic single-24GB-card answer. Weights eat 20.6GiB, what remains covers roughly 32K of context — do not also load the vision tower.
unsloth UD-IQ4_XS GGUF	35B-A3B	IQ4_XS 17.7GB (16.5GiB)	128K workable	Same 24GB card, but you want long context. Qwen3.6 has only 10 full-attention layers, so FP16 KV runs about 20KB per token — 128K is roughly 2.5GB, which the 4GiB you just freed covers.
mmproj-F16 (vision tower, optional)	vision encoder	+0.9GB	—	Qwen3.6 ships vision, so the agent can read screenshots and UI. For pure code work, skip it with vLLM's --language-model-only and hand that memory to the KV cache.

02 —

Picking a card: four ways to start on NexGPU

Prices are list rate per GPU per hour, metered per second, compute billing stops when the instance stops.

Single 24GB card, Q4_K_M at 32K context, Ollama or LM Studio
RTX 3090 24GB$0.193/GPU-hr
24GB is the documented floor, GGUF Q4 on a llama.cpp backend does not need FP8 math, and the 3090 is the cheapest 24GB card that clears the bar.
Q4 / IQ4 at 128K context, or a vLLM instance a small team shares
RTX 5090 32GB$0.723/GPU-hr
32GB holds 20.6GiB of weights, the 0.9GB vision tower and a 128K KV cache with room to spare, and it has native FP8 if you switch to the official FP8 checkpoint.
Q8_0 or official FP8 on one card, with the full 262K context enabled
RTX A6000 48GB$0.817/GPU-hr
34-35GiB of weights plus roughly 5GB of KV at 262K fits in 48GB on a single card, which removes the entire tensor-parallel configuration problem.
bf16 native weights for evaluation, benchmark runs, or serving a whole team
A100 SXM4 80GB x2$1.088/GPU-hr ($2.176/hr for the pair)
71.9GB of weights needs two cards, which is exactly the --tensor-parallel-size 2 in the docs. If you want it on one card, H200 141GB at $6.660/GPU-hr runs the full 262K without arithmetic.

03 —

Four steps to point OpenHands at your own model

About fifteen minutes from a fresh NexGPU instance to a running agent, most of it spent pulling weights.

01
Serve the model
Expose Qwen3.6-35B-A3B as an OpenAI-compatible endpoint with vLLM. You need vllm 0.19.0 or newer, or sglang 0.5.10 or newer. --enable-prefix-caching pays off heavily for agents, which resend the same long system prompt on every turn, and --tool-call-parser qwen3_coder is mandatory or tool calls will not parse. NexGPU's 2,000+ prebuilt images include vLLM and PyTorch, so there is no CUDA setup to do.
```
vllm serve Qwen/Qwen3.6-35B-A3B --host 0.0.0.0 --port 8000 --api-key mykey --tensor-parallel-size 2 --served-model-name Qwen3.6-35B-A3B --enable-prefix-caching --enable-auto-tool-choice --tool-call-parser qwen3_coder
```
02
Install OpenHands
The CLI installs through uv and pins Python 3.12. --mount-cwd mounts your current directory into the Docker sandbox, which is where the agent edits code; SANDBOX_VOLUMES=$PWD:/workspace:rw does the same thing explicitly. If you want the web control center instead, run the Agent Canvas Docker image on port 8000 with the UI at /canvas.
```
uv tool install openhands --python 3.12 && openhands serve --mount-cwd
```
03
Point it at your endpoint
The openai/ prefix is the whole trick. OpenHands dispatches through LiteLLM, so a self-hosted endpoint must be written as openai/<served-model-name>; drop the prefix and you get a model-not-found error. This is the single most common self-hosting failure. If OpenHands runs in a container and the model runs on the host, use http://host.docker.internal:8000/v1 and add --add-host host.docker.internal:host-gateway to your docker run.
```
export LLM_MODEL=openai/Qwen3.6-35B-A3B LLM_BASE_URL=http://127.0.0.1:8000/v1 LLM_API_KEY=mykey
```
04
Run it headless and capture JSONL
--headless always runs in always-approve mode — the agent never stops to ask, which is what you want in CI and why the sandbox matters. The docs are explicit that --llm-approve is unavailable in this mode. Add --json and every action and observation event streams out as one JSON object per line, easy to parse for pass/fail downstream.
```
openhands --headless --json -t "Add unit tests for src/parser.py and make them pass" > run.jsonl
```

What a month actually costs, with the arithmetic

Take a five-person team sharing one inference box. An RTX 5090 32GB running vLLM with the IQ4_XS weights, up eight hours a day on working days, twenty-two days a month, is 176 hours: 176 x $0.723 = $127.25. Keep 60GB of weights and images on disk: 60 x $0.414 = $24.84/month. Total $152.09, or just under $31 per engineer — less than one person's cloud API bill. Need a bf16 benchmark reproduction? Two A100 SXM4 80GB for six hours: 6 x 2 x $1.088 = $13.06, and compute billing stops the moment you stop the instance. Want it cheaper? Push the same workload onto an RTX 3090 24GB at Q4_K_M and 176 hours costs 176 x $0.193 = $33.97 — you pay for it with a 32K context ceiling and no vision tower. One warning worth internalising: compute and storage are separate bills. Compute stops when the instance stops; storage runs until you destroy the volume. Delete that 71.9GB bf16 checkpoint after the benchmark or it becomes a silent 71.9 x $0.414 = $29.77 every month. No minimum, no setup fee, no quota request.

04 —

FAQ

Does self-hosting OpenHands require a GPU?

Not for OpenHands itself. The official host requirement is a modern processor and a minimum of 4GB RAM. It orchestrates, dispatches through LiteLLM and executes commands in a Docker sandbox — it never runs inference. The GPU is for the model you choose to host. If you are calling a cloud API, a cheap always-on node is enough. If you want full privacy, NexGPU spans RTX 3090 24GB at $0.193/GPU-hr up to H200 141GB at $6.660/GPU-hr, all started and stopped by the second.

What is the minimum VRAM for running OpenHands with a local model?

The documented bar is at least 24GB for quantised variants and multiple GPUs for full precision. In actual file sizes: Qwen3.6-35B-A3B at Q4_K_M is 20.6GiB, so a 24GB card has two or three gigabytes left after weights — enough for 32K of context, not enough to also load the vision tower. If you want 128K on 24GB, switch to IQ4_XS at 16.5GiB. NexGPU's RTX 3090 24GB ($0.193/GPU-hr) and RTX 4090 24GB ($0.540/GPU-hr) sit right on that line; the RTX 5090 32GB ($0.723/GPU-hr) removes the decision entirely.

Why does OpenHands behave like a plain chatbot and refuse to call tools with my local model?

Nine times out of ten the context window is too small. Ollama defaults to 4096, and the docs say plainly that not even the system prompt fits — set OLLAMA_CONTEXT_LENGTH=32768, and never below 22000. The other common trap is LM Studio on Linux binding only to 127.0.0.1, which a containerised OpenHands cannot reach; enable "Serve on Local Network" to bind 0.0.0.0. If both are ruled out, the model itself is not reliable at tool use — the docs cite community reports that qwen2.5-coder-14b-instruct fixes exactly this chatbot-like behaviour. Swapping cards or models on NexGPU takes minutes and the experiment is billed by the second.

Did OpenHands get renamed? How does it relate to All Hands?

Same code, new signage. The repo moved from All-Hands-AI/OpenHands to github.com/OpenHands/OpenHands, the docs 308 from docs.all-hands.dev to docs.openhands.dev, and the community moved from Discord to Slack. The project also split across four repos: OpenHands/OpenHands owns the Agent Canvas frontend and local-stack orchestration, software-agent-sdk owns the Python SDK, Agent Server and canonical API, with typescript-client and automation each covering their own slice. Still MIT, still CodeActAgent by default. Old links redirect, but the V1 docs were rewritten — do not follow an old tutorial.

Can OpenHands drive Claude Code or Codex directly?

Yes. Agent Canvas hosts third-party agents over ACP (Agent Client Protocol) — Claude Code, Codex and Gemini CLI are all supported, switchable alongside the built-in CodeActAgent in the same console, with LLM profiles for different situations. The practical pattern is a cheap local model for routine edits and a stronger model for planning and review. Running a second model on NexGPU is another per-second instance; nodes go up to 14 GPUs and 2,152GB of VRAM, so keeping both resident on one box is well within reach.

Do I have to build the environment from scratch to run OpenHands on NexGPU?

No. The 2,000+ prebuilt images include PyTorch and vLLM, so you boot and run vllm serve without touching CUDA or drivers. Access is via SSH, Jupyter, web terminal, REST API and CLI, and both Agent Canvas on port 8000 and the OpenHands web app on port 3000 map straight out. The network spans 51 countries and regions, 1,175 verified rentable nodes, 2,498 GPUs and 75 GPU models, so a 71.9GB bf16 checkpoint or a two-way tensor-parallel setup both find a home. The console is at console.nexgpu.net — no minimum, no setup fee, no quota request, and bilingual support over Telegram with no ticket queue.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.