Agent framework

LangChain costs you nothing in VRAM — the model behind it is what costs

langchain 1.3.16 is pure CPU orchestration. What decides the size of card you rent is the local model behind create_agent that has to emit tool_calls turn after turn. This page separates the two bills.

LangChain · self-hosted

LangChain is no longer the chain-of-chains library from 2024. The main package sits at langchain 1.3.16, langchain-core at 1.6.0, LangGraph at 1.2.11, all MIT, all maintained by langchain-ai at github.com/langchain-ai/langchain. Since v1 the langchain namespace holds five things — agents, messages, tools, chat_models, embeddings — and the single entry point for building an agent is `from langchain.agents import create_agent`, which replaced create_react_agent from langgraph.prebuilt. Customisation no longer means subclassing or stacking Chains; it means middleware, each component hooking into execution points like before_model and after_model and owning exactly one concern: execution environment, context management, planning and delegation, fault tolerance, guardrails, steering. Provider differences are absorbed into content_blocks, so reasoning, text and tool_call blocks read the same way across langchain-anthropic, langchain-openai and langchain-ollama.

So the honest answer to "how much VRAM does LangChain need" is 0GB. It is orchestration logic in a Python process, running on CPU, supported on Python 3.10 through 3.14. The VRAM bill lands entirely on the backends you stand up yourself: a chat model that calls tools reliably, an embedding model for retrieval, optionally a reranker. Those are what need cards. Working only from verifiable figures: openai/gpt-oss-20b is a 21B-total / 3.6B-active MoE with MXFP4 MoE weights, and its model card states plainly that it runs within 16GB of memory, which a 24GB card handles comfortably; gpt-oss-120b is 117B total / 5.1B active and is explicitly designed to fit into a single 80GB GPU; Qwen3.8-27B is 27B dense with a native 262,144-token context extensible to 1M via YaRN, and its bf16 weights alone come to roughly 54GB, which the official FP8 build compresses to around 27GB. Multiply the parameters, look at the number, rent that card. There is no magic in between.

What actually breaks when you self-host LangChain is not the framework either. Three failures dominate. First, you wire up a local model, bind_tools looks fine, and the agent never calls a tool — because vLLM was started without `--enable-auto-tool-choice` and a matching `--tool-call-parser`, so the model's tool JSON comes back as plain text and the loop never fires. Second, older code upgraded to 1.x throws ImportError on `from langchain.chains import LLMChain`, because legacy chains, legacy retrievers, the indexing API, the hub module and the langchain-community re-exports all moved into langchain-classic 1.0.8. Third, the one that matters most for a private deployment: the moment LANGSMITH_TRACING is true, your prompts and tool outputs travel to LangSmith. You only remember these after hitting them on real hardware — and hitting them needs a card you can start on demand, meter by the second, and stop when you are done. NexGPU has 1,175 verified rentable nodes across 51 countries and regions, 2,498 GPUs, and PyTorch and vLLM sitting ready among 2,000+ prebuilt images.

01 —

What you install, and what you feed it

The top half is the LangChain stack itself and takes no GPU at all; the bottom half is what actually eats the VRAM.

Version	Parameters	VRAM	Context	Notes
langchain 1.3.16	framework, no weights	0GB, CPU-only orchestration	set by the backend model	The main package for create_agent and middleware. MIT, Python 3.10–3.14; namespace narrowed to agents / messages / tools / chat_models / embeddings.
langgraph 1.2.11 + langgraph-cli 0.4.31	runtime and CLI	0GB, state lives in Postgres	checkpointer decides replay depth	The runtime under stateful multi-turn agents. CLI ships langgraph new / dev / up / build / dockerfile; `langgraph up` puts the server straight into Docker.
langchain-classic 1.0.8	migration compatibility package	0GB	—	Home of the legacy chains, legacy retrievers (MultiQueryRetriever included), the indexing API, the hub module and the langchain-community re-exports. Install it when a 1.x upgrade throws ImportError.
openai/gpt-oss-20b (backend candidate)	21B total / 3.6B active MoE	MXFP4 weights; card states it runs within 16GB	long, trimmed via --max-model-len	Apache 2.0. A single 24GB card closes the whole agent loop, making it the cheapest honest backend for validating create_agent, middleware and your tool chain.
Qwen/Qwen3.8-27B and -FP8 (backend candidate)	27B dense (HF counts ~28B including the vision tower)	bf16 weights ~54GB / FP8 ~27GB	262,144 native, YaRN to 1M	Apache 2.0, natively handles images and video, exposes reasoning_effort tiers (xhigh / medium / low) and preserve_thinking. One line to serve: `vllm serve "Qwen/Qwen3.8-27B"`.
Qwen/Qwen3-Embedding-0.6B / 4B / 8B (retrieval side)	0.6B / 4B / 8B	fp16 ~1.2GB / ~8GB / ~16GB	documents chunked before embedding	The vectorisation backend for LangChain RAG, with official GGUF builds alongside. The 0.6B tier is small enough to share a card — don't let it eat VRAM your inference card needs for KV cache.

02 —

Pick the card for what you're actually running

LangChain doesn't appear in this table — the models behind it do, each tier matched to verifiable weight sizes.

Close the create_agent loop: gpt-oss-20b on one card, validating tool calls
RTX 4090 24GB$0.540/GPU-hr
MXFP4 weights fit inside the stated 16GB, leaving roughly 8GB of the 24GB for KV cache — enough for a ten-step agent at 32K context to finish without OOM.
Retrieval only: batch-embedding a knowledge base with Qwen3-Embedding-0.6B
Tesla T4 16GB$0.298/GPU-hr
The 0.6B model is about 1.2GB in fp16. Split embedding onto a cheap card instead of stealing the few GB your inference card was holding back for KV cache.
Production agent backend: Qwen3.8-27B-FP8 running continuously
RTX A6000 48GB$0.817/GPU-hr
FP8 weights are about 27GB. A 32GB card does hold the weights, but leaves almost nothing for KV cache, so multi-turn tool calling blows up once context accumulates. 48GB is the safe line here.
Top tier: gpt-oss-120b, or Qwen3.8-27B in bf16 with long context
A100 PCIE 80GB$0.824/GPU-hr
gpt-oss-120b is designed for exactly one 80GB GPU, and Qwen3.8-27B's ~54GB of bf16 weights land in the same tier. Same 80GB, about 23% of H100 SXM at $3.582/GPU-hr.

03 —

Empty instance to a tool-calling agent, in four steps

The whole thing hinges on two flags in step one — miss them and steps two through four look correct while the agent silently never calls a tool.

01
Start the backend, and turn auto tool choice on explicitly
`--enable-auto-tool-choice` is mandatory in vLLM, and `--tool-call-parser` is chosen per model family: llama3_json for Llama 3.1/3.2, llama4_pythonic for Llama 4, mistral for Mistral, hermes for the Qwen family, plus a generic pythonic. Miss either flag and the model returns tool JSON as ordinary text, LangChain receives a plain message, and the agent loop never enters the tool branch. Fire one manual request with a tool attached and confirm finish_reason really is tool_calls before you trust the agent.
```
vllm serve "Qwen/Qwen3.8-27B" --host 0.0.0.0 --port 8000 --enable-auto-tool-choice --tool-call-parser hermes --max-model-len 32768 --gpu-memory-utilization 0.90
```
02
Install LangChain 1.x with the versions pinned
Pinning is not fussiness here: langchain-ollama 1.1.0 requires langchain-core ≥ 1.2.21, deepagents 0.7.8 requires langchain ≥ 1.3.15 and langchain-core ≥ 1.6.0, and its Python floor is 3.11 — one notch above langchain's own 3.10, so installing deepagents into a 3.10 image fails resolution outright. Pointing at a local endpoint needs nothing more than the OpenAI-compatible client from langchain-openai: base_url is your vLLM /v1, api_key is any placeholder.
```
pip install -U "langchain==1.3.16" "langgraph==1.2.11" "langchain-openai==1.6.0"
```
03
Assemble with create_agent, not create_react_agent
Since v1, create_agent is the recommended entry point, taking model, tools, system_prompt, response_format, state_schema, checkpointer, context_schema, middleware and name. Attach a checkpointer for multi-turn memory — InMemorySaver locally, the Postgres saver in production. Pass response_format for structured output, but note your local backend has to support constrained decoding or Pydantic validation will fail intermittently. Hosted APIs paper over that; self-hosting does not.
```
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="Qwen/Qwen3.8-27B", base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
agent = create_agent(model=llm, tools=[search], system_prompt="You are a research agent.")
```
04
Cut the outbound path, then bring the service up
The last gate on a private deployment: with LANGSMITH_TRACING set to true, prompts, tool arguments and tool results all travel to LangSmith. Either turn it off or point LANGSMITH_ENDPOINT at your own self-hosted instance — do not assume the default is closed. Once it's clean, use langgraph-cli to put the server into Docker and leave it running. SSH or Jupyter both work on the instance; NexGPU gives you those plus a web terminal, REST API and CLI out of the box.
```
export LANGSMITH_TRACING=false && langgraph up
```

What a week of getting a LangChain agent working actually costs

Take a typical self-hosted pair: RTX 4090 24GB running gpt-oss-20b on the inference side at $0.540/GPU-hr, Tesla T4 16GB running Qwen3-Embedding-0.6B on the retrieval side at $0.298/GPU-hr — $0.838 per hour together. A small team works five days, eight hours of uptime a day, 40 hours total: 0.838 × 40 = $33.52. Weights plus corpus take 40GB of storage at the $0.414/GB-month median: 40 × 0.414 = $16.56/month. Pull 10GB of logs and eval output back home at the $0.0081/GB median egress: 10 × 0.0081 = $0.08. First month, about $50.16. The same rig left running 24/7 would be 720 × 0.838 = $603.36 — the order-of-magnitude gap is not the rate, it's whether you stop the instance when the agent isn't running. NexGPU meters per second and prices per hour, with no minimum, no setup fee and no quota request; compute billing stops when the instance stops, and only storage keeps accruing until you destroy it. Stepping up to something like gpt-oss-120b, designed for a single 80GB GPU, an A100 PCIE 80GB is $0.824/GPU-hr — roughly 23% of H100 SXM 80GB at $3.582/GPU-hr.

04 —

FAQ

How much VRAM does a LangChain deployment actually need?

The framework needs 0GB. langchain 1.3.16 is orchestration code running on CPU; you don't even need CUDA present to install it. All the VRAM goes to the backends you stand up: gpt-oss-20b's MXFP4 weights are stated to run within 16GB, Qwen3.8-27B is about 27GB in FP8 and ~54GB in bf16, Qwen3-Embedding-0.6B is about 1.2GB in fp16. Choose the backend first, then size the card against those numbers — NexGPU runs from RTX 3090 24GB at $0.193/GPU-hr up to H200 141GB at $6.660/GPU-hr, and across 75 GPU models one tier will sit exactly on your weight footprint.

My old project throws ImportError on `from langchain.chains import LLMChain` after upgrading to langchain 1.x. How do I fix it?

Not a bug — the v1 namespace narrowed. Legacy chains, legacy retrievers (MultiQueryRetriever and the whole langchain.retrievers module), the indexing API, the hub module and the langchain-community re-exports all moved into langchain-classic 1.0.8. Install it and rewrite imports from langchain.chains to langchain_classic.chains and you're running again. Don't write new code against Chains; use create_agent. If you'd rather rehearse the migration on a real environment first, NexGPU instances meter per second and start and stop on demand, so breaking one and rebuilding costs small change.

I wired up a local model and called bind_tools, but the agent never triggers a tool. Why?

Nine times out of ten the server never enabled auto tool choice. In vLLM `--enable-auto-tool-choice` is mandatory, and it must be paired with a `--tool-call-parser` matching the model family (llama3_json, llama4_pythonic, mistral, hermes, pythonic). Without them the model does generate correctly formatted tool JSON, but the server doesn't parse it and returns it verbatim as text, so LangChain receives an ordinary AIMessage and the loop skips the tool branch. Diagnose it by bypassing LangChain entirely and sending one OpenAI-compatible request to check whether finish_reason is tool_calls. vLLM is ready-made in NexGPU's 2,000+ prebuilt images, so spinning up an instance to get the parser right takes about ten minutes.

Does a private LangChain deployment quietly send prompts to LangSmith?

It comes down to one environment variable. With LANGSMITH_TRACING set to true, prompts, tool arguments and tool results are reported to LangSmith, alongside the rest of the config — LANGSMITH_API_KEY, LANGSMITH_ENDPOINT, LANGSMITH_PROJECT. If you're serious about a private deployment, set `LANGSMITH_TRACING=false` explicitly, or point LANGSMITH_ENDPOINT at your own self-hosted instance; don't assume the default is safe. To keep the data inside your own boundary entirely, NexGPU covers 1,175 verified nodes across 51 countries and regions, so you can pick a region that satisfies your compliance requirements and keep the whole chain inside it.

My agent OOMs on CUDA around the seventh or eighth turn. Is the model too big?

Usually it isn't the weights, it's context. Every turn appends the raw tool result to messages, and a tool that fetches a page or reads a file can drop thousands of tokens in one go, so after ten turns KV cache grows faster than the weights ever did. Three moves: set `--max-model-len` to the length you actually need rather than the model's ceiling, tune `--gpu-memory-utilization` so KV cache has room, and use LangChain's context-management middleware on before_model to trim or summarise. If it genuinely won't fit, move up a VRAM tier — NexGPU supports up to 14 GPUs per node and a maximum node VRAM of 2,152GB, so there's headroom above you.

Is LangGraph's create_react_agent still usable, and how does it relate to create_agent?

create_agent is its successor. The docs state that create_agent is the standard way to build agents from LangChain 1.0 onward, replacing create_react_agent from langgraph.prebuilt with a simpler interface where the extension point shifts from function arguments to middleware. LangGraph 1.2.11 is still the runtime underneath, handling state, checkpoints and multi-actor orchestration — you just no longer pull the agent constructor out of its prebuilt module. The matching langgraph-cli 0.4.31 gives you `langgraph dev` with hot reload and `langgraph up` for Docker, both of which run straight over SSH or the web terminal on a NexGPU instance, with bilingual support on Telegram and no ticket queue.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.