Multi-agent framework

Self-hosting AutoGen: the VRAM bill belongs to the model behind it

AutoGen is a pure-Python, event-driven agent runtime. It consumes no VRAM at all. The thing you actually rent a GPU for is the OpenAI-compatible endpoint you point it at.

AutoGen · self-hosted

AutoGen is Microsoft's open-source multi-agent orchestration framework, MIT licensed, at github.com/microsoft/autogen. Start with its actual status: the Python packages autogen-core, autogen-agentchat and autogen-ext stopped at 0.7.5 (30 September 2025), the project is in maintenance mode and community-managed, and Microsoft is no longer adding features. The baton went to Microsoft Agent Framework (PyPI package agent-framework, now 1.15.0, MIT, Python ≥ 3.10), positioned in AutoGen's own README as the enterprise-ready successor, with a migration guide shipped in the repo. Meanwhile the old v0.2 lineage — ConversableAgent, GroupChatManager, OAI_CONFIG_LIST — was forked into AG2, Apache-2.0, now at 1.0.2, with the classic namespace relocated to ag2-classic. Frozen is not dead: for self-hosting, a frozen API is the friendliest thing that can happen to you, because your dependency tree stops moving underneath you.

So "how much VRAM does AutoGen need" asks the wrong component. autogen-core is an actor-model message runtime — agents pass messages through mailboxes, the whole framework process runs on CPU, and it can live on a different machine from the GPU entirely. The OpenAIChatCompletionClient and OllamaChatCompletionClient in autogen-ext are HTTP clients, nothing more. The VRAM bill lands on the inference server: Qwen3-8B in GGUF is 4.68GB at Q4_K_M, 8.11GB at Q8_0, and roughly 16.4GB in bf16 (8.2B params × 2 bytes). Move to an MoE like Qwen3-30B-A3B and it is 18.6GB at Q4_K_M, 25.1GB at Q6_K, 32.5GB at Q8_0, and about 61GB in bf16. Pick your card against those numbers, not against AutoGen's version string.

What actually burns money in multi-agent work is tokens, not weights. RoundRobinGroupChat replays the entire conversation history to the next speaker on every turn, so consumption trends toward O(n²); SelectorGroupChat is worse, spending an extra model call each turn just to pick who speaks. Qwen3's GGUF context is natively 40,960 tokens — three agents and a dozen turns will punch straight through it. Which is why self-hosted AutoGen wants a per-second-billed GPU you can stop at will, plus a hard MaxMessageTermination and TokenUsageTermination wired in from the start. NexGPU has 1,175 verified rentable nodes and 2,498 GPUs across 75 models in 51 countries and regions, from a $0.188/GPU-hr Tesla V100 32GB up to a 141GB H200, all startable and stoppable by the second.

01 —

Which AutoGen package do you actually install

0.7.5 is the final stable line — with AG2 and Microsoft's successor standing next to it

Version	Parameters	VRAM	Context	Notes
autogen-agentchat 0.7.5	High-level API: AssistantAgent, RoundRobinGroupChat, SelectorGroupChat, Swarm, MagenticOneGroupChat, GraphFlow	0GB (plain Python process); all VRAM is spent by the backing LLM server	Bounded by the backing model, and every team turn replays the full history	This is what most people install: pip install -U "autogen-agentchat" "autogen-ext[openai]". Note that since 0.4, tools are executed by the same AssistantAgent inside the same run() call — the v0.2 UserProxyAgent pairing is gone, and that is the first thing migrators trip over.
autogen-core 0.7.5	Event-driven actor runtime and foundational interfaces, Python ≥ 3.10	0GB; can run entirely on a CPU node while only the inference server holds a card	No context of its own — messages are delivered through agent mailboxes	Officially framed as an event-driven, distributed, scalable, resilient agent runtime. Its biggest deployment win is decoupling: orchestration logic never touches the GPU, so you rent a card only for the hours you are actually running inference.
autogen-ext 0.7.5	Model clients, tools, code executors and MCP extension layer	0GB; the openai and ollama clients only issue HTTP requests	Clients declare capability via model_info; the context budget is unchanged	Pick the right extra: [openai] for OpenAI-compatible endpoints such as vLLM, SGLang or LM Studio; [ollama] for OllamaChatCompletionClient; [magentic-one] for the Magentic-One generalist agents, which additionally need a Playwright browser and a vision-capable model.
AutoGen Studio 0.4.2.2	Low-code web UI, package name autogenstudio	0GB itself, but it expects a resident backend — budget at least the 8.11GB of a Q8_0 8B	Determined by whichever model client you select in the UI	pip install -U "autogenstudio" then autogenstudio ui --port 8081. Great for showing an orchestration flow to colleagues who do not write Python, but it is a prototyping tool — do not make it your production gateway.
AG2 1.0.2	Community fork carrying the v0.2 lineage, Apache-2.0	0GB; same external inference server, identical VRAM conclusions	Determined by the external model	If your code is the 2024-era ConversableAgent / GroupChat / config_list style, AG2 is the heir. Be aware 1.0 restructured things: the classic namespace moved into ag2-classic, so a plain pip install ag2 gets you the newer protocol-driven Agent architecture instead.
Microsoft Agent Framework 1.15.0	Microsoft's official successor, merging the AutoGen and Semantic Kernel lines, MIT	0GB; migrating changes nothing about VRAM — the backing model still decides	Same again, set by the chosen model client	AutoGen's own README calls it the enterprise-ready successor with stable APIs and long-term support. Start new projects there. But the migration question does not change which card you rent today: both sides talk to the same OpenAI-compatible endpoint.

02 —

Which GPU to rent for self-hosted AutoGen

Size the card against the model you put behind it — NexGPU meters per second with no minimum

Getting the orchestration right: three agents, a few tool schemas, termination conditions, running Qwen3-8B at Q8_0 (8.11GB) or Q4_K_M (4.68GB)
RTX 4090 24GB$0.540/GPU-hr
A Q8_0 8B takes only 8.11GB, leaving well over a dozen GB of KV cache to absorb the long histories group chats replay every turn, and the 4090's time-to-first-token makes interactive debugging bearable.
Real workloads: Qwen3-30B-A3B at Q4_K_M (18.6GB) as the primary agent, with a generous context window
RTX 5090 32GB$0.723/GPU-hr
18.6GB of weights inside 32GB still leaves ten-plus GB for KV, which is exactly what absorbs SelectorGroupChat's extra speaker-selection call each turn — an order of magnitude cheaper than reaching for an 80GB card.
Precision first: a resident Q8_0 (32.5GB) or a 32B-class bf16 backend serving several agents concurrently
RTX A6000 48GB$0.817/GPU-hr
A 32.5GB Q8_0 on a 32GB card thrashes as soon as KV cache grows; 48GB lets weights and a real batch coexist, and Q8 measurably reduces malformed tool-call JSON compared with Q4.
Production, a 70B-class bf16 backend, or one shared endpoint serving a whole team's AutoGen experiments
A100 SXM4 80GB$1.088/GPU-hr
70B in bf16 is roughly 140GB of weights, so it needs two 80GB cards in tensor parallel ($2.176/hr); NexGPU nodes go up to 14 GPUs and 2,152GB of node VRAM, so scaling out does not mean changing provider.

03 —

Pointing AutoGen at your own model on NexGPU

Four steps from a bare instance to a multi-agent team that genuinely calls tools

01
Start a vLLM endpoint with a tool-call parser
Pick an RTX 4090 24GB or RTX 5090 32GB in the NexGPU console and boot the prebuilt vLLM image. Two flags matter: --enable-auto-tool-choice is mandatory, and --tool-call-parser should be hermes for Qwen models, whose tokenizer config already uses Hermes-style tool calls. Omit them and vLLM emits tool calls as ordinary text, so AutoGen never sees a tool call at all.
```
vllm serve Qwen/Qwen3-8B --enable-auto-tool-choice --tool-call-parser hermes --max-model-len 32768 --port 8000
```
02
Install AutoGen with only the extras you need
Use [openai] for OpenAI-compatible endpoints such as vLLM, SGLang or LM Studio. If you plan to pull GGUFs through Ollama instead, install autogen-ext[ollama] and switch to OllamaChatCompletionClient — the two client surfaces differ, so do not mix them.
```
pip install -U "autogen-agentchat" "autogen-ext[openai]"
```
03
Write model_info by hand — this step is not optional
OpenAIChatCompletionClient only has a built-in capability table for official OpenAI model names. Give it something like Qwen/Qwen3-8B and it has no idea whether the model supports function calling, so it errors out demanding model_info. All five fields are required: family, function_calling, json_output, vision, structured_output. Setting function_calling to False makes AssistantAgent silently ignore its tools list — by far the most common cause of "my tools never fire" in self-hosted AutoGen. Point base_url at your instance and pass any non-empty api_key.
```
model_client = OpenAIChatCompletionClient(model="Qwen/Qwen3-8B", base_url="http://127.0.0.1:8000/v1", api_key="EMPTY", model_info={"family": "unknown", "function_calling": True, "json_output": True, "vision": False, "structured_output": True})
```
04
Form the team, and wire the brakes first
Get RoundRobinGroupChat working before reaching for SelectorGroupChat or Swarm. Compose termination conditions with |: relying on TextMentionTermination("APPROVE") alone is risky, because small local models routinely forget to emit the keyword and your two agents will compliment each other until the GPU hours run out. Always OR in a hard MaxMessageTermination or TokenUsageTermination. For code execution, if nested Docker is unavailable on your instance, use LocalCommandLineCodeExecutor with an isolated venv rather than fighting DockerCommandLineCodeExecutor.
```
team = RoundRobinGroupChat([primary, critic], termination_condition=TextMentionTermination("APPROVE") | MaxMessageTermination(12))
```

What a three-agent team actually costs for a week

Phase one is getting the orchestration right: Qwen3-8B at Q8_0 is only 8.11GB, which an RTX 4090 24GB swallows easily — $0.540/GPU-hr × 8 hours = $4.32 spent on tool schemas, termination conditions and checking that agents really are passing messages. Phase two swaps in Qwen3-30B-A3B at Q4_K_M (18.6GB) on an RTX 5090 32GB, where the remaining ten-plus GB feeds the conversation history being replayed each turn — $0.723/GPU-hr × 12 hours = $8.68. Call it 50GB for weights, vLLM image layers and logs: storage is 50 × $0.414 = $20.70/month. Pull 5GB of results and traces home and egress is 5 × $0.0081 = $0.04. First week: $4.32 + $8.68 + $20.70 + $0.04 ≈ $33.74, of which actual compute is just $13.00. One billing detail to internalise: compute billing stops the moment the instance stops, while storage keeps running until the volume is destroyed. Multi-agent development spends most of its wall-clock time editing prompts, reading logs and redrawing flow diagrams — hours you should not be paying GPU rates for. NexGPU meters per second, prices per hour, has no minimum, no setup fee and no quota request, which fits that rhythm exactly. For contrast: leave the same work parked on an H100 SXM 80GB and $3.582/GPU-hr × 20 hours is $71.64 — more than five times the compute above, for models that never needed that card.

04 —

FAQ

How much VRAM does self-hosting AutoGen actually require?

AutoGen itself requires 0GB. It is a pure-Python event-driven runtime and can even live on a separate machine from the GPU. The VRAM is entirely determined by the model you point it at: Qwen3-8B is 4.68GB at Q4_K_M, 8.11GB at Q8_0 and roughly 16.4GB in bf16; Qwen3-30B-A3B is 18.6GB at Q4_K_M, 25.1GB at Q6_K and 32.5GB at Q8_0. Add the KV cache that multi-agent history replay inflates, and you want at least 50% headroom above the weights. On NexGPU that maps to an RTX 4090 24GB ($0.540/GPU-hr) for 8B-class, an RTX 5090 32GB ($0.723/GPU-hr) for 30B-A3B at Q4, and an RTX A6000 48GB ($0.817/GPU-hr) for a resident Q8_0 — with vLLM and PyTorch among the 2,000+ prebuilt images.

Is AutoGen deprecated? Should I still use it in 2026?

The Python side stopped at 0.7.5, the project is in maintenance mode and community-managed, and Microsoft points new users at Microsoft Agent Framework (agent-framework, now 1.15.0, MIT) with a migration guide in the repo; the v0.2 lineage lives on as AG2 (Apache-2.0, 1.0.2). For self-hosting, a frozen API is genuinely an advantage — your dependency tree stops moving and 0.7.5's behaviour stays reproducible. And whether you land on AutoGen, AG2 or Agent Framework, all three talk to the same OpenAI-compatible endpoint, so hardware sizing carries over unchanged. On NexGPU you can run two instances side by side to compare a migration, then stop both and pay only for the seconds used.

I connected AutoGen to a local vLLM server — why do AssistantAgent's tools never fire?

Two causes account for almost all of it. First, vLLM was started without --enable-auto-tool-choice and --tool-call-parser (use hermes for Qwen models); without them the model's tool calls come back as plain text. Second, function_calling is set to False in the model_info passed to OpenAIChatCompletionClient, which makes AssistantAgent silently ignore the tools list. Also note that smaller local models handle parallel tool calls poorly — the smaller the model, the more you should expose one tool at a time. Verifying all of this is cheap on NexGPU: spin up an RTX 4090 24GB at $0.540/GPU-hr, change one flag, restart vLLM, and you have your answer. SSH, Jupyter, web terminal, REST API and CLI are all available.

Is a single RTX 4090 24GB enough to run AutoGen multi-agent workflows?

Yes, and it is the best value starting point. 24GB comfortably holds an 8B-class model at Q8_0 (8.11GB) or even bf16 (~16.4GB), leaving the rest for KV cache — plenty for a three-to-five agent RoundRobinGroupChat. What actually breaks first is context: Qwen3 GGUF is natively 40,960 tokens and every team turn replays the whole history, so termination conditions are mandatory. Step up to an RTX 5090 32GB ($0.723/GPU-hr) for 30B-A3B at Q4 (18.6GB), or an RTX A6000 48GB ($0.817/GPU-hr) for higher concurrency. All three are stock nodes on NexGPU, and switching cards just means launching a different instance — no quota request involved.

Why do multi-agent token costs spiral, and how do I control them?

Because group-chat orchestration resends the complete conversation to the next speaker every turn, so growth trends toward O(n²), and SelectorGroupChat adds an extra model call per handoff just to choose the speaker. Fix it in this order: set hard ceilings with MaxMessageTermination and TokenUsageTermination, tighten your system_message lengths, add per-agent context trimming, and only then reach for a smaller model. The hardware lever is just as real — self-hosting replaces a per-token API bill with a whole card billed by the second. On NexGPU an 8B-class setup runs $0.540/GPU-hr, and compute billing stops the moment you stop the instance, so the hours you spend editing prompts cost nothing.

AutoGen's code execution wants Docker — will that work inside a rented GPU instance?

It depends which executor you need. DockerCommandLineCodeExecutor requires a usable Docker daemon inside the instance, and nested Docker is often unavailable in containerised environments. In that case LocalCommandLineCodeExecutor with a dedicated venv is simpler — a rented instance is already an isolated environment you destroy when you are done. If you genuinely need full Docker-in-Docker, confirm the node type before you launch. NexGPU support is bilingual, runs over Telegram with no ticket queue, so a single message settles it, and PyTorch, vLLM, Jupyter and Ubuntu CLI are all ready to go among the 2,000+ prebuilt images.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.