Skip to main content
NexGPU

Agent Framework

Self-host CrewAI, and spend your VRAM budget on the model

CrewAI itself is a pure Python orchestration framework that occupies no GPU memory at all. What decides the card you rent is the local model you point it at — and multi-agent call density makes that a very different choice than sizing for a chatbot.

Sign up freeView pricing

First, the thing everyone searches wrong: CrewAI is not a model, it is the scheduling layer above one. Maintained by CrewAI Inc at crewAIInc/crewAI under the MIT licence, it requires Python >=3.10 and <3.14. Once installed it is a process using a few hundred MB of RAM and exactly zero VRAM. So the honest answer to "how much VRAM does CrewAI need" is: whatever the model behind your agent's `llm` needs. Point it at a hosted API and the answer is 0GB. Point it at a locally served gpt-oss:20b and you are starting at 14GB of weights.

Second, versions. CrewAI is well past 1.0 — 1.0.0 shipped on 20 October 2025, and the line is now at 1.15.17. If the tutorial you found still writes `config/agents.yaml` with `@CrewBase` decorators, that is the old scaffolding from the 0.28 / 0.86 era. In 1.x, `crewai create crew` generates JSONC by default: agents live in `agents/<name>.jsonc`, crew settings in `crew.jsonc`, with top-level fields merging into the `settings` object and `settings` taking precedence. Pass `--classic` if you want the YAML layout back. The other big shift is that LiteLLM is no longer a hard dependency — OpenAI, Anthropic, Gemini, Azure and Bedrock go through native SDKs, LiteLLM is the fallback for everyone else, and local inference servers can connect directly with `custom_openai=True` and skip it entirely.

Third, the part that actually determines which card you rent: CrewAI is a multi-agent framework, not single-turn Q&A. An agent's `max_iter` defaults to 20, meaning one task can fire up to twenty model calls — multiply that by the number of agents in the crew, by tool-call round trips, and by the extra summarisation passes that `respect_context_window=True` triggers when you overflow. The same business logic that costs a chatbot three requests can cost a crew sixty. So the criterion for self-hosting CrewAI is not "do the weights fit" but "how much KV cache is left after the weights, and does throughput survive this access pattern." Below is every realistic tier with real VRAM figures and real NexGPU rates.

01 —

CrewAI itself, and the local models it can drive

Row one is the framework; the rest is where the money goes. VRAM figures are the official Ollama distribution weights — leave headroom for KV cache on top.

VersionParametersVRAMContextNotes
CrewAI 1.15.17 (the framework)Pure Python, no weights0GB VRAM, ~300–500MB RAMSet by whichever model you attachCurrent stable line. Ships two orchestration paradigms — Crews for role-based collaboration, Flows for event-driven execution with `@persist` checkpointing. MIT licensed, installed with a single `uv tool install crewai`.
gpt-oss:20b (MXFP4)20B MoE, 4.25 bits/param14GB weights, 24GB card advised128KThe default recommendation for local CrewAI. Native function calling and structured outputs — precisely what a crew leans on hardest. Officially it starts on as little as 16GB, but you want 24GB once KV cache is in play.
Qwen3-14B14B denseQ4_K_M 9.3GB, 24GB card advised40KDense, so latency is steady with none of the routing jitter MoE brings. 9.3GB of weights leaves a large margin on a 24GB card — good when you have many agents or several crews running concurrently.
Qwen3-30B-A3B30B total / ~3B activeQ4 19GB, 48GB card advised256KThe sweet spot for long-context production crews. A native 256K window means `respect_context_window` rarely fires, saving you the extra summarisation round trip, and only ~3B active parameters keeps throughput well ahead of a dense model of the same footprint.
gpt-oss:120b (MXFP4)120B MoE65GB weights, fits one 80GB card128KThe quality tier. Officially documented as fitting on a single 80GB GPU, so no tensor parallelism — deployment complexity is essentially identical to the 20b, the card is just pricier.
Qwen3-235B-A22B235B total / ~22B activeQ4 ~142GB, needs 2×80GB256KTop end. At 142GB the weights do not fit a single 141GB H200, so two 80GB cards is both safer and cheaper. For teams running CrewAI as core business infrastructure where output quality is non-negotiable.

02 —

Pick a card by crew size

Four tiers, from one engineer tuning a crew to a multi-agent system in production. Prices are NexGPU's per-second-metered rates.

  • Solo development: run gpt-oss:20b or Qwen3-14B while you get the roles and tasks working

    RTX 3090 24GB$0.193/GPU-hr

    14GB of weights in 24GB leaves roughly 8–9GB of KV cache, enough for CrewAI's twenty-iteration loop to thrash around in — and it is the cheapest card in our fleet that runs a 20B-class model at all.

  • Fast iteration: same 20B tier, but you want each agent call quick enough to edit code and watch results live

    RTX 4090 24GB$0.540/GPU-hr

    Same 24GB as the 3090 but substantially more bandwidth and compute. A single crew run is dozens of model calls, so the per-call latency gap gets multiplied into something you genuinely feel.

  • Long-context production crew: Qwen3-30B-A3B at 256K with multiple agents resident

    RTX A6000 48GB$0.817/GPU-hr

    After 19GB of weights you still have nearly 29GB purely for KV cache — in long-context work the bottleneck was never the weights, and a 32GB card runs out first at 256K.

  • Quality first: gpt-oss:120b on a single card, for crews whose output ships to customers

    A100 PCIE 80GB$0.824/GPU-hr

    65GB of weights lands inside the officially stated single-80GB envelope, so no tensor parallel config and no changed launch flags — your 20b deployment script mostly carries over unchanged.

03 —

Getting CrewAI onto your own GPU from scratch

Four steps, all on one NexGPU instance, with no model call ever leaving the machine.

  1. 01

    Spin up a GPU, pick an inference image, pull weights

    Provision at console.nexgpu.net using the table above. With 2,000+ prebuilt images you can take the vLLM or PyTorch one and skip CUDA and driver version matching entirely. Then pull the weights — 14GB usually lands in a minute or two.

    ollama pull gpt-oss:20b && ollama serve
  2. 02

    Verify the OpenAI-compatible endpoint before anything else

    Do not skip this. The single most common failure in the CrewAI issue tracker is `litellm.APIConnectionError: OllamaException` and `[Errno 65] No route to host`. Nine times out of ten it is not CrewAI — the inference server bound to 127.0.0.1 while the caller is in a container, or the port never came up. Curl it, see the model list, then move on.

    curl http://127.0.0.1:11434/v1/models
  3. 03

    Install CrewAI and scaffold the project

    CrewAI 1.x manages dependencies through uv. `crewai create crew` emits JSONC scaffolding — agents in `agents/<name>.jsonc`, crew config in `crew.jsonc`. If the guide you are following still assumes the older layout, append `--classic` to get `config/agents.yaml` back.

    uv tool install crewai && crewai create crew nex_crew && cd nex_crew && crewai install
  4. 04

    Point the agent at your local endpoint and run it

    The key flag is `custom_openai=True` — it routes CrewAI through the native OpenAI-compatible client and bypasses LiteLLM completely. The same path works for Ollama, vLLM and LM Studio. Any non-empty api_key will do since local servers do not check it. Then `crewai run`.

    llm = LLM(model="gpt-oss:20b", custom_openai=True, base_url="http://127.0.0.1:11434/v1", api_key="local")

What a month of private CrewAI development actually costs

Take the most common path: RTX 3090 24GB running gpt-oss:20b at $0.193/GPU-hr. Six hours a day across twenty working days is 120 hours × $0.193 = $23.16. Storage at 30GB (14GB of weights plus project code plus the LanceDB memory store) is 30 × $0.414 = $12.42/month. Total: roughly $35.58/month — less than many people's monthly model API bill, with nothing leaving the machine. Per run it is even starker: a twelve-minute crew execution is 0.2 hours × $0.193 ≈ $0.04. Four cents. The quality tier prices out just as cleanly: A100 PCIE 80GB running gpt-oss:120b is $0.824 × 120 hours = $98.88, plus 80GB × $0.414 = $33.12 storage, for exactly $132.00 a month. Billing is metered per second and priced per hour, with no minimum, no setup fee and no quota request. Compute billing stops the moment the instance stops — storage keeps accruing until you destroy the volume, so clean up the ones you are done with.

04 —

FAQ

How much VRAM does CrewAI actually need to run locally?

The CrewAI framework itself needs 0GB of VRAM — it is a pure Python process using roughly 300–500MB of RAM. All GPU memory goes to the local model you attach: gpt-oss:20b is 14GB of weights on a 24GB card; Qwen3-14B at Q4_K_M is 9.3GB; Qwen3-30B-A3B at Q4 is 19GB and wants a 48GB card for long context; gpt-oss:120b is 65GB and needs 80GB. On NexGPU those four tiers map to RTX 3090 at $0.193, RTX 4090 at $0.540, RTX A6000 48GB at $0.817 and A100 PCIE 80GB at $0.824 — all metered per second, so if you size it wrong you just stop and switch cards.

My local open-source model keeps failing CrewAI tool calls. What is going on?

This is the classic self-hosting trap, and the issue tracker is full of it. The root cause is that many open models have substandard function calling: CrewAI checks capability via `supports_function_calling()`, and when that check misfires you can see `output_pydantic` injected as a native tool; some newer OSS models such as Olmo and Nemotron-3-nano ignore the system prompt outright and fail tool calling altogether; and some inference backends do not accept the `response_format` parameter at all. Practically: choose models with real native function calling and structured output (the gpt-oss and Qwen3 families both qualify), drop `max_iter` below its default of 20 so a loop cannot burn tokens forever, and turn on `verbose=True` to see exactly which iteration broke. The thing that makes this debugging painful is friction in swapping hardware — on NexGPU a different model on a different card is just a new instance, metered per second, so trial and error costs almost nothing.

pip cannot install litellm. Can I still use CrewAI?

Yes, and for local deployment you do not want it anyway. The `litellm` package was quarantined on PyPI, and CrewAI published a migration guide for exactly this: OpenAI, Anthropic, Google Gemini, Azure OpenAI and AWS Bedrock now go through native SDKs, so installing a provider extra like `crewai[openai]` is enough. For local inference, connect directly with `LLM(model=..., custom_openai=True, base_url=..., api_key=...)` and LiteLLM is never involved. That path covers Ollama, vLLM and LM Studio alike. All three backends ship ready to go in NexGPU's prebuilt images, so you can verify it the minute the box boots.

What is the difference between a Crew and a Flow, and which belongs in production?

A Crew is role-based collaboration: you define agents with a role, goal and backstory and let them divide work autonomously — good for exploratory problems where the path is not known ahead of time. A Flow is event-driven orchestration giving you conditional branching and deterministic execution paths, and the official production guidance is explicit that you should start with a Flow. Flows support the `@persist` decorator to save state, `restore_from_state_id` to resume after a crash or fork a new execution lineage, and `kickoff_async` for long-running work. In real projects the two nest: a Flow as the backbone, calling Crews at the nodes that need to diverge. Long-running work like this is exactly what per-second billing suits — you pay for the minutes you run, and on NexGPU compute billing stops when the instance does.

CrewAI memory is throwing an embedding dimension mismatch. How do I fix it?

Known upgrade compatibility issue. CrewAI's unified memory system defaults to LanceDB, stored under `./.crewai/memory` in your project (override with `CREWAI_STORAGE_DIR`). The OpenAI-side default embedder moved from `text-embedding-ada-002` at 1536 dimensions to `text-embedding-3-large` at 3072, so stores built under the old default cannot be read. Either reset the memory store and rebuild the index, or explicitly configure the older embedder. If you want nothing touching the network, switch the embedder to a local one: Ollama's `mxbai-embed-large` is 670MB at 335M parameters, or use HuggingFace's `all-MiniLM-L6-v2` — either uses almost no VRAM and can share the card with your main model. That co-resident setup fits comfortably on a 24GB card, which on NexGPU is an RTX 3090 at $0.193/GPU-hr.

Can CrewAI be fully private, with no data leaving the machine?

Yes, provided all three pieces are local: model inference, embeddings, and memory storage. Serve the model and embedder locally via the Ollama or vLLM setup above; memory is already LanceDB on local disk by default, with no managed vector database involved. As long as your agent's `llm` and embedder both point at 127.0.0.1 and no cloud provider API key is configured, inference data never leaves the box. NexGPU gives you SSH, Jupyter, a web terminal, a REST API and a CLI across 51 countries and regions, 1,175 verified rentable nodes, up to 14 GPUs per node and a maximum node VRAM of 2,152GB — so you can place the machine in whatever region your compliance requirements demand and then decide what network egress it gets. Questions go straight to us on Telegram, in English or Chinese, with no ticket queue.

More in AI agents and workflows

Every model guide

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.

Sign up freeTalk to sales

Sign up to browse live network pricing. No payment method required.