Autonomous agent framework

Self-hosting SuperAGI: the framework, the model, and where the VRAM actually goes

SuperAGI is an MIT-licensed autonomous agent framework that comes up with one docker compose command. It barely touches a GPU on its own — the card you rent is decided by the model you bolt onto the back of it.

SuperAGI · self-hosted

TransformerOptimus shipped SuperAGI in 2023 and it collected 17.7k GitHub stars under an MIT licence. What it really did was put a product shell around agents: a FastAPI backend, a Celery task queue, PostgreSQL 15, Redis Stack for long-term memory, a Next.js GUI, and an Nginx proxy on port 3000 — all of it brought up by a single compose file. Toolkits, an agent-template marketplace, an APM performance console, webhook event triggers, scheduled runs, Python and Node client SDKs: things other frameworks were still hand-rolling in notebooks, SuperAGI turned into buttons.

Which means "how much VRAM does SuperAGI need" is the wrong question as asked. On the default path it just calls a remote API through openai==0.27.7 and the whole stack runs on CPU. VRAM only enters the picture when you localise the model, and SuperAGI gives you two doors. The built-in one is llama-cpp: Dockerfile-gpu is based on nvidia/cuda:12.1.1-devel-ubuntu22.04 and compiles llama_cpp_python==0.2.7 with CMAKE_ARGS="-DLLAMA_CUBLAS=on"; the model file sits at /app/local_model_path inside the container, GPU_LAYERS defaults to -1 (offload everything), the LocalLLM class defaults to context_length 4096, and the config template pins MAX_MODEL_TOKEN_LIMIT at 4032. The second door is a sidecar OpenAI-compatible endpoint you point OPENAI_API_BASE at. That second door is the one that actually works today, for reasons below.

One more thing has to be said plainly. The last tagged release of the main repo is v0.0.14 from 2024-01-16 (whose headline feature was local LLM support with multi-GPU), and the last commit on main lands on 2025-01-22. The people did not vanish — they moved. Ishaan Bhola, Adithyan Krishnan and Mukunda NS published SuperScout in August 2026 (arXiv:2608.04804). Its core weight, SuperScout-7B, is a LoRA fine-tune of Qwen2.5-Coder-7B-Instruct that solves 159 of 266 tasks on the full Python slice of SWE-bench Pro — edging the best single model's 158 — at roughly one fifth the total cost per solve. If you want to put something from the SuperAGI lineage on your own card today, this 7.6B is the weight worth the slot.

01 —

The SuperAGI lineage: frameworks and weights

Framework, models and successor projects have wildly different hardware needs — read them separately

Version	Parameters	VRAM	Context	Notes
SuperAGI v0.0.14	Framework only, no model weights	No VRAM; budget 8GB+ system RAM for the Docker stack	MAX_MODEL_TOKEN_LIMIT 4032	The last tagged release, 2024-01-16, and it was mostly about local LLM integration plus multi-GPU support. MIT licence; the last commit on main is 2025-01-22. If you want to reproduce that generation of agent workflows, this is the version to install.
SuperAGI/SuperScout-7B	7.6B (LoRA fine-tune of Qwen2.5-Coder-7B-Instruct, r=64, alpha=128)	bf16 ~15.2GB / FP8 ~7.6GB / AWQ INT4 ~5.6GB; add ~1.9GB for a full 32K KV cache	32,768	A repository scout: it explores the codebase, localises the implicated files, attempts a failing reproduction, and emits a structured handoff for a frontier fixer. It deliberately does not write patches. Apache-2.0, bf16 across four safetensors shards.
SuperAGI/SAM (SAM-7B)	7B (Mistral-7B fine-tune)	F16 ~14.5GB / community GGUF Q4_K_M ~4.4GB	~8K effective (Mistral-7B sliding window)	A December 2023 reasoning-specialised model trained entirely on data generated by open-source LLMs. The card is blunt about it: not suitable for conversation or simple Q&A, better at task breakdown and reasoning. GGUF quantisations exist from RichardErkhov and featherless-ai.
SuperAGI/mistral-7B-PoSE-32k	7B	fp16 ~14.5GB	32K (PoSE positional extrapolation)	An experiment that stretched Mistral-7B's window to 32K with PoSE. Agent long-term memory blows through context faster than anything else, and this was the long-context control arm. Mostly of research interest now.
SuperCoder (the team's successor project)	No weights of its own; bring your own LLM key	No VRAM for the app itself; the optional Context Engine runs under docker compose	Whatever your chosen model provides	Now rewritten as a local-first Tauri 2 desktop coding agent that speaks the OpenAI chat-completions and Anthropic Messages APIs natively, with your code staying on your machine. MIT. This is the SuperAGI-family project still moving.

02 —

Which card to rent

Pick by what you are actually running — don't buy VRAM for the framework itself

SuperAGI framework only, with a 7B GGUF as the local LLM (Q4_K_M ~4.7GB, 4096 context)
Tesla V100 32GB$0.188/GPU-hr
llama-cpp's cuBLAS backend is perfectly happy on Volta, and 32GB leaves enormous headroom over a 4-bit 7B — this is the cheapest line on the whole rate card.
Single-card SuperScout-7B evaluation on AWQ INT4 or your own Q4_K_M quant
RTX 3090 24GB$0.193/GPU-hr
INT4 weights land around 5.6GB, so 24GB swallows the model and a full 32K context without thinking, and $0.193 an hour makes a complete eval run essentially free.
SuperScout-7B in production at full bf16 with the whole 32,768 context
RTX 4090 24GB$0.540/GPU-hr
15.2GB of weights plus ~1.9GB for one 32K sequence is about 17.1GB — it fits on 24GB with room for concurrency, and Ada is the best-optimised vLLM target in this class.
Batch scouting with many concurrent sequences and a deep queue
RTX A6000 48GB$0.817/GPU-hr
SuperScout's KV cache costs about 56KiB per token, so 48GB minus 15.2GB of weights holds well over a dozen concurrent 32K sequences — it halves wall-clock time on a SWE-bench Pro sweep.

03 —

From zero to running

Bring up the framework, attach the model, then deal with the two traps everyone hits

01
Bring up the full SuperAGI stack
Clone the repo, copy config_template.yaml to config.yaml, and build with the GPU compose file. It starts six services — backend, celery, gui, super__redis, super__postgres, proxy — with backend and celery both holding an nvidia GPU reservation (count: all). Open http://localhost:3000 when the build finishes and you can create agents through the UI. The first build compiles llama-cpp-python against cuBLAS, so give it fifteen minutes.
```
git clone https://github.com/TransformerOptimus/SuperAGI.git && cd SuperAGI && cp config_template.yaml config.yaml && docker compose -f docker-compose-gpu.yml up --build
```
02
Serve SuperScout-7B as an OpenAI-compatible endpoint with vLLM
This is the least painful way to get a local model into SuperAGI. One hard rule from the model card: do not serve SuperScout greedily. Greedy decoding measures 0.1104 file-localisation accuracy against 0.3058 for a single sampled draw at temperature 0.9 — a 2.65x gap. Pin top_p=1.0 and top_k=-1 and take exactly one draw. The 32768 max-model-len is the max_position_embeddings ceiling in its own config.
```
vllm serve SuperAGI/SuperScout-7B --served-model-name superscout-7b --dtype bfloat16 --max-model-len 32768 --gpu-memory-utilization 0.90 --port 8000
```
03
Point SuperAGI at the local endpoint
SuperAGI ships the 0.27.x generation of the OpenAI SDK, which honours an api_base override, so rewriting OPENAI_API_BASE in config.yaml to your local vLLM /v1 is all it takes; put any non-empty string in OPENAI_API_KEY and restart backend and celery. Keep the model name aligned with --served-model-name, then register it in the Models console so it shows up when you build an agent.
```
sed -i 's|^OPENAI_API_BASE:.*|OPENAI_API_BASE: http://host.docker.internal:8000/v1|' config.yaml && docker compose -f docker-compose-gpu.yml restart backend celery
```
04
Optional: use the built-in GGUF path, and dodge the version trap
The built-in local inference is llama_cpp.Llama loading the file at /app/local_model_path, with GPU_LAYERS defaulting to -1 to offload the whole model, and it constrains output through superagi/llms/grammar/json.gbnf so small models emit valid JSON — a genuinely smart touch that makes tool calls far more reliable. The trap: requirements pin llama_cpp_python==0.2.7, a September 2023 build that only reads GGUF files from that era. Anything quantised by a recent llama.cpp will usually fail with unknown model architecture or an unsupported GGUF version. Either pick a period-correct quant, or shell into the container and rebuild llama-cpp-python with cuBLAS against a current version.
```
docker compose -f docker-compose-gpu.yml cp ./model-q4_k_m.gguf backend:/app/local_model_path
```

What the run actually costs

Take an RTX 4090 24GB at $0.540/GPU-hr and push SuperScout-7B through the full 266-task Python slice of SWE-bench Pro. Pulling 15.2GB of weights takes roughly 10 minutes, vLLM cold start about 3 minutes, and 266 tasks at an assumed average of 4 minutes of single-card time each is 1064 minutes. That is (10 + 3 + 1064) / 60 = 17.95 hours, and 17.95 x $0.540 = $9.69 — about $0.036 per task. The 4-minutes-per-task figure is an assumption; your repository sizes will move it, but not by an order of magnitude. If you only want to click through the SuperAGI UI and see what an agent workflow looks like, drop to an RTX 3090 24GB at $0.193/GPU-hr: one hour is $0.193, metered per second, no minimum, no setup fee, no quota request, and compute billing stops the moment you stop the instance. Weights plus images run around 40GB, which at the $0.414/GB-month median is $16.56/month — and note that storage keeps billing after the instance stops, until you destroy it. Pulling handoff JSON back out costs $0.0081/GB in egress, so a few hundred megabytes rounds to nothing. NexGPU has 1,175 verified rentable nodes and 2,498 GPUs across 51 countries and regions, up to 14 GPUs per node and 2,152GB max node VRAM, so batch runs never wait for capacity.

04 —

FAQ

Is SuperAGI still maintained? Is there any point installing it now?

Depends what you want from it. The last tagged release is v0.0.14 from 2024-01-16 and the last commit on main is 2025-01-22; the 17.7k stars and the MIT licence are still there, the code still runs, and the toolkit/marketplace design remains a genuinely good study in agent orchestration. But if you want something the team is actively shipping, look at SuperCoder (rewritten as a Tauri 2 desktop app) and SuperScout. We cover both paths: spin up an RTX 3090 24GB on NexGPU at $0.193/GPU-hr and you can install both inside an hour before deciding.

How much VRAM does self-hosting SuperAGI really need?

The framework needs none. It is a CPU stack — FastAPI, Celery, PostgreSQL 15, Redis Stack, Next.js — that calls a remote API by default. All the VRAM goes to whatever model you attach: a 7B Q4_K_M GGUF is about 4.7GB, SuperScout-7B in bf16 is 15.2GB, plus roughly 1.9GB of KV for a full 32K context. So the real question is which model you're attaching. Once you know, pick the card by VRAM on NexGPU — 75 GPU models from a Tesla T4 16GB at $0.298/GPU-hr up to an H200 141GB at $6.660/GPU-hr.

Can SuperAGI run fully local without an OpenAI key?

Yes, two ways. The built-in path has llama_cpp.Llama load a GGUF from /app/local_model_path with GPU_LAYERS at -1 for full offload and json.gbnf grammar constraints on the output; the catch is the llama_cpp_python==0.2.7 pin, which chokes on modern quants. The sturdier route is a vLLM sidecar exposing an OpenAI-compatible endpoint with OPENAI_API_BASE in config.yaml pointed at it — the bundled openai 0.27.7 honours that override. NexGPU's 2,000+ prebuilt images already include vLLM and PyTorch, so there's no CUDA setup to do.

How much VRAM does SuperScout-7B need — will 24GB do?

It will, comfortably. It is a 7.6B-parameter fine-tune of Qwen2.5-Coder-7B-Instruct, 15.2GB in bf16. The architecture is 28 layers with grouped-query attention mapping 28 query heads onto 4 KV heads, which works out to about 56KiB of KV per token — roughly 1.9GB for one sequence at the full 32,768 context, so about 17.1GB all in. An RTX 4090 24GB at $0.540/GPU-hr holds that with room for several concurrent streams. For batch work step up to an RTX A6000 48GB at $0.817/GPU-hr and the KV pool gets an order of magnitude roomier.

Why can't I serve SuperScout-7B with greedy decoding?

Because it was trained as a scout, and sampling diversity is what lets it find the right files. The card publishes the numbers: 0.1104 file-localisation accuracy greedy versus 0.3058 with a single sampled draw at temperature 0.9, a 2.65x difference. Pin temperature=0.9, top_p=1.0, top_k=-1, one draw. One more trap worth knowing: of 266 handoffs, 249 claimed a verified reproduction and only 50 were genuine, so sandbox-verify before you trust a claim. Re-running those sweeps on NexGPU is metered per second, so a full parameter scan costs a few dollars.

Is SAM-7B still worth downloading?

As a research control, yes; as a production model, no. It is a December 2023 Mistral-7B fine-tune trained entirely on open-source-generated data, and the card states outright that it is not for conversation or simple Q&A — only task breakdown and reasoning. F16 is about 14.5GB and the community GGUF Q4_K_M about 4.4GB. The same org also released Veagle (multimodal, arXiv:2403.08773) and mistral-7B-PoSE-32k (a 32K context-extension experiment). To benchmark all of them side by side, a Tesla V100 32GB at $0.188/GPU-hr is the cheapest bench you'll find — the console is at console.nexgpu.net, with SSH, Jupyter, web terminal, REST API and CLI access, and bilingual support over Telegram with no ticket queue.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.