Text-to-image model

Self-hosting AuraFlow v0.3: 16.7GB of fp16 weights that fit on one 24GB card

fal's 6.8B flow-matching text-to-image model, with code and weights both under Apache 2.0. This page lays out the real memory footprint, the hard 1536 resolution ceiling, every GGUF tier, and the fine-tuning floor — then tells you exactly which card to rent.

AuraFlow · self-hosted

AuraFlow was built by fal together with @cloneofsimo (Simo Ryu) and @isidentical as a pure flow-matching text-to-image model. The skeleton starts from Stable Diffusion 3 but does not copy MMDiT wholesale: most joint-attention blocks were replaced with wider single-tower DiT blocks, landing on 4 MMDiT layers plus 32 single DiT layers — 36 in total, hidden dim 3072 (12 heads × 256 head dim) — which bought roughly 15% better model FLOPs utilisation. There is no CLIP anywhere: text conditioning comes from the EleutherAI/pile-t5-xl encoder (joint_attention_dim 2048), the VAE is SDXL's 4-channel AutoencoderKL, and sampling runs on FlowMatchEulerDiscreteScheduler at a default 50 steps and guidance_scale 3.5. Training used muP for zero-shot learning-rate transfer across scales and stepped through 256 → 512 → 1024, reaching 0.64 on GenEval and 0.703 with a prompt-enhancement pipeline.

Straight talk: the official version line stops at v0.3. The fal/AuraFlow-v0.3 weights have not been touched since August 2024 and the README still says beta. The ecosystem around it did not stop, though. diffusers merged AstraliteHeart's rewrite of AuraFlowTransformer2DModel in April 2025 so torch.compile no longer recompiles on every resolution change; city96 published a full GGUF ladder; p1atdev ships fp8_e4m3fn and bnb-nf4 conversions; SimpleTuner maintains a dedicated AuraFlow quickstart whose "adamw_bf16 is the most reliable" advice comes straight from the author of Pony Flow. The reason is simple: if you need to genuinely own the weights — ship them commercially, fork the architecture, strip it down — there are very few large text-to-image models with Apache 2.0 on both code and weights, and AuraFlow is the most workable of them.

So the answer to "how much VRAM does AuraFlow need" is concrete. In fp16 the transformer shards are 9.94GB + 3.77GB = 13.7GB, the Pile-T5-XL encoder adds 2.95GB, and with the VAE the official single-file aura_flow_0.3.safetensors comes to 16.5GB. A 16GB card cannot hold that without quantisation or offloading — 24GB is the real floor. If you need less, the ladder already exists: GGUF runs from Q8_0 at 7.35GB down to Q2_K at 2.4GB, with Q4_K_M at 4.05GB as the sweet spot. On NexGPU an RTX 3090 24GB is $0.193 per GPU-hour, billed per second with no minimum and no quota request, so pulling 16.5GB of weights and rendering your first image costs a few cents.

01 —

AuraFlow releases and quantisation tiers

The official line ends at v0.3, but quantised and pruned forks push the floor down to 2.4GB

Version	Parameters	VRAM	Context	Notes
fal/AuraFlow-v0.3	6.8B (tagged 7B on Hugging Face)	fp16 ~16.7GB (transformer 13.7GB + Pile-T5 2.95GB) / single file 16.5GB / fp32 transformer 27.4GB	1536 max side / 256-token prompt	The end of the line. Trained with more compute than v0.2 and fine-tuned on aesthetic data; handles any aspect ratio up to 1536 per side. Still labelled beta.
fal/AuraFlow-v0.2	6.8B	fp16 transformer ~13.7GB	Mostly 1024×1024	The checkpoint ComfyUI's official example page still points at — drop aura_flow_0.2.safetensors into ComfyUI/checkpoints and the native nodes work.
fal/AuraFlow (v0.1)	6.8B	fp16 transformer ~13.7GB	1024×1024	The launch release; the bitsandbytes 8-bit example in the diffusers docs still loads it. Useful for architecture reproduction and A/B baselines.
city96/AuraFlow-v0.3-gguf	6.8B (transformer only)	F16 13.7GB / Q8_0 7.35GB / Q6_K 5.7GB / Q5_K_M 4.85GB / Q4_K_M 4.05GB / Q3_K_M 3.13GB / Q2_K 2.4GB	Same as v0.3, 1536 max side	Loads natively in diffusers via GGUFQuantizationConfig + from_single_file. Note it quantises only the transformer — the Pile-T5 encoder and VAE still come from fal/AuraFlow-v0.3.
p1atdev/AuraFlow-v0.3-fp8 / -bnb-nf4	6.8B	fp8_e4m3fn roughly halves fp16; nf4 drops another tier	Same as v0.3	The fp8 build casts every linear weight in the flow transformer to torch.float8_e4m3fn except t_embedder, final_linear and modF. Best value on Ada and Blackwell cards.
jimmycarter/auraflow-3.8b / auraflow-4.8b	3.8B / 4.8B	Proportionally lower — roughly 7.6GB / 9.6GB in fp16	Same architecture	An experimental prune that excises the middle 18 of the 36 layers. The author is explicit that it emits corrupt images until you fine-tune it for a few thousand steps — a training starting point, not a finished model.

02 —

Which GPU to rent for AuraFlow

Matched to VRAM honestly: a 16GB card will not hold the full fp16 stack, so don't try

GGUF Q4_K_M or fp8 on one card, to check whether the style suits you
RTX 3090 24GB$0.193/GPU-hour
The cheapest 24GB card on the platform; Q4_K_M's 4.05GB plus the 2.95GB text encoder peaks under 10GB, leaving the rest of the card free for a 1536 long side.
Full fp16 weights resident, producing 1024×1024 at 50 steps
RTX 4090 24GB$0.540/GPU-hour
16.7GB of fp16 weights sit on the card with no enable_model_cpu_offload, and Ada's fp16/bf16 throughput keeps a 36-layer DiT pass well under a minute per image.
Full 1536×1536 output with torch.compile and batched concurrency
RTX 5090 32GB$0.723/GPU-hour
Attention activations at the full 9216 patch tokens plus the compile cache need headroom beyond 24GB; 32GB lets you run a real batch and fullgraph compilation at once.
LoKr / LyCORIS fine-tuning with SimpleTuner
RTX A6000 48GB$0.817/GPU-hour
The documented 24GB floor only works with grouped offloading; 48GB lets you switch offloading off, enable int8-quanto and train 1024 buckets directly.

03 —

AuraFlow running on NexGPU in four steps

From an empty instance to your first 1536×768 image, with weights and inference on the same box

01
Spin up an instance and install dependencies
The console is at console.nexgpu.net. Pick a PyTorch prebuilt image, start an RTX 3090 or 4090, and connect over SSH, Jupyter or the web terminal. AuraFlow's tokenizer needs sentencepiece and protobuf — leaving those out is the single most common ImportError people hit here.
```
pip install -U "diffusers>=0.31" transformers accelerate protobuf sentencepiece
```

Load the pipeline in full fp16

Passing variant="fp16" pulls only the 9.94GB + 3.77GB fp16 shards instead of the 27.4GB fp32 trio, halving both download time and disk. The official defaults are 50 steps and guidance_scale 3.5; any aspect ratio works as long as neither side exceeds 1536.

from diffusers import AuraFlowPipeline
import torch

pipe = AuraFlowPipeline.from_pretrained("fal/AuraFlow-v0.3", torch_dtype=torch.float16, variant="fp16").to("cuda")
image = pipe("rampage of the iguana character riding F1, cinematic movie poster", width=1536, height=768, num_inference_steps=50, guidance_scale=3.5).images[0]
image.save("out.png")

Short on VRAM? Swap in a GGUF transformer

Q4_K_M compresses the 13.7GB transformer to 4.05GB while the rest of the pipeline still loads from fal/AuraFlow-v0.3. On Turing hardware such as the T4, change compute_dtype from bfloat16 to float16 — Turing has no native bf16.

from diffusers import AuraFlowPipeline, AuraFlowTransformer2DModel, GGUFQuantizationConfig
import torch

transformer = AuraFlowTransformer2DModel.from_single_file(
    "https://huggingface.co/city96/AuraFlow-v0.3-gguf/blob/main/aura_flow_0.3-Q4_K_M.gguf",
    quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16),
    torch_dtype=torch.bfloat16,
)
pipe = AuraFlowPipeline.from_pretrained("fal/AuraFlow-v0.3", transformer=transformer, torch_dtype=torch.bfloat16).to("cuda")

04
Turn on torch.compile — or just use ComfyUI
use_duck_shape must be disabled or every resolution change triggers a recompile; the documented gain is up to 100% at low resolutions and around 30% at 1536×1536. If you would rather not write code, boot the ComfyUI prebuilt image, drop aura_flow_0.3.safetensors into ComfyUI/checkpoints and use the comfy_workflow.json shipped in the repo — GGUF weights additionally need city96's ComfyUI-GGUF loader node.
```
import torch
torch.fx.experimental._config.use_duck_shape = False
pipe.transformer = torch.compile(pipe.transformer, fullgraph=True, dynamic=True)
```

What a full day of AuraFlow actually costs

Here is the whole bill. Evaluation: RTX 3090 24GB at $0.193/GPU-hour; pulling the 16.5GB single-file checkpoint bills roughly 0.1 hour, about $0.019. At 1024×1024, 50 steps, fp16, budgeting a conservative 30 seconds per image gives 120 images an hour, so $0.193 ÷ 120 ≈ $0.0016 per image, and a 1000-image run takes about 8.3 hours × $0.193 = $1.60. Fine-tuning: move to an RTX A6000 48GB at $0.817/GPU-hour for a SimpleTuner LoKr run — 6 hours × $0.817 = $4.90. Storage: keep a 20GB volume for weights and outputs at $0.414/GB-month × 20 = $8.28/month, which over 3 days is 8.28 ÷ 30 × 3 ≈ $0.83, and billing stops the moment you destroy the volume. Together that is 1.60 + 4.90 + 0.83 ≈ $7.33 — a complete evaluate-then-fine-tune-then-generate loop for under eight dollars. One billing note: compute is metered per second and priced per hour and stops when the instance stops, while storage keeps billing until the volume is destroyed. No minimum, no setup fee, no quota ticket.

04 —

FAQ

What is the minimum VRAM to run AuraFlow locally?

There are two answers. Full fp16: the transformer's two shards are 9.94GB + 3.77GB, the Pile-T5-XL encoder is 2.95GB, and with the VAE that is about 16.7GB of weights before activations — 24GB is the real floor. GGUF route: Q4_K_M's transformer is only 4.05GB, but you cannot escape the 2.95GB text encoder, so peak usage lands around 8–10GB and a 12GB card can generate. On NexGPU an RTX 3090 24GB is $0.193/GPU-hour and runs either path, so you can measure instead of guessing before you buy hardware.

Can a 16GB Tesla T4 run AuraFlow?

Not in full fp16 — 16.7GB of weights simply exceeds 16GB, so you must quantise the transformer or use enable_model_cpu_offload to swap the encoder out, which costs a lot of speed. Turing also has no native bf16, so the compute_dtype in the GGUF snippet from the diffusers docs needs to become torch.float16. The more practical objection is the price: on NexGPU a T4 16GB is $0.298/GPU-hour, more than the $0.193 RTX 3090 24GB. Just take the 3090.

What is AuraFlow's maximum resolution, and what happens above it?

The ceiling is written into the transformer config: pos_embed_max_size = 9216. With patch_size 2 on top of an 8× downsampling VAE, that allows at most a 96 × 96 grid of patch tokens, which is exactly 1536 × 1536. The official README only promises arbitrary aspect ratios within 1536 per side, because beyond that token budget there is no positional embedding left to select. If you want the full 1536 alongside torch.compile and a real batch size, NexGPU's RTX 5090 32GB at $0.723/GPU-hour has the headroom.

How does AuraFlow compare with FLUX.1 or SD3.5 — which should I pick?

The hardest difference is licensing: AuraFlow ships code and weights under Apache 2.0, so commercial use, redistribution and architectural forks carry no extra terms to read. The cost is specific too — it uses SDXL's 4-channel VAE (in_channels = 4 in the config), so fine detail and in-image text top out below newer 16-channel latent models, and max_sequence_length = 256 truncates very long prompts. Whether that trade is worth it is fastest to settle empirically: rent one card for an hour, run the same prompt set through all three on the same machine, and stop the instance when you are done. NexGPU bills per second, so the comparison costs pocket change.

Can I fine-tune AuraFlow or train a LoRA on it?

Yes — SimpleTuner ships a dedicated AuraFlow quickstart. The recommended path is LoKr (LyCORIS) rather than plain LoRA; the documented floor is a 24GB GPU with grouped offloading, with the leanest configuration around 20–22GB, while full-rank tuning needs DeepSpeed and the docs themselves discourage it. Quantisation options include int8-quanto, int4-quanto, nf4-bnb and fp8-torchao; Lion trains faster but adamw_bf16 is the most stable. Two traps worth remembering: aspect bucketing is constrained by the positional embeddings so multi-resolution training gives unpredictable results, and cards at 16GB or below OOM during pre-caching. NexGPU's RTX A6000 48GB at $0.817/GPU-hour sidesteps the offloading entirely, and A100 PCIE 80GB at $0.824/GPU-hour covers full-rank runs.

Is AuraFlow still maintained, or has it been superseded?

The official line genuinely stops at v0.3 — the weights have not been updated since August 2024, the README still says beta, and no v0.4 appeared. The ecosystem kept moving, though: diffusers merged the AuraFlowTransformer2DModel rewrite in April 2025 to make torch.compile resolution-agnostic, the GGUF, fp8 and nf4 forks are all complete, and downstream projects like Pony Flow picked it as their base. In other words its role today is a clean-licensed, clearly structured, well-tooled base you can modify — not a SOTA chase. The cheapest way to decide whether it fits your use case is an hour on a NexGPU RTX 3090 at $0.193 running your own prompt set, then stopping the instance.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.