3D generative models

DreamFusion, deployed locally: what SDS text-to-3D actually costs in VRAM

The paper's DreamFusion runs on Imagen, which Google never released — so there is no official code and no checkpoint. What you can actually run are two Apache-2.0 reimplementations, threestudio and stable-dreamfusion: 6GB to start, 24GB to be comfortable, billed by the hour.

DreamFusion · self-hosted

DreamFusion came out of Google Research and UC Berkeley in September 2022 — Ben Poole, Ajay Jain, Jonathan T. Barron and Ben Mildenhall, arXiv 2209.14988 — and took an ICLR 2023 Outstanding Paper award. Its core idea is Score Distillation Sampling: train no 3D network at all, instead treat a randomly initialised NeRF (built on mip-NeRF 360, five ResNet blocks, 128 hidden units, Swish activation) as the parameters to optimise, render shaded 64×64 views from random camera poses, hand them to a frozen Imagen 64×64 base model for a denoising gradient, and backpropagate that gradient into the NeRF. Classifier-free guidance is cranked to ω=100, and timesteps are sampled from U(0.02, 0.98) to dodge the numerically unstable extremes. In the paper one object takes 15,000 iterations and about 1.5 hours on a 4-chip TPUv4 machine, each chip rendering a separate view at per-device batch size 1.

Here is the thing every tutorial should say first: DreamFusion is not open source and never will be. Imagen's weights were never published, so there is no official code and no downloadable model — anything named dreamfusion on Hugging Face is a community reimplementation. Two Apache-2.0 repos are what you actually install. threestudio swaps the prior for Stable Diffusion or DeepFloyd IF: dreamfusion-sd trains in ~6GB VRAM, dreamfusion-if needs ~15GB to extract text embeddings plus ~10GB in training, both defaulting to 10,000 iterations. ashawkey's stable-dreamfusion uses an Instant-NGP backbone at roughly 16GB, or ~24GB with --IF, and adds a DMTet refinement stage that hands you a mesh. Its README opens by warning that generation quality cannot match the original paper and that many prompts still fail badly — believe it.

Zoom out and the SDS lineage is now a whole tree: Magic3D's coarse-plus-refine split (the refine stage needs only ~5GB), ProlificDreamer's VSD (~30GB at 512×512, ~15GB at 64×64, under 10GB if you reuse the same base model for the LoRA), Fantasia3D, TextMesh, SJC, Magic123. MVDream generates four synchronised 256×256 views at once and largely puts the Janus multi-face problem to bed; DreamGaussian switches to 3D Gaussian Splatting and gets a single object down to minutes. A separate branch skips SDS entirely: TRELLIS (MIT, text-base 342M / text-large 1.1B / text-xlarge 2B, at least 16GB of VRAM required) and Hunyuan3D 2.1 (3.3B shape model at ~10GB, 2B PBR texture model at ~21GB, ~29GB combined) produce assets in a single forward pass. So the reason to run DreamFusion today is reproduction, ablation, teaching, or because you are modifying the SDS loss itself — not because it is the fastest path to an .obj. That kind of work is exactly what hourly rental is for.

01 —

Which DreamFusion you can actually run

There are no official weights. These are the reimplementation configs people really use, with the VRAM figures their own docs publish.

Version	Parameters	VRAM	Context	Notes
DreamFusion (original paper, Google Research)	Imagen 64×64 base + mip-NeRF 360	No downloadable weights; the paper ran on a 4-chip TPUv4	64×64 rendering, 15,000 iterations ≈ 1.5 hours	Method and paper only — Imagen was never released. The 397 prompts from the project gallery did survive: threestudio ships them in load/prompt_library.json, callable with the lib: prefix.
threestudio · dreamfusion-sd	Stable Diffusion guidance + HashGrid NeRF	~6GB in training	10,000 iterations by default, batch_size 1	The entry point — it starts on a 6GB card. But 6GB only means it runs; the paper used batch size 4, and more VRAM buys you geometric consistency.
threestudio · dreamfusion-if	DeepFloyd IF-I-XL + T5-XXL text encoder	~15GB to extract text embeddings, ~10GB in training	10,000 iterations, guidance scale 20	The docs say plainly that IF performs way better than Stable Diffusion. T5-XXL eats ~15GB even at 8-bit and is the bottleneck of the whole pipeline; you must accept the IF-I-XL-v1.0 licence on Hugging Face first.
stable-dreamfusion · -O (Instant-NGP)	SD 2.1 by default + Instant-NGP backbone	~16GB; --vram_O adds fp16 and attention slicing to shrink it	10,000 iterations by default; --backbone vanilla gives a pure-PyTorch path	Requires compiling the raymarching and gridencoder CUDA extensions on the box (bash scripts/install_ext.sh) — mismatch your CUDA toolkit and torch version and it simply will not build.
stable-dreamfusion · --IF / --dmtet	DeepFloyd IF guidance, DMTet 32/64/128 tetrahedral grids	~24GB with --IF	5,000 refinement iterations, initialised from df.pth	The only route that carries you from coarse NeRF through DMTet refinement to --test --save_mesh inside one repo. Take this path if you want an .obj at the end.
threestudio · prolificdreamer (the SDS successor)	VSD (variational score distillation), SD + LoRA pair	~30GB at 512×512; ~15GB at 64×64; under 10GB sharing one base model for the LoRA	Three stages; only the first 5,000 iterations render at 64×64	None of the oversaturated ω=100 look — noticeably better detail and colour. The price is memory: this is the one config on this page that genuinely wants a 48GB-plus card.

02 —

Pick the card from the config, not from a guess

Matched to the VRAM figures the repos publish themselves. A 30GB job does not go on a 24GB card.

Getting threestudio dreamfusion-sd running, with headroom to raise the batch
RTX 3090 24GB$0.193/GPU-hour
The official 6GB is the floor for starting, not for quality — data.batch_size is what stabilises the geometry. 24GB lets you go straight to 4-8 (the paper used 4) for under twenty cents an hour.
stable-dreamfusion -O (~16GB) plus 5,000 DMTet refinement steps to a mesh
Tesla V100 32GB$0.188/GPU-hour
The repo's author tested on a V100, so compatibility is the safest bet you can make with that codebase; 32GB holds the 16GB training run with room left for DMTet, at the lowest rate on the network.
The full dreamfusion-if route: ~15GB for T5-XXL embeddings, ~10GB training
RTX 4090 24GB$0.540/GPU-hour
The embedding extraction step is the memory peak, and 24GB absorbs it without CPU offload — offloading makes training extremely slow. Ada's fp16 throughput visibly shortens the 10,000 iterations.
ProlificDreamer VSD at 512×512 (~30GB), or sweeping the prompt library for seeds
A100 PCIE 80GB$0.824/GPU-hour
Priced within a cent of the 48GB RTX A6000 ($0.817/GPU-hour) for twice the memory: the 30GB 512×512 run fits with room for a second trial on another seed. Up to 14 GPUs per node, 2,152GB max node VRAM.

03 —

From zero to a mesh on NexGPU

Four steps, from boot to an .obj you can drag into Blender.

01
Spin up an instance with CUDA
Pick a PyTorch prebuilt image at console.nexgpu.net — PyTorch, vLLM, ComfyUI and Stable Diffusion are all among the 2,000+ images. threestudio's hard requirement is an NVIDIA card with at least 6GB VRAM and CUDA installed, Python >= 3.8 and PyTorch >= 1.12; the team tested on torch1.12.1+cu113 and torch2.0.0+cu118. Reach the box over SSH, Jupyter, or the web terminal.
```
ssh root@<node> -p <port>   # or open Jupyter / the web terminal
```
02
Install threestudio, ninja first
ninja cuts the CUDA extension compile time significantly, so install it before the requirements. If tiny-cuda-nn refuses to build, the documented fix is downgrading pip to 23.0.1. For the better-looking DeepFloyd IF route, accept the licence on the IF-I-XL-v1.0 model card on Hugging Face and run huggingface-cli login first. After the first run pulls the weights, set HF_HUB_OFFLINE=1 to stop it phoning home every launch.
```
git clone https://github.com/threestudio-project/threestudio && cd threestudio && pip install ninja && pip install -r requirements.txt
```
03
Run your first prompt
dreamfusion-sd trains for 10,000 iterations by default and drops validation renders into the trial directory as it goes. random_aug=true replaces the background with a random colour half the time, which stops the model from treating the background as part of the object. Pressing Ctrl+C once jumps straight to the test stage and renders the 360-degree orbit video; a second press actually quits. To reproduce the paper's 397 prompts, pass them as lib:keyword1_keyword2.
```
python launch.py --config configs/dreamfusion-sd.yaml --train --gpu 0 system.prompt_processor.prompt="a delicious hamburger" data.batch_size=4 system.background.random_aug=true
```
04
Export the mesh, then stop the box
--export with mesh-exporter writes obj+mtl. Raise isosurface_threshold if the scene is full of floaters; switch to mc-cpu at resolution 256 for a finer surface. Pull the files off, then stop the instance: compute is metered per second and stops billing the moment the instance stops, while storage keeps billing until it is destroyed — so clear out the intermediate checkpoints while you are there.
```
python launch.py --config outputs/dreamfusion-sd/<trial>/configs/parsed.yaml --export --gpu 0 resume=outputs/dreamfusion-sd/<trial>/ckpts/last.ckpt system.exporter_type=mesh-exporter system.geometry.isosurface_method=mc-cpu system.geometry.isosurface_resolution=256
```

What one object actually costs

Run threestudio's dreamfusion-sd for its default 10,000 iterations on an RTX 3090 24GB at $0.193/GPU-hour. Budget 1.5 hours, the same order as the paper's 15,000 iterations at ~1.5 hours: 0.193 × 1.5 ≈ $0.29 per object. Since stable-dreamfusion's own README admits many prompts fail badly, real practice is sweeping twenty and keeping the ones that worked: 20 × 0.29 ≈ $5.79, less than lunch. For paper-grade quality, move to dreamfusion-if on an RTX 4090 24GB ($0.540/GPU-hour) — call it 2 hours, so 0.540 × 2 = $1.08 an object. ProlificDreamer's 512×512 VSD wants ~30GB, which means an A100 PCIE 80GB ($0.824/GPU-hour): three hours is 0.824 × 3 ≈ $2.47. Keeping 20GB of artefacts (checkpoints, orbit videos, meshes) for a month runs 20 × $0.414 = $8.28, about $0.28 a day; compute is metered per second and stops when the instance stops, while storage runs until you destroy it — so clean up once the mesh is exported. No minimum rental, no setup fee, no quota request. The honest framing: SDS work means re-rolling seeds and re-tuning lambda_sparsity over and over, and renting that by the hour beats buying a card for one ablation study.

04 —

FAQ

Is DreamFusion open source? Why can't I find the weights anywhere?

It is not open source, and it will not be. DreamFusion depends on Google's Imagen as its 2D prior, and Imagen's weights were never published — so there is no official code and no checkpoint, only result videos on the project page. What you can install are two Apache-2.0 community reimplementations: threestudio and ashawkey's stable-dreamfusion, both of which substitute Stable Diffusion or DeepFloyd IF for Imagen. Both run straight out of NexGPU's PyTorch prebuilt images, with no quota request needed.

What is the minimum VRAM for running DreamFusion locally? Will a 6GB card really do it?

threestudio states dreamfusion-sd trains in ~6GB VRAM and its install notes only require a 6GB NVIDIA card, so yes, in principle. But 6GB means batch_size 1, where the paper used 4, and small batches are exactly where the geometry falls apart. The realistic figures are the DeepFloyd IF route at ~15GB for embeddings plus ~10GB training, stable-dreamfusion's -O mode at ~16GB, and --IF at ~24GB. If you want results, 24GB is the comfortable floor — NexGPU's RTX 3090 24GB is $0.193/GPU-hour, cheaper in both money and hours than fighting a 6GB card.

My object came out with several faces (the Janus multi-face problem). How do I fix it?

Change the seed first — it is the top official suggestion, and often the overall geometry is right and you just drew a bad roll. Then enable Perp-Neg with system.prompt_processor.use_perp_neg=true, supported for both SD and IF guidance. Still broken? Reach for D-SDS: set system.guidance.grad_clip=[0,0.5,2.0,10000] for score debiasing alongside use_prompt_debiasing=true. For a structural fix, switch to MVDream, which generates four synchronised 256×256 views at once and never gives the model the chance to grow a second face. Seed sweeps mean many reruns — running several per-second-billed instances in parallel on NexGPU beats waiting through them one at a time.

Why does one generation take an hour or two? Can't it be seconds like text-to-image?

Because SDS is not inference, it is training. DreamFusion optimises a fresh NeRF from scratch for every prompt — 15,000 iterations and about 1.5 hours on a 4-chip TPUv4 in the paper, 10,000 iterations by default in threestudio. The documented speedups are token_merging=true (dramatically faster and lighter on Stable Diffusion guidance, at some quality cost), attention slicing which costs about 20% throughput, and sequential CPU offload which the docs call extremely slow — avoid that last one if you can. If you want assets in seconds, use a feed-forward model like TRELLIS or Hunyuan3D instead. NexGPU meters per second, so a run that takes 1 hour 3 minutes is not billed as two.

Is DreamFusion still worth running, or should I just use TRELLIS or Hunyuan3D?

If you want assets, take the short path: TRELLIS is MIT-licensed with text-base 342M, text-large 1.1B, text-xlarge 2B and image-large 1.2B variants, requires at least 16GB of VRAM, and emits Radiance Fields, 3D Gaussians and meshes from one forward pass; Hunyuan3D 2.1 pairs a 3.3B shape model at ~10GB with a 2B PBR texture model at ~21GB, ~29GB together. DreamFusion's value today is the method itself — SDS is the foundation Magic3D, ProlificDreamer and DreamGaussian are all built on, so reproduction, ablation, teaching or loss-function surgery all run through it. Both kinds of work fit on NexGPU: 16GB needs go on an RTX 3090 24GB at $0.193/GPU-hour, 29GB needs on an RTX 5090 32GB at $0.723/GPU-hour.

How do I turn the trained result into an .obj I can open in Blender?

In threestudio, pass --export with system.exporter_type=mesh-exporter for obj+mtl, or add system.exporter.fmt=obj for an obj with vertex colours; system.exporter.save_uv=false skips UV unwrapping when you just want a quick look. Raise system.geometry.isosurface_threshold if floaters came along for the ride, and use isosurface_method=mc-cpu with isosurface_resolution=256 when you want detail. In stable-dreamfusion it is --test --save_mesh, and running the --dmtet refinement for 5,000 steps first gives a noticeably cleaner mesh. Export leans on CPU and RAM more than VRAM — on NexGPU you scp the result off and stop the instance, and compute billing stops with it.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.