3D Generation / Novel View Synthesis

Self-hosting Zero123: one image, a full orbit of views

The original Zero123 demo asks for roughly 22GB of VRAM. Zero123++ v1.2 does its job in about 5GB at fp16 — a 4x spread, and picking the wrong card just burns money. Here is every variant's real footprint, its licence boundary, and the path that actually ships.

Zero123 · self-hosted

Zero123 — paper name Zero-1-to-3 — comes out of Columbia University's CVLab together with Toyota Research Institute, authored by Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Carl Vondrick and others. The code lives at github.com/cvlab-columbia/zero123 under MIT. The idea is deceptively plain: hand it one photo of an object plus a relative camera pose (R, T), and it diffuses what that object looks like from the new viewpoint. It is built on Lambda Labs' image-conditioned Stable Diffusion checkpoint, then fine-tuned on Objaverse renders (views_release.tar.gz, ODC-By 1.0). Four checkpoints shipped — 105000.ckpt, 165000.ckpt, 230000.ckpt and 300000.ckpt — one per fine-tuning milestone, with the longest run costing roughly 6,000 A100 hours.

The trouble is that "Zero123" now names four different things, made by different teams. The original is Columbia's. Zero123-XL is the same architecture retrained on the ten-million-scale Objaverse-XL. Stable Zero123 is Stability AI's rebuild with reworked render data and elevation conditioning, shipping as stable_zero123.ckpt under a non-commercial research licence — with a separate Stable Zero123C under the StabilityAI Community License for commercial use. And Zero123++ is an entirely different group: SUDO-AI, with Ruoxi Shi and Hao Su among the authors, at github.com/SUDO-AI-3D/zero123plus, Apache 2.0 code and CC-BY-NC 4.0 weights. VRAM across these four runs from 5GB to 22GB and licences from MIT to no-commercial-use-at-all. Check before you download.

The blunter framing: if what you want today is "image in, usable 3D mesh out", the original Zero123 is a reference implementation, not a production pipeline. It emits one novel view at a time with no consistency guarantee between views, so turning it into geometry means threestudio's per-object SDS optimisation — tens of minutes per object, minimum. What is still called constantly is Zero123++ v1.2 (0.9B, six views in a single 3x2 grid), the multi-view backbone inside InstantMesh, FreeSplatter and friends. The end-to-end job, meanwhile, has been taken over by native 3D latent models like TRELLIS (Microsoft, MIT, arXiv 2412.01506) and Hunyuan3D 2.x (0.6B to 3.0B, ~6GB for shape, ~16GB with texture). That is good news for your budget: a single RTX 3090 runs the Zero123++ line comfortably.

01 —

Four lines in the Zero123 family — don't grab the wrong weights or the wrong card

Same name, different teams, a 4x VRAM spread, and four different licences

Version	Parameters	VRAM	Context	Notes
Zero123 · 105000.ckpt (official default)	SD image-variation backbone, 15.5GB per .ckpt file	~22GB for the fp32 gradio demo	Single image + relative pose (R, T) -> one novel view	The README names RTX 3090 / 4090(Ti) as viable. MIT code, and the reference implementation you should use when reproducing the paper's numbers.
Zero123 · 165000 / 230000 / 300000.ckpt	Same architecture, more Objaverse fine-tuning steps	Same ~22GB class	wget cv.cs.columbia.edu/zero123/assets/$iteration.ckpt	The longest run took roughly 6,000 A100 hours. More steps does not automatically mean better on your data — 105000 remains the official default, so benchmark them yourself.
Zero123-XL	Same architecture, retrained on Objaverse-XL	Same ~22GB class as the original	Single image + relative pose -> one novel view	Moving from Objaverse to the ten-million-object Objaverse-XL noticeably steadies generalisation to real photographs. The first big data-side upgrade after the original.
Stable Zero123 / Stable Zero123C	stable_zero123.ckpt, from Stability AI	~12GB coarse stage, ~10GB refine, via threestudio SDS	Requires background-removed RGBA input (files suffixed _rgba.png)	Reworked render data and elevation conditioning; Stability reports it beating both Zero123 and Zero123-XL. The standard release is research-only — only the C variant's Community License permits commercial use. This is the easiest trap in the family.
Zero123++ v1.1	0.9B	~5GB at fp16	Single image -> 3x2 six-view grid, elevations 30° / -20°	SUDO-AI's first release, loaded through a diffusers custom_pipeline. Pairing it with controlnet-zp11-depth-v1 for depth control takes it to roughly 5.7GB.
Zero123++ v1.2 (the one to start with)	0.9B, safetensors	~5GB at fp16; ~5.7GB with ControlNet	Six views at fixed azimuths 30/90/150/210/270/330°, elevations 20° / -10°, unified 30° FOV	More deliberate camera-intrinsics handling and normalised object size. This is the variant genuinely in use: InstantMesh runs a customised version of its UNet for white-background multi-view, and FreeSplatter builds on it too. Weights are CC-BY-NC 4.0, so commercial use needs a separate conversation.

02 —

Pick the card by scenario — match the VRAM and the money works

All NexGPU list rates, billed per second, destroy whenever you like

Zero123++ v1.2 inference, six views from one image (~5GB at fp16)
RTX 3090 24GB$0.193/GPU-hour
A 0.9B UNet leaves 24GB so roomy you can hold a ControlNet and rembg alongside it — the best value on the list for this workload.
Original Zero123 weights through gradio_new.py (~22GB per the README)
RTX 5090 32GB$0.723/GPU-hour
22GB inside a 24GB card leaves 2GB of headroom; nudge the batch or open a visualiser and you OOM. 32GB means you stop restarting.
threestudio + stable-zero123, per-object SDS optimisation
RTX 4090 24GB$0.540/GPU-hour
The coarse stage only needs ~12GB, so VRAM is not the constraint — this is an hours-long optimisation where a faster card directly means a shorter bill.
Zero123++ fine-tuning (configs/zero123plus-finetune.yaml) or batch InstantMesh mesh generation
RTX A6000 48GB$0.817/GPU-hour
Fine-tuning has to hold weights, gradients and optimiser state at once, and 48GB buys a batch size worth having. InstantMesh's two-GPU memory-saving mode also collapses back to a single card here.

03 —

Zero to first render in four steps

Zero123++ v1.2 as the main line, with side roads for the original weights and the SDS route

01
Spin up the instance and install dependencies
Pick an RTX 3090 24GB in the NexGPU console, boot the prebuilt PyTorch image, and connect over SSH or Jupyter. Zero123++ lives in the diffusers ecosystem so the dependency list is light. rembg matters more than it looks: every model in this family expects a clean background-removed RGBA object image, and feeding it an unsegmented photo produces mush.
```
pip install -U diffusers transformers accelerate rembg && python -c "import torch; print(torch.cuda.get_device_name(0))"
```
02
Run Zero123++ v1.2 and get six views in one pass
Two details people miss. First, you must pass custom_pipeline="sudo-ai/zero123plus-pipeline" — a plain from_pretrained will not load it. Second, swap in EulerAncestralDiscreteScheduler with timestep_spacing="trailing"; get that argument wrong and quality visibly drops. Feed it at least 320x320, square if you can, and use 28 to 100 steps, pushing higher for complex objects. Output is a single 3x2 grid at fixed azimuths of 30/90/150/210/270/330°.
```
pipe = DiffusionPipeline.from_pretrained("sudo-ai/zero123plus-v1.2", custom_pipeline="sudo-ai/zero123plus-pipeline", torch_dtype=torch.float16).to("cuda:0")
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")
pipe(Image.open("input_rgba.png"), num_inference_steps=75).images[0].save("six_views.png")
```
03
Side road: the original weights, for reproduction work
The original repo is pinned to Python 3.9, and both taming-transformers and CLIP have to be installed from source — pip alone will not get you there. Each checkpoint is a single 15.5GB file, so check your disk first. Only two groups should take this road: people reproducing the paper's numbers, and people modifying the Zero123 architecture itself. If you just want multi-view images, go back to step two.
```
conda create -n zero123 python=3.9 -y && conda activate zero123
pip install -r requirements.txt
git clone https://github.com/CompVis/taming-transformers.git && pip install -e taming-transformers/
git clone https://github.com/openai/CLIP.git && pip install -e CLIP/
wget https://cv.cs.columbia.edu/zero123/assets/105000.ckpt
python gradio_new.py
```
04
Downstream: turning six views into actual 3D
Two options. The feed-forward route is InstantMesh (Apache 2.0, CUDA>=12.1, PyTorch 2.1+), which internally runs a customised Zero123++ UNet for white-background multi-view and then reconstructs a mesh with the LRM/Instant3D sparse-view approach — seconds to minutes, across four configs including instant-mesh-large and instant-nerf-large, with --export_texmap for a textured mesh. The optimisation route is threestudio's stable-zero123 SDS: a higher quality ceiling, but hours per object, and the input must be a background-removed image with the _rgba.png suffix.
```
python launch.py --config configs/stable-zero123.yaml --train --gpu 0 data.image_path=./load/images/hamburger_rgba.png
```

What it costs to test all three routes in one day

Priced honestly against the NexGPU list. Route one, Zero123++ v1.2 inference at ~5GB fp16 on an RTX 3090 24GB at $0.193/GPU-hour: install, pull the 0.9B weights, push a few dozen inputs through, call it 40 minutes — $0.193 x 40 / 60 = $0.13. Route two, the original 105000.ckpt at 15.5GB for a single file, two hours to download and tune the gradio demo, on an RTX 5090 32GB so that the documented 22GB has room to breathe — $0.723 x 2 = $1.45. Route three is the expensive one: threestudio's stable-zero123 optimises per object, so budget an hour per object on an RTX 4090 and twelve objects in a day — $0.540 x 12 = $6.48. All three together come to $8.06. Under nine dollars tells you which route suits your data, instead of buying a card first and regretting it later. Storage is separate: keeping 105000.ckpt and the Zero123++ weights on disk is roughly 20GB, and at $0.414/GB-month that is $8.28 a month — so delete what you don't need when you're done. One billing detail worth internalising: compute stops billing the moment the instance stops, but storage keeps billing until the volume is destroyed. Egress is a rounding error — exporting 200 meshes and preview videos is about 2GB, and $0.0081 x 2 is under two cents.

04 —

FAQ

How much VRAM do I actually need to self-host Zero123?

It depends entirely on which Zero123 you mean. Columbia's original gradio demo is documented at roughly 22GB, so only 24GB-and-up cards survive it. Stable Zero123 through threestudio's SDS pipeline wants about 12GB for the coarse stage and 10GB for refinement. Zero123++ v1.2 is the frugal one: about 5GB at fp16, or about 5.7GB with the depth ControlNet attached. That is a spread of more than 4x, which is why "how much VRAM does Zero123 need" has no single answer. On NexGPU you can boot one RTX 3090 24GB at $0.193/GPU-hour, work through all four variants, and destroy it — billed per second throughout.

Are Zero123 and Zero123++ two versions of the same project?

No, and this is the most common mix-up in the field. Zero123 comes from Columbia CVLab with Toyota Research Institute, at cvlab-columbia/zero123 under MIT. Zero123++ is a separate effort by SUDO-AI (Ruoxi Shi, Hao Su and colleagues) at SUDO-AI-3D/zero123plus, Apache 2.0 code with CC-BY-NC 4.0 weights. Even the output shape differs: the original returns one novel view per call, while Zero123++ returns a consistent 3x2 six-view grid in a single pass. Similar names, incompatible weights — don't substitute one for the other. A single 3090 on NexGPU is enough to try both, and it boots in minutes.

Is Zero123 still worth deploying, or has it been superseded?

Split the question. The original Zero123 is now mostly reference and teaching value — single-view output that needs SDS to become geometry — and the end-to-end job has moved to native 3D latent models like TRELLIS (Microsoft, MIT) and Hunyuan3D 2.x (0.6B to 3.0B, ~6GB for shape and ~16GB with texture). Zero123++ v1.2, though, has not aged out at all: it is the multi-view backbone behind InstantMesh and FreeSplatter, still pulling five figures of downloads a month. The sensible posture is to treat Zero123++ as a component and benchmark the newer end-to-end models beside it. NexGPU carries prebuilt images for both, so you can run the comparison on one machine.

Can I use the Zero123 family commercially?

Licensing is this family's biggest trap — four lines, four answers. The original Zero123 code is MIT and its Objaverse renders are ODC-By 1.0. Standard Stable Zero123 is a Stability AI non-commercial research licence; only Stable Zero123C's Community License allows commercial use. Zero123++ is Apache 2.0 in code but CC-BY-NC 4.0 in weights, which explicitly rules out commercial use. InstantMesh itself is Apache 2.0 yet still calls Zero123++ weights internally, so that layer has to be untangled before you ship. None of this stops you evaluating: renting on NexGPU needs no quota request and has no minimum, so prove the quality first and negotiate licensing after.

Will a 24GB RTX 4090 run the original Zero123?

Yes — the README names RTX 3090 / 4090(Ti) specifically. Just go in knowing that 22GB inside 24GB leaves two gigabytes of slack, so an extra visualiser window, a slightly larger batch, or a background-segmentation model running alongside will OOM you. If you only need to see it work, NexGPU's RTX 3090 24GB at $0.193/GPU-hour is the cheapest way to find out. If you want uninterrupted tuning sessions, the RTX 5090 32GB at $0.723/GPU-hour or the RTX A6000 48GB at $0.817/GPU-hour will save you a lot of re-runs. Both are available on demand with us.

What is the full path from one image to a mesh I can import into an engine?

Shortest version is three steps: rembg to produce a background-free RGBA image, Zero123++ v1.2 for the 3x2 six-view grid, then InstantMesh's sparse-view reconstruction straight to a mesh — add --export_texmap for textures. That whole path runs in minutes. For higher quality, swap in the threestudio + stable-zero123 SDS route and pay for it in hours of GPU time per object. The two routes have very different VRAM profiles: the first is happy on one RTX 3090 24GB at $0.193/GPU-hour, the second is better served by an RTX 4090 24GB at $0.540/GPU-hour trading compute for wall-clock. NexGPU spans 51 countries and regions, 1,175 verified rentable nodes and 2,498 GPUs, with 2,000+ prebuilt images including PyTorch and ComfyUI and bilingual Telegram support with no ticket queue — so your first render is minutes away.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.