Image generation model

Run the full DeepFloyd IF three-stage cascade on one 24GB card

Q: How much VRAM does DeepFloyd IF actually need to run locally?

There are three official tiers: with the diffusers default model_cpu_offload the whole cascade fits in as little as 14GB; keeping stage 1 and stage 2 resident takes 16GB; running all three stages through to 1024x1024 takes 24GB; fine-tuning is around 28GB. The path of least resistance is renting an honest 24GB card - NexGPU's RTX 3090 24GB is $0.193/GPU-hour and also happens to be the cheapest GPU we rent.

Q: Why is DeepFloyd IF so memory-hungry when stage 1 is only 4.3B?

Because the weight sits in the text encoder, not the UNet. IF uses a frozen T5-v1_1-xxl whose fp16 shards total 11.5GB, larger than IF-I-XL's 8.61GB UNet, while Stable Diffusion's CLIP encoder is only a few hundred megabytes. Switching to IF-I-L or IF-I-M saves you a few gigabytes of UNet and nothing at all on T5. To squeeze further, use the official 8-bit T5 checkpoint (7.92GB), or load only the encoder, compute prompt_embeds, delete it, then load the UNet. On NexGPU the simpler answer is an RTX A6000 48GB at $0.817/GPU-hour that holds everything at once.

Q: Are there GGUF or Q4 quantized builds of DeepFloyd IF?

No. DeepFloyd IF never grew a GGUF ecosystem and there are no community Q4_K_M-style weights - that world belongs to llama.cpp and, later, the FLUX ComfyUI crowd. The only quantized artifact DeepFloyd shipped is the 8-bit T5 encoder checkpoint (7.92GB, via bitsandbytes LLM.int8()); the UNets exist only in fp16 and fp32. Do not carry LLM quantization instincts over here - VRAM is a hard constraint for this model, so start at 24GB when you pick a card on NexGPU.

Q: Can DeepFloyd IF be used commercially? What is the license?

No. The stage 1 and stage 2 weights are under deepfloyd-if-license, explicitly restricted to non-commercial research and prohibiting military, surveillance and biometric-processing use. The code repo is modified MIT, and only the stage 3 stabilityai/stable-diffusion-x4-upscaler is openrail++, which is a separate matter. The open release promised at the 2023 launch never arrived. For commercial text rendering the live options are FLUX.1, Qwen-Image and SD 3.5 - and on NexGPU switching models just means switching prebuilt images, with PyTorch, ComfyUI, Stable Diffusion and AUTOMATIC1111 among the 2,000+ ready to boot.

Q: Is DeepFloyd IF still maintained, and can I use the official pip package?

The project is effectively frozen: the last commit to deep-floyd/IF was April 2024, and deepfloyd_if on PyPI stopped at 1.0.1 in April 2023 with torch<2.0.0, transformers~=4.25.1, diffusers~=0.16.0 and xformers==0.0.16 pinned - a stack that will not install against current CUDA drivers or newer architectures like Ada and Blackwell. The correct entry point today is diffusers' IFPipeline, IFSuperResolutionPipeline, IFImg2ImgPipeline and IFInpaintingPipeline, all maintained on the diffusers mainline. So skip the archaeology: a NexGPU PyTorch prebuilt image plus one pip install -U diffusers is the fastest route.

Q: Why do my generated images have a watermark and keep getting blocked?

That is the pipeline default, not a misconfiguration. IFPipeline is constructed with an IFWatermarker and an IFSafetyChecker; the final step calls watermarker.apply_watermark() to stamp the image, and the safety checker substitutes anything it flags. For reproduction work and FID evaluation, pass watermarker=None, safety_checker=None and requires_safety_checker=False to from_pretrained. This kind of stop-tweak-restart debugging is exactly what per-second billing is for - on NexGPU compute billing halts the moment you stop the instance, so the minutes you spend editing code cost nothing.

Pixel-space cascaded diffusion, 64x64 at stage one, and a T5-v1_1-xxl encoder that outweighs the UNet. Here is the real memory math and the real dependency trap before you rent a card.

DeepFloyd · self-hosted

DeepFloyd IF, built by Stability AI with the DeepFloyd research lab, is a pixel-space cascaded diffusion model, and it works nothing like Stable Diffusion's latent denoising. A frozen T5-v1_1-xxl encodes the prompt into hidden states, stage 1 paints a 64x64 image directly in pixel space, stage 2 upscales it to 256x256, and Stability's stable-diffusion-x4-upscaler carries it to 1024x1024. IF-I-XL-v1.0 posts a zero-shot FID-30K of 6.66, and its claim to fame was rendering the actual words from your prompt legibly inside the image.

The thing that trips people up when self-hosting is not the 4.3B UNet, it is the text encoder. IF-I-XL's UNet is 8.61GB in fp16, while the T5-v1_1-xxl encoder shared across stages 1 and 2 comes to 11.5GB in fp16 shards. The encoder is larger than the backbone, which is exactly the assumption Stable Diffusion users arrive with and get wrong. The official numbers: diffusers enables model_cpu_offload by default and runs the entire pipeline in as little as 14GB of VRAM; keeping stage 1 and stage 2 resident needs 16GB; all three stages through 1024x1024 needs 24GB; fine-tuning is roughly 28GB.

One more thing to say plainly: this project is no longer being updated. The last commit to deep-floyd/IF on GitHub landed in April 2024, and the deepfloyd_if package on PyPI is frozen at 1.0.1 from April 2023, pinning torch<2.0.0, transformers~=4.25.1, diffusers~=0.16.0 and xformers==0.0.16 - a stack that simply will not install on a current CUDA setup. The maintained entry point is the IFPipeline family inside Hugging Face diffusers, which has tracked the diffusers mainline ever since. The weights are still under deepfloyd-if-license, research use only; the open release promised at launch never happened. Running IF today usually means reproducing the cascaded pixel-diffusion results, building an FID baseline, or chasing its particular look. If you need commercially usable text rendering, look at FLUX.1, Qwen-Image or SD 3.5 instead.

01 —

The model family and what each piece weighs

Three stages means three separate repos - budget VRAM and disk for each one

Version	Parameters	VRAM	Context	Notes
DeepFloyd/IF-I-XL-v1.0	4.3B (stage 1)	UNet fp16 8.61GB + T5 fp16 11.5GB; 24GB for the full chain, 14GB with offload	64x64 output; T5 truncated at 77 tokens	The flagship stage 1, zero-shot FID-30K 6.66. This is the one that reproduces the paper. Its fp32 weights are 17.2GB, so always pass variant='fp16'.
DeepFloyd/IF-I-L-v1.0	900M (stage 1)	UNet fp16 1.87GB; T5 is still the 11.5GB elephant	64x64 output; T5 truncated at 77 tokens	FID 8.06. The UNet shrinks more than fourfold, but since the T5 encoder is unchanged you save far less memory than you would expect.
DeepFloyd/IF-I-M-v1.0	400M (stage 1)	UNet fp16 743MB; the bottleneck is still T5	64x64 output; T5 truncated at 77 tokens	FID 8.86, the smallest stage 1. Use it to validate the three-stage wiring, the license gate and your cache paths, then swap in XL.
DeepFloyd/IF-II-L-v1.0	1.2B (stage 2)	UNet fp16 2.49GB	64x64 to 256x256 super-resolution	The stage 2 the official examples pair with IF-I-XL. Always load it with text_encoder=None - this repo ships its own 19GB copy of the T5 shards.
DeepFloyd/IF-II-M-v1.0	450M (stage 2)	UNet fp16 922MB	64x64 to 256x256 super-resolution	The lightweight stage 2, for a cheap validation chain alongside IF-I-M or IF-I-L.
stabilityai/stable-diffusion-x4-upscaler	~950M (stage 3)	UNet fp16 947MB + VAE fp16 111MB	256x256 to 1024x1024 super-resolution	Strictly this is not a DeepFloyd model at all - it is Stability's latent x4 upscaler under openrail++, a different license from the two research-only stages. It is also the stage that pushes the requirement from 16GB to 24GB.

02 —

Pick the card that matches the run

24GB is the hard floor for the full cascade, and our cheapest card happens to be 24GB

Full three-stage cascade to 1024x1024 with model_cpu_offload
RTX 3090 24GB$0.193/GPU-hour
It hits the official 24GB requirement exactly and it is the cheapest card in the fleet - unbeatable for a workload that is memory-hungry but not compute-bound.
Sweeping prompts in batch, building an FID comparison
RTX 4090 24GB$0.540/GPU-hour
Stage 1 only paints 64x64 so compute is barely the constraint; wall-clock goes into T5 encoding and the two upscalers, and the 4090's fp16 throughput shows up most at the 1024x1024 stage.
Turn offload off and keep T5 plus all three UNets resident
RTX A6000 48GB$0.817/GPU-hour
The 11.5GB T5 plus the three UNets lands in the low twenties of gigabytes, so 48GB holds everything at once and kills the per-step weight shuffling between CPU and GPU. It also covers the ~28GB fine-tuning figure.
Fine-tuning IF-I-XL, or running several prompt-encoding streams on one card
A100 PCIE 80GB$0.824/GPU-hour
Practically the same hourly rate as the 48GB A6000 but with 80GB, so optimizer states and gradients stop forcing you to cut batch size.

03 —

From blank instance to first image

Four steps, all through diffusers - never touch the deepfloyd_if package frozen back in 2023

01
Boot a 24GB instance and add the dependencies
Pick RTX 3090 24GB at console.nexgpu.net, boot the prebuilt PyTorch image, SSH in and add the diffusers-side packages. sentencepiece is mandatory for the T5 tokenizer - without it from_pretrained fails while loading the tokenizer. bitsandbytes only matters if you take the 8-bit T5 route.
```
pip install -U diffusers transformers accelerate safetensors sentencepiece bitsandbytes
```
02
Accept the license and log into Hugging Face
The IF weights are gated. Accept the terms on the DeepFloyd/IF-I-XL-v1.0 model page first - accepting on stage 1 auto-accepts the other IF repos - then write a token onto the instance. Without it from_pretrained returns a bare 401 rather than a readable error.
```
huggingface-cli login
```
03
Pull only the fp16 variant, cache onto the data volume
Omit variant='fp16' and you download the 17.2GB fp32 UNet, doubling both disk and download time for nothing. Point HF_HOME at the data volume so the weights survive an instance rebuild. The full fp16 IF-I-XL chain is roughly 25GB: T5 11.5GB + stage 1 UNet 8.61GB + safety checker 1.2GB + stage 2 UNet 2.49GB + x4 upscaler about 1.1GB.
```
export HF_HOME=/workspace/hf && python -c "import torch; from diffusers import DiffusionPipeline; DiffusionPipeline.from_pretrained('DeepFloyd/IF-I-XL-v1.0', variant='fp16', torch_dtype=torch.float16)"
```
04
Chain the three stages together
Stages 2 and 3 must be loaded with text_encoder=None or you pull a second copy of T5 into memory and blow past 24GB immediately. Call enable_model_cpu_offload() on every stage. Stage 1 defaults to num_inference_steps=100 and guidance_scale=7.0 - drop to 50 steps to prove the chain works, then dial back up. Note that the pipeline watermarks output and runs a safety checker by default; for research evaluation pass watermarker=None and safety_checker=None at construction.
```
stage_2 = DiffusionPipeline.from_pretrained('DeepFloyd/IF-II-L-v1.0', text_encoder=None, variant='fp16', torch_dtype=torch.float16); stage_2.enable_model_cpu_offload()
```

What one round of DeepFloyd IF actually costs

Take RTX 3090 24GB at $0.193/GPU-hour. Downloading 25GB of weights and setting up the environment runs about half an hour: 0.193 x 0.5 = $0.10. Then six hours of debugging and generation: 0.193 x 6 = $1.16. The line people forget is storage - 25GB at the $0.414/GB-month median is 25 x 0.414 = $10.35 for a full month, or 25 x 0.414 x 0.1 = $1.04 if you keep it three days. That puts the round at roughly 0.10 + 1.16 + 1.04 = $2.30, with the card and the storage costing about the same. That balance is unusually skewed for DeepFloyd IF because a single T5-XXL encoder weighs as much as an entire competing model. Exporting 200 PNGs at 1024x1024 is around 2GB, so egress is 2 x $0.0081 = $0.02, effectively nothing. Billing is metered per second: compute stops the moment the instance stops, while storage keeps billing until you destroy the volume, so decide up front whether those 25GB stay. If you want offload gone entirely and every UNet plus T5 resident, RTX A6000 48GB for the same six hours is 0.817 x 6 = $4.90. No minimum, no setup fee, no quota request.

04 —

FAQ

How much VRAM does DeepFloyd IF actually need to run locally?

There are three official tiers: with the diffusers default model_cpu_offload the whole cascade fits in as little as 14GB; keeping stage 1 and stage 2 resident takes 16GB; running all three stages through to 1024x1024 takes 24GB; fine-tuning is around 28GB. The path of least resistance is renting an honest 24GB card - NexGPU's RTX 3090 24GB is $0.193/GPU-hour and also happens to be the cheapest GPU we rent.

Why is DeepFloyd IF so memory-hungry when stage 1 is only 4.3B?

Because the weight sits in the text encoder, not the UNet. IF uses a frozen T5-v1_1-xxl whose fp16 shards total 11.5GB, larger than IF-I-XL's 8.61GB UNet, while Stable Diffusion's CLIP encoder is only a few hundred megabytes. Switching to IF-I-L or IF-I-M saves you a few gigabytes of UNet and nothing at all on T5. To squeeze further, use the official 8-bit T5 checkpoint (7.92GB), or load only the encoder, compute prompt_embeds, delete it, then load the UNet. On NexGPU the simpler answer is an RTX A6000 48GB at $0.817/GPU-hour that holds everything at once.

Are there GGUF or Q4 quantized builds of DeepFloyd IF?

No. DeepFloyd IF never grew a GGUF ecosystem and there are no community Q4_K_M-style weights - that world belongs to llama.cpp and, later, the FLUX ComfyUI crowd. The only quantized artifact DeepFloyd shipped is the 8-bit T5 encoder checkpoint (7.92GB, via bitsandbytes LLM.int8()); the UNets exist only in fp16 and fp32. Do not carry LLM quantization instincts over here - VRAM is a hard constraint for this model, so start at 24GB when you pick a card on NexGPU.

Can DeepFloyd IF be used commercially? What is the license?

No. The stage 1 and stage 2 weights are under deepfloyd-if-license, explicitly restricted to non-commercial research and prohibiting military, surveillance and biometric-processing use. The code repo is modified MIT, and only the stage 3 stabilityai/stable-diffusion-x4-upscaler is openrail++, which is a separate matter. The open release promised at the 2023 launch never arrived. For commercial text rendering the live options are FLUX.1, Qwen-Image and SD 3.5 - and on NexGPU switching models just means switching prebuilt images, with PyTorch, ComfyUI, Stable Diffusion and AUTOMATIC1111 among the 2,000+ ready to boot.

Is DeepFloyd IF still maintained, and can I use the official pip package?

The project is effectively frozen: the last commit to deep-floyd/IF was April 2024, and deepfloyd_if on PyPI stopped at 1.0.1 in April 2023 with torch<2.0.0, transformers~=4.25.1, diffusers~=0.16.0 and xformers==0.0.16 pinned - a stack that will not install against current CUDA drivers or newer architectures like Ada and Blackwell. The correct entry point today is diffusers' IFPipeline, IFSuperResolutionPipeline, IFImg2ImgPipeline and IFInpaintingPipeline, all maintained on the diffusers mainline. So skip the archaeology: a NexGPU PyTorch prebuilt image plus one pip install -U diffusers is the fastest route.

Why do my generated images have a watermark and keep getting blocked?

That is the pipeline default, not a misconfiguration. IFPipeline is constructed with an IFWatermarker and an IFSafetyChecker; the final step calls watermarker.apply_watermark() to stamp the image, and the safety checker substitutes anything it flags. For reproduction work and FID evaluation, pass watermarker=None, safety_checker=None and requires_safety_checker=False to from_pretrained. This kind of stop-tweak-restart debugging is exactly what per-second billing is for - on NexGPU compute billing halts the moment you stop the instance, so the minutes you spend editing code cost nothing.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.