Video generation

Self-hosting AnimateDiff: VRAM, version selection, and what a render run actually costs

AnimateDiff isn't a video model. It's a motion module you slot into the SD1.5 weights you already have — so your character LoRA and your art style start moving, on a card as small as 8GB.

AnimateDiff · self-hosted

The paper is "AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning" (arXiv 2307.04725, ICLR 2024 Spotlight) by Yuwei Guo, Ceyuan Yang, Bo Dai and colleagues, released under Apache-2.0 at github.com/guoyww/AnimateDiff. The trick is counterintuitive: the base model is frozen, untouched, and a set of temporal attention layers — the roughly 0.4B-parameter motion module — is inserted into the UNet. Any SD1.5 checkpoint you pulled off Civitai, any character LoRA you trained, animates with its style intact. That is the entire reason people still deploy this in 2026, and it is what separates it from native video models.

Be clear-eyed about the state of it: the main repo's feature line stopped at v3 (December 2023 — v3_sd15_mm.ckpt plus SparseCtrl and the Domain Adapter), and no new motion module has shipped since. The living work happens in the ecosystem. Kosinkadink's ComfyUI-AnimateDiff-Evolved added sliding context windows, fp8, AnimateLCM, CameraCtrl and PIA support; Hugging Face diffusers exposes AnimateDiffPipeline, AnimateDiffSDXLPipeline, AnimateDiffSparseControlNetPipeline and AnimateDiffVideoToVideoPipeline. If raw text-to-video fidelity is what you want, the native DiT video models — Wan, HunyuanVideo, LTX-Video — passed AnimateDiff a while ago. But none of them will load your SD1.5 character LoRA. AnimateDiff will.

AnimateDiff has one memory property worth internalising before you pick a card: video length barely moves VRAM. ComfyUI's Context Options slice the timeline at context_length=16 with context_overlap=4, so the UNet only ever sees 16 frames at once; FreeNoise does the same thing on the diffusers side. Peak sampling memory for 160 frames is close to that for 16. The real spike relocates to VAE decode, which vae.enable_slicing() or tiling flattens. Once you see that, your GPU choice changes: you are not buying memory for longer clips, you are buying resolution, control-net headroom, and throughput.

01 —

Versions and weights

Filenames and sizes as actually listed in the official guoyww/animatediff Hugging Face repo

Version	Parameters	VRAM	Context	Notes
AnimateDiff v3 (v3_sd15_mm.ckpt)	~0.4B motion module / 1.67GB	~4GB resident in fp16; ~6–8GB peak at 512×512 × 16 frames	16-frame window, extensible indefinitely	The current default. Pairs with v3_sd15_adapter.ckpt (102MB), the domain adapter LoRA — keep its weight near 0.5, since running it at full strength drags the cheap WebVid look into your output.
AnimateDiff v2 (mm_sd_v15_v2.ckpt)	453M motion module / 1.82GB	Same class as v3; runs on 8GB at 512px	16-frame window	The only module the official camera MotionLoRAs bind to. Eight v2_lora_*.ckpt files (ZoomIn/ZoomOut/PanLeft/PanRight/TiltUp/TiltDown/RollingClockwise/RollingAnticlockwise), 77.5MB each, v2 only.
AnimateDiff v1 (mm_sd_v14 / mm_sd_v15)	1.67GB each	Same as v2	16 frames	Superseded. mm_sd_v15 learned Shutterstock watermarks from WebVid-10M training footage and they bleed faintly into output. Stop using it as a default.
AnimateDiff SDXL-Beta (mm_sdxl_v10_beta.ckpt)	950MB	Official README: inference usually requires ~13GB	16 frames @ 1024×1024	Beta means beta — motion stability is visibly behind the SD1.5 line. Plenty of SDXL users go back to HotshotXL's hsxl_temporal_layers.safetensors instead.
SparseCtrl RGB / Scribble (v3_sd15_sparsectrl_*.ckpt)	1.99GB each	Adds roughly 2GB resident on top of v3	Arbitrary number of condition frames	v3-only control encoders. The RGB variant is AnimateDiff's image-to-video path and works from a single first frame; Scribble takes line art. In diffusers this is AnimateDiffSparseControlNetPipeline with controlnet_frame_indices.
AnimateLCM (wangfuyun/AnimateLCM)	~0.5B	Comparable to v3, but far fewer steps	16-frame window	Consistency-distilled acceleration: 4–8 steps instead of 25–50. Used with AnimateLCM_sd15_t2v_lora.safetensors, with an I2V variant available. Best value per dollar for bulk style exploration.

02 —

Which NexGPU card to rent

Pick by resolution and control-chain complexity, not by clip length — length is a sliding-window problem, not a memory one

SD1.5 + v3 motion module, 512×512, 16–32 frames, prompt and style exploration
RTX 3090 24GB$0.193/GPU-hr
A 6–8GB peak on a 24GB card leaves enormous headroom, and this is the cheapest 24GB option on the network — a full style-search run costs less than a coffee.
Production renders at 768px with stacked ControlNet / SparseCtrl / IP-Adapter chains
RTX 4090 24GB$0.540/GPU-hr
Same memory as the 3090, an entirely different throughput class. Once the control chain grows, the bottleneck shifts from VRAM to compute — and the pricier card becomes the cheaper run.
AnimateDiff-SDXL or HotshotXL at 1024×1024, or renders followed by heavy upscaling
RTX 5090 32GB$0.723/GPU-hr
The official SDXL figure is ~13GB; add an upscale pass and VAE decode and you are scraping the 24GB ceiling. 32GB buys you the margin to skip tiling entirely.
Training your own motion module, large-scale video-to-video rewrites, or many concurrent render streams
A100 PCIE 80GB$0.824/GPU-hr
Training temporal layers means holding the base model, motion module, optimiser state and a full frame batch at once; 80GB avoids gradient checkpointing. It costs under a cent more per hour than the 48GB A6000 at $0.817.

03 —

Four steps to a running instance

ComfyUI for the graph workflow, diffusers for scripted batches — both paths below

01
Boot an instance and install the AnimateDiff-Evolved nodes
Launch from the prebuilt ComfyUI image in the NexGPU console, connect over SSH or Jupyter, and clone the Evolved node pack into custom_nodes. That pack is what carries sliding context windows, fp8, AnimateLCM and CameraCtrl — the original repo's scripts.animate is really only for reproducing the paper.
```
git clone https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved.git ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved
```
02
Pull the motion module and your own base checkpoint
Motion modules go in ComfyUI/models/animatediff_models, MotionLoRAs in ComfyUI/models/animatediff_motion_lora, base checkpoints stay where they always were. Download inside the instance rather than over your home connection — with nodes in 51 countries and regions you can pick an entry point near Hugging Face and 1.67GB lands in minutes.
```
huggingface-cli download guoyww/animatediff v3_sd15_mm.ckpt v3_sd15_adapter.ckpt --local-dir ComfyUI/models/animatediff_models
```
03
Skip the UI and batch from diffusers
Load the v1-5-3 weights into MotionAdapter and point the pipeline at your own SD1.5 checkpoint. The scheduler must be constructed explicitly with beta_schedule="linear", clip_sample=False, timestep_spacing="linspace" and steps_offset=1 — stock DDIM settings produce a flickering smear, and this is the single most common first-day mistake.
```
adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-v1-5-3", torch_dtype=torch.float16)
```
04
Extend the clip and flatten the memory curve
In ComfyUI, attach a Context Options◆Standard Uniform node (context_length 16, context_stride 1, context_overlap 4). In diffusers, enable FreeNoise — it reaches past 256 frames and supports prompts keyed to frame indices. VAE decode is the genuine memory spike, so pair slicing with CPU offload and it stays flat.
```
pipe.enable_free_noise(context_length=16, context_stride=4); pipe.vae.enable_slicing(); pipe.enable_model_cpu_offload()
```

What a render run actually costs

Take an RTX 4090 24GB at $0.540/GPU-hr running SD1.5 + v3_sd15_mm, 512×512, 16 frames, 25 DDIM steps. Estimating 30 seconds per two-second clip, that is 3600 ÷ 30 = 120 clips an hour, or $0.540 ÷ 120 ≈ $0.0045 per clip. A 200-clip style search occupies about 100 minutes: $0.540 × (100 ÷ 60) ≈ $0.90. Swap in AnimateLCM at 6 steps and the same budget buys roughly three to four times the output. To go cheaper still, drop to an RTX 3090 24GB at $0.193/GPU-hr — slower sampling, but it handles 512px comfortably, and the same 100 minutes costs $0.193 × 1.67 ≈ $0.32. Storage is separate: 20GB for base model, motion module and output is $0.414/GB-month × 20 = $8.28/month, and pulling 5GB of finished clips down runs $0.0081/GB ≈ $0.04. Billing is metered per second and priced per hour; compute stops the moment the instance stops, but storage bills until the volume is destroyed — so delete the volumes you are done with.

04 —

FAQ

How much VRAM does AnimateDiff actually need? Is 8GB enough?

It is. An SD1.5 base plus the v3 motion module sits around 4GB resident in fp16, and sampling at 512×512 × 16 frames peaks in the 6–8GB range, which an 8GB card handles with VAE slicing on. Push to 768px with ControlNet and plan for 12–16GB; AnimateDiff-SDXL is stated at ~13GB in the official README. On NexGPU an RTX 3090 24GB at $0.193/GPU-hr covers the first two tiers outright, billed per second, so you stop paying the moment you stop testing.

With Wan and HunyuanVideo available, is there still a reason to deploy AnimateDiff?

Depends what you need. On pure text-to-video fidelity and physical plausibility, the native DiT video models won, and not narrowly. But they cannot consume the character LoRA and style checkpoint you trained on SD1.5 — and that is AnimateDiff's whole value: the base model is frozen, only temporal layers are inserted, so your character comes out as your character. For IP-consistent short animation, adding motion to an existing illustration style, or bulk asset generation, it remains the cheapest route. If you want to evaluate both, NexGPU's 2,000+ prebuilt images include ComfyUI and PyTorch, so it is one instance swap.

Why do my AnimateDiff clips have a faint Shutterstock watermark?

You are almost certainly on mm_sd_v15. The original motion module was trained on WebVid-10M, much of which carries Shutterstock watermarks, and the module absorbed them as an intrinsic texture of video. Switching to v3_sd15_mm.ckpt or mm_sd_v15_v2.ckpt clears it up; if you are running the v3 domain adapter, dropping its weight from 1.0 to around 0.5 also removes that stock-footage flavour. Re-running the comparison on a 3090 costs a few cents.

My MotionLoRA loads but does nothing. What did I configure wrong?

Nothing — it is a version mismatch. The eight official camera-motion LoRAs are literally named v2_lora_*, and they only bind to the v2 motion module, mm_sd_v15_v2. Attach one to v3_sd15_mm and you get no error and no camera movement. For camera motion, either fall back to the v2 module or move to CameraCtrl. If you want v3 quality and v2 camera work together you will be compositing two chains — a good use for NexGPU nodes that take up to 14 GPUs each, so you can validate both in parallel.

What is the maximum length AnimateDiff can generate, and does a long clip need more VRAM?

Length costs almost no extra memory, which is the most underrated thing about AnimateDiff. ComfyUI's Context Options slide a window at context_length=16 with context_overlap=4, so the UNet only ever processes 16 frames; FreeNoise in diffusers works the same way and the documentation explicitly supports 256+ frames with prompts assigned to frame indices. The genuine spike is VAE decode, handled by enable_slicing or split inference. So a long film needs a time budget, not a memory budget — and under per-second billing, time is money, which makes a fast card cheaper than a big one.

Can I use AnimateDiff output commercially? How does licensing work?

Two separate layers. AnimateDiff itself — motion modules, SparseCtrl, code — is Apache-2.0, which is permissive. The base model you attach it to is a different matter: stock SD1.5 ships under CreativeML Open RAIL-M, community fine-tunes vary wildly, and every Civitai checkpoint needs its commercial terms checked individually. A motion module never changes the base model's licence. What we can guarantee is the technical side: a clean environment, weights you pulled yourself, a dedicated instance. NexGPU runs 1,175 verified rentable nodes and 2,498 GPUs, reachable over SSH, Jupyter, web terminal, REST API or CLI, with bilingual support on Telegram and no ticket queue. Console at console.nexgpu.net.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.