Avatar Video Model

One photo, one audio track — SadTalker self-hosted on 8GB of VRAM

SadTalker translates audio into 3DMM coefficients through ExpNet and PoseVAE, then drives a still portrait to speak. It barely touches VRAM — what it consumes is render time, which is exactly what per-second GPU billing is for.

SadTalker · self-hosted

SadTalker comes out of Xi'an Jiaotong University, Tencent AI Lab and Ant Group. The paper, "Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation," landed at CVPR 2023 (arXiv 2211.12194). Its approach differs from its contemporaries: instead of generating pixels directly, ExpNet regresses 3DMM expression coefficients from audio, PoseVAE samples head pose separately, and MappingNet hands both to a 3D-aware renderer descended from face-vid2vid. Expression and pose are disentangled — which is why a SadTalker head actually turns and blinks rather than just flapping its mouth. That is a fundamentally different lineage from Wav2Lip, which inpaints the lip region of existing footage, or MuseTalk, which targets real-time lip sync.

The honest caveat: this repository is frozen. OpenTalker/SadTalker carries roughly 14,000 stars, 2,600+ forks and over 600 open issues, but the last functional commit to main landed in October 2023, and the whole OpenTalker org — video-retalking, StyleHEAT, DPE, ToonTalker — went quiet with it. The official Hugging Face Space, with 1,400+ likes, currently sits in BUILD_ERROR: even the authors' own demo environment no longer builds. The reason is mundane. requirements.txt pins numpy to 1.23.4 and basicsr to 1.4.2, and the README asks for Python 3.8 with torch 1.12.1+cu113 — but Python 3.8 is end-of-life and modern CUDA wheels ship no cp38 builds at all. The real difficulty with SadTalker today is not the model; it is reassembling a 2023 dependency tree.

The good news is that the model itself is tiny. The full weight set is about 3GB on disk, the SadTalker_V0.0.2_256.safetensors renderer is just 691MiB, and resident parameters come to roughly 1.4GB. The actual cost is wall-clock time: output is fixed at 25fps, so a one-minute clip is 1,500 frames, and `--enhancer gfpgan` runs per frame — that single stage eats about nine tenths of the pipeline. Which reframes the whole GPU question. It is not "will this fit in VRAM," it is "which card renders 1,500 frames most cheaply." On NexGPU, an RTX 3090 24GB runs $0.193 per GPU-hour, putting a one-minute enhanced clip at a couple of cents. Don't like the result? Spin up another box; billing stops by the second.

01 —

Variants, Weights and Measured VRAM

SadTalker has no parameter-count tiers — only combinations of render resolution, enhancement chain and preprocessing mode. Here is what each actually costs

Version	Parameters	VRAM	Context	Notes
SadTalker V0.0.2 · 256 (default path)	691MiB safetensors renderer, ~3GB full weight set	~4GB fp32 inference (--batch_size 2)	256×256 @ 25fps, unlimited audio length	`--size 256` is the default and the most reliable route. This is the baseline for lip-sync and head-pose quality — get it working here before chasing fidelity.
SadTalker V0.0.2 · 512 (beta)	SadTalker_V0.0.2_512.safetensors, also 691MiB	~6GB at --batch_size 2, ~8GB at batch 8	512×512 @ 25fps	The beta renderer shipped in June 2023. Curiously, the 512 and 256 safetensors are byte-identical in size (725,066,984) — same architecture, different training resolution — so moving up to 512 costs almost nothing in weight memory.
With GFPGAN / RestoreFormer face enhancement	GFPGANv1.4.pth ~350MB plus facexlib detection and parsing weights	+2–3GB on top of the renderer; ~8–10GB for the full 512 chain	Per-frame post-processing, face detail up to the 1024 class	`--enhancer gfpgan` is the single biggest quality win and the single slowest stage. In batch production it is the term that sets your per-clip cost.
full / extfull paste-back mode	Reuses the same renderer weights, no extra model	Scales with source resolution — ~10–12GB for a 1080p half-body image, more with `--background_enhancer realesrgan`	Output resolution matches the source image	`--preprocess full --still` is the mandatory combination for half-body presenter videos: render only the cropped face region, paste it back into the original frame, and freeze large head movements so the paste-back does not tear.
--old_version legacy weights	facevid2vid_00189-model.pth.tar, 1.97GiB in a single file	~5–6GB	256×256 @ 25fps	The pre-V0.0.2 .pth.tar weights, an order of magnitude larger than the safetensors. Only needed to reproduce older results or when safetensors loading misbehaves — skip the download for normal deployments.
Successors and alternatives after 2024	Varied architectures, mostly diffusion / DiT backbones	Lightweight LivePortrait and MuseTalk run in 8–12GB; diffusion-based Hallo2 and EchoMimicV2 generally want 24GB+	Longer duration, higher resolution, stronger identity preservation	Once SadTalker's main branch stalled, community momentum shifted to LivePortrait (Kuaishou), MuseTalk (Tencent ARC), EchoMimicV2 (Ant Group) and the Hallo series (Fudan). They look better and cost an order of magnitude more compute — SadTalker still owns the "one consumer card, a few minutes per clip" niche.

02 —

Which GPU to Rent for SadTalker

Two things decide it: enough VRAM to raise `--batch_size`, and a CUDA architecture that old torch stack will actually run on

Reproduce the README verbatim on the smallest budget
Tesla V100 32GB$0.188/GPU-hour
The cheapest card in the fleet, and it carries 32GB. Volta is natively supported by the torch 1.12.1+cu113 stack the README specifies, making it the one option where the official environment runs unmodified.
512 + GFPGAN batch production, the daily driver
RTX 3090 24GB$0.193/GPU-hour
Ampere sm_86, cleanly covered by cu118 wheels. 24GB lets you push `--batch_size` from the default 2 up to 8–16, maximising frames rendered per unit time and finished clips per dollar.
Deadline-driven work where a single clip must land fast
RTX 4090 24GB$0.540/GPU-hour
Ada sm_89 runs the per-frame GFPGAN pass roughly twice as fast as a 3090, halving per-clip time. The trade is that you must move to torch 2.x with cu118 or newer — the cu113 environment will not run here.
TTS + SadTalker + a successor model in one pipeline
RTX A6000 48GB$0.817/GPU-hour
48GB keeps GPT-SoVITS or CosyVoice resident in VRAM feeding audio straight into SadTalker while several render workers run in parallel, eliminating repeated weight-loading overhead.

03 —

From Empty Instance to Finished Clip, in Four Steps

Step two is the whole game — dragging a 2023 dependency tree back into installable shape

01
Launch an instance, pick a prebuilt image, clone the repo
Spin up an Ampere-or-newer card in the NexGPU console and pick a PyTorch image so you skip driver and CUDA setup entirely. The repository itself is only tens of megabytes — the weights are the bulk, and they come later.
```
git clone https://github.com/OpenTalker/SadTalker.git && cd SadTalker
```
02
Build the environment: Python 3.10 + torch 2.1.2, and pin torchvision to 0.16.2
The README's Python 3.8 simply cannot be provisioned any more. The combination that works is Python 3.10 with torch 2.1.2+cu118. Pin torchvision at 0.16.2 specifically: version 0.17 removed torchvision.transforms.functional_tensor, and basicsr==1.4.2 from requirements.txt imports exactly that module — this is the single most common ModuleNotFoundError people hit installing SadTalker. If some other constraint forces a newer torchvision, edit basicsr/data/degradations.py and change that import from functional_tensor to functional. Also keep numpy on the 1.x line; letting NumPy 2 in causes an outright ABI failure.
```
conda create -n sadtalker python=3.10 -y && conda activate sadtalker && pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu118 && conda install -y ffmpeg && pip install -r requirements.txt
```
03
Fetch the weights into checkpoints/ and gfpgan/weights/
The official script pulls two renderer safetensors (691MiB each) and two mapping_*.pth.tar files (~148MiB each) from the v0.0.2-rc release, then grabs enhancement weights from the GFPGAN and facexlib repos. You also need the 3DMM assets: epoch_20.pth (275MiB), BFM_Fitting/01_MorphableModel.mat (230MiB), the s3fd face detector and friends — about 3GB on disk in total. One trap: the audio-branch checkpoints are genuinely named auido2exp_00300-model.pth and auido2pose_00140-model.pth upstream. "audio" is misspelled as "auido" — do not helpfully correct it, or the loader will not find them.
```
bash scripts/download_models.sh
```
04
Render your first clip, then decide whether you want the WebUI
Validate the environment with the bundled example audio before switching to your own assets. The knobs that matter: `--size 512` for resolution, `--batch_size` for render throughput (8–16 is comfortable on a 24GB card), `--expression_scale` for expression intensity, `--pose_style` from 0 to 45 for head-motion style, and `--still` to freeze large head movement. For a GUI, run app_sadtalker.py — but note it calls a bare demo.launch(), which binds only to 127.0.0.1. On a remote box you either forward the port over SSH or change that line to demo.launch(server_name='0.0.0.0'). If you do hit OOM, the official FAQ's remedy is setting PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:128.
```
python inference.py --driven_audio examples/driven_audio/bus_chinese.wav --source_image examples/source_image/full_body_1.png --result_dir ./results --preprocess full --still --enhancer gfpgan --size 512 --batch_size 8
```

What a Presenter Clip Actually Costs

SadTalker outputs a fixed 25fps, so cost is straightforward frame arithmetic. One minute of video = 60 × 25 = 1,500 frames. On an RTX 3090 24GB at $0.193/GPU-hour running 512 with GFPGAN enhancement, estimating about 4 frames per second, rendering takes 1,500 ÷ 4 = 375 seconds; add roughly 45 seconds for 3DMM extraction, audio features and muxing, for about 420 seconds total = 0.1167 hours. 0.1167 × $0.193 ≈ $0.0225 — call it two and a quarter cents for a one-minute enhanced clip. Drop the enhancer and run at 256 and throughput rises to roughly 10 frames per second: 1,500 ÷ 10 = 150 seconds, plus 45 = 195 seconds = 0.0542 hours, × $0.193 ≈ $0.0105, about a penny. Producing 200 one-minute clips: 200 × 0.1167 = 23.34 GPU-hours × $0.193 = $4.50. Switching to an RTX 4090 at $0.540/GPU-hour roughly halves per-clip time to 0.0583 hours, × $0.540 ≈ $0.0315 — about 40% more expensive but delivered twice as fast, which is worth it on a deadline and not worth it in bulk. Storage is separate: 3GB of weights plus a ~12GB conda environment plus source assets and renders, call it 20GB at 20 × $0.414 = $8.28/month. Note that compute billing stops the moment the instance stops, while storage keeps billing until you destroy it. Downloading 200 finished clips at 30MB each is 6GB × $0.0081 ≈ $0.05 in egress — effectively noise. Everything is metered per second, with no minimum, no setup fee and no quota request.

04 —

FAQ

How much VRAM does SadTalker actually need? Is an 8GB card enough?

Yes, comfortably. SadTalker keeps only about 1.4GB of parameters resident, and the default `--size 256 --batch_size 2` runs in roughly 4GB. Moving to 512 with GFPGAN face enhancement puts the full chain around 8–10GB. Two things push it higher: raising `--batch_size`, and using `--preprocess full` on a 1080p source while also enabling realesrgan background enhancement, which can reach past 12GB. So VRAM is rarely the bottleneck — render time is. On NexGPU an RTX 3090 24GB costs $0.193/GPU-hour, giving you headroom to spare and room to max out batch size, which makes it the best value pairing for this model.

Is SadTalker still maintained? Is it still worth using?

The last functional commit to main was October 2023, more than 600 issues sit unaddressed, the official Hugging Face Space is in BUILD_ERROR, and the entire OpenTalker org has gone quiet. It is still worth using anyway: Apache 2.0 licensed, 3GB of weights total, minutes per clip on a consumer card. Nothing has taken over that "cheap and good enough" niche — LivePortrait, MuseTalk, EchoMimicV2 and Hallo2 all look better and all cost an order of magnitude more compute. The pragmatic path is to prove out your pipeline on SadTalker, then evaluate a diffusion approach once volume justifies it. With per-second billing on NexGPU, running both against the same batch of source material as an A/B costs a few dollars.

How do I fix No module named 'torchvision.transforms.functional_tensor'?

This is the most frequent SadTalker install failure, and the cause is the pinned basicsr==1.4.2 in requirements.txt: basicsr/data/degradations.py imports torchvision.transforms.functional_tensor, a module removed in torchvision 0.17. Two fixes. Pin torchvision back to 0.16.2 (recommended — it pairs cleanly with torch 2.1.2+cu118), or edit that import line directly and change functional_tensor to functional. While you are there, keep numpy on 1.x; requirements specifies 1.23.4, and letting NumPy 2 in produces an immediate ABI crash. Working from a clean NexGPU PyTorch image, sorting these version conflicts takes a few minutes instead of repeatedly polluting your local conda setup.

Can an RTX 4090 or RTX 5090 run SadTalker?

The 4090 can; the 5090 takes real work. The torch 1.12.1+cu113 build the README specifies only compiles up to Ampere (sm_86), so running it on a 4090 (Ada, sm_89) throws "no kernel image is available for execution on the device" — you must move to torch 2.x with cu118 or later. The RTX 5090 is Blackwell (sm_120), requiring torch 2.7+ on CUDA 12.8, which drags basicsr, facexlib and the rest of that aging dependency chain forward with it. Unless you have another reason to want Blackwell, stay on the Ampere generation for SadTalker — NexGPU's RTX 3090 24GB at $0.193/GPU-hour and A10 24GB at $0.414/GPU-hour are both zero-friction choices.

How is SadTalker different from Wav2Lip and MuseTalk, and which should I pick?

The inputs differ fundamentally. Wav2Lip inpaints the lip region of existing footage — it changes the mouth, never the head. MuseTalk targets real-time lip sync and also needs a base video. SadTalker needs only a single still photo, generating head pose through PoseVAE and expression through ExpNet, so it turns and blinks and can manufacture a presenter clip from nothing. If all you have is an ID photo or an AI-generated portrait, SadTalker is essentially the only option; if you already have real footage and just want new dialogue, Wav2Lip or MuseTalk will look more natural and run faster. NexGPU carries 2,000+ prebuilt images across 2,498 rentable GPUs, so running all three against the same audio on separate instances gets you an answer within the hour.

How long does a one-minute avatar video take, and can I run it in batch?

At 25fps a minute is 1,500 frames. On an RTX 3090 at 512 with GFPGAN, expect roughly seven minutes per clip; drop the enhancer and run 256 and it is closer to three. An RTX 4090 roughly halves both. The key to batch throughput is raising `--batch_size` above the default of 2 — 8 to 16 is stable on a 24GB card and improves throughput noticeably. At real scale, NexGPU nodes take up to 14 GPUs with a maximum node VRAM of 2,152GB, so you can shard the asset list and render in parallel. With 1,175 verified rentable nodes across 51 countries and regions, launching near your source material also cuts upload time. SSH, Jupyter, web terminal, REST API and CLI are all available, so polling a job queue from a script is trivial. Questions go to bilingual support over Telegram — no ticket queue.

More in Digital humans and face animation

Every model guide

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.