Avatar · Portrait animation

Self-hosting LivePortrait: one photo, one driving video, and a 24GB card with room to spare

An implicit-keypoint framework of roughly 130M parameters that lives in under 300MB of fp16 VRAM — and already ships inside Kuaishou, Douyin, Jianying and WeChat Channels. On NexGPU it starts at $0.193 per GPU-hour.

LivePortrait · self-hosted

LivePortrait is not a diffusion model. It extends the FOMM-lineage implicit-keypoint framework, scales training to roughly 69 million high-quality frames with a mixed image-video strategy, and adds one stitching module plus two retargeting MLPs so eyes and lips can be driven independently and the result pastes back into the source frame without a visible seam. That choice sets the whole cost curve. The repo's own speed.py, measured on an RTX 4090 with torch.compile, reports 0.82ms for the appearance feature extractor, 0.84ms for the motion extractor, 7.59ms for the SPADE generator, 5.21ms for the warping module and 0.31ms for the stitching and retargeting heads — 14.77ms a frame, about 68 FPS of pure model throughput. A diffusion-based lip-sync model spends dozens of denoising steps on the same frame.

The repo has changed nameplates twice. The original KwaiVGI/LivePortrait now redirects to github.com/KlingAIResearch/LivePortrait, and the weights live in the HuggingFace repo KlingTeam/LivePortrait, 2.14GB in total. The code is MIT — but the InsightFace face-detection models it ships with are licensed for non-commercial research only, which is the trap most self-hosters fall into, and it is legal rather than technical. The last substantive model change was Animals v1.1 on 1 January 2025 (more training data, dogs' mouths no longer mistaken for noses, enabled by default); commits since then are documentation and link maintenance. That is not abandonment — it is a model stable enough that production video platforms ship it as-is.

VRAM is never what stops you. What stops you is the X-Pose custom CUDA op, MultiScaleDeformableAttention, that animals mode makes you compile on the spot; the hard version pins in requirements.txt (numpy 1.26.4, onnxruntime-gpu 1.18.0, transformers 4.38.0); FFmpeg that must be on PATH before inference.py will even start; driving videos that need a 1:1 crop, a head-focused frame and a neutral frontal expression on frame one; and the fact that CUDA 12.4 or 12.6 on Windows causes known problems, so the maintainers tell you to drop back to 11.8. You either hit all of that once, or you turn your workstation into a dependency swamp. A clean Linux GPU box you can spin up and throw away is the cheap way out.

01 —

Variants and weights: which files you actually need

The KlingTeam/LivePortrait repo on HuggingFace is 2.14GB. Here is what each part weighs and what it is for.

Version	Parameters	VRAM	Context	Notes
LivePortrait Humans (liveportrait/base_models)	~130M total (appearance 0.84M + motion 28.12M + warping 45.53M + SPADE 55.37M + stitch/retarget 0.23M)	522MB fp32 on disk, ~261MB resident in fp16	256×256 implicit-keypoint input, source max dimension 1280, 25fps CRF-15 mp4 output by default	The default mode, driving either a still portrait or a portrait video (v2v). Runs on Linux, Windows and macOS Apple Silicon — though Apple Silicon is roughly 20x slower than an RTX 4090.
LivePortrait Animals v1.1 (default since 2025/01/01)	Same ~130M architecture, plus xpose.pth for keypoint detection	520MB of base weights + 435MB xpose.pth	Same 256×256 input; cats and dogs only	Retrained on more data. Cats improve marginally, dogs noticeably — the older checkpoint kept reading a dog's mouth as its nose. Only tested on Linux and Windows with an NVIDIA GPU, and it depends on X-Pose compiling successfully.
LivePortrait Animals original (base_models, no _v1.1 suffix)	Same ~130M architecture	520MB of weights	Same as v1.1	Set version_animals back to an empty string in src/config/inference_config.py to reproduce the August 2024 behaviour. Both checkpoints still sit side by side in the HF repo.
Stitching & retargeting modules (stitching_retargeting_module.pth)	0.23M across three chained MLPs	2.3MB	Three heads: stitching, eye retargeting, lip retargeting	The cheapest part of the pipeline — 0.31ms for all three on a 4090 — and the reason the animation pastes back seamlessly and lets you drive eyes or lips on their own. Note that animals mode currently reuses the humans copy of this file.
Support models: landmark.onnx + InsightFace buffalo_l	det_10g detector + 2d106det landmarks	landmark.onnx 114.7MB, det_10g 16.9MB, 2d106det 5.0MB	Executed through onnxruntime-gpu 1.18.0 on the CUDA EP, allocating separately from the PyTorch path	This is where the licence bites: LivePortrait's code is MIT, but the InsightFace models are non-commercial research only. Commercial deployments swap in something like MediaPipe — kijai's ComfyUI node ships that switch — at a small cost in small-face detection accuracy.
FasterLivePortrait (community TensorRT / ONNX build)	Same weights exported to ONNX and TRT engines	Engine files are the same order of magnitude as the source weights	30+ FPS measured on an RTX 3090 including pre- and post-processing; webcam realtime, multi-face and audio-driven modes	Hard constraints: TensorRT must be 8.x (10.x and up are incompatible) and cuDNN must be 8.x (9.x is not), and grid_sample needs a custom plugin build. The path of least resistance is docker pull shaoguo/faster_liveportrait:v3.

02 —

Picking a card: LivePortrait eats compute, not VRAM

With under 300MB of fp16 weights, the question on a 24GB card is how many workers you run, not whether the model fits.

First run of humans mode, plus Gradio sessions for tuning expression and pose
RTX 3090 24GB$0.193/GPU-hour
Ampere sm_86 lands squarely inside the CUDA 11.8 / 12.1 + torch 2.3 combination the repo validates, 24GB is enormous overkill for this model, and it wins outright on frames per dollar.
TensorRT pipeline chasing 25fps-plus interactive or live output
RTX 4090 24GB$0.540/GPU-hour
12.8ms of the 14.77ms per frame goes into the SPADE generator and warping module, exactly what Ada's FP16 throughput compresses; pair it with a FasterLivePortrait TRT engine and one card carries several concurrent streams.
A batch rendering API that stays up 24/7
A10 24GB$0.414/GPU-hour
A passively cooled datacentre card holds its clocks under sustained load, and 24GB fits 8 to 10 worker processes side by side at roughly 2GB each (fp16 weights + its own CUDA context + the onnxruntime arena).
Consolidating dozens of concurrent streams onto a single card
A100 PCIE 80GB$0.824/GPU-hour
80GB holds dozens of independent CUDA contexts and the HBM bandwidth suits the grid_sample-heavy warping module — but it only pays off if you genuinely saturate it. Below that, several 3090s cost less for the same throughput.

03 —

From rented card to first rendered clip, in four steps

SSH into a NexGPU PyTorch image and these lines are the whole process.

01
Spin up a clean Linux instance with Python 3.10 and torch 2.3
The repo pins its dependencies hard: numpy 1.26.4, onnxruntime-gpu 1.18.0, transformers 4.38.0, gradio 5.1.0. Do not let pip drift to numpy 2.x. FFmpeg must also be on PATH — inference.py checks for it before anything else and exits if it is missing.
```
conda create -n LivePortrait python=3.10 -y && conda activate LivePortrait && pip install torch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 --index-url https://download.pytorch.org/whl/cu121 && pip install -r requirements.txt
```
02
Pull the 2.14GB of weights
Official weights live at KlingTeam/LivePortrait on HuggingFace. Confirm you end up with pretrained_weights/liveportrait, liveportrait_animals and insightface. If you only need human portraits, skip the animals folder and save the 435MB xpose.pth.
```
huggingface-cli download KlingTeam/LivePortrait --local-dir pretrained_weights --exclude "*.git*" "README.md" "docs"
```
03
Render your first clip, and crop the driving video properly
Driving video should be 1:1 (512×512 or 256×256), framed on the head, with minimal shoulder movement, and frame one must be a frontal face with a neutral expression — otherwise the relative-motion baseline is skewed from the start. If your own footage does not fit, add --flag_crop_driving_video and adjust with --scale_crop_driving_video and --vy_ratio_crop_driving_video. Use --animation_region to drive only exp, pose, lip or eyes. Converting the driving video to a .pkl motion template both speeds up inference and keeps the original footage off your disk.
```
python inference.py -s assets/examples/source/s9.jpg -d assets/examples/driving/d0.mp4 --flag_crop_driving_video
```
04
Enable animals mode, then claw back another 20-30%
Cats and dogs require compiling X-Pose's MultiScaleDeformableAttention op locally, and your nvcc version must match the CUDA version torch was built against — this is where most self-hosting attempts die. Once it runs, add --flag_do_torch_compile: the first inference triggers about a minute of optimisation, after which everything is 20-30% faster. Linux only.
```
cd src/utils/dependencies/XPose/models/UniPose/ops && python setup.py build install && cd - && python inference_animals.py -s assets/examples/source/s39.jpg -d assets/examples/driving/wink.pkl --driving_multiplier 1.75 --no_flag_stitching
```

What 1,000 fifteen-second talking-head clips actually cost

A 15-second clip at 25fps is 375 frames. FasterLivePortrait's TensorRT pipeline measures 30+ FPS on an RTX 3090 with pre- and post-processing included, so 375 ÷ 30 ≈ 12.5 seconds of GPU time; budget 20 seconds per clip once face detection, cropping and ffmpeg encoding are counted. NexGPU's RTX 3090 24GB is $0.193/GPU-hour, so one hour yields 3600 ÷ 20 = 180 clips at $0.193 ÷ 180 ≈ $0.0011 each — about $1.07 for a thousand. Storage is separate: the full 2.14GB weight set at the $0.414/GB-month median is $0.89 a month, while keeping only the humans stack (522MB of weights + 114.7MB landmark.onnx + 22MB of detectors ≈ 0.66GB) costs $0.27. At roughly 3MB per clip, egressing all 1,000 is 3GB × $0.0081/GB ≈ $0.024. Billing is metered per second and priced per hour with no minimum and no setup fee; compute stops billing the moment the instance stops, and only storage keeps accruing until you destroy it.

04 —

FAQ

How much VRAM does LivePortrait actually need?

Add it up: 522MB of fp32 weights become roughly 261MB resident once the default half-precision flag kicks in; landmark.onnx at 114.7MB and the two InsightFace detectors at 22MB allocate inside onnxruntime's arena; add a CUDA context and 256×256 activations and a single stream fits comfortably inside 4GB. This has never been a memory-bound model — the real question is how many streams you run and how fast they finish. On NexGPU an RTX 3090 24GB is $0.193/GPU-hour, enough to saturate concurrency on one card, metered per second.

Has LivePortrait been renamed, and is it still maintained?

KwaiVGI/LivePortrait now redirects to github.com/KlingAIResearch/LivePortrait, having passed through KlingTeam along the way; old links resolve automatically, and the weights repo on HuggingFace is still KlingTeam/LivePortrait. The last substantive model update was Animals v1.1 on 1 January 2025; everything since has been docs and links. The maintainers note it is the portrait-animation solution running inside Kuaishou, Douyin, Jianying and WeChat Channels — stable rather than stalled. Spin up a 3090 on NexGPU and you can verify the output yourself in about ten minutes.

Can I use LivePortrait commercially?

The code is MIT, so yes. The models are where it catches: the default InsightFace buffalo_l detectors (det_10g.onnx and 2d106det.onnx) are licensed for non-commercial research only. Commercial deployments replace the detection stage — usually with Apache-2.0 MediaPipe, which kijai's ComfyUI-LivePortraitKJ exposes as a switch — at a modest cost in small-face accuracy. That swap needs side-by-side comparison, which is exactly what a per-second-billed instance running two environments is for.

Why is my throughput nowhere near the 12.8ms the paper quotes?

Because that figure is pure forward pass. The repo's own speed.py on a 4090 measures 0.82 + 0.84 + 7.59 + 5.21 + 0.31 = 14.77ms, and still excludes face detection, crop alignment, paste-back and ffmpeg encoding. The realistic end-to-end reference is FasterLivePortrait's 30+ FPS on an RTX 3090 with pre- and post-processing counted. Three levers: pre-generate .pkl motion templates, run --flag_do_torch_compile on Linux, and move to TensorRT. All three are worth benchmarking on a NexGPU 4090 at $0.540/GPU-hour before you commit to a card long term.

Will an RTX 5090 run LivePortrait?

Yes, but not by following the repo's install steps verbatim. Blackwell is sm_120 and needs CUDA 12.8 or newer with torch 2.7+ wheels, whereas the repo pins torch 2.3 and onnxruntime-gpu 1.18, and FasterLivePortrait additionally demands TensorRT 8.x and cuDNN 8.x. You end up lifting the entire dependency chain, and the numpy 1.26.4 pin goes with it. Unless that migration is the point, the 3090 ($0.193), 4090 ($0.540) and A10 ($0.414) are far less trouble — and on NexGPU all three start per-second, so trying them costs almost nothing.

Animals mode won't install — MultiScaleDeformableAttention fails to build. Now what?

That is X-Pose's custom CUDA op, and it only compiles when nvcc matches the CUDA version torch was built with. Run nvcc -V first and install the matching torch wheel. The maintainers also warn that higher CUDA versions on Windows (12.4, 12.6) cause unknown issues and suggest dropping to 11.8, and macOS cannot run animals mode at all because X-Pose does not support it. Building from scratch on a clean Linux box has by far the highest success rate — NexGPU's Ubuntu and PyTorch images ship with nvcc, and if you wreck the environment you destroy it and start again.

More in Digital humans and face animation

Every model guide

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.