Avatar / Audio-Driven Video

EMO local deployment: the weights were never released — here's the path that actually works

EMO is Alibaba's audio-driven portrait video model. The paper, the demos and 7.6k GitHub stars are all there. The code and the checkpoints are not. This page tells you which GPU to rent and which model to actually run to get the EMO result.

EMO · self-hosted

EMO stands for Emote Portrait Alive, arXiv 2402.17485, by Linrui Tian, Qi Wang, Bang Zhang and Liefeng Bo at Alibaba Group's Institute for Intelligent Computing, published at ECCV 2024. The design is two-stage: a ReferenceNet encodes the reference image and motion frames, then the diffusion backbone carries three kinds of attention — Reference-Attention to hold identity, Audio-Attention driven by wav2vec features to modulate lip movement and expression, and Temporal Modules for frame-to-frame coherence. You give it one portrait and one vocal track; the output length simply follows the audio. Singing, speech, multiple languages, oil paintings and AI-generated faces all drive it. That is why it went viral.

Then everyone who tries to build with it hits the same wall: the HumanAIGC/EMO repository contains no inference code. The README is badges, demo video links and a BibTeX entry. No checkpoint was ever uploaded, and the bulk of the 200-plus open issues ask the same question — when is the code coming. Officially there is exactly one way to use EMO itself: the emo-detect-v1 and emo-v1 models on Alibaba Cloud Model Studio (DashScope). The limits are hard — reference image between 400px and 7,000px on an edge, audio in WAV or MP3 capped at 15MB and 60 seconds, output locked to 512×512 for a 1:1 dynamic region or 512×704 for 3:4, priced at ¥0.08 per second for 1:1 and ¥0.16 per second for 3:4, QPS 5, one concurrent task, and result URLs that expire after 24 hours. The sequel, EMO2 (arXiv 2501.10687), maps audio to hand end-effector poses first and then drives a half-body performance with gestures — and is likewise paper-and-demo only.

So the honest answer to "how do I self-host EMO" is not to hunt for EMO weights. It is to pick an architecturally similar model that actually ships. Three lines work today. Ant Group's EchoMimicV3: 1.3B parameters, Apache 2.0, accepted to AAAI 2026, built on Wan2.1-Fun-V1.1-1.3B-InP, with a Flash variant that generates in 8 steps, needs no face mask, and starts at 12GB of VRAM. Alibaba's Wan2.2-S2V-14B: Apache 2.0, cinematic 480P/720P speech-to-video, but the official single-GPU script wants 80GB. And MeiGen's InfiniteTalk, built on Wan2.1-I2V-14B-480P, purpose-built for long-form dubbing and lip re-sync with an unbounded streaming mode. What they have in common is that your VRAM bill depends entirely on which line you choose — and on NexGPU, per-second billing with no minimum means bouncing between a 3090 and an A100 to find out costs almost nothing.

01 —

The EMO family, and the open models you can actually run

The first two rows are the state of the original EMO. The rest are what you can clone onto your own box.

Version	Parameters	VRAM	Context	Notes
EMO (Emote Portrait Alive)	SD1.5 diffusion backbone + ReferenceNet + temporal modules; parameter count never published	No weights to download; cloud emo-v1 API only	Audio ≤ 60s / ≤ 15MB, output 512×512 or 512×704	arXiv 2402.17485, ECCV 2024. The repo holds only a README and citation; the authors state it is for academic research and demonstration.
EMO2 (End-Effector Guided)	Two stages: audio → hand end-effector pose → half-body video	Also unreleased, no weights	Half-body with gestures, length follows audio	arXiv 2501.10687. The key insight: audio correlates weakly with full-body motion but strongly with hand pose, so generate the hands first.
EchoMimicV3-Flash	1.3B on a Wan2.1-Fun-V1.1-1.3B-InP base + wav2vec2	bf16 from roughly 12GB; 24GB recommended for long 768×768 clips	Up to 768×768, 8-step sampling	From Ant Group, Apache 2.0. No face mask required — currently the most realistic way to get an EMO-like result on a single card.
EchoMimicV3-Preview	1.3B, bf16 safetensors	~16GB via the ComfyUI workflow; ~12GB via the quantized Gradio UI	Up to 768×768; 5 steps for talking head, 15–25 for talking body	Officially validated on A100 80G, RTX4090D 24G and V100 16G. Accepted to AAAI 2026. Needs CUDA 12.1+ and Python 3.10/3.11.
Wan2.2-S2V-14B	14B speech-to-video model	Official single-GPU inference asks for ≥ 80GB; the ComfyUI wan2.2_s2v_14B_fp8_scaled weights need considerably less	480P / 720P, frame count auto-aligned to the audio, controllable with --num_clip	Alibaba's open Wan line, Apache 2.0. The best-looking output of the group — paid for in VRAM.
InfiniteTalk / Sonic / Hallo2	InfiniteTalk: Wan2.1-I2V-14B-480P + audio-condition LoRA; Sonic: SVD-XT + Whisper-tiny; Hallo2: SD1.5 + AnimateDiff + wav2vec2	InfiniteTalk supports FP8 single-GPU and a low-VRAM mode via --num_persistent_param_in_dit 0; Sonic is validated on a single 32GB GPU; Hallo2 was tested on A100	InfiniteTalk defaults to 1,000 frames (~40s) with unlimited streaming; Hallo2 reaches 4K and up to an hour	Check the licences: InfiniteTalk is Apache 2.0, Sonic is CC BY-NC-SA 4.0 and non-commercial, and Hallo2's upscaling stage inherits CodeFormer's S-Lab License 1.0.

02 —

Pick the card for the line you're running

A 1.3B EchoMimic and a 14B Wan2.2-S2V are an order of magnitude apart. Don't force them onto the same GPU.

Get EchoMimicV3-Flash running and produce a few sample clips
RTX 3090 24GB$0.193/GPU-hour
Flash needs 12GB; 24GB leaves comfortable headroom for 768×768 — and this is the cheapest 24GB card on the list.
Batch production through the ComfyUI workflow at full 768×768 without lowering partial_video_length
RTX 4090 24GB$0.540/GPU-hour
RTX4090D 24G is one of the officially validated configs, and Ada throughput makes 15–25 step talking-body sampling noticeably faster.
Run Sonic (only validated on a single 32GB GPU) or InfiniteTalk in FP8 single-GPU mode
RTX 5090 32GB / RTX A6000 48GB$0.723 / $0.817 per GPU-hour
32GB matches Sonic's reference configuration exactly; 48GB gives a 14B FP8 base and long sequences room to breathe.
Wan2.2-S2V-14B single-GPU inference via the official generate.py, or Hallo2 producing 4K long-form video
A100 PCIE 80GB$0.824/GPU-hour
The docs state single-GPU inference needs at least 80GB; this is the cheapest card that clears the bar. Step up to H100 SXM 80GB ($3.582/GPU-hour) for speed.

03 —

Four steps to an audio-driven avatar on NexGPU

The first three steps cover the open-weight route. Step four is for when you genuinely need EMO itself.

01
Spin up an instance with CUDA 12.1+
Choose a PyTorch or ComfyUI image from the 2,000+ prebuilt images — EchoMimicV3 requires CUDA 12.1 or newer and Python 3.10 or 3.11. Confirm the GPU and driver line up before anything else. SSH, Jupyter and the web terminal are all available.
```
nvidia-smi && python -c "import torch; print(torch.__version__, torch.cuda.get_device_name(0))"
```
02
Clone EchoMimicV3 and install dependencies
Validated on CentOS 7.2 and Ubuntu 22.04. Use a dedicated conda environment so it does not collide with the diffusers version baked into the image.
```
git clone https://github.com/antgroup/echomimic_v3 && cd echomimic_v3 && conda create -n echomimic_v3 python=3.10 -y && conda activate echomimic_v3 && pip install -r requirements.txt
```
03
Pull three sets of weights, then generate
You need three things: the Wan2.1-Fun-V1.1-1.3B-InP base, an audio encoder (wav2vec2-base, or chinese-wav2vec2-base for Mandarin material), and either the EchoMimicV3-Flash or Preview weights, each in its own folder. The maintainers publish the sweet spots: audio CFG 1.8–2.0, text CFG 3–6, 5 sampling steps for a talking head and 15–25 for a talking body. If VRAM gets tight, drop partial_video_length to 81 or 65.
```
bash run_flash.sh    # or python infer_preview.py; python app_mm.py for the web UI
```
04
Fall back to the async API when you truly need EMO itself
Call emo-detect-v1 first to obtain face_bbox and the dynamic-region ext_bbox, then submit those alongside image_url and audio_url to emo-v1. The X-DashScope-Async: enable header is mandatory; poll GET /api/v1/tasks/{task_id} with the returned id. style_level accepts normal, calm or active. Both the task id and the result URL expire after 24 hours, so copy finished videos into your own object storage immediately.
```
curl -X POST 'https://dashscope.aliyuncs.com/api/v1/services/aigc/image2video/video-synthesis' -H 'X-DashScope-Async: enable' -H "Authorization: Bearer $DASHSCOPE_API_KEY" -H 'Content-Type: application/json' -d '{"model":"emo-v1","input":{"image_url":"...","audio_url":"...","face_bbox":[],"ext_bbox":[]},"parameters":{"style_level":"normal"}}'
```

What this evaluation actually costs

Real arithmetic. Line one: rent an RTX 3090 24GB at $0.193/GPU-hour. Environment setup plus pulling three sets of weights runs about 40 minutes; generating 20 sample clips with 8-step EchoMimicV3-Flash takes roughly 2 hours — 2.7 hours total, so 0.193 × 2.7 ≈ $0.52. Want to sanity-check the quality against the 14B line? Move to an A100 PCIE 80GB for Wan2.2-S2V-14B: $0.824/GPU-hour × 3 hours = $2.47. Both lines together come to $2.99 — less than a coffee for a side-by-side comparison on your own footage. Storage is separate: roughly 40GB of weights and assets at $0.414/GB-month × 40 = $16.56 per month, but compute billing stops the moment the instance stops, and storage only keeps running until the volume is destroyed — delete the weights when you're done and that line goes to zero. Exporting 20 clips is about 1GB, so egress is $0.0081/GB × 1 ≈ $0.01. For contrast, the official EMO API: ¥0.08 per second for 1:1, so 20 clips of 5 seconds is 100 seconds, or ¥8. Cheaper and simpler, yes — but capped at 512×512, capped at 60 seconds, one concurrent task, and you cannot change a single parameter or attach a single LoRA.

04 —

FAQ

Is EMO open source? Can I download the weights and deploy locally?

No. The HumanAIGC/EMO repository has only ever contained a README, demo videos and a BibTeX entry — no inference code, no checkpoints — and most of the 200-plus open issues are still asking for a release. Anything online claiming to be "EMO weights" is not official. What you can genuinely self-host is EchoMimicV3, Wan2.2-S2V-14B or InfiniteTalk, all Apache 2.0. Spin up one GPU on NexGPU and you'll know within ten minutes whether they look close enough to EMO for your use case.

How much VRAM does EMO need locally?

The question has to be reframed, because EMO has no runnable weights — your VRAM budget depends entirely on which model replaces it. EchoMimicV3-Flash states a 12GB floor, the ComfyUI workflow wants about 16GB, and long 768×768 clips are comfortable at 24GB. Sonic is only validated on a single 32GB GPU. Wan2.2-S2V-14B's official script asks for at least 80GB on one card. NexGPU covers the whole span, from RTX 3090 24GB at $0.193/GPU-hour to A100 SXM4 80GB at $1.088 to H200 141GB, billed per second — so guessing wrong just means switching cards.

What is the difference between EMO and EMO2, and which should I read?

EMO (arXiv 2402.17485) works at portrait scale: one face plus audio, producing a head-and-shoulders video with matched expression and lip sync. EMO2 (arXiv 2501.10687) tackles the upper body — the authors found audio correlates weakly with full-body motion but strongly with hand pose, so they generate hand end-effector poses from audio first and use those to guide a gesture-driven half-body synthesis. For e-commerce presenters or explainer video with gestures, EMO2's framing is the right one — but neither released code. To reproduce that idea, InfiniteTalk and Wan2.2-S2V-14B are the closest open bases, and NexGPU's A6000 48GB and A100 80GB will run them from a cold boot.

Any licence traps if I build a commercial avatar product on these?

Yes, and this is where teams most often trip. EchoMimicV3, Wan2.2-S2V-14B and InfiniteTalk are Apache 2.0 and commercially usable. Sonic is CC BY-NC-SA 4.0 — explicitly non-commercial, with Tencent Cloud's paid service as the commercial route. Hallo2's high-resolution stage uses a modified CodeFormer and inherits the S-Lab License 1.0. All of them also make you responsible for generated content, so likeness rights and platform compliance are still on you. Once you've chosen a clean licence, run it as your own private inference service on NexGPU — the weights and the footage never leave your instance.

Is a 12GB or 16GB card enough for audio-driven avatars?

For EchoMimicV3, yes. Flash starts at 12GB, ComfyUI at 16GB, and dropping partial_video_length to 81 or 65 saves more still — the maintainers even validated V100 16G. Do not expect the 14B line (Wan2.2-S2V, full-precision InfiniteTalk) to fit in 16GB. NexGPU's Tesla V100 32GB is $0.188/GPU-hour and RTX 3090 24GB is $0.193 — rather than fighting OOM on a local 12GB card, rent a roomy one by the second and get the clips out.

I need hundreds of presenter clips. What's the cheapest way to parallelise?

Audio-driven video is embarrassingly parallel — one clip per process, no model parallelism needed. The 1.3B EchoMimicV3 scales out beautifully: run several RTX 4090 24GB cards ($0.540/GPU-hour) each chewing through its own queue, and throughput beats a single H100 running serially by a wide margin. Only reach for A100/H100 when you need 720P cinematic quality. NexGPU supports up to 14 GPUs per node with a max node VRAM of 2,152GB, across 1,175 verified rentable nodes and 2,498 GPUs in 51 countries and regions — no quota request, no minimum, and compute billing stops the moment the queue finishes and the instance stops.

More in Digital humans and face animation

Every model guide

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.