Skip to main content
NexGPU

Avatar & Image Animation

Run FOMM yourself: one still image, one driving video, under 2GB of VRAM

First Order Motion Model is the NeurIPS 2019 first-order motion model — MIT licensed, 10 self-supervised keypoints, 256x256 output. Inference weights total just 228MB. On NexGPU that is a Tesla V100 32GB at $0.188/GPU-hr or an RTX 3090 24GB at $0.193/GPU-hr, billed per second.

Sign up freeView pricing

FOMM stands for First Order Motion Model for Image Animation, by Aliaksandr Siarohin, Stephane Lathuiliere, Sergey Tulyakov, Elisa Ricci and Nicu Sebe, out of the University of Trento and Snap, published at NeurIPS 2019. The repo lives at github.com/AliaksandrSiarohin/first-order-model under an MIT license. Its counter-intuitive trick is that it uses no facial landmark supervision at all: it learns 10 keypoints without labels, takes a first-order Taylor expansion around each one to get a local affine transform, stitches those into a dense optical flow field, and hands the result to an occlusion-aware generator that inpaints the pixels the warp could not cover. Because it knows nothing about what a face is, the same architecture drives faces, full-body tai chi, fashion models, animal silhouettes and even a robot arm.

The 2020 wave of talking-photo apps, the Dame da ne meme videos and Avatarify's live Zoom face swaps were nearly all running FOMM's vox-adv weights underneath. The lineage has moved three generations since: the same lead author built MRAA at Snap Research (CVPR 2021, aimed at articulated objects — that "Check out our new paper" banner at the top of the FOMM README points to it), the community shipped Thin-Plate Spline Motion Model (CVPR 2022, swapping thin-plate splines in for the first-order affine transforms), and Kuaishou's KwaiVGI released LivePortrait (2024), which pushed the same implicit-keypoint warping into production-grade lip and gaze retargeting. So be clear about what you are downloading: today FOMM is the baseline and the textbook, not the state of the art. It has not been deleted, renamed or archived, and it still runs end to end.

That is exactly what makes it the best model in this family to run yourself. The full inference stack is 228MB of weights — 54.2MB for the keypoint detector and 174MB for the generator, per Qualcomm AI Hub's float ONNX export — so the cheapest card in the fleet can render clips all day. And its training recipe is honest about what it costs: the official vox-256.yaml specifies batch_size=40, num_epochs=100, num_repeats=75, and the README says outright that this batch "is tuned to run on 2 or 4 Titan-X gpu". NexGPU covers both ends: a Tesla V100 32GB from $0.188/GPU-hr to validate, or four RTX 4090s aggregating 96GB to reproduce — metered per second, no minimum, no setup fee, and compute billing stops the moment the instance stops.

01 —

The official checkpoints, and what each one actually drives

All 256x256, all MIT — the filenames and byte sizes below match the Google Drive / Yandex Disk listings exactly

VersionParametersVRAMContextNotes
vox-256 (vox-cpk.pth.tar)10 keypoints / 728.8MB checkpointUnder 2GB peak at fp32 inference; training at batch_size=40 needs 40GB-class VRAM or multiple cards256x256 output; driving video must be cropped to a face-centred squareThe VoxCeleb face model. Default choice for talking-head avatars and photo revival, and the one checkpoint that appears in nine out of ten tutorials.
vox-adv-256 (vox-adv-cpk.pth.tar)10 keypoints / same architecture plus adversarial lossSame sub-2GB inference footprint as vox-256; training additionally keeps a discriminator resident256x256 output; same face-crop requirementThe GAN-trained face variant — sharper texture, fewer smeared edges. This is the checkpoint Avatarify-style live face-swap tools shipped with.
taichi-256 / taichi-adv-256 (taichi-cpk.pth.tar)10 keypoints / 750.9MB checkpointUnder 2GB inference; leave a few extra GB for decode and write-out buffers on busy full-body frames256x256 output; needs the full body in frame, one subject, simple backgroundTrained on tai chi video. One of the few FOMM weights that animates an entire person rather than just a head — reach for it for full-body pose transfer demos.
fashion-256 (fashion.pth.tar)10 keypoints / 750.8MB checkpointUnder 2GB inference; in batch rendering the bottleneck is ffmpeg encoding, not VRAM256x256 output; plain background, model standing and turningTrained on the Fashion Video dataset. The shortest path to making a static e-commerce model shot turn around, and it handles skirt and fabric deformation far more stably than the face weights.
mgif-256 (mgif-cpk.pth.tar)10 keypoints / quadruped side-view silhouettesUnder 2GB inference; the source material is low-resolution GIF, so VRAM is never the constraint256x256 output; flat-background side-profile silhouettesThe MGif dataset is all animal silhouettes. Cheapest way to sanity-check that FOMM generalises past humans, and it produces a watchable result in minutes.
bair-256 (bair-cpk.pth.tar)10 keypoints / 750.8MB checkpointUnder 2GB inference; sequences are short, so one card can run several streams concurrently256x256 output; fixed overhead camera on a tabletopThe BAIR robot-pushing dataset, included to show the first-order motion representation does not care whether the subject is alive. Still cited as a control in robot video-prediction work.

02 —

Picking a card: check which GPU architectures torch 1.12.1 covers before you check VRAM

The official requirements.txt pins PyTorch to 1.12.1, and that constraint rules out cards long before memory does

  • Run inference on the exact pinned environment, at the lowest possible cost

    Tesla V100 32GB$0.188/GPU-hr

    V100 is sm_70, which the official torch 1.12.1 wheels ship native kernels for — clone, pip install -r requirements.txt, run, without editing a single line for compatibility. And 32GB against a 228MB model is pure headroom.

  • Batch rendering, with a 256x256-to-1024 upscaler resident on the same card

    RTX 3090 24GB$0.193/GPU-hr

    sm_86 is also inside 1.12.1's build targets, and 24GB comfortably holds FOMM inference plus a GFPGAN or Real-ESRGAN pass at the same time, so frames never round-trip through disk.

  • Reproduce the official vox-256 training recipe (batch_size=40) on a single card

    RTX A6000 48GB$0.817/GPU-hr

    The README states that batch was tuned for 2 to 4 Titan X cards at 12GB each; 48GB on one card absorbs in a single device what used to need four. FOMM is also fp32 throughout with no AMP path in the code, which is exactly where the A6000's fp32 throughput pays off.

  • Replicate the official --device_ids 0,1,2,3 multi-GPU run and finish faster

    4 x RTX 4090 24GB$0.540/GPU-hr ($2.160/hr for four)

    96GB aggregate dwarfs the original 4x Titan X, and DataParallel over PCIe is plenty at 256x256. One caveat: Ada is sm_89 and is not in 1.12.1's compiled targets, so a 4090 means moving to PyTorch 2.x and dealing with the torch.load and numpy breaking changes that come with it.

03 —

Instance to finished clip, in four steps

Every command below is straight from the official repo, unmodified

  1. 01

    Boot an instance and install the pinned dependencies

    Start from a PyTorch prebuilt image in the NexGPU console and get in over SSH or Jupyter. requirements.txt is only fifteen packages, all hard-pinned: torch==1.12.1, torchvision==0.13.1, numpy==1.23.3, scikit-image==0.19.3, imageio==2.22.0. numpy stops at 1.23.3 for a reason — 1.24 removed the np.float and np.int aliases, and upgrading on your own initiative breaks the code immediately.

    git clone https://github.com/AliaksandrSiarohin/first-order-model.git && cd first-order-model && pip install -r requirements.txt
  2. 02

    Fetch a checkpoint and crop the driving video to a face square

    The author hosts weights on Google Drive and Yandex Disk. vox-cpk.pth.tar is 728.8MB and taichi-cpk.pth.tar is 750.9MB — those figures include the generator, the discriminator, the keypoint detector and three optimizer states, of which pure inference touches only about 228MB. The driving video has to be cropped to a face-centred square first or the keypoints drift; the repo's crop-video.py prints the exact ffmpeg command to run.

    python crop-video.py --inp driving_raw.mp4
  3. 03

    Run demo.py — and do not forget --relative

    This is where most first attempts go wrong. demo.py's parser.set_defaults sets both relative and adapt_scale to False, so the default run uses absolute keypoint coordinates and transplants the driver's face geometry onto your output, which reads as a different person entirely. --relative transfers only the motion delta, --adapt_scale normalises motion magnitude using the convex hull of the keypoints, and --find_best_frame picks the driving frame whose pose best matches your source. Note that face-alignment is not in requirements.txt and has to be installed separately.

    pip install face-alignment && python demo.py --config config/vox-256.yaml --checkpoint vox-cpk.pth.tar --source_image source.png --driving_video driving.mp4 --relative --adapt_scale --find_best_frame --result_video result.mp4
  4. 04

    Either chain an upscaler, or scale out and train on your own data

    Every official checkpoint outputs 256x256, so anything destined for production needs a face-restoration or general upscaling pass behind it. To train your own data, follow the official command: vox-256.yaml carries num_epochs=100, num_repeats=75 and batch_size=40, using DataParallel with the sync_batchnorm implementation vendored in the repo. Card count is just a list in --device_ids — add cards, add numbers.

    CUDA_VISIBLE_DEVICES=0,1,2,3 python run.py --config config/vox-256.yaml --device_ids 0,1,2,3

Pennies per clip, about $150 to reproduce a training run

Inference first. An RTX 3090 24GB is $0.193/GPU-hr, which is $0.193 / 3600 = about $0.0000536 per second. Boot, install, pull the 728.8MB vox-cpk.pth.tar, render a 30-second driving clip — cap that at 20 minutes and you pay 0.193 x (20 / 60) = $0.064. Want it cheaper? A Tesla V100 32GB costs 0.188 x (20 / 60) = $0.063, and because it is sm_70 the pinned torch 1.12.1 supports it natively, so the environment time you save is worth far more than the cent you save. Training next. Replicating the official four-card setup: 4 x RTX 4090 24GB = 4 x $0.540 = $2.160/hr. Budget 72 hours and that is 2.160 x 72 = $155.52. The same 72 hours on a single RTX A6000 48GB is 0.817 x 72 = $58.82 — cheaper but slower, which makes it the sensible way to validate the config and the data pipeline before you scale out. Storage last. Call it 20GB for the repo, one checkpoint and a cropped dataset: at the $0.414/GB-month median that is 20 x 0.414 = $8.28/month. Pulling 2GB of finished renders back down costs 2 x 0.0081 = $0.016 at the $0.0081/GB median egress rate. One thing to keep in mind: compute billing stops the second the instance stops, but storage keeps billing until the volume is destroyed — clean up after a training run.

04 —

FAQ

How much VRAM does First Order Motion Model actually need?

Inference is very light. Qualcomm AI Hub's float ONNX export puts the keypoint detector at 54.2MB and the generator at 174MB, roughly 228MB of weights in total, and a 256x256 single-frame pass stays comfortably under 2GB of VRAM — any 16GB card has room to spare. Training is where the memory goes: the official vox-256.yaml uses batch_size=40, and the README states that batch "is tuned to run on 2 or 4 Titan-X gpu", meaning 24 to 48GB of aggregate memory just to start. On NexGPU that maps to a Tesla V100 32GB at $0.188/GPU-hr for inference and either an RTX A6000 48GB at $0.817/GPU-hr or four RTX 4090s at $0.540/GPU-hr for training — all metered per second, so you stop paying the moment you are done.

Is FOMM stuck at 256x256? How do I get a 1080p avatar out of it?

It is stuck. vox, vox-adv, taichi, fashion, mgif and bair were all trained at 256x256 — the -256 in the config filename is a hard constraint, and no official high-resolution checkpoint exists. The standard workaround is to treat FOMM purely as a motion engine and chain a face-restoration or general upscaler behind it (GFPGAN, CodeFormer, Real-ESRGAN and friends) to lift 256 to 1024 or beyond: motion naturalness from FOMM, image quality from the upscaler. An RTX 3090 24GB at $0.193/GPU-hr holds both models resident with no round-trip through disk, and NexGPU's 2,000+ prebuilt images already include PyTorch and ComfyUI, so you are not building the environment from scratch.

Is FOMM still worth using, or should I go straight to LivePortrait or TPSMM?

It depends on the subject. For high-quality portrait talking heads with lip and gaze retargeting, go straight to LivePortrait (KwaiVGI, 2024). For articulated subjects, both the original author's MRAA (CVPR 2021) and the community's Thin-Plate Spline Motion Model (CVPR 2022, replacing the first-order affine transforms with thin-plate splines) beat FOMM — and that "Check out our new paper" banner on the FOMM README points at MRAA. But FOMM remains the cleanest baseline in the family: MIT licensed, fifteen dependencies, 228MB of inference weights, and able to drive full-body tai chi or a robot arm, which face-specific models simply cannot touch. For a paper control, a teaching reproduction, or any non-face subject, it is still the first pick. You can install all four generations on the same NexGPU instance and compare them back to back — and the second you stop the instance, compute billing stops.

vox-cpk.pth.tar is 728MB — why do you say the model is only 228MB?

Because a .pth.tar is a training checkpoint, not an inference weight file. It bundles the generator, the discriminator and the keypoint detector together with three optimizer states, which is why vox-cpk.pth.tar (728.8MB), taichi-cpk.pth.tar (750.9MB), fashion.pth.tar (750.8MB) and bair-cpk.pth.tar (750.8MB) are all within a rounding error of each other — the differences have nothing to do with model capacity. After loading, demo.py only ever uses generator and kp_detector, which is precisely Qualcomm's 174MB + 54.2MB. To slim a deployment down, save just those two state_dicts back out after loading. NexGPU storage runs at a $0.414/GB-month median, so every half-gigabyte you drop is real money over a year.

I installed the official requirements.txt and everything breaks — torch.load and numpy errors. What now?

This is 2019 code meeting a 2026 ecosystem, and it fails in three predictable places. First, PyTorch 2.6 flipped torch.load's weights_only default to True, which refuses FOMM's pickled checkpoints outright — pass weights_only=False explicitly. Second, numpy removed the np.float and np.int aliases in 1.24, which is exactly why the repo pins numpy==1.23.3; do not upgrade it out of habit. Third, the official 1.12.1 wheels compile up to sm_86, so RTX 4090 (sm_89), H100 (sm_90) and RTX 5090 (sm_120) are not covered — either move to PyTorch 2.x or simply rent an sm_70/sm_86 card. On NexGPU the Tesla V100 32GB ($0.188), RTX 3090 24GB ($0.193) and RTX A6000 48GB ($0.817) all sit inside 1.12.1's native support range and run out of the box, and support is bilingual over Telegram with no ticket queue if you get stuck.

Why does my output look like the person in the driving video instead of my source photo?

Because you left off --relative. demo.py's parser.set_defaults sets relative and adapt_scale to False, so the default run uses absolute keypoint coordinates, which transplants the driver's face geometry onto the output and erases the identity in your source image. The fix is to pass --relative --adapt_scale together: the first transfers only the relative motion delta, the second normalises motion magnitude to your source's scale using the convex hull of the keypoints. Stack --find_best_frame on top to auto-select the driving frame whose pose most closely matches the source and the result improves again — just remember face-alignment is not in requirements.txt and needs its own pip install. Dialling this in takes several attempts, and on NexGPU those attempts cost cents: per-second metering, no minimum, no quota request.

More in Digital humans and face animation

Every model guide

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.

Sign up freeTalk to sales

Sign up to browse live network pricing. No payment method required.