Skip to main content
NexGPU

Document AI · OCR and layout understanding

Self-host LayoutLM: 133M params, 8GB of VRAM and you own a document extraction model

The cheapest path in document AI isn't throwing invoices at a frontier model — it's feeding LayoutLMv3 the right bounding boxes. The biggest model in the family is 368M params; one 24GB card covers both training and serving.

Sign up freeView pricing

LayoutLM comes out of Microsoft's UniLM group, and the idea is refreshingly plain: on top of BERT's 1D position embeddings, give every token a 2D coordinate embedding (a bbox) so the model knows where on the page a word sits. By LayoutLMv3, Microsoft dropped the detectron2 CNN visual backbone that made LayoutLMv2 painful and replaced it with ViT-style 16x16 patch embeddings, pre-trained with a unified MLM + MIM + WPA (word-patch alignment) objective. That single change collapsed the deployment difficulty, and v3 is the version people actually run today.

One thing to be upfront about: LayoutLMv3 shipped in April 2022, and Microsoft's UniLM Document AI line never released a LayoutLMv4. Anyone hunting for the newest LayoutLM will be disappointed — but it is not abandoned. microsoft/layoutlmv3-base still pulls seven figures of downloads a month on Hugging Face, Transformers 5.x still maintains all four models (LayoutLM, LayoutLMv2, LayoutLMv3, LayoutXLM), and Optimum's ONNX export list names LayoutLM and LayoutLM-v3 explicitly. The reason is pragmatic: a 133M encoder doing token classification emits a fixed label, not generated text, so it cannot hallucinate a tax ID that isn't on the page; latency is measured in milliseconds; one card serves dozens of concurrent requests. Generative document VLMs like DeepSeek-OCR and PaddleOCR-VL are better at open-ended parsing, but for pulling twenty fixed fields off a form with an auditable source coordinate on each one, LayoutLMv3 is still the cheapest answer that works.

This page covers what you will actually hit when you self-host it: how big each variant really is, what the VRAM numbers look like in practice, why bboxes must be normalised to 0-1000 (skip it and you get a CUDA device-side assert), how to get around the CC BY-NC-SA 4.0 licence line, what to do when 512 tokens won't hold one A4 page, and which NexGPU card to rent for each configuration — with the actual dollar cost of one fine-tuning run.

01 —

The LayoutLM family, side by side

Four generations with genuinely different architectures — pick the wrong one and you inherit a different set of problems

VersionParametersVRAMContextNotes
microsoft/layoutlmv3-base133M (12 layers / 768 hidden / 12 heads)fp32 weights ~0.53GB; ~2GB peak at inference; 8-12GB for full AMP fine-tuning at batch 8512 text tokens + 196 image patchesThe default choice. 90.29 F1 on FUNSD, 96.56 F1 on CORD, 95.44% on RVL-CDIP. No detectron2 dependency — pip installs and runs — and it is the only v3 branch Optimum can export to ONNX.
microsoft/layoutlmv3-large368M (24 layers / 1024 hidden / 16 heads)fp32 weights ~1.5GB; ~4GB peak at inference; budget 16-24GB for full fine-tuning512 text tokens + 196 image patchesThe leaderboard model: 92.08 F1 on FUNSD, 97.46 F1 on CORD, 83.37 ANLS on DocVQA. Worth 1-2 points over base at roughly double the training time and memory, which is why most production teams end up back on base.
microsoft/layoutlmv3-base-chinese133M, Chinese pre-training branchSame class as base: ~2GB inference, 8-12GB fine-tuning512 text tokens + 196 image patchesUse this for Chinese receipts, forms and contracts. 92.02 F1 on XFUND Chinese, 99.21% mean on EPHOIE. Note that its download volume is a fraction of a percent of the English base, so community examples are thin — expect to write your own data pipeline.
microsoft/layoutlmv2-base-uncased / microsoft/layoutxlm-baseBase scale: 12 layers / 768 hidden / 12 heads30-50% above v3 because of the extra ResNeXt-FPN visual backbone512 text tokensOnly for reproducing older papers or when you need LayoutXLM's 53 languages. The killer is the detectron2 requirement, which frequently fails to build against current CUDA and PyTorch; images are also BGR, not RGB; and Optimum's ONNX list has no entry for v2.
microsoft/layoutlm-base-uncased / layoutlm-large-uncased113M / 343MLightest of the family — no visual branch: <1.5GB inference, 4-6GB fine-tuning512 text tokensThe original: text plus bbox, no image modality. Weaker than v3 on every benchmark, but it is the only generation released under MIT. If your project is commercial and the licence cannot move, this row is your answer.

02 —

Which card to rent

LayoutLM is an encoder, not a generator — VRAM is driven by batch size and image preprocessing, not by parameter count

  • Full fine-tune of LayoutLMv3-base (FUNSD, CORD, or your own labelled invoices)

    RTX 3090 24GB$0.193/GPU-hr

    Peaks at 8-12GB under AMP, so 24GB lets you push batch size past 16; Ampere gives you bf16 and TF32, and it is both cheaper and several times faster than a Tesla T4 — the best price-performance on this list for LayoutLM.

  • Full fine-tune of LayoutLMv3-large, or larger batches to speed convergence

    Tesla V100 32GB$0.188/GPU-hr

    32GB is more than enough headroom for a 368M encoder, fp16 tensor cores are plenty at this scale, and it is the lowest cost-per-GB-of-VRAM training card in the catalogue.

  • Production batch extraction (ONNX or fp16, always-on service)

    Tesla T4 16GB$0.298/GPU-hr

    Inference peaks under 4GB, so 16GB holds the LayoutLMv3 extractor and the upstream OCR model together; the T4's INT8 tensor cores and low power draw suit a queue that runs 24/7.

  • Whole pipeline on one box — OCR to LayoutLMv3 to layout detection, or PubLayNet Cascade R-CNN layout analysis

    RTX 4090 24GB$0.540/GPU-hr

    The detection half still runs through detectron2 and is hungry for both memory and compute; a single 4090 fits PaddleOCR, LayoutLMv3 and the detection head together and saves you shipping page images between machines.

03 —

LayoutLMv3 self-hosted in four steps

From a bare instance to a deployable ONNX extraction model

  1. 01

    Spin up the box and install dependencies

    Pick an RTX 3090 24GB in the NexGPU console, launch it from the prebuilt PyTorch image, and connect over SSH or Jupyter. LayoutLMv3 needs no detectron2, so the dependency tree stays clean — you only need the system package if you plan to use the processor's built-in Tesseract.

    pip install transformers datasets seqeval pytesseract pillow && apt-get update && apt-get install -y tesseract-ocr
  2. 02

    Run OCR and normalise every bbox to 0-1000

    This is the single biggest trap in LayoutLM. max_2d_position_embeddings is 1024, and the processor requires every box as [x0, y0, x1, y1] integers scaled into 0-1000. Feed raw pixel coordinates from a 200 DPI A4 scan (routinely 1654x2339) and you get IndexError: index out of range in self on CPU, or a device-side assert with no useful traceback on GPU. For Chinese documents, take word boxes from PaddleOCR and pass apply_ocr=False with your own words and boxes — the bundled Tesseract is effectively unusable on Chinese.

    bbox = [int(1000 * x0 / W), int(1000 * y0 / H), int(1000 * x1 / W), int(1000 * y1 / H)]
  3. 03

    Fine-tune the token classification head

    Information extraction uses LayoutLMv3ForTokenClassification with num_labels set to your BIO tag count. The tokenizer defaults to only_label_first_subword=True, meaning that when a word splits into several subwords only the first carries the label and the rest are filled with -100; misaligned labels are the second most common bug here. For documents past 512 tokens, use return_overflowing_tokens with a stride to window the page into overlapping segments and merge the predictions back.

    processor = AutoProcessor.from_pretrained("microsoft/layoutlmv3-base", apply_ocr=False)
  4. 04

    Export to ONNX and move to a serving card

    Do not ship the raw PyTorch checkpoint to production. Optimum supports ONNX export for LayoutLM and LayoutLM-v3 officially; follow it with INT8 dynamic quantisation and the model compresses to a couple of hundred megabytes, small enough to run on CPU and into single-digit millisecond latency on GPU. Then stop the training instance — NexGPU meters per second, so compute billing halts the moment it stops.

    optimum-cli export onnx --model ./layoutlmv3-invoice --task token-classification ./onnx/

What one invoice-extraction fine-tune actually costs

Say you have 2,000 labelled invoice pages, batch size 8, 20 epochs: 2000 x 20 / 8 = 5,000 steps. LayoutLMv3-base is 133M params with a total sequence length of 708 (512 text + 196 patches), so on an RTX 3090 with AMP those 5,000 steps are a half-hour job. Cost the whole workflow: 1 hour of OCR preprocessing and label alignment, a six-point learning-rate/warmup grid at roughly 25 minutes each for 2.5 hours, plus 1 hour of evaluation and export — 4.5 instance-hours total. Compute: 4.5 hrs x $0.193/GPU-hr = $0.87. Egress: the INT8 ONNX artefact is about 0.13GB, so 0.13 x $0.0081/GB is roughly $0.001. Storage: dataset plus checkpoints around 8GB at $0.414/GB-month is $3.31/month — but destroy the volume once the ONNX is out and that line goes to zero. So training your own invoice field extractor from scratch costs under a dollar in GPU time. Move up to LayoutLMv3-large on a Tesla V100 32GB at $0.188/GPU-hr and training roughly doubles to 9 hours: $1.69. No minimum spend, no setup fee, no quota request, and billing stops when the instance does.

04 —

FAQ

How much VRAM does LayoutLMv3 need to self-host? Is a 24GB 4090 enough?

More than enough — arguably wasteful. LayoutLMv3-base is 133M params, 0.53GB of fp32 weights, roughly 2GB peak at inference and 8-12GB for full AMP fine-tuning at batch 8; large is 368M and wants 16-24GB to fine-tune comfortably. What actually consumes memory is batch size and image preprocessing, not parameter count. So don't reach for a 4090 by reflex: NexGPU's RTX 3090 24GB is $0.193/GPU-hr and beats both the 4090 and the T4 on price-performance for this model, and if you want 32GB of headroom the Tesla V100 32GB is $0.188/GPU-hr.

Can LayoutLMv3 be used commercially? What is the licence?

This is the easiest compliance landmine to step on. LayoutLMv2, all of LayoutLMv3 (base, large, base-chinese) and LayoutXLM are released under CC BY-NC-SA 4.0 — non-commercial, with share-alike on anything derived. Two clean routes exist. First, drop back to the original microsoft/layoutlm-base-uncased (113M) or layoutlm-large-uncased (343M), which are MIT. Second, switch to LiLT (SCUT-DLVCLab/lilt-roberta-en-base, also MIT), which stitches a language-independent layout Transformer onto any pre-trained RoBERTa encoder — pair it with a Chinese RoBERTa for Chinese documents and you land in the same accuracy band. Both routes fine-tune inside 10GB, which on NexGPU is still a single RTX 3090.

Why do I immediately get IndexError: index out of range in self, or a CUDA device-side assert?

Almost always an unnormalised bbox. LayoutLM's 2D position embedding table has 1024 rows and the processor's contract is that every coordinate is an integer scaled into 0-1000. Pass raw OCR pixel coordinates and a 1654x2339 scan blows past the table on the first sample — you see an IndexError on CPU and a traceback-free device-side assert on GPU. The fix is int(1000 * x / page_width) per coordinate, plus asserting x0 <= x1, y0 <= y1 and no negatives. Debug one batch on CPU to locate it, then move the full run onto a NexGPU GPU instance — per-second billing means those debugging minutes cost essentially nothing.

Is there a GGUF or Q4 quantised build of LayoutLM?

No, and you shouldn't go looking. The GGUF and Q4_K_M ecosystem exists for multi-billion-parameter generative LLMs; the largest LayoutLM is 368M with 1.5GB of fp32 weights, which fits on any card, so quantising for memory buys you nothing. The optimisation that does pay is ONNX: Optimum supports LayoutLM and LayoutLM-v3 export officially (note: not v2), and INT8 dynamic quantisation on top of that delivers far more in throughput and latency than any VRAM saving. Run that INT8 ONNX on NexGPU's Tesla T4 16GB at $0.298/GPU-hr and you're using its INT8 tensor cores exactly as intended.

An A4 contract has far more than 512 tokens of text. How does LayoutLM handle long documents?

512 text tokens is a hard ceiling on the LayoutLMv3 text side; add 196 image patches and a CLS and the total sequence is 709. That number does not move. The standard engineering answer is sliding windows: pass truncation=True, max_length=512, stride=128 and return_overflowing_tokens=True, run each overlapping segment, then merge predictions back by word_id and take the higher-confidence label in overlaps. Three to five windows per page is normal for dense tabular documents. If your documents are genuinely long-form prose rather than forms, LayoutLM was the wrong tool from the start and you want a generative document model — NexGPU ships prebuilt images for both, so switching is a new instance, not a new project.

Should anyone still use LayoutLM, or go straight to a document VLM like DeepSeek-OCR?

It depends on the shape of the task. LayoutLMv3 really did stop in April 2022 and there is no v4 — but the problem it solves hasn't changed: a fixed schema, a bounded field set, every extracted value traceable to a page coordinate, millisecond latency, high concurrency per card. In that shape a 133M discriminative encoder still beats a 3B generative model outright, because it cannot invent a field value and you aren't paying for thousands of output tokens per page. Invert the requirements — wildly variable scans, Markdown or free-form JSON output — and a VLM like DeepSeek-OCR (3B, MIT licensed) is the right call. The cheapest way to settle it is to run both: an RTX 3090 at $0.193/GPU-hr for LayoutLMv3 and an RTX 4090 at $0.540/GPU-hr for the VLM makes a two-hour bake-off cost under two dollars — considerably less than the meeting where you argue about it.

More in OCR and document understanding

Every model guide

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.

Sign up freeTalk to sales

Sign up to browse live network pricing. No payment method required.