Skip to main content
NexGPU

Document OCR model

Self-host Donut: skip the OCR engine, read documents straight into JSON

Roughly 200M parameters and 809MB of fp32 weights — Donut is still the most VRAM-frugal end-to-end document parsing model there is. A single $0.193/GPU-hr RTX 3090 24GB on NexGPU runs a full fine-tune, and you can go from cold start to your first JSON in under an hour.

Sign up freeView pricing

Donut stands for Document Understanding Transformer. It came out of NAVER CLOVA, the paper "OCR-free Document Understanding Transformer" landed at ECCV 2022 (arXiv 2111.15664), the code lives at clovaai/donut on GitHub, and it is MIT licensed. The thesis is right there in the name: no OCR. The classic extraction stack is a three-stage pipeline — OCR engine reads characters, layout analysis groups them, then rules or a model assemble fields — and every stage leaks error into the next, with a re-tune required every time the layout changes. Donut collapses the whole pipeline into a single model: a Swin Transformer encoder feeding a four-layer mBART decoder. Feed it an image, it autoregressively emits a tagged sequence, and processor.token2json() turns that into the JSON you asked for. The entire thing is 809MB of fp32 weights, an order of magnitude smaller than the 3B-and-up OCR models that dominate today.

The honest half of the story: upstream Donut is no longer maintained. The donut-python package stopped at 1.0.9 in November 2022, the official Colab demos were last refreshed in June 2023, and issues like "Provide a working list of deps" and "Colab Demo CORD is Broken" are still sitting open as of February 2026. The naver-clova-ix org on Hugging Face has exactly nine models, all Donut family, nothing new since 2022. If your job is "convert arbitrary documents to full-text Markdown," the current generation of OCR-free VLMs — DeepSeek-OCR (3B, MIT, October 2025), GLM-OCR, GOT-OCR2.0 — has left Donut behind, so use those instead. But if your job is "fixed layout in, fixed JSON schema out" — e-commerce receipts, fuel invoices, lab reports, customs declarations, shipping labels — Donut is still the right answer. 200M parameters means its VRAM footprint and inference cost sit an order of magnitude below a 3B model, its output is hard-locked to the fields you defined during fine-tuning rather than occasionally inventing a new one the way a general VLM will, and the MIT license means you can ship it commercially and closed-source without a second thought.

VRAM is barely a gate here; the real cost is fine-tuning. Pre-training donut-base took 64 A100s for about 2.5 days, and donut-proto took 8 V100s for about 5 days — but that is training from scratch, and you do not need to repeat it. For downstream fine-tuning from an official checkpoint, the most widely cited community run was done on a single 16GB V100: resolution dropped to 720x960, fp16, batch size 2, 624 labelled images, 3 epochs. On NexGPU a Tesla V100 32GB is $0.188/GPU-hr and an RTX 3090 24GB is $0.193/GPU-hr, both roomier than that machine. Billing is metered per second with no minimum, no setup fee and no quota request, the PyTorch image is one of 2,000+ prebuilt images that boot ready to go, and 1,175 verified nodes across 51 countries and regions let you land close to wherever your documents already live.

01 —

Every official Donut checkpoint

All six usable weights under naver-clova-ix sit around 200M params — what differs is input resolution and the decoder's output ceiling

VersionParametersVRAMContextNotes
naver-clova-ix/donut-base~200M (Swin encoder, depths [2,2,14,2] + 4-layer mBART, d_model 1024, vocab 57525)809MB fp32 weights / ~400MB fp16; fp16 inference peaks around 3-5GB at 2560x19201536 decoder tokensThe bare backbone, pre-trained only on SynthDoG synthetic data. Run it as-is and it emits nothing meaningful — this is a fine-tuning starting point, not a product. Begin here for a brand-new language or an unusual layout family.
naver-clova-ix/donut-base-finetuned-cord-v2~200M (vocab 57580, 806MB weights)~1.5-2GB fp16 inference; ~10GB to fine-tune at 1280x960 fp16 batch 4768 decoder tokensReceipt parsing at 1280x960 (height x width), task prompt <s_cord-v2>. Works out of the box, turning a till receipt into menu / sub_total / total JSON, and it is the checkpoint the community most often uses as the starting point for a custom receipt schema.
naver-clova-ix/donut-base-finetuned-docvqa~200M (vocab 57532)~3-5GB fp16 inference — the 2560x1920 Swin activations dominate, not the weightsonly 128 decoder tokensDocument question answering, prompt format <s_docvqa><s_question>your question</s_question><s_answer>. That 128-token ceiling means short answers only; ask it to summarise a page and the output gets hard-truncated.
naver-clova-ix/donut-base-finetuned-rvlcdip~200M (vocab 57544)~3-5GB fp16 inference; decoding emits a single class label, so KV cache is effectively nilonly 8 decoder tokensRVL-CDIP 16-way document classification (invoice, resume, advertisement, handwritten...), prompt <s_rvlcdip>, output like {'class': 'advertisement'}. Trying to use it as an extraction model always fails — 8 tokens will not hold any JSON.
naver-clova-ix/donut-base-finetuned-zhtrainticket~200M (vocab 57542)~1.5-2GB fp16 inference (960x1280 landscape, a quarter the pixels of base)256 decoder tokensThe one official Chinese checkpoint, extracting fields from Chinese rail tickets. Its 960x1280 landscape geometry matches the ticket shape. For CJK documents it sits closer to your distribution than cord-v2 does, so it is worth one run as a fine-tuning start.
naver-clova-ix/donut-proto~200M (Swin depths [2,2,18,2], full Swin-B depth, vocab 57524)~810MB fp32 weights; ~3-4GB fp16 inference at 2048x1536768 decoder tokensThe early prototype backbone at 2048x1536, trained on 8 V100s for about 5 days. Its third stage has 18 blocks where donut-base trimmed to 14, so the architectures are not fully interchangeable. Unless you are reproducing early paper results, just use donut-base.

02 —

Which NexGPU card to rent

At 200M parameters VRAM is never the bottleneck — what matters is activation cost from input resolution and the speed of autoregressive decoding

  • Run an official checkpoint and see whether Donut can read your documents at all

    Tesla V100 32GB$0.188/GPU-hr

    The cheapest card on the platform, and Volta's fp16 tensor cores are plenty for Donut — the widely cited community fine-tune ran on exactly this silicon. 32GB holds donut-base at its native 2560x1920, so twenty minutes gets you a yes-or-no answer on feasibility.

  • Fine-tune a custom schema on a few hundred to a few thousand labelled documents (1280x960, fp16, batch 4-8)

    RTX 3090 24GB$0.193/GPU-hr

    AdamW optimizer state for a 200M model is only about 3.2GB; add gradients and 1280x960 activations and 24GB still leaves room to push batch size to 8. Under twenty cents an hour means the compute for a full training round costs less than a coffee.

  • Keep the native 2560x1920 resolution, or cut training wall-clock in half

    RTX 4090 24GB$0.540/GPU-hr

    At native resolution Swin's first stage carries around 300K tokens and four times the activations of 1280x960, so you will need gradient checkpointing to fit 24GB. Ada delivers roughly double the fp16/bf16 throughput of a 3090, and with per-second billing finishing sooner often costs less overall.

  • Re-pretrain a backbone for a new language or domain using SynthDoG

    A100 SXM4 80GB$1.088/GPU-hr

    The official donut-base took 64 A100s over 2.5 days, and rebuilding a backbone is the one scenario that genuinely needs scale. NexGPU offers up to 14 GPUs per node and a maximum node VRAM of 2,152GB, so an 8-GPU NVLink job against synthdog-zh's 63.7K synthetic Chinese documents is a reasonable starting point.

03 —

Four steps to running Donut on NexGPU

Do not touch the donut-python package — its dependencies are frozen against 2022-era timm and pytorch-lightning. Use the Transformers VisionEncoderDecoder path instead.

  1. 01

    Rent a card and install a clean Transformers stack

    Pick an RTX 3090 24GB with the PyTorch prebuilt image in the console, then connect over SSH or Jupyter. Nearly every "it won't install" and "the Colab is broken" thread in the upstream issue tracker is a donut-python problem; Transformers ships a complete Donut implementation, so you need none of the original repo's code. Note that Transformers v5 retired the image-to-text pipeline task in favour of image-text-to-text, with AutoModelForImageTextToText or VisionEncoderDecoderModel as the model class.

    pip install -U "transformers>=5.0" torch torchvision pillow sentencepiece protobuf datasets
  2. 02

    Run one real receipt through cord-v2 to sanity-check the card and resolution

    Load donut-base-finetuned-cord-v2 in fp16 on the GPU and feed it a photo of your own receipt with the task prompt <s_cord-v2>. Always pass bad_words_ids=[[tokenizer.unk_token_id]] and use_cache=True to generate — the first stops the decoder looping on <unk> when it meets an out-of-vocabulary glyph, the second is where all the speed comes from. Convert the output sequence with processor.token2json(). Once this runs, you know exactly how well Donut handles your layout family out of the box.

    model = VisionEncoderDecoderModel.from_pretrained("naver-clova-ix/donut-base-finetuned-cord-v2", dtype=torch.float16).to("cuda")
  3. 03

    Swap in your own field vocabulary: task token, resized embeddings, longer output

    This is where Donut fine-tunes go wrong most often — upstream issue #332, "TED and F1 both 0.0 on test data," is this bug. After adding your task-start token and field tokens to the tokenizer, you must call decoder.resize_token_embeddings() so the embedding matrix grows with the vocab, and point config.decoder_start_token_id at the new token. Miss either step and the model starts decoding from an id it has never seen: training loss looks like it is converging while evaluation reads exactly zero. Check max_position_embeddings at the same time — cord-v2 gives you 768, rvlcdip only 8 — and raise it if your JSON is longer.

    processor.tokenizer.add_special_tokens({"additional_special_tokens": ["<s_myreceipt>"]}); model.decoder.resize_token_embeddings(len(processor.tokenizer)); model.config.decoder_start_token_id = processor.tokenizer.convert_tokens_to_ids("<s_myreceipt>")
  4. 04

    Drop the resolution, enable fp16, start training

    Resolution is the single knob trading VRAM against accuracy: change image_processor.size and encoder.image_size together, and let Swin's relative position bias interpolate by loading with ignore_mismatched_sizes=True. Keep both dimensions divisible by the window_size of 10. The community run that dropped 2560x1920 to 720x960 fit batch 2 on a 16GB V100; on a 24GB 3090 you can keep 1280x960 and push batch to 8. Call save_pretrained when you are done and pull the 800MB checkpoint home — egress is billed at a $0.0081/GB median.

    processor.image_processor.size = {"height": 1280, "width": 960}; model.config.encoder.image_size = (1280, 960)

What a custom Donut actually costs to train

Take a realistic job: 600 of your own labelled documents, fine-tuned into a model that only ever needs to understand your layout. All figures are NexGPU list price. Training runs on an RTX 3090 24GB at $0.193/GPU-hr at 1280x960 with fp16 and batch 4; budget 3 hours to cover the epochs plus mid-run evaluation, so 0.193 x 3 = $0.579. Add an hour afterwards to run inference and TED scoring across the full test set: 0.193 x 1 = $0.193. Dataset, Hugging Face cache and a handful of checkpoints come to roughly 20GB, held for 3 days at the $0.414/GB-month median: 20 x 0.414 x 3 / 30 = $0.828. Finally, pull the 800MB finished weights home at the $0.0081/GB egress median: 0.8 x 0.0081 = about $0.01. Total: roughly $1.61 — breakfast money for an extraction model that runs on your own hardware and never shows a single document to a third party. One billing detail worth internalising: compute is metered per second and stops the moment your training script exits, but storage keeps billing until you destroy the instance. That $0.828 is the line people forget, so delete the volume when you are done. And if you only want to check whether donut-base-finetuned-cord-v2 can already read your receipts, a Tesla V100 32GB at $0.188/GPU-hr gives you twenty minutes for 0.188 x 1/3 = about $0.06.

04 —

FAQ

How much VRAM does Donut actually need? Is a 24GB card enough?

More than enough — Donut's appetite is probably an order of magnitude lower than you expect. It is roughly 200M parameters, and donut-base ships an 809MB fp32 pytorch_model.bin that halves to about 400MB in fp16. What actually consumes memory is the Swin encoder activations driven by input resolution: cord-v2 at 1280x960 peaks around 1.5-2GB for inference, while donut-base and docvqa at 2560x1920 land around 3-5GB. Fine-tuning adds AdamW optimizer state (about 3.2GB for 200M params) plus gradients, so batch 4 at 1280x960 sits near 10GB. That means NexGPU's cheapest options — Tesla V100 32GB at $0.188/GPU-hr and RTX 3090 24GB at $0.193/GPU-hr — are both comfortable, and even a Tesla T4 16GB at $0.298/GPU-hr handles inference once you drop the resolution. Donut is about the best value-per-dollar model on our platform: nearly everything you pay buys throughput rather than capacity you are being held hostage by.

It's 2026 and DeepSeek-OCR and GLM-OCR exist. Is Donut still worth using?

It depends on the job. For general OCR — arbitrary layouts to Markdown, full-text scan extraction, complex table reconstruction — skip Donut. Its 2022 architecture and sub-300M parameter count have been comprehensively beaten there; reach for DeepSeek-OCR (3B, MIT) or GOT-OCR2.0 (0.7B). But when the input is a fixed layout and the output is a fixed JSON schema, Donut still wins on three counts: a fifteenth to a quarter of the parameters, so single-card throughput and cost differ by an order of magnitude; output hard-locked to the fields you defined during fine-tuning, rather than a general VLM occasionally inventing an extra one; and an MIT license with no strings on commercial or closed-source distribution. The practical move is to rent a card for each approach and run the same sample batch through both. Per-second billing with no minimum rental means twenty minutes on each side costs under $0.50 combined.

Why is my fine-tuned model scoring 0.0 on TED and F1?

This is the classic Donut fine-tuning trap — it is literally upstream issue #332. Nine times out of ten the special-token dance is incomplete. After adding your task-start token and field tokens to the tokenizer, you must call model.decoder.resize_token_embeddings(len(processor.tokenizer)) so the embedding matrix covers the new vocab, and set model.config.decoder_start_token_id to your new start token. Skip the resize and the new ids go out of range; skip decoder_start_token_id and the model begins decoding from a completely different anchor than it trained on. Either way, training loss looks perfectly healthy while evaluation reads zero. The second common cause is max_position_embeddings truncating your target JSON (cord-v2 is 768, docvqa only 128, rvlcdip only 8). The fastest way to debug this is short repeated runs on a $0.193/GPU-hr 3090 — change one line, verify, repeat — which beats waiting on local CPU by a wide margin.

donut-python won't install and the official Colab is broken. What now?

Don't fix it, route around it. donut-python's last release was 1.0.9 in November 2022, with dependencies pinned to that era's timm, pytorch-lightning and transformers, which are simply incompatible with modern PyTorch. Threads like "Provide a working list of deps," "Colab Demo CORD (Document Parsing) is Broken" and "app.py Failing for Python 3.9" have been sitting open into February 2026 because NAVER CLOVA has effectively stopped maintaining the repository. The correct path is pure Hugging Face Transformers: DonutProcessor plus VisionEncoderDecoderModel (or AutoModelForImageTextToText). The official weights are unchanged and both the inference and fine-tuning code paths are actively maintained. Also remember that Transformers v5 dropped the image-to-text pipeline task in favour of image-text-to-text. The PyTorch image among NexGPU's 2,000+ prebuilt images boots into a clean modern environment, saving you the half day you would otherwise spend fighting CUDA versions locally.

Can I use Donut as a general OCR engine to read all the text on a page?

No, and this is the single most common misunderstanding about it. Donut is not a recognition engine; it is an end-to-end image-to-structured-output model whose output is entirely determined by the task prompt and the fine-tuning data. Give it <s_cord-v2> and it emits receipt menu/total fields; give it <s_rvlcdip> and it emits one class name; give it <s_docvqa> plus a question and it emits a short answer. The donut-base backbone cannot even do those — it has only seen SynthDoG pre-training, so un-fine-tuned inference produces meaningless sequences. For full-page text, use GOT-OCR2.0, Surya or DeepSeek-OCR. For "what are the twelve fields on this document," that is Donut's territory. If you are not sure which bucket you are in, rent a V100 32GB on NexGPU and try both in twenty minutes — it costs about $0.06.

How much labelled data does a Donut fine-tune need, and what will it cost?

Far less than an LLM fine-tune. The most cited community run used just 624 labelled images over 3 epochs and produced usable results; in practice fixed-layout documents typically converge somewhere between a few hundred and a couple of thousand samples. The labelling format is just an image paired with the target JSON — no bounding boxes, no character-level positions, which is precisely the labour the OCR-free approach saves you. On the compute side, 600 images with a 3-hour budget on an RTX 3090 24GB at $0.193/GPU-hr covers training and evaluation for $0.579; add scoring, storage and the download and the whole round lands under $2. NexGPU bills per second with no minimum, no setup fee and no quota request, so you can label 200 images first, watch the loss curve, and only continue labelling if it looks promising — stop the instance and the compute billing stops with it. That kind of short-iteration loop is exactly how a model this small should be used.

More in OCR and document understanding

Every model guide

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.

Sign up freeTalk to sales

Sign up to browse live network pricing. No payment method required.