Skip to main content
NexGPU

Document OCR model

Turn academic PDFs back into Markdown with real LaTeX — Nougat local deployment needs one 24GB card

0.3B parameters, under 1GB of weights in bf16. It is the lowest-VRAM paper OCR you can self-host today. Here is what it does, what it refuses to do, and which card actually pays off.

Sign up freeView pricing

Nougat (Neural Optical Understanding for Academic Documents) comes from the Meta AI paper arXiv:2308.13418 by Lukas Blecher, Guillem Cucurull, Thomas Scialom and Robert Stojnic. It skips the classic OCR pipeline of layout detection plus line recognition and instead uses a Donut-style end-to-end vision encoder-decoder: a Swin Transformer encodes a full 896×672 page image into visual tokens, and an mBART decoder autoregressively emits Mathpix Markdown (.mmd). Display equations come out as LaTeX and tables come out as tabular environments — the single reason a lot of paper pipelines still keep it around.

Both official checkpoints are tiny by 2026 standards. facebook/nougat-base is 0.3B parameters with a 10-layer mBART decoder, d_model 1024, a 4096-token per-page context, and 1.4GB of safetensors on disk. facebook/nougat-small is 0.2B with only 4 decoder layers, 3584 tokens per page, and 990MB on disk. The repo's own nougat/utils/device.py casts the model to bfloat16 by default, so base sits at roughly 0.7GB resident — nearly all your VRAM pressure comes from the stack of page-image activations in the batch, not from the weights. You genuinely do not need a big card for this.

The honest status report: nougat-ocr on PyPI is frozen at 0.1.17, released 2023-10-04, and the last commit on GitHub main is a dependency swap dated 2025-02-21 that never shipped as a new package. Marker's current benchmark baselines are Chandra 2, Gemini Flash 3.5, MinerU, docling and liteparse — Nougat is not on that list anymore; olmOCR v0.4.0 moved to a 7B olmOCR-2 model whose README demands at least 12GB of VRAM. But if your corpus is English arXiv/PMC-style papers and what you want is LaTeX for equations and tables, 0.3B Nougat is still the cheapest single-card option in existence.

01 —

Checkpoints and releases: which one to pull, and how big

Two official checkpoints, one community formula fine-tune, plus the distribution differences that bite.

VersionParametersVRAMContextNotes
facebook/nougat-base (0.1.0-base)0.3B1.4GB fp32 weights / ~0.7GB bf164096 tokens per pageThe accuracy pick. 10-layer mBART decoder, d_model 1024, 50000-token vocab; noticeably steadier than small on long layouts and dense reference pages. The HF copy is the transformers conversion of the 0.1.0-base GitHub release.
facebook/nougat-small (0.1.0-small)0.2B990MB fp32 weights / ~0.5GB bf163584 tokens per pageOnly 4 decoder layers, so it is faster and leaner, at the cost of dropping content in long equations and multi-column spreads. Around 27,700 downloads a month on HF — still more used than base.
Norm/nougat-latex-base0.3B (fine-tuned from base)Same class as base, ~0.7GB bf16One equation, not a full pageA community fine-tune of facebook/nougat-base on im2latex-100k that does equation-image-to-LaTeX only, using adaptive padding so rescaling does not smear sub- and superscripts. Licensed Apache-2.0, which is far looser than the official weights.
nougat-ocr 0.1.17 (PyPI release)Reuses the checkpoints aboveNo extra VRAMInherits the chosen checkpointShips the nougat CLI and the nougat_api server. Nothing new has been published since 2023-10-04. Its metadata hard-pins timm==0.5.4 and declares transformers>=4.25.1 with no upper bound — the main reason installs break today.
GitHub main (2025-02-21)Same weights as 0.1.17SameSameThe last commit replaced python-Levenshtein with rapidfuzz (issue #255), but that fix was never released to PyPI. If Levenshtein will not build on your machine, installing straight from git is the only way to get it.

02 —

Picking a card: do not rent an H100 for a 0.3B model

The official CLI derives batch size from VRAM itself — the formula lives in nougat/utils/device.py: int(total_memory_bytes / 1024 / 1024 / 1000 × 0.3).

  • One paper to a few dozen, converted to .mmd on the default bf16 path

    RTX 3090 24GB$0.193/GPU-hour

    The cheapest card in the fleet with native bfloat16 (Ampere and up only). The official heuristic yields batch 7 at 24GB, weights take 0.7GB, and everything left over is headroom for page activations.

  • Thousands of pages in one run, wall-clock matters

    RTX 4090 24GB$0.540/GPU-hour

    Still batch 7, but Ada's decode throughput clearly beats the 3090. Nougat is page-by-page autoregressive decoding, so the decoder is the bottleneck, not memory — and at volume the total bill works out in its favour.

  • Ten-thousand-page pipeline with nougat_api running concurrently

    RTX A6000 48GB$0.817/GPU-hour

    The heuristic goes straight to batch 14 at 48GB, so a single card keeps the queue behind port 8503 saturated without splitting into processes that fight over VRAM.

  • Absolute minimum budget, or a workflow stuck on fp32

    Tesla V100 32GB$0.188/GPU-hour

    The lowest rate on the list. Volta has no native bf16, so you must pass --full-precision and run fp32 — 1.4GB of weights still fits comfortably in 32GB, but you absorb the speed hit and the extra [MISSING_PAGE] false positives.

03 —

Getting Nougat running: four steps, dependency traps included

From boot to .mmd coming out of port 8503. Every command below is copy-pasteable.

  1. 01

    Boot a bf16-capable card, install nougat-ocr, and pin transformers

    Pick a PyTorch prebuilt image in the NexGPU console and SSH in. Do not install nougat-ocr bare — its transformers dependency has no upper bound, so pip resolves transformers 5.x while 0.1.17's code was written against 4.x. timm is simultaneously hard-pinned to 0.5.4, so use an isolated virtualenv. If python-Levenshtein refuses to build, install from git main instead: pip install "git+https://github.com/facebookresearch/nougat", which already uses rapidfuzz.

    pip install "nougat-ocr==0.1.17" "transformers<5" "timm==0.5.4"
  2. 02

    Measure throughput on the first 20 pages before committing to a batch strategy

    The first run auto-downloads the 0.1.0-base weights from the GitHub release. Pass -b explicitly; without it the CLI computes batch from VRAM: 4 at 16GB, 7 at 24GB, 9 at 32GB, 14 at 48GB, 24 at 80GB. Use -p to process only the first 20 pages and time it. Do not start with a whole volume — Nougat decodes page by page, denser pages are slower, and any generic pages-per-second figure is worthless to you.

    nougat paper.pdf -o ./out -m 0.1.0-base -b 7 -p 1-20
  3. 03

    Start the HTTP service and wire batch conversion into your own pipeline

    nougat_api brings up a FastAPI service on 127.0.0.1:8503; multipart-POST a PDF and Markdown comes back, with start and stop query parameters bounding the page range. It only listens on localhost by default, so tunnel over SSH rather than exposing 8503 to the internet. For batch jobs, --recompute discards previous predictions and reruns from scratch.

    nougat_api & curl -X POST -F "[email protected]" "http://127.0.0.1:8503/predict/?start=1&stop=20"
  4. 04

    Or skip the distribution entirely and use the native transformers path

    If you are already on transformers 5.x, do not downgrade: Nougat is a first-class model there, so NougatProcessor plus AutoModelForImageTextToText is all you need. Note that pipeline("image-to-text") was removed in v5, so you must load the model directly. After decoding, call processor.post_process_generation() — it fixes table formatting and runs remove_hallucinated_references to strip the fabricated citations.

    model = AutoModelForImageTextToText.from_pretrained("facebook/nougat-base", dtype="bfloat16", device_map="auto")

A real bill: what 2,000 pages of papers actually costs

Honest caveat first: Nougat decodes autoregressively page by page, up to 4096 tokens per page, and throughput depends entirely on layout density — no generic pages-per-second number is trustworthy, so time -p 1-20 on your own sample. Now the wall-clock arithmetic. Say you measure 4 hours for 2,000 pages on an RTX 3090 24GB at $0.193/GPU-hour: 4 × $0.193 = $0.772. Move to an RTX 4090 24GB at $0.540/GPU-hour — same batch 7, faster decode — and say it takes 2 hours: 2 × $0.540 = $1.08. That is $0.31 more for two hours of your life back. Saturate batch 14 on an RTX A6000 48GB at $0.817/GPU-hour and finish in 1.5 hours: 1.5 × $0.817 = $1.23. All three land inside the price of a coffee, so the real question is how long you are willing to wait, not which card is cheaper. Storage is separate: 1.4GB for the fp32 base weights, 990MB for small, plus source PDFs and pypdfium2 render cache — a 20GB volume runs 20 × $0.414 = $8.28/month. The .mmd output is plain text, a few MB for 2,000 pages, which at the $0.0081/GB median egress rate rounds to nothing. Billing is metered per second and priced per hour, with no minimum and no setup fee; compute billing stops the moment the instance stops, while storage keeps accruing until you destroy the volume.

04 —

FAQ

How much VRAM does Nougat actually need? Is 24GB enough?

The weights are small: facebook/nougat-base ships 1.4GB of fp32 safetensors and sits at roughly 0.7GB once the repo casts it to bfloat16, while nougat-small is 990MB on disk and about 0.5GB in bf16. What consumes memory is the stack of 896×672 page-image activations in the batch. The official CLI heuristic lives in nougat/utils/device.py: batch = int(total_memory_bytes / 1024 / 1024 / 1000 × 0.3), which lands on 4 at 16GB, 7 at 24GB, 9 at 32GB, 14 at 48GB and 24 at 80GB. So 24GB is not merely enough — it is the sweet spot. A NexGPU RTX 3090 24GB is $0.193/GPU-hour, and ten minutes on one is enough to confirm all of this yourself.

Does Nougat support Chinese papers? Can I use it for Chinese PDF-to-Markdown?

No, and the official FAQ is blunt about it: Nougat was trained on scientific papers from arXiv and PMC, works best on English, may cope with other Latin-script languages, and states that Chinese, Russian and Japanese will not work. Feeding it Chinese PDFs produces walls of [MISSING_PAGE] or invented English. Multilingual documents need a vision-language model instead, and those typically start at 7B and 12GB of VRAM. On NexGPU you can go from an RTX 4090 24GB at $0.540/GPU-hour up to an A100 PCIE 80GB at $0.824/GPU-hour, swapping cards in the same console with no quota request.

Why is my output nothing but [MISSING_PAGE_FAIL] or [MISSING_PAGE_EMPTY]?

That is Nougat's failure-detection heuristic firing; it exists to cut off pages where the decoder falls into a repetition loop. The official FAQ's order of investigation is: confirm the document really is an English arXiv/PMC-style paper, then suspect false positives on CPU or older GPUs — rerun with --no-skipping to disable the heuristic. The other common cause is precision: the repo casts to bfloat16 by default, and Tesla T4, Tesla V100 and Tesla P40 (Turing, Volta, Pascal) have no native bf16, so those need --full-precision to run fp32. To sidestep the whole class of problem, use Ampere or newer — the cheapest on NexGPU is the RTX 3090 24GB at $0.193/GPU-hour.

Is Nougat still maintained? Is it still worth self-hosting?

Straight answer: nougat-ocr on PyPI is frozen at 0.1.17, released 2023-10-04, and the last GitHub main commit is a dependency swap from 2025-02-21 that never became a release. Marker's current baselines are Chandra 2, Gemini Flash 3.5, MinerU, docling and liteparse, with Nougat absent; olmOCR v0.4.0 switched to a 7B olmOCR-2 whose README requires at least 12GB of VRAM. On the other hand, if your corpus is English arXiv papers and you want LaTeX for display equations and tables, 0.3B Nougat is still the lowest-VRAM, cheapest single-card option available, and nougat-small still pulls close to 28,000 downloads a month. Spend $0.193 on one hour of an RTX 3090 on NexGPU and benchmark it against a modern model on your own sample — that beats reading ten comparison posts.

pip install nougat-ocr fails immediately. How do I fix the dependency conflicts?

Three known landmines. First, 0.1.17 declares transformers>=4.25.1 with no upper bound, so modern pip resolves transformers 5.x against code written for 4.x — pass "transformers<5" explicitly. Second, timm is hard-pinned to ==0.5.4, which will not coexist peacefully with a current torch stack, so use an isolated virtualenv or a clean image. Third, 0.1.17 still depends on python-Levenshtein, which fails to build on some machines; the 2025-02-21 commit swapped it for rapidfuzz but never shipped to PyPI, so install from git when you hit it. NexGPU has 2,000+ prebuilt images, and the PyTorch ones boot into a clean CUDA environment so you are not spending half a day paying down someone else's dependency debt.

Can I use the Nougat weights commercially? What is the licence?

Treat code and weights separately, and be aware the official labelling is inconsistent. The GitHub code is MIT (Meta Platforms), the README describes the model weights collectively as CC-BY-NC, the Hugging Face card for facebook/nougat-base says cc-by-nc-4.0 (non-commercial), and the card for facebook/nougat-small says cc-by-4.0. That discrepancy is real, so verify against the current repo and model cards before shipping anything commercial — this page is not legal advice. The community fine-tune Norm/nougat-latex-base is Apache-2.0 and much more permissive. Running that evaluation is cheap on NexGPU: metered per second, no minimum, no setup fee, and $0.193 for an hour of RTX 3090 is enough to try both checkpoints before you decide. The console is at console.nexgpu.net, and support is bilingual over Telegram with no ticket queue.

More in OCR and document understanding

Every model guide

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.

Sign up freeTalk to sales

Sign up to browse live network pricing. No payment method required.