OCR & Document Parsing

Self-hosting PaddleOCR: from a 1.9MB tiny detector to a 0.9B VL pipeline, all on one card

PaddleOCR stopped being just detection-plus-recognition a while ago. Since 3.7.0 it maintains both the millisecond-scale PP-OCRv6 classic pipeline and the 0.9B PaddleOCR-VL document parsing pipeline, and the two have completely different GPU requirements. Pick the wrong card and the service starts fine, then times out forever.

PaddleOCR · self-hosted

PaddleOCR is the open-source OCR toolchain maintained by Baidu's PaddlePaddle team, under Apache 2.0, so commercial use needs no extra licence. What it is for has shifted fundamentally: before 2025 people installed PaddleOCR to get boxes and strings back; in 2026 they install it to turn a PDF straight into structurally correct Markdown. Release 3.7.0 (11 June 2026) shipped PP-OCRv6 with +4.6% detection and +5.1% recognition over PP-OCRv5_server, one model covering Chinese, English, Japanese and 46 Latin-script languages for 50 total, plus a 5.2x CPU speedup on the OpenVINO path.

The other line is PaddleOCR-VL. It first shipped in 3.3.0 (October 2025), pairing a NaViT-style dynamic-resolution vision encoder with the ERNIE-4.5-0.3B language model across 109 languages. Version 1.5 in 3.4.0 pushed that to 111 languages, and PaddleOCR-VL-1.6 in 3.6.0 (28 May 2026) scored 96.33% on OmniDocBench v1.6. The 1.6 recipe is region-aware data optimisation: identify the weak regions of the previous model, then apply targeted enhancement there. Its architecture is fully compatible with 1.5, so the upgrade is a zero-cost, plug-and-play swap of the model name. On 22 July 2026 the team added HPD-Parsing, a 1B model on an InternVL3.5-1B backbone that uses hierarchical parallel decoding to hit 4,752 tokens/s peak throughput on an A800 80GB at batch size 512 under vLLM.

What actually decides which card you rent is the backend, not the parameter count. The vLLM and SGLang backends for PaddleOCR-VL require compute capability of at least 8.0 and CUDA 12.6 or newer. The docs say it plainly: on compute-capability 7.x cards like the T4 and V100 the service will start, but it is prone to timeouts or OOM and is therefore not recommended. This is the single most common trap in self-hosting PaddleOCR-VL: the box comes up, the logs look clean, and everything just hangs. Below are the cards that work, split by pipeline.

01 —

What is actively maintained today

The classic pipeline is measured in megabytes, the VL pipeline in billions of parameters, and they solve different problems

Version	Parameters	VRAM	Context	Notes
PP-OCRv6 (PaddleOCR 3.7.0)	tiny 1.5M / small 7.7M / medium 34.5M	Detection weights 1.9MB / 9.6MB / 59.4MB, recognition 4.4MB / 20.4MB / 73.3MB; VRAM measured in hundreds of MB	Chinese, English, Japanese + 46 Latin-script languages, 50 total	The current default for the classic pipeline. Medium hits 86.2% detection and 83.2% recognition, beating PP-OCRv5_server while being smaller; the tiny tier exists for edge devices and high-concurrency pipelines.
PP-OCRv5 server / mobile	server and mobile tiers	Server: 84.3MB detection + 81MB recognition, about 165MB of weights; a 24GB card runs very large batches	Simplified Chinese, Traditional Chinese, English, Japanese, pinyin handwriting	Superseded by PP-OCRv6 but still maintained, and it has the most mature docs and third-party integrations. Per-image GPU latency is 89.55ms + 8.46ms for server, 10.67ms + 5.43ms for mobile, the only generation with a complete official latency table.
PaddleOCR-VL-1.6 (PaddleOCR 3.6.0)	0.9B (the Hugging Face card counts tensors and lists 1.0B)	About 2GB of bf16 weights (0.9B x 2 bytes); actual vLLM footprint is set by gpu-memory-utilization, officially recommended at 0.3	109 languages, NaViT dynamic-resolution input	The workhorse for document parsing, at 96.33% on OmniDocBench v1.6. Architecturally identical to 1.5, so upgrading means changing the model name and nothing else in your inference code.
PaddleOCR-VL-1.5 (PaddleOCR 3.4.0)	0.9B	Same as 1.6, about 2GB of bf16 weights	111 languages	Covers two more languages than 1.6, so if your workload includes a low-resource script that 1.6 dropped, this version is worth keeping around. Reported at 94.5% on OmniDocBench.
HPD-Parsing (22 July 2026)	1B, built on InternVL3.5-1B	About 2GB of bf16 weights; official throughput was measured on an A800 80GB at batch size 512	Dynamic tiling, up to 24 tiles of 448x448	Built for high-throughput batch parsing. Hierarchical parallel decoding plus P-MTP speculative decoding reaches 4,752 tokens/s, which is 2.62x the fastest existing document parser and 3.06x its own autoregressive baseline, at 94.91% on OmniDocBench v1.6.
PP-StructureV3	Multi-model pipeline	Depends on the detection, recognition and table sub-models you select	Full-page layout + tables + formulas	The non-VLM route to structure: layout analysis, table recognition and formula recognition each handled by a dedicated small model wired into a pipeline. Since 3.5.0 it exports Markdown and DOCX, which suits teams that need an explainable, module-by-module replaceable stack.

02 —

Which card to rent

Check compute capability first, VRAM second — PaddleOCR-VL's gate is CC 8.0, not memory

PP-OCRv6 / PP-OCRv5 classic pipeline for invoices, IDs and bulk image recognition
RTX 3090 24GB$0.193/GPU-hr
Weights are tens of megabytes, so all 24GB goes into batch size; compute capability 8.6 also clears the bar if you later move to the VL pipeline, and this is the cheapest card we rent.
Single-card vLLM service for PaddleOCR-VL-1.6, turning documents into Markdown
RTX 4090 24GB$0.540/GPU-hr
Compute capability 8.9 and native CUDA 12.6+ means FlashAttention compiles cleanly, and the recommended gpu-memory-utilization of 0.3 gives roughly a 7GB pool on 24GB, plenty for a 0.9B model.
High-concurrency parsing service needing a larger KV cache and higher vl_rec_max_concurrency
RTX 5090 32GB$0.723/GPU-hr
The extra 8GB converts directly into concurrency, and the layout model plus the VLM can both stay resident on one card instead of splitting the two-stage pipeline across two machines.
HPD-Parsing batch throughput, reproducing 4,752 tokens/s at batch 512
A100 SXM4 80GB$1.088/GPU-hr
The published throughput was measured on an A800 80GB, and the A100 SXM4 80GB is its same-generation, same-spec, same-memory counterpart; a batch-512 KV cache only fits on an 80GB-class card.

03 —

From bare machine to Markdown

Four steps, every command taken verbatim from the official docs

01
Start an instance with compute capability 8.0+, install PaddlePaddle and PaddleOCR
In the NexGPU console pick an RTX 4090 or RTX 3090 and start from a PyTorch or Ubuntu CLI image out of the 2,000+ prebuilt ones. Note that you install paddlepaddle-gpu from the CUDA 12.6 channel: the docs require framework 3.2.1 or above, and PaddleOCR itself must be 3.6.0 or newer to include PaddleOCR-VL-1.6.
```
python -m pip install paddlepaddle-gpu==3.2.1 -i https://www.paddlepaddle.org.cn/packages/stable/cu126/ && python -m pip install -U "paddleocr[doc-parser]>=3.6.0"
```
02
Bring up the VLM inference service
The least painful route is the official GenAI server image, roughly 13GB for vLLM and 43GB for FastDeploy (the -offline suffix gives you the offline variants). If you would rather control the environment yourself, run paddleocr install_genai_server_deps vllm; the supported backends are vllm, sglang and fastdeploy.
```
docker run -it --rm --gpus all --network host ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddleocr-genai-vllm-server:latest-nvidia-gpu paddleocr genai_server --model_name PaddleOCR-VL-1.6-0.9B --host 0.0.0.0 --port 8118 --backend vllm
```
03
Run the full pipeline, not the VLM alone
The docs are emphatic about this: PaddleOCR-VL is a two-stage system. Stage one does layout analysis, detects elements like tables and formulas, determines reading order and crops sub-images; stage two feeds those crops to the VLM and merges the results by reading order. Skipping stage one and throwing a whole page at the VLM is the most common cause of disappointing output. Point the pipeline client at the service from step two.
```
paddleocr install_genai_server_deps vllm
```
04
Tune two parameters against your VRAM
On the vLLM side the docs recommend gpu-memory-utilization of 0.3 and max-num-seqs of 128, rising to 0.7 on memory-tight cards (they use an RTX 3060 as the example). On the client side, vl_rec_max_concurrency controls parallelism: lower it when the server has to handle multiple clients or has limited compute. When the job is done, docker stop — compute billing stops that second.
```
paddleocr genai_server --model_name PaddleOCR-VL-1.6-0.9B --backend vllm --port 8118
```

A real invoice, worked out

Say you need to convert a batch of contract PDFs to Markdown with PaddleOCR-VL-1.6. Setup: an RTX 4090 24GB at $0.540/GPU-hr, pulling the 13GB vLLM image, installing dependencies and getting a demo running takes about 40 minutes, so 0.67 x $0.540 is about $0.36. The batch job then runs for 12 hours straight: 12 x $0.540 = $6.48. Intermediate files and output take 20GB, kept for 3 days after the job for spot checks: 20GB x $0.414/GB-month x 3/30 is about $0.83. Finally you pull 5GB of Markdown and JSON back down: 5 x $0.0081 is about $0.04. That is roughly $7.71 for the whole batch. If you are running the classic PP-OCRv6 pipeline instead, the same 12 hours on an RTX 3090 costs 12 x $0.193 = $2.32. Neither number carries a minimum, a quota request or a setup fee; compute billing stops the second the instance stops, and storage keeps billing only until you destroy the volume.

04 —

Frequently asked questions

How much VRAM does self-hosting PaddleOCR actually need?

Two different answers. The classic PP-OCRv6 pipeline is measured in megabytes: the tiny detector is 1.9MB and even medium recognition is only 73.3MB, so VRAM sits in the hundreds of MB and the real constraint is how big a batch you want. PaddleOCR-VL-1.6 is a 0.9B model with about 2GB of bf16 weights, but the footprint that matters is set by vLLM's gpu-memory-utilization, officially recommended at 0.3, which is roughly a 7GB pool on a 24GB card. That makes 24GB a very comfortable starting point, and both our RTX 3090 24GB at $0.193/GPU-hr and RTX 4090 24GB at $0.540/GPU-hr sit in that tier.

I have a T4 or a V100 — can they run PaddleOCR-VL?

Not advisable. The docs state it directly: the vLLM backend requires compute capability of at least 8.0 and CUDA 12.6 or newer, and although the service will start on compute-capability 7.x cards like the T4 and V100, it is prone to timeouts or OOM and is therefore not recommended. The T4 is CC 7.5, the V100 is 7.0, and the P40 is only 6.1 — all below the bar. That is exactly why we do not point the VL pipeline at our Tesla T4 16GB at $0.298/GPU-hr: it costs more than an RTX 3090 and still misses the requirement. Open an RTX 4090 or RTX 5090 on NexGPU instead, at CC 8.9 and up, and skip that whole class of debugging.

Should I use PP-OCRv6 or PaddleOCR-VL?

It depends on the output you need. If you only want boxes and strings with millisecond latency at minimal cost, use PP-OCRv6 — a single RTX 3090 absorbs a serious amount of throughput. If you need a full page reconstructed into structurally correct Markdown with tables, formulas and reading order intact, you need PaddleOCR-VL-1.6 and its 96.33% on OmniDocBench v1.6. Plenty of teams want both, which is exactly what per-second billing is for on NexGPU: run one of each, evaluate, then stop the one you do not need and its compute billing stops immediately.

Can I use PaddleOCR commercially, and what does licensing cost?

PaddleOCR, PaddleOCR-VL and HPD-Parsing are all Apache 2.0. Commercial use needs no additional licence and no application to Baidu. That is precisely why so many teams move off third-party OCR APIs and self-host: the data never leaves your own machine, and the cost model shifts from per-page to per-GPU-hour. Rent a card on NexGPU and the machine is yours, the weights are ones you pulled yourself, and we only charge for the compute.

Why are my results so much worse than the official demos when I feed images straight to PaddleOCR-VL?

Because you skipped stage one. PaddleOCR-VL is a two-stage design: layout analysis detects tables and formulas, fixes reading order and crops sub-images, and only then does the VLM recognise those crops before everything is merged back in reading order. The docs state that to fully leverage PaddleOCR-VL you must adopt the complete pipeline integrating layout analysis and VLM-based recognition, rather than using the VLM component alone. They also warn that local direct inference may not meet production requirements and strongly recommend a dedicated VLM inference service — on NexGPU that is just one more port on the same box.

The vLLM backend keeps failing to compile. Is there a way around it?

Yes. Both vLLM and SGLang depend on FlashAttention and therefore need a CUDA compile toolchain, which is where unclean environments tend to die; the FastDeploy backend skips that step entirely. You can pull the official FastDeploy image (about 43GB offline) instead of the vLLM one. The genuinely fastest fix, though, is to relaunch on NexGPU from one of the 2,000+ prebuilt images that already has CUDA 12.6 and PyTorch configured — with per-second billing, spinning up a fresh box costs a few cents, far less than debugging a broken one. If you get stuck, message us on Telegram in English or Chinese; there is no ticket queue.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.