Instructions to use typhoon-ai/typhoon-ocr2-0.8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use typhoon-ai/typhoon-ocr2-0.8b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="typhoon-ai/typhoon-ocr2-0.8b") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("typhoon-ai/typhoon-ocr2-0.8b") model = AutoModelForMultimodalLM.from_pretrained("typhoon-ai/typhoon-ocr2-0.8b", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use typhoon-ai/typhoon-ocr2-0.8b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "typhoon-ai/typhoon-ocr2-0.8b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "typhoon-ai/typhoon-ocr2-0.8b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/typhoon-ai/typhoon-ocr2-0.8b
- SGLang
How to use typhoon-ai/typhoon-ocr2-0.8b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "typhoon-ai/typhoon-ocr2-0.8b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "typhoon-ai/typhoon-ocr2-0.8b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "typhoon-ai/typhoon-ocr2-0.8b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "typhoon-ai/typhoon-ocr2-0.8b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use typhoon-ai/typhoon-ocr2-0.8b with Docker Model Runner:
docker model run hf.co/typhoon-ai/typhoon-ocr2-0.8b
Typhoon OCR V2 0.8B
Typhoon OCR V2 0.8B is a 0.8B-parameter document parser for Thai and English. Given an image of a page, it returns the page's layout elements in reading order, each with a category, a bounding box and its text. Tables come out as HTML, formulas as LaTeX and other text as Markdown. It was trained on the same tasks as the 2B model, but it is much weaker outside full-page parsing, so we recommend it as a page parser only (see Limitations).
The model is a fine-tune of OvisOCR2. We trained it with supervised fine-tuning on the same curated Thai data and public English and Chinese data as the 2B model, then with GRPO, using rewards that give zero to outputs that run into the token limit. On MDPBench, GRPO reduced looping pages from 12 to 6 and raised the score from 75.8 to 77.4. The larger Typhoon OCR V2 2B is more accurate and covers the other tasks.
Our goal was the same as for the 2B model: start from a parser that is already strong on English, raise its Thai accuracy, and keep what it already knew about English. Thai accuracy rose sharply, but some English and Chinese parsing accuracy was lost; see Comparison with the base model. On olmOCR-bench, which is English only, it scores 81.7, against 84.7 for the 2B model.
- Demo: Typhoon OCR Playground
- API: Typhoon OCR V2 is on the Typhoon API as model
typhoon-ocr-v2(OpenAI-compatible endpointhttps://api.opentyphoon.ai/v1) - GGUF (llama.cpp) and MLX versions: GGUF, MLX 8-bit (bf16 and 4-bit MLX versions are also available)
- Ollama: SuraponOCR/typhoon-ocr2-0.8b (
ollama run SuraponOCR/typhoon-ocr2-0.8b;bf16,int8andint4tags) - Python package:
typhoon-ocr2(pip install typhoon-ocr2), code at srn-source/typhoon-ocr2 - Evaluation code (to reproduce the Results): srn-source/evaluation_thai_ocr
- Website: https://opentyphoon.ai
- Previous version: Typhoon OCR 1.5 2B
Use of this model is subject to the OpenTyphoon Terms and Conditions.
- Use vLLM 0.30.0 or newer (
pip install "vllm>=0.30.0").- Enlarge every image whose long side is under 1,600 px to 1,600 px before sending it (see Enlarge small images). The
typhoon-ocr2package does this for you.
Key enhancements over Typhoon OCR 1.5
- Layout with bounding boxes. Each element comes with a category (13 types, such as title, text, table, formula, header and footer) and a box, in reading order. Typhoon OCR 1.5 returned the page as Markdown without positions.
- Smaller: 0.8B parameters, against 2B for Typhoon OCR 1.5.
- Higher page-parsing accuracy on public Thai and English benchmarks: MDPBench from 66.0 to 77.4 (Thai pages from 68.9 to 77.2), olmOCR-bench from 62.0 to 81.7, OmniDocBench v1.6 from 70.97 to 89.37, and SEA-DocBench NED from 0.1726 to 0.1081 (lower is better; the lowest of all models we evaluated).
- Better on photographed pages: MDPBench photographed pages from 64.7 to 75.9.
- Markdown is still available. A few lines of code convert the JSON to Markdown, with or without headers and footers (see Usage).
The 0.8B model was trained on the same general question-answering tasks as the 2B model, but at this size it scores lower on them than Typhoon OCR 1.5 (ThaiOCRBench 36.4 against 40.5). For questions about documents, use Typhoon OCR V2 2B.
Model details
| Parameters | 0.8B, BF16 |
| Architecture | Qwen3.5 (Qwen3_5ForConditionalGeneration), hidden size 1,024 |
| Base model | ATH-MaaS/OvisOCR2 |
| Training | Supervised fine-tuning, then GRPO with a LoRA adapter, merged into the weights in fp32 and stored in bf16 |
| Languages | Thai and English. The training data also contains Chinese, which we did not evaluate separately. |
| Input | One page image. Trained with images of up to 4 MP; the benchmark numbers below use up to 16.7 MP. Images whose long side is under 1,600 px must be enlarged first (see Enlarge small images). |
| Output | A JSON list of layout elements in a json code block |
Comparison with the base model
We ran OvisOCR2 through the same benchmark scripts with the prompt and post-processing from its model card (including its trimming of repeated text at the end of long outputs), images of up to 8.3 MP, greedy decoding, at most 16,384 new tokens and vLLM 0.33.0 in its default mode. Typhoon OCR V2 0.8B was run as described in Results, with our doc2json prompt and vLLM 0.27.1 in eager mode.
| Benchmark | OvisOCR2 | Typhoon OCR V2 0.8B | Change |
|---|---|---|---|
| MDPBench, Thai pages β | 14.7 | 77.2 | +62.5 |
| MDPBench, English pages β | 79.9 | 77.7 | β2.2 |
| MDPBench, photographed pages β | 45.5 | 75.9 | +30.4 |
| MDPBench, digital pages β | 54.3 | 82.0 | +27.7 |
| MDPBench, overall β | 47.7 | 77.4 | +29.7 |
| ThaiOCRBench β | 10.2 | 36.4 | +26.2 |
| SEA-DocBench, Thai NED β | 0.6683 | 0.1004 | β0.5679 (better) |
| SEA-DocBench, English NED β | 0.1003 | 0.1157 | +0.0154 (worse) |
| olmOCR-bench β | 72.3 | 81.7 | +9.4 |
| OmniDocBench v1.6 β | 96.34 | 89.37 | β6.97 |
OvisOCR2 does not read Thai: its Thai output mixes misread Thai with Chinese characters, and on ThaiOCRBench it returns a page parse instead of answering the question. Fine-tuning made Thai usable. Most of the olmOCR-bench gain is in the header and footer tests (18.6 to 94.6), which check that headers and footers are left out of the output: our Markdown conversion drops them and OvisOCR2's post-processing keeps them.
English and Chinese parsing got worse. On OmniDocBench v1.6 (English and Chinese) the score fell by 6.97 points: table TEDS from 93.47 to 82.49, formula CDM from 98.11 to 89.80 and text edit distance from 0.0254 to 0.0416. English pages also got worse on SEA-DocBench and MDPBench. For documents without Thai text, OvisOCR2 is the more accurate of the two.
Results
We ran every model below ourselves with the same evaluation scripts, which are public at srn-source/evaluation_thai_ocr. The numbers are not copied from other papers or model cards, so some differ from what the other authors report.
- Typhoon OCR V2 and Infinity-Parser2-Flash use the doc2json prompt shown in Usage, images of up to 16.7 MP, greedy decoding, at most 16,384 new tokens and vLLM 0.27.1 in eager mode. Their JSON is converted to Markdown with Infinity-Parser2's conversion code, which drops headers, footers and page footnotes. The model still outputs these elements; only the conversion leaves them out. Usage shows how to keep them. These runs did not enlarge small images; see Enlarge small images for the ThaiOCRBench gain with it.
- The other models use the prompts and post-processing published with them.
- Typhoon OCR 1.5's MDPBench scores were measured with vLLM 0.33.0 in its default mode.
- MDPBench and SEA-DocBench are limited to their English and Thai subsets.
- OpenThai2.0-OCR-Markdown and its base model, Nemotron 3.5 Parse, were run as their model cards recommend (NVIDIA's
DocumentParser, one pass per page) with vLLM 0.33.0 in its default mode. We made two changes to their output that do not alter the content: we removed therowspan="1"andcolspan="1"attributes the post-processing puts on every table cell, and we wrote the Thai vowel sara am as the standard character (ΰΈ³, U+0E33), which their tokenizer cannot produce. - OpenThai2.0-OCR-Markdown's low olmOCR-bench, SEA-DocBench and OmniDocBench scores come mostly from formulas and from page headers and footers, not from how well it reads text and tables. It writes formulas as plain text, and these benchmarks only accept LaTeX: it scores 2.3 and 0.0 on the two olmOCR-bench math categories, 24.6 on OmniDocBench's formula metric and about 1.0 (the worst value) on SEA-DocBench's formula NED. SEA-DocBench averages four parts with equal weight, so the formula part alone moves its NED from 0.114 (text, tables and reading order) to 0.334. It also keeps page headers and footers, which olmOCR-bench expects to be left out (23.4 on that category). On the other five olmOCR-bench categories it averages 76.8.
- DeepSeek-OCR-2 was run with the prompt, greedy decoding, no-repeat n-gram logits processor and post-processing of its official OmniDocBench script, with vLLM 0.33.0. On photographed pages it often repeats itself until the token limit, which lowers its MDPBench score most.
- PaddleOCR-VL-1.6, Nemotron 3.5 Parse and OpenThai2.0-OCR-Markdown do not support VQA, so they have no ThaiOCRBench score.
- β Nemotron 3.5 Parse's SEA-DocBench NED is not ranked. It writes most display formulas inline, and on the Thai pages the scorer counted all of them as text, so its Thai NED averages three parts instead of four. Counting only text, tables and reading order for every model, it scores 0.075, close to the best (Typhoon OCR V2 2B and dots.mocr, both 0.074).
In each column, bold is the best result and underlined the second best. β means higher is better and β lower is better.
| Model | Params | MDPBench β | MDPBench TH β | olmOCR-bench β | ThaiOCRBench β | SEA-DocBench NED β | OmniDocBench v1.6 β |
|---|---|---|---|---|---|---|---|
| Chandra-OCR-2 | 5B | 74.2 | 71.1 | 84.4 | 26.4 | 0.1141 | 89.65 |
| dots.mocr | 3B | 76.2 | 73.6 | 85.2 | 37.9 | 0.1135 | 91.09 |
| PaddleOCR-VL-1.6 | 0.9B | 72.5 | 68.4 | 80.8 | n/a | 0.1323 | 95.83 |
| OvisOCR2 | 0.8B | 47.7 | 14.7 | 72.3 | 10.2 | 0.3843 | 96.34 |
| Infinity-Parser2-Flash | 2B | 68.3 | 61.3 | 84.4 | 35.0 | 0.1266 | 91.03 |
| Nemotron 3.5 Parse | 0.6B | 74.7 | 75.9 | 62.7 | n/a | 0.1046β | 80.13 |
| OpenThai2.0-OCR-Markdown | 0.6B | 68.6 | 69.7 | 51.2 | n/a | 0.3342 | 62.02 |
| DeepSeek-OCR-2 | 3B | 48.9 | 40.7 | 78.2 | 10.1 | 0.1356 | 90.05 |
| Typhoon OCR 1.5 | 2B | 66.0 | 68.9 | 62.0 | 40.5 | 0.1726 | 70.97 |
| Typhoon OCR V2 0.8B | 0.8B | 77.4 | 77.2 | 81.7 | 36.4 | 0.1081 | 89.37 |
| Typhoon OCR V2 2B | 2B | 79.8 | 81.3 | 84.7 | 59.9 | 0.1153 | 90.97 |
Per-benchmark component scores will be in the technical report (coming soon).
Examples
All three examples are outputs of this model on benchmark items, run with vLLM 0.27.1 in eager mode. The two parsing examples use max_pixels at 4 MP; the handwriting example comes from our ThaiOCRBench run, where the item's question is the prompt. The same items are shown for the 2B model on its card.
1. Full-page parsing: a dense Thai exam page
Left to right: the input (MDPBench), the predicted boxes numbered in reading order, and the start of the output rendered as Markdown. 21 of the 27 predicted boxes match the benchmark's layout boxes (IoU β₯ 0.5). The layout is right, but the text has a few small Thai errors that the 2B model does not make, for example ΰΈΰΈ£ΰΈ°ΰΈΰΈ²ΰΈ for ΰΈΰΈ£ΰΈ°ΰΈΰΈ²ΰΈ© and ΰΈ«ΰΈ₯ΰΈ±ΰΈΰΉΰΈΰΈΰΈ²ΰΈ£ΰΈΰΈΰΈ for ΰΈ«ΰΈ₯ΰΈ±ΰΈΰΉΰΈΰΈΰΈ²ΰΈ£ΰΈΰΈ³ΰΈΰΈΰΈ in the first elements below.
Model output (first 5 of 27 elements)
[{"bbox": [939, 14, 979, 27], "category": "header", "text": "8 / 29"},
{"bbox": [409, 74, 591, 92], "category": "title", "text": "ΰΈΰΈ²ΰΈ£ΰΈͺΰΈ£ΰΈΈΰΈΰΈΰΈ§ΰΈ²ΰΈ‘ΰΈΰΈ²ΰΈΰΈ ΰΈ²ΰΈ©ΰΈ²"},
{"bbox": [128, 96, 870, 169], "category": "text", "text": "ΰΈΰΈ³ΰΈͺΰΈ±ΰΉΰΈ ΰΉΰΈΰΉΰΈΰΉΰΈ₯ΰΈ°ΰΈΰΉΰΈΰΈΰΈ°ΰΈΰΈ£ΰΈ°ΰΈΰΈΰΈΰΈΰΉΰΈ§ΰΈ’ΰΉΰΈΰΈ·ΰΉΰΈΰΈΰΉΰΈΰΈ«ΰΈ£ΰΈ·ΰΈΰΈΰΉΰΈΰΈ‘ΰΈΉΰΈ₯ΰΈΰΈ΅ΰΉΰΈΰΈ³ΰΈ«ΰΈΰΈΰΉΰΈ«ΰΉ ΰΉΰΈ₯ΰΈ°ΰΈΰΉΰΈΰΈͺΰΈ£ΰΈΈΰΈΰΉΰΈΰΉΰΈΰΈΰΈΉΰΉ ΰΉ ΰΈΰΈ²ΰΈΰΉΰΈΰΈ·ΰΉΰΈΰΈΰΉΰΈΰΈΰΈ±ΰΉΰΈΰΉΰΈ«ΰΉΰΈ¨ΰΈΆΰΈΰΈ©ΰΈ²ΰΉΰΈΰΈ·ΰΉΰΈΰΈΰΉΰΈΰΈΰΈ΅ΰΉΰΈΰΈ³ΰΈ«ΰΈΰΈΰΉΰΈ«ΰΉΰΈΰΉΰΈΰΈ ΰΉΰΈ₯ΰΉΰΈ§ΰΈΰΈΆΰΈΰΈΰΈ²ΰΈ¨ΰΈ±ΰΈ’ΰΈΰΈ§ΰΈ²ΰΈ‘ΰΈ£ΰΈΉΰΉΰΈΰΈ΅ΰΉΰΉΰΈΰΉΰΈΰΈ²ΰΈΰΉΰΈΰΈ·ΰΉΰΈΰΈΰΉΰΈΰΈΰΈ±ΰΈΰΈΰΈ₯ΰΉΰΈ²ΰΈ§ΰΈ‘ΰΈ²ΰΉΰΈΰΉΰΈΰΈ΄ΰΈΰΈ²ΰΈ£ΰΈΰΈ²ΰΈΰΉΰΈΰΈͺΰΈ£ΰΈΈΰΈΰΈΰΈ±ΰΉΰΈΰΈͺΰΈΰΈΰΈΰΈΰΈΰΉΰΈΰΉΰΈ₯ΰΈ°ΰΈΰΉΰΈ ΰΉΰΈ₯ΰΉΰΈ§ΰΈΰΈ³ΰΈΰΈΰΈΰΈ₯ΰΈΰΉΰΈΰΈΰΈ£ΰΈ°ΰΈΰΈ²ΰΈΰΈΰΈ³ΰΈΰΈΰΈ ΰΉΰΈΰΈ’ΰΈ’ΰΈΆΰΈΰΈ«ΰΈ₯ΰΈ±ΰΈΰΉΰΈΰΈΰΈ²ΰΈ£ΰΈΰΈΰΈΰΈΰΈ±ΰΈΰΈΰΈ΅ΰΉ"},
{"bbox": [129, 200, 273, 219], "category": "title", "text": "ΰΈ«ΰΈ₯ΰΈ±ΰΈΰΉΰΈΰΈΰΈ²ΰΈ£ΰΈΰΈΰΈ"},
{"bbox": [129, 225, 474, 245], "category": "text", "text": "ΰΈΰΈΰΈ 1. ΰΈΰΉΰΈ²ΰΈΰΉΰΈΰΈͺΰΈ£ΰΈΈΰΈΰΈΰΈ±ΰΉΰΈΰΈͺΰΈΰΈΰΉΰΈΰΉΰΈΰΈΰΈ£ΰΈ΄ΰΈΰΈΰΈ²ΰΈ‘ΰΉΰΈΰΈ·ΰΉΰΈΰΈΰΉΰΈ"},
...]
2. Full-page parsing: a Thai nurse-training handout with references
18 of the 20 predicted boxes match the benchmark's layout boxes (IoU β₯ 0.5).
Model output (first 5 of 20 elements)
[{"bbox": [496, 43, 883, 77], "category": "header", "text": "ΰΉΰΈΰΈΰΈͺΰΈ²ΰΈ£ΰΈΰΈ£ΰΈ°ΰΈΰΈΰΈΰΈΰΈ²ΰΈ£ΰΈΰΈΰΈ£ΰΈ‘ ΰΈΰΈ²ΰΈ£ΰΈΰΈΉΰΉΰΈ₯ΰΈΰΈΉΰΉΰΈΰΈ²ΰΈΰΉΰΈΰΉΰΈΰΈΰΈ±ΰΉΰΈΰΈͺΰΈΉΰΈ ΰΈ£ΰΈΈΰΉΰΈΰΈΰΈ΅ΰΉ 13 Advanced trauma emergency care for nurse :ATEN"},
{"bbox": [897, 45, 929, 61], "category": "header", "text": "127"},
{"bbox": [176, 94, 724, 157], "category": "text", "text": "ΰΈͺΰΈΉΰΈΰΈ£ΰΈΰΈ²ΰΈ£ΰΈΰΈ³ΰΈΰΈ§ΰΈΰΈΰΈ₯ΰΈ±ΰΈΰΈΰΈ²ΰΈΰΉΰΈΰΈ₯ΰΈΰΈ£ΰΈ΅ΰΉ (25 Kcal x kg.)+(40 Kcal x % BSA)\nΰΈΰΈ±ΰΈ§ΰΈΰΈ’ΰΉΰΈ²ΰΈΰΈΰΈ²ΰΈ’ΰΉΰΈΰΈ’ ΰΈΰΈ. 50 ΰΈΰΈ. Burn 80%TBSA Total Calories/day\n= (25x50)+(40x80)=4450 Kcal/day"},
{"bbox": [178, 181, 362, 198], "category": "title", "text": "10.ΰΈΰΈ²ΰΈ£ΰΈΰΈ’ΰΈ²ΰΈΰΈ²ΰΈ₯ΰΈΰΉΰΈ²ΰΈΰΈΰΈ΄ΰΈΰΉΰΈ"},
{"bbox": [208, 202, 460, 220], "category": "text", "text": "β ΰΈ‘ΰΈ΅ΰΈΰΈ²ΰΈ£ΰΈΰΈ£ΰΈ°ΰΈΰΈ±ΰΈΰΈΰΈ£ΰΈ°ΰΈΰΈΰΈΰΈΰΉΰΈ²ΰΈΰΈΰΈ΄ΰΈΰΉΰΈ"},
...]
3. Handwriting: lecture notes
Prompt: ΰΈΰΉΰΈΰΈΰΈ΅ΰΉ 4 ΰΉΰΈΰΈ ΰΈ²ΰΈΰΈΰΈ΅ΰΉ ΰΉΰΈΰΉΰΈΰΈΰΉΰΈΰΈΰΈ§ΰΈ²ΰΈ‘ΰΈΰΈ΅ΰΉΰΉΰΈΰΈ΅ΰΈ’ΰΈΰΈΰΉΰΈ§ΰΈ’ΰΈ‘ΰΈ·ΰΈΰΈ§ΰΉΰΈ²ΰΈΰΈ°ΰΉΰΈ£ΰΈΰΉΰΈ²ΰΈ ΰΈΰΈΰΈΰΉΰΈΰΈΰΈ²ΰΈ°ΰΈΰΉΰΈΰΈΰΈ§ΰΈ²ΰΈ‘ΰΈΰΈ²ΰΈ‘ΰΈ ΰΈ²ΰΈΰΈΰΈ£ΰΈ ΰΉ ΰΈΰΈ’ΰΉΰΈ²ΰΈΰΈΰΈ£ΰΈ°ΰΈΰΈ±ΰΈ ΰΉΰΈΰΈ’ΰΉΰΈ‘ΰΉΰΉΰΈͺΰΉΰΈΰΈ³ΰΈΰΈΰΈ΄ΰΈΰΈ²ΰΈ’ΰΉΰΈΰΈ΄ΰΉΰΈ‘ΰΉΰΈΰΈ΄ΰΈ‘ (What is handwritten in item 4?)
Output, score 0.84 (Typhoon OCR V2 2B: 0.96; best of the four other open models we ran on ThaiOCRBench: 0.38). One phrase is misread: ΰΈ‘ΰΈ²ΰΈΰΉΰΈ₯ΰΈ’ instead of ΰΈ‘ΰΈ²ΰΈΰΈΰΉΰΈΰΈ.
β£ ΰΈΰΈΉΰΈΰΈΰΈ²ΰΈ // ΰΈΰΈΉΰΉΰΈΰΈ²ΰΈ’ΰΈ£ΰΈ²ΰΈ’ΰΉΰΈΰΈ΅ΰΈ’ΰΈ§ ΰΉΰΈ‘ΰΉΰΈ‘ΰΈ΅ΰΈͺΰΈ΄ΰΈΰΈΰΉΰΈ²ΰΈΰΈ·ΰΉΰΈΰΈ‘ΰΈ²ΰΈΰΉΰΈ₯ΰΈ’ ex. ΰΉΰΈΰΈΰΉΰΈ² ΰΈΰΈ£ΰΈ°ΰΈΰΈ²
Usage
Python package
The typhoon-ocr2 package (pip install typhoon-ocr2) wraps everything below: it sends the prompt, enlarges small images, reads PDFs and returns each page as layout elements and Markdown. It works with the Typhoon API, a vLLM server (see vLLM server), Ollama and llama.cpp.
from typhoon_ocr2 import TyphoonOCR
ocr = TyphoonOCR(
base_url="http://localhost:8000/v1", # the vLLM server below; or "https://api.opentyphoon.ai/v1" for the Typhoon API
model="typhoon-ocr2", # the --served-model-name of that server
version="v2",
upscale=True, # required: enlarges images under 1,600 px
)
pages = ocr.parse("report.pdf") # a PDF or an image
print(pages[0].markdown) # Markdown, without headers and footers
print(pages[0].elements[0]) # {"bbox": [...], "category": "...", "text": "..."}
See the package README for questions, key-information extraction and the command line. The older typhoon-ocr package runs Typhoon OCR 1.5 only. To use the model without the package, follow the sections below.
The model was trained on a fixed set of prompts. For full-page parsing, use this prompt exactly as written; it is the prompt from Infinity-Parser2.
PROMPT = """- Extract layout information from the provided PDF image.
- For each layout element, output its bbox, category, and the text content within the bbox.
- Bbox format: [x1, y1, x2, y2].
- Allowed layout categories: ['header', 'title', 'text', 'figure', 'table', 'formula', 'figure_caption', 'table_caption', 'formula_caption', 'figure_footnote', 'table_footnote', 'page_footnote', 'footer'].
- Text extraction and formatting:
1) For 'figure', the text field must be an empty string.
2) For 'formula', format text as LaTeX.
3) For 'table', format text as HTML.
4) For all other categories (e.g., text, title), format text as Markdown.
- The output text must be exactly the original text from the image, with no translation or rewriting.
- Sort all layout elements in human reading order.
- Final output must be a single JSON object."""
Although the last line asks for a JSON object, the model returns a JSON list inside a json code block:
[{"bbox": [62, 48, 938, 96], "category": "title", "text": "Annual Report 2025"},
{"bbox": [62, 120, 938, 410], "category": "text", "text": "The company ..."},
{"bbox": [62, 432, 938, 700], "category": "table", "text": "<table>...</table>"},
{"bbox": [480, 950, 520, 968], "category": "footer", "text": "12"}]
Bounding boxes use a 0β1000 grid relative to the page width and height, so they do not depend on the image resolution. These helpers parse the output, convert it to Markdown and map a box back to pixels:
import json, re
def parse_elements(raw: str) -> list[dict]:
"""Parse the model output: a JSON list of elements inside a ```json fence."""
m = re.search(r"```json\n(.*?)(?:\n```|$)", raw, re.DOTALL)
text = (m.group(1) if m else raw).strip()
try:
return json.loads(text)
except json.JSONDecodeError:
# Output stopped at the token limit: drop the last, incomplete element.
cut = text.rfind('{"bbox":')
return json.loads(text[:cut].rstrip().rstrip(",") + "]")
def to_markdown(elements: list[dict], keep_header_footer: bool = False) -> str:
skip = set() if keep_header_footer else {"header", "footer", "page_footnote"}
return "\n\n".join(e["text"] for e in elements if e.get("text") and e["category"] not in skip)
def to_pixels(bbox, width, height):
x1, y1, x2, y2 = bbox
return [x1 * width / 1000, y1 * height / 1000, x2 * width / 1000, y2 * height / 1000]
The JSON from the model always contains every element. to_markdown leaves out header, footer and page_footnote elements by default, which is how we evaluated the model. To keep them, pass keep_header_footer=True:
elements = parse_elements(raw) # raw: the text the model returned
md = to_markdown(elements) # drops header, footer, page_footnote (as in our evaluation)
md_all = to_markdown(elements, keep_header_footer=True) # keeps every element
Enlarge small images (required)
Before sending a page to the model, enlarge it so that its long side is at least 1,600 pixels, keeping the aspect ratio. The model reads small images poorly, Thai text most of all, and the image processor does not enlarge them for you. On ThaiOCRBench, enlarging raised this model from 37.3 to 41.3 (and the 2B model from 59.9 to 62.9); MDPBench, where almost every page is already larger, did not change. All examples below use load_image, which does this.
import base64, io
from PIL import Image
def upscale_min_long(image: Image.Image, min_long: int = 1600) -> Image.Image:
"""Enlarge the image so that its long side is at least `min_long` pixels, keeping the
aspect ratio. Images that are already large enough are returned unchanged."""
scale = min_long / max(image.size)
if scale <= 1:
return image
return image.resize((round(image.width * scale), round(image.height * scale)), Image.LANCZOS)
def load_image(path: str) -> Image.Image:
return upscale_min_long(Image.open(path).convert("RGB"))
def to_data_url(image: Image.Image) -> str:
"""PNG data URL for the vLLM examples."""
buf = io.BytesIO()
image.save(buf, format="PNG")
return "data:image/png;base64," + base64.b64encode(buf.getvalue()).decode()
The bounding boxes are on the 0β1000 grid, so to_pixels with the original width and height maps them onto the original image.
The examples below need PROMPT and the helpers above. Use the other settings in the code as they are; the benchmark numbers were produced with them. Use vLLM 0.30.0 or newer:
pip install "vllm>=0.30.0"
vLLM server
vllm serve typhoon-ai/typhoon-ocr2-0.8b \
--served-model-name typhoon-ocr2 \
--max-model-len 40960 \
--limit-mm-per-prompt '{"image": 1}' \
--mm-processor-kwargs '{"min_pixels": 2048, "max_pixels": 16777216}' \
--default-chat-template-kwargs '{"enable_thinking": false}'
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
image_url = to_data_url(load_image("page.jpg"))
resp = client.chat.completions.create(
model="typhoon-ocr2",
messages=[{"role": "user", "content": [
{"type": "image_url", "image_url": {"url": image_url}},
{"type": "text", "text": PROMPT},
]}],
temperature=0.0,
max_tokens=16384,
)
elements = parse_elements(resp.choices[0].message.content)
print(to_markdown(elements))
vLLM offline
from vllm import LLM, SamplingParams
def main():
llm = LLM(
model="typhoon-ai/typhoon-ocr2-0.8b",
max_model_len=40960,
limit_mm_per_prompt={"image": 1},
mm_processor_kwargs={"min_pixels": 2048, "max_pixels": 16777216},
)
sampling = SamplingParams(temperature=0.0, max_tokens=16384)
image_url = to_data_url(load_image("page.jpg"))
messages = [{"role": "user", "content": [
{"type": "image_url", "image_url": {"url": image_url}},
{"type": "text", "text": PROMPT},
]}]
out = llm.chat(messages, sampling, chat_template_kwargs={"enable_thinking": False})
print(to_markdown(parse_elements(out[0].outputs[0].text)))
if __name__ == "__main__":
main()
Faster generation with a draft head (optional, vLLM only)
typhoon-ai/typhoon-ocr2-0.8b-eagle3 is an EAGLE-3 draft head for this model. With speculative decoding in vLLM, it made generation 1.5 to 1.7 times faster in our tests, depending on the load. To use it, add one line to the server command above:
--speculative-config '{"method": "eagle3", "model": "typhoon-ai/typhoon-ocr2-0.8b-eagle3", "num_speculative_tokens": 3}'
For LLM(...), add speculative_config={"method": "eagle3", "model": "typhoon-ai/typhoon-ocr2-0.8b-eagle3", "num_speculative_tokens": 3}. We tested it with vLLM 0.33.0. Outputs can differ slightly from decoding without it, as they also do between repeated runs without it; on MDPBench the score was 76.9 with the draft head, against 76.3 to 76.9 in four runs without it. The draft head does not work with Transformers, GGUF, Ollama or MLX, and it is for the 0.8B model only. See its model card for details.
Transformers
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "typhoon-ai/typhoon-ocr2-0.8b"
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")
processor = AutoProcessor.from_pretrained(model_id)
image = load_image("page.jpg")
messages = [{"role": "user", "content": [
{"type": "image", "image": image},
{"type": "text", "text": PROMPT},
]}]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt",
enable_thinking=False,
processor_kwargs={"images_kwargs": {"min_pixels": 2048, "max_pixels": 16777216}},
).to(model.device)
out = model.generate(**inputs, max_new_tokens=16384, do_sample=False)
raw = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(to_markdown(parse_elements(raw)))
Questions about a document
To ask a question about a document, send the question as the text prompt in place of PROMPT. The answer comes back as plain text.
Extracting key information as JSON
To pull the fields out of a receipt, form, label or poster without listing them, use this prompt (with this model, check the output carefully; see Limitations):
Perform open-schema key information extraction (WYSIWYG) on this image: one JSON object, printed labels as keys, values copied exactly, "" for blank fields, arrays for repeated rows. Return only JSON.
To get specific fields, list them in the prompt instead.
Limitations
- Outside full-page parsing the model is weak. Its ThaiOCRBench score is 36.4, with chart parsing at 7.4, full-page OCR at 47.9 and key-information extraction at 10.9.
- The model can fall into repetition loops, repeating text until it reaches the token limit. When that happens,
parse_elementsabove keeps only the complete elements. - Extracting all fields as JSON without a field list (see Usage) works much less well than with the 2B model: on 40 training documents, 28 outputs were valid JSON (2B: 39) and 5 ran into the token limit. Use the 2B model for this task.
- Tables and formulas are weaker than in models of a similar size: on OmniDocBench, PaddleOCR-VL-1.6 reaches table TEDS 93.38 and formula CDM 97.48, and the base model OvisOCR2 93.47 and 98.11, against 82.49 and 89.80 for this model.
- We did not hold out a validation set for parsing. The SFT checkpoint this model continues from was chosen by MDPBench score, so its MDPBench numbers are likely somewhat optimistic. The GRPO checkpoint is the last saved checkpoint of its run and was not selected by score. The other four benchmarks were not used for any choice.
- Bounding boxes and image size. The model was trained with images of up to 4 MP, and the Usage examples allow up to 16.7 MP. On 13 digital MDPBench pages larger than 4 MP, the share of predicted boxes matching the benchmark's layout boxes (F1 at IoU β₯ 0.5) was 0.75 at 4 MP and 0.72 at 16.7 MP; the difference comes from one dense newspaper page whose output at 16.7 MP was not valid JSON. Text is more accurate at 16.7 MP: with
max_pixelsat 4,194,304 (4 MP), MDPBench drops from 77.4 to 75.6, mostly on Thai pages (77.2 to 74.1) and tables (TEDS 0.663 to 0.580). Keep 16.7 MP unless you need less GPU memory or faster inference. Better box accuracy is a goal for the next version. - Accuracy can depend on the inference engine. Use vLLM 0.30.0 or newer. The GGUF Q8_0 (llama.cpp) and MLX 8-bit versions score 77.4 on MDPBench, the same as the original weights, and the Ollama
int8version 75.7; their pages list the scores of every quantization. SGLang has not been tested. - We studied Thai and English only. Chinese is in the training data and in OmniDocBench but was not evaluated on its own, and other languages were not tested.
- The model has no safety filtering and can output text that is not in the image. It is meant for reading documents, not for open-ended chat.
Citation
Typhoon OCR V2 technical report: coming soon.
Typhoon OCR 1.5:
@misc{nonesung2026typhoonocropenvisionlanguage,
title = {Typhoon OCR: Open Vision-Language Model For Thai Document Extraction},
author = {Surapon Nonesung and Natapong Nitarach and Teetouch Jaknamon and Pittawat Taveekitworachai and Kunat Pipatanakul},
year = {2026},
eprint = {2601.14722},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
url = {https://arxiv.org/abs/2601.14722}
}
Support
- Discord: https://discord.gg/us5gAYmrxw
- X (Twitter): https://twitter.com/opentyphoon
- Downloads last month
- 600
