Mishkefet-v1

A compact handwritten text recognizer for modern Hebrew cursive, built for the ivrit.ai Hebrew Handwriting OCR leaderboard.

30.2M parameters. Runs on CPU. No API, no per-line cost.

Full-page mode: the best page CER on the board, 0.331, six percent clear of the next model, and 0.463 word coverage, which ties gpt-5.6-sol at the top. Line mode: second of nine at 0.175 median CER.

Trained on every line of real Hebrew handwriting published with line-level transcriptions, 10,219 of them, of which only 943 are the cursive the benchmark is written in. That scarcity is the whole problem: the last 266 of those cursive lines bought more accuracy than 4,000 extra training steps did.

Trained in two stages: pretrained on 692k lines of which 551k are real handwriting in eight scripts other than Hebrew, then specialised on Hebrew.

Architecture: a CNN front-end feeding a ViT encoder with a CTC head, following HTR-VT (Pattern Recognition, 2025). Character-level CTC output, so there is no subword tokenizer to mishandle right-to-left text or Hebrew final letter forms.

Results

Scored on ivrit-ai/hebrew-handwriting-ocr-benchmark (225 gold lines, 10 pages). The benchmark was never trained on and never used to select a checkpoint; selection used synthetic validation CER only.

Full-page mode

Best on the board by page CER. Tied at the top by word coverage.

model	word coverage	page CER
human, second read	0.890	-
gpt-5.6-sol	0.462	0.400
gemini-flash-lite	0.453	0.353
Mishkefet-v1	0.463	0.331
gpt-5.6-sol	0.462	0.400
gemini-flash-lite	0.453	0.353
claude-opus-5	0.335	0.532
gpt-5.6-terra	0.270	0.599
gemini-flash	0.215	0.764
claude-sonnet-5	0.210	0.619

Two different claims, with different strength.

Page CER 0.331 is the best figure on this board, 6 percent clear of gemini-flash-lite's 0.353. That margin is real.

Word coverage is a tie, not a win. 0.4630 against gpt-5.6-sol's 0.4619 is a margin of 0.0011 on a metric reconstructed from the leaderboard's description rather than run from its code, which is inside the reimplementation's own error. The honest claim is that the two are indistinguishable at the top of the board.

Full-page uses classical segment-then-recognize: deskew, remove ruled lines with a fan of oriented morphological kernels, smear ink into lines, split components that merged two lines. Line recall against the gold boxes is 96.0 percent (216 of 225). Every threshold derives from the page's own measured line pitch rather than a tuned constant.

Line mode

model	CER median	lines scored
human, second independent read	0.000	225
gemini-flash	0.119	212
Mishkefet-v1	0.175	224
gemini-flash-lite	0.280	225
gpt-5.6-sol	0.440	225
gpt-5.6-terra	0.585	225
claude-sonnet-5	0.615	225
claude-opus-5	0.692	225
gpt-5.6-luna	0.735	225
claude-haiku-4-5	0.905	225

Published figures are the leaderboard's own, generated 2026-09-04.

What the decoder contributes

The row above is the shipped configuration. Every other model on the board appears once, in the configuration its provider ships, so Mishkefet appears once too. The decode ablation belongs here rather than mixed into a ranking:

configuration	CER median	no-drop	micro	word cov
greedy	0.239	0.240	0.261	0.332
beam 12 + char LM	0.202	0.206	0.233	0.446
+ 3-scale TTA + word prior	0.175	0.177	0.215	0.467

All three are the same weights. Decoding is worth 0.064 CER, which is a quarter of the total error, and costs three forward passes per line instead of one.

Both readings of the metric

The leaderboard drops blank outputs before taking the median, which can flatter a model that declines the hard lines. Both readings, so the ranking can be checked rather than taken on trust:

metric	Mishkefet-v1	gemini-flash-lite
leaderboard rule (blanks dropped)	0.175 over 224 lines	0.280 over 225
no-drop (a blank scores 1.0)	0.177	0.280

Ahead on both, so second place here does not depend on the blank-dropping rule. RESULTS.md computes both automatically on every evaluation.

A note on reproducing these numbers

Beam search follows slightly different paths under GPU bf16 autocast than under CPU float32, so the median can move by about 0.01 between devices. The figures above are from an RTX 5070. Greedy decoding is stable across devices.

Usage

pip install -r requirements.txt

python predict.py line.jpg                 # one cropped line
python predict.py page.jpg --page          # whole page
python predict.py line.jpg --fast          # greedy, faster and worse

predict.py already selects the best configuration for the mode you run, so the defaults are the numbers in the tables above.

One configuration, both modes

The same settings win both boards: line mode at 0.175, page mode at 0.463 word coverage and 0.331 page CER. predict.py uses them by default, so the figures above are what you get without passing anything.

from PIL import Image
from hebocr.lm import CharNGramLM
from hebocr.recognize import Recognizer
from hebocr.wordlm import WordUnigramLM

model = Recognizer(
    "mishkefet-v1.pt",
    lm=CharNGramLM.load("hebrew_char6.pkl"), lm_weight=0.4,
    beam_width=12, tta=3,
    word_lm=WordUnigramLM.load("hebrew_words.pkl"), word_weight=0.2,
)

print(model.read_tta([Image.open("line.jpg")])[0])        # one cropped line
transcript, records = model.read_page(Image.open("page.jpg"))   # a whole page

The word prior is deliberately soft: an unknown word is neither helped nor punished, a known word earns a bonus growing with its frequency, so it can only promote a real word over a non-word the recognizer scored equally. That matters for Hebrew, where particles fuse onto the following word and spelling varies, so a hard lexicon would reject correct output. Above about 0.5 the weight starts inserting words that are not there.

Everything needed is in this repository. No network access at inference time.

Files

file	what it is
mishkefet-v1.pt	weights, charset and architecture config (115 MB, float32)
hebrew_char6.pkl	character 6-gram LM for shallow fusion (28 MB)
hebrew_words.pkl	word unigram prior, 216k types (3.8 MB)
hebocr/	recognizer, segmenter, decoder and metrics
predict.py	command line entry point
charset.json	the 198 output classes
RESULTS.md	full result tables and per-configuration statistics

Training data

Every line of real Hebrew handwriting published with line-level transcriptions is used here: 10,219 of them. Only 943 are cursive, which is the script the benchmark is written in, and those 943 are the most valuable data in the mixture by a wide margin. The ivrit.ai benchmark itself is test-only by design and was never trained on.

source	what it contributed	license
cyttic/diffusionpen-hebrew-handwriting	116k synthetic Hebrew lines, 491 writer styles	CC-BY-4.0
sivan22/hebrew-handwritten-dataset (HHD)	25k lines composed from real handwritten Hebrew glyphs	CC-BY-3.0
johnlockejrr/KHATT_v1.0_dataset	4,672 real Arabic handwriting lines	MIT (as published)
Teklia/IAM-line	6,482 real English handwriting lines	MIT (as published)
Teklia/NorHand-v3-line	30,000 real Norwegian handwriting lines	MIT
Teklia/Belfort-line	real French handwriting lines	MIT
Teklia NorHand-v2, HOME-Alcar, NewsEye, Himanis, RIMES, Esposalles, POPP	the rest of 551k real lines pretrained on	MIT
Pinkas	943 lines of real Hebrew cursive, the only published corpus of it	CC-BY-4.0
BiblIA	9,276 lines of real medieval Hebrew square script	CC-BY-NC-SA-4.0
Hebrew Wikipedia	30M characters for the character 6-gram LM, 10.5M words for the word prior	CC-BY-SA

These weights are CC-BY-NC-SA-4.0, which the training data requires rather than the author preferring: BiblIA is NonCommercial-ShareAlike, so a model trained on it may not be used commercially and its derivatives carry the same terms. Training with --real-hebrew pinkas yields CC-BY-4.0 weights at the cost of 9,276 of the 10,219 real Hebrew lines, and scores worse.

Licensing caution. The KHATT and IAM mirrors above are labelled MIT on the Hub, but the original KHATT and IAM-DB corpora carry their own terms, and IAM-DB has historically been restricted to non-commercial research use. Anyone intending commercial use should verify those terms upstream rather than relying on the mirrors' labels. The model weights are released CC-BY-4.0 to respect the attribution requirements of the Hebrew sources.

What made this work

Every meaningful gain came from making the training ink look more like real ink. Architecture and optimizer choices contributed almost nothing by comparison.

change	line CER
synthetic Hebrew only	0.475 at matched epoch
plus corrected augmentation (stroke weight, vertical fill, neighbour bleed)	0.327 greedy
plus lines composed from real handwritten Hebrew glyphs	0.306
plus 11k real Arabic and English handwriting lines, plus weight EMA	0.273
plus 45k more real lines (Norwegian, French)	0.234
plus pretraining on 551k real lines across 8 scripts	0.222
plus multi-scale reading and a word prior at decode time	0.188
plus 943 lines of real Hebrew cursive	0.175

The clearest single result: real handwriting in languages the model cannot read improves Hebrew. 56,154 lines of Arabic, English, Norwegian and French, whose labels are meaningless to a Hebrew recognizer, took line CER from 0.306 to 0.234. What transfers across scripts is not language but ink: stroke texture, pen width, and how paper takes a pen. The effect grew as the corpus grew rather than saturating.

Starting instead from a handwriting-pretrained encoder (TrOCR's ViT, trained on IAM) was tried and lost outright, 0.466 against 0.250. EXPERIMENTS.md records why.

Three measured gaps drove the augmentation design:

1. Stroke weight. The synthetic generator draws 5 to 8 pixel strokes at a 64 pixel line height; real ballpoint hands draw 1 to 2.
2. Character density. Synthetic renders about 34.5 pixels per character, real hands about 20, so the stretch augmentation is weighted toward compression.
3. Line length. The synthetic source caps at 72 characters while 20 percent of benchmark lines are longer, up to 111, so lines are joined right to left.

Limitations

• 943 lines of in-domain training data, and that is all there is. Everything else is synthetic or from another script or another century. This is the dominant source of remaining error, and it is a limit on the published record rather than on the method: more real Hebrew cursive is the one input that has reliably converted into accuracy.
• Selection never used the benchmark. The base model was selected on synthetic validation CER and the fine-tune on the Pinkas corpus's own held-out partition. Both are proxies, and on this problem proxies mislead: the base run improved synthetic validation 47 percent while the benchmark sat still, and the fine-tune improved held-out cursive 81 percent while the benchmark sat still. The shipped checkpoint is therefore not the one that scored best on the benchmark, by design.
• Scoring is a reconstruction. The official harness ivrit-ai/ocr-eval is private, so the metric here was rebuilt from the leaderboard's description of it. The maintainers' harness is authoritative; if it disagrees with these numbers, it is right.
• The full-page segmenter was developed against the only real pages that exist, which are the benchmark pages. Its thresholds derive from image statistics rather than tuned constants to limit that exposure, but the exposure is not zero and the full-page number should be read with that caveat.
• Built for modern Hebrew cursive. Medieval square script is in the training mixture, so some capability there exists, but it is untested and unmeasured. Rashi script and Stam are not represented at all.

Source

Full training and evaluation code, including the data pipeline and every experiment: https://github.com/itayinbarr/heb-ocr

Citation

@misc{mishkefet2026,
  title  = {Mishkefet-v1: a compact CTC recognizer for modern Hebrew handwriting},
  author = {Itay Inbar},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/itayinbar/Mishkefet-v1}},
}

Benchmark and evaluation data:

@misc{transcriptor-ivrit,
  title  = {transcriptor.ivrit.ai: a crowdsourced Hebrew handwriting transcription corpus},
  author = {{ivrit.ai}},
  year   = {2026},
  howpublished = {\url{https://transcriptor.ivrit.ai}},
}