danush99 commited on
Commit
e68a24a
·
verified ·
1 Parent(s): 3372846

update readme file

Browse files
Files changed (1) hide show
  1. README.md +70 -19
README.md CHANGED
@@ -1,36 +1,86 @@
1
  ---
2
  license: mit
 
 
3
  base_model: kavg/TrOCR-SIN-DeiT
 
4
  tags:
5
- - generated_from_trainer
6
- model-index:
7
- - name: TrOCR-SIN-DeiT-Handwritten
8
- results: []
9
  ---
10
 
11
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
- should probably proofread and complete it, then remove this comment. -->
13
 
14
- # TrOCR-SIN-DeiT-Handwritten
15
 
16
- This model is a fine-tuned version of [kavg/TrOCR-SIN-DeiT](https://huggingface.co/kavg/TrOCR-SIN-DeiT) on an unknown dataset.
17
- It achieves the following results on the evaluation set:
18
- - Loss: 2.9839
19
- - Cer: 0.5253
20
 
21
- ## Model description
 
 
 
 
22
 
23
- More information needed
24
 
25
- ## Intended uses & limitations
 
 
26
 
27
- More information needed
28
 
29
- ## Training and evaluation data
30
 
31
- More information needed
 
 
 
32
 
33
- ## Training procedure
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
 
35
  ### Training hyperparameters
36
 
@@ -46,6 +96,8 @@ The following hyperparameters were used during training:
46
 
47
  ### Training results
48
 
 
 
49
  | Training Loss | Epoch | Step | Validation Loss | Cer |
50
  |:-------------:|:-----:|:----:|:---------------:|:------:|
51
  | 0.2915 | 3.45 | 100 | 1.8613 | 0.6450 |
@@ -78,7 +130,6 @@ The following hyperparameters were used during training:
78
  | 0.0 | 96.55 | 2800 | 3.1309 | 0.5273 |
79
  | 0.0 | 100.0 | 2900 | 2.9839 | 0.5253 |
80
 
81
-
82
  ### Framework versions
83
 
84
  - Transformers 4.35.2
 
1
  ---
2
  license: mit
3
+ language:
4
+ - si
5
  base_model: kavg/TrOCR-SIN-DeiT
6
+ pipeline_tag: image-to-text
7
  tags:
8
+ - ocr
9
+ - handwriting
10
+ - sinhala
11
+ - trocr
12
  ---
13
 
14
+ # TrOCR-Sin-Handwritten
 
15
 
16
+ A Sinhala text-line recognizer for **handwritten** text, released with the SinFUND and SinOCR datasets ([SriDoc/datasets](https://github.com/SriDoc/datasets)).
17
 
18
+ ## Model
 
 
 
19
 
20
+ - TrOCR-style vision encoder-decoder (`VisionEncoderDecoderModel`).
21
+ - The encoder was initialized from [`facebook/deit-base-distilled-patch16-224`](https://huggingface.co/facebook/deit-base-distilled-patch16-224) (ImageNet weights). The decoder was initialized from [`NLPC-UOM/SinBERT-large`](https://huggingface.co/NLPC-UOM/SinBERT-large), a RoBERTa model pre-trained on Sinhala text. The cross-attention layers started from random weights.
22
+ - No OCR-pretrained TrOCR weights were used.
23
+ - **Input:** one pre-cropped text-line image, resized to 224 x 224.
24
+ - **Decoding:** beam search with 4 beams and a maximum length of 64 tokens.
25
 
26
+ ## Training data
27
 
28
+ This model was trained in two stages:
29
+ 1. On the SinOCR printed training split. That model is [SriDoc/TrOCR-Sin-Printed](https://huggingface.co/SriDoc/TrOCR-Sin-Printed).
30
+ 2. Then on the SinOCR handwritten training split (908 lines) for 100 epochs (2,900 steps), with a batch size of 32 and a learning rate of 5e-5.
31
 
32
+ ## Results
33
 
34
+ Character error rate (CER; lower is better) on the SinOCR test splits:
35
 
36
+ | Test split | CER |
37
+ |---|---|
38
+ | SinOCR handwritten (227 lines) | 0.5253 |
39
+ | SinOCR printed (10,000 lines) | 0.2246 |
40
 
41
+ How the CER is computed:
42
+ - Characters are Unicode code points.
43
+ - The CER is pooled over the whole test split.
44
+
45
+ SinOCR has no separate validation split, so the test split was also the evaluation set during training. The lowest test CER occurred at the final checkpoint, so checkpoint selection did not change this result.
46
+
47
+ ## Limitations
48
+
49
+ - **Errors remain common.** A CER of about 0.53 means roughly one character error for every two characters, so the output needs human checking.
50
+ - **Test writers were seen in training.** The handwritten test split is not writer-disjoint: 72% of its lines come from writers who also appear in training. The CER is therefore probably optimistic for new writers.
51
+ - **Do not use this model to measure OCR accuracy on the SinFUND test forms.** 132 of its training lines are crops of those forms. The dataset repository lists them in `sinocr_handwritten_provenance.csv`.
52
+ - The model reads single, pre-cropped lines. It does not find text on a page.
53
+
54
+ ## Usage
55
+
56
+ ```python
57
+ from transformers import VisionEncoderDecoderModel, TrOCRProcessor, AutoTokenizer, ViTImageProcessor
58
+ from PIL import Image
59
+
60
+ model = VisionEncoderDecoderModel.from_pretrained("SriDoc/TrOCR-Sin-Handwritten")
61
+ tokenizer = AutoTokenizer.from_pretrained("NLPC-UOM/SinBERT-large")
62
+ image_processor = ViTImageProcessor.from_pretrained("google/vit-base-patch16-224")
63
+ processor = TrOCRProcessor(image_processor=image_processor, tokenizer=tokenizer)
64
+
65
+ image = Image.open("line.png").convert("RGB") # one cropped text line
66
+ pixel_values = processor(image, return_tensors="pt").pixel_values
67
+ generated_ids = model.generate(pixel_values)
68
+ print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])
69
+ ```
70
+
71
+ ## Citation
72
+
73
+ ```bibtex
74
+ @article{gunathilaka2025sinfund,
75
+ title = {SinFUND and SinOCR: Benchmarks for Sinhala Handwritten OCR and Template-Free Form Understanding},
76
+ author = {Gunathilaka, Kavishka and Hewagama, Danusha and Pushpakumara, Supul and Ambegoda, Thanuja D.},
77
+ journal = {Research Square preprint},
78
+ year = {2025},
79
+ doi = {10.21203/rs.3.rs-6976719/v1}
80
+ }
81
+ ```
82
+
83
+ ## Training log
84
 
85
  ### Training hyperparameters
86
 
 
96
 
97
  ### Training results
98
 
99
+ The "Cer" and "Validation Loss" columns were measured on the SinOCR handwritten **test** split.
100
+
101
  | Training Loss | Epoch | Step | Validation Loss | Cer |
102
  |:-------------:|:-----:|:----:|:---------------:|:------:|
103
  | 0.2915 | 3.45 | 100 | 1.8613 | 0.6450 |
 
130
  | 0.0 | 96.55 | 2800 | 3.1309 | 0.5273 |
131
  | 0.0 | 100.0 | 2900 | 2.9839 | 0.5253 |
132
 
 
133
  ### Framework versions
134
 
135
  - Transformers 4.35.2