Post
68
Okay so, we at 
udgamlabs just released Lekhani, a Hindi transliterator that is 0.87 MB in total. You type Hindi in English letters, like "kaise ho", and it writes कैसे हो.
The whole thing is a 0.81M-parameter model at 4 bits, a 200,000-word list and a 5 KB JavaScript runner with no dependencies. It runs in the browser or in Node, so nothing you type leaves your device.
On the Aksharantar Hindi test it gets 65.9% of words exactly right, 7 points more than IndicXlit's model on its own (58.9%), which has 14x more parameters. IndicXlit's full package is still ahead at 68.7%, but its word list alone takes 358 MB of memory, about 400 times the size of all of Lekhani. I think for a web page or an app, that difference matters a lot.
It reads one word at a time, so "main" and "mein" can come out the same, and a few common words still come out wrong. The card has the full table, the other open models we measured and where it falls short.
It's Apache-2.0 for the model and runner, and CC0 for the word list. If you type Hindi in English letters, try it and tell me which words it gets wrong.
Model: udgamlabs/Lekhani
Try it: https://udgamlabs.com/models/lekhani
Code: https://github.com/udgamlabs/lekhani
The whole thing is a 0.81M-parameter model at 4 bits, a 200,000-word list and a 5 KB JavaScript runner with no dependencies. It runs in the browser or in Node, so nothing you type leaves your device.
On the Aksharantar Hindi test it gets 65.9% of words exactly right, 7 points more than IndicXlit's model on its own (58.9%), which has 14x more parameters. IndicXlit's full package is still ahead at 68.7%, but its word list alone takes 358 MB of memory, about 400 times the size of all of Lekhani. I think for a web page or an app, that difference matters a lot.
It reads one word at a time, so "main" and "mein" can come out the same, and a few common words still come out wrong. The card has the full table, the other open models we measured and where it falls short.
It's Apache-2.0 for the model and runner, and CC0 for the word list. If you type Hindi in English letters, try it and tell me which words it gets wrong.
Model: udgamlabs/Lekhani
Try it: https://udgamlabs.com/models/lekhani
Code: https://github.com/udgamlabs/lekhani