Add Boris-1.3-75M and Boris-1.3-125M (OpenCerebral)
Submitting two from-scratch pretrained base models from OpenCerebral (formerly NMAI).
Both are GPT-2 architecture base checkpoints trained from scratch on one RTX 3060, open weights, Apache-2.0, and are training checkpoints (not merges) — they meet the stated qualification criteria.
| Boris-1.3-75M | Boris-1.3-125M | |
|---|---|---|
| Params | 77,431,680 | 124,475,904 |
| HellaSwag (acc_norm) | 27.57 | 29.59 |
| PIQA (acc_norm) | 59.36 | 61.70 |
| ARC-Easy (acc_norm) | 40.11 | 41.46 |
| ARC-Challenge (acc_norm) | 23.89 | 24.40 |
| ArithMark-3 (acc_norm) | 33.40 | 35.20 |
| Intelligence Index | 10.62 | 13.41 |
How these were produced
- HellaSwag / PIQA / ARC-Easy / ARC-Challenge:
lm-evalv0.4.9, 0-shot,--model hf,dtype=float32, batch size 64, on a single RTX 3060. - ArithMark-3: the official
bencharithmark-3.pyfromAxiomicLabs/ArithMark-3.0,--dtype float32 --device cuda --batch-size 64. Dataset SHA-256bf8ab1a5193d52cdf0e05ff0b3ca226bdfcf416cb6e75562dcbe72e7e4559435. - ArithMark-2 was not run, matching other recent entries that omit it.
One note for reproduction
bencharithmark-3.py loads tokenizers with use_fast=False. GPT-NeoX-style tokenizers ship no slow implementation, so this raises Tokenizer class GPTNeoXTokenizer does not exist for any model using that tokenizer — I confirmed it fails identically on EleutherAI/gpt-neox-20b and EleutherAI/pythia-70m under transformers 4.56.2, so it is not specific to these repos. I ran with a local one-line fallback to use_fast=True (the only implementation that exists for these tokenizers); nothing else in the scoring path was changed. Happy to open a separate PR on the dataset repo for that if useful.
The chart color for the new opencerebral org entry is a placeholder — please adjust if it clashes.
Both models are available for independent verification: