Connais 1
Connais 1 is a small decision model. It reads a state (text) and one or more typed questions, and returns a probability for every allowed answer in a single encoder pass. It has 66,826,753 parameters, reads text only and generates no text.
Public Decision Index: 25.94, from this project's own complete run of the public suite. It is not a leaderboard entry: the leaderboard's maintainers have not reviewed or rerun it.
Summary
- What it does. Multiple choice among options you name (
choice), yes or no (noul), and an ordered scale (score). Options are free text, set per request, and may carry a few examples. - Size. 66,826,753 parameters, 267 MB of weights in 32-bit floats.
- Context. Accepts up to 32,768 tokens per question; a longer request is refused, never cut. It is most reliable when the deciding information is within a few thousand tokens of the question (see Long requests).
- Public Decision Index (0.3 suite): 25.94, from this project's own complete run of the public suite. Not a leaderboard entry (see Benchmarks).
- It knows few facts. It recognises which offered option fits a text. It is not a source of knowledge, and its scores on knowledge benchmarks are low.
Run it
Everything needed is in this repository: the weights, the tokenizer and base-encoder settings, and the inference code. Nothing else is downloaded.
hf download Connais/connais-1-68m --local-dir connais-1
pip install ./connais-1 # the `connais` package; add "[kit]" for the Decision Index kit
python connais-1/example.py # answers examples/request.json
from connais import load_engine
engine = load_engine("connais-1") # a folder, or "Connais/connais-1-68m"
response, timing = engine(
"User: is it raining in Zurich?",
{"tool": {"type": "choice", "instructions": "Which tool?",
"criteria": {"get_weather": "Current weather for a city",
"send_email": "Send an email to a contact",
"none": None}}},
)
As a server with the common POST /v1/systemone shape:
python -m connais.server --checkpoint connais-1 --port 8000
requirements.txt lists the exact versions the package was checked with. pip install alone
takes compatible ones; transformers and tokenizers are pinned either way.
Request and response
{
"model": "connais-1",
"state": "User: is it raining in Zurich?",
"questions": {
"tool": {
"type": "choice",
"instructions": "Which tool?",
"criteria": {
"get_weather": "Current weather for a city",
"send_email": "Send an email to a contact",
"none": null
}
},
"urgent": {"type": "noul", "instructions": "Is it urgent?"},
"tone": {"type": "score", "instructions": "How polite?", "criteria": ["rude", "neutral", "polite"]}
}
}
The model's answer to that request (captured from these weights on a CPU; the last decimals vary by device):
{
"model": "connais-1",
"answers": {
"tool": {
"type": "choice",
"choice": "get_weather",
"probabilities": {
"get_weather": 0.970036680744708,
"send_email": 0.0005388215954726673,
"none": 0.02942449765981942
},
"confidence": 0.970036680744708
},
"urgent": {
"type": "noul",
"noul": 0.053177199069175146,
"confidence": 0.9468228009308248
},
"tone": {
"type": "score",
"score": 1.037020503783832,
"probabilities": [
0.014370147023889206,
0.9342392021683895,
0.05139065080772126
],
"confidence": 0.9342392021683895
}
},
"usage": {
"input_tokens": 97
}
}
noulis the probability of yes.scoreis the expected level (0 is the first level).confidenceis the probability of the reported answer. Set a threshold on it to trade coverage for precision.- A question may carry
examples: a few known cases per option, such as{"billing": ["I was charged twice"], "tech": ["The app crashes on login"]}. They are written into the option they belong to. - A
choicetakes 2 to 255 options, ascore2 to 10 levels. - A request that is malformed or too long raises an error (HTTP 422 from the server). Nothing is truncated and no option is dropped.
Intended use
What it is for:
- choosing a tool, or deciding whether to call one at all, inside an agent;
- routing a request to an intent or a handler from a list you describe;
- yes/no checks and judgements about a text you supply;
- sorting cases into your own categories, with or without a few examples per category.
What it is not for:
- answering questions from memory (it knows few facts);
- images, audio or anything but text;
- multi-step reasoning;
- decisions with real consequences without a confidence threshold and a fallback: it can be confidently wrong.
Benchmarks
Public Decision Index (0.3 suite): 25.94 (before the correction for chance: 43.82).
This is this project's own complete run of the public suite of the Decision Index, scored with the benchmark's kit. It is not an entry on the leaderboard and not the leaderboard's Full score, which comes from private tests that only its maintainers run. They have not reviewed or rerun this result.
- 140,178 scored requests, all answered; 0 refused, 0 errors. One run, no error bars.
- Settings: at most 32,768 tokens per question, 32-bit floats, one request at a time, on an Apple M4 Mac mini. Time per request over the run: median 19.7 ms, mean 145.4 ms, 95th percentile 416.8 ms.
By area (0 is guessing, 100 is perfect):
| Area | Skill (0 to 100) | Raw | Coverage |
|---|---|---|---|
| Knowledge & Reasoning | 3.8 | 28.1 | 100.0% |
| Language Understanding | 23.4 | 44.0 | 100.0% |
| Retrieval & Classification | 54.4 | 65.3 | 100.0% |
| Tools & Automation | 37.0 | 43.9 | 100.0% |
| Arts & Human Taste | 12.5 | 40.9 | 100.0% |
For comparison, the public index of other small decision models, read from the leaderboard's data on 16 November 2026:
| Model | Public index |
|---|---|
| Amber 68M | 37.89 |
| Crivo-68M | 32.82 |
| Mola-68M | 27.27 |
| Connais 1 (this project's own run) | 25.94 |
| LiquidAI d1-omni-600M (587M parameters) | 17.90 |
| Bekko System One v0 68M | 13.01 |
Every benchmark
| Benchmark | Metric | Score | Chance | Skill (0 to 100) | Answered | In the index |
|---|---|---|---|---|---|---|
| BFCL | case exact accuracy | 0.714 | 0.259 | 61.4 | 1,694 of 1,694 | yes |
| ToolRet | nDCG@10 | 0.308 | 0.134 | 20.1 | 685 of 685 | yes |
| API-Bank | accuracy | 0.510 | 0.019 | 50.0 | 508 of 508 | yes |
| BANKING77 | macro-F1 | 0.912 | 0.013 | 91.1 | 3,080 of 3,080 | yes |
| CLINC150+OOS | macro-F1 | 0.885 | 0.006 | 88.5 | 5,500 of 5,500 | yes |
| RouterBench | selected quality (quality objective) | 0.602 | 0.574 | 6.6 | 10,000 of 10,000 | shown, not counted |
| Home appliance simulator | case exact accuracy | 0.000 | 0.000 | 0.0 | 88 of 88 | yes |
| SGD/SGD-X | macro-F1 | 0.508 | 0.399 | 18.2 | 2,500 of 2,500 | shown, not counted |
| ContractNLI | macro-F1 | 0.459 | 0.308 | 21.8 | 123 of 123 | yes |
| ANLI | macro-F1 | 0.417 | 0.332 | 12.7 | 3,200 of 3,200 | yes |
| BPoMP | accuracy | 0.677 | 0.500 | 33.9 | 5,000 of 5,000 | yes |
| Humicroedit | accuracy | 0.528 | 0.500 | 5.7 | 2,628 of 2,628 | yes |
| POP909-CL | accuracy | 0.005 | 0.008 | 0.0 | 2,000 of 2,000 | yes |
| cfcolor | accuracy | 0.484 | 0.500 | 0.0 | 5,000 of 5,000 | yes |
| MMLU | accuracy | 0.297 | 0.250 | 6.3 | 14,033 of 14,033 | shown, not counted |
| GPQA Diamond | accuracy | 0.219 | 0.250 | 0.0 | 196 of 196 | yes |
| ARC-Easy | accuracy | 0.555 | 0.250 | 40.6 | 2,376 of 2,376 | shown, not counted |
| ARC-Challenge | accuracy | 0.361 | 0.250 | 14.8 | 1,172 of 1,172 | shown, not counted |
| WinoGrande | accuracy | 0.527 | 0.500 | 5.4 | 1,267 of 1,267 | yes |
| HellaSwag | accuracy | 0.328 | 0.250 | 10.4 | 10,042 of 10,042 | yes |
| GSM8K | accuracy | 0.172 | 0.250 | 0.0 | 2,638 of 2,638 | yes |
| ChessBench | accuracy | 0.092 | 0.082 | 1.1 | 5,000 of 5,000 | yes |
| MuSR | accuracy | 0.480 | 0.371 | 17.3 | 752 of 752 | yes |
| SATA-Bench | case exact accuracy | 0.049 | 0.013 | 3.6 | 1,650 of 1,650 | yes |
| SimpleBench | accuracy | 0.200 | 0.167 | 4.0 | 10 of 10 | shown, not counted |
| BRIGHT | nDCG@10 | 0.171 | 0.116 | 6.2 | 220 of 220 | yes |
| Amazon ESCI | macro-F1 | 0.367 | 0.203 | 20.6 | 5,000 of 5,000 | yes |
| ACOS | per-review F1 | 0.273 | 0.031 | 25.0 | 1,565 of 1,565 | yes |
| FinEntity | macro-F1 | 0.804 | 0.320 | 71.2 | 979 of 979 | yes |
| iSarcasmEval | Sarcasm F1 · track A, English | 0.286 | 0.223 | 8.1 | 4,600 of 4,600 | yes |
| VAST | macro-F1 | 0.678 | 0.333 | 51.7 | 3,006 of 3,006 | yes |
| NLI4CT | macro-F1 | 0.560 | 0.486 | 14.3 | 5,500 of 5,500 | yes |
| CRUXEval | accuracy | 0.421 | 0.370 | 8.1 | 570 of 570 | yes |
| CLadder | accuracy | 0.499 | 0.500 | 0.0 | 5,000 of 5,000 | yes |
| HLE | accuracy | 0.162 | 0.164 | 0.0 | 501 of 501 | yes |
| Habermas Machine | accuracy | 0.396 | 0.311 | 12.4 | 1,676 of 1,676 | yes |
| PhishNChips phishing decisions | accuracy | 0.809 | 0.500 | 61.9 | 2,000 of 2,000 | yes |
| MMLU-Pro | accuracy | 0.131 | 0.111 | 2.3 | 12,032 of 12,032 | yes |
| BBH fixed-option tasks | accuracy | 0.345 | 0.310 | 5.0 | 5,507 of 5,507 | yes |
| RAGTruth response-level hallucination | F1 on hallucinated class | 0.181 | 0.518 | 0.0 | 2,700 of 2,700 | yes |
| HoVer claim verification | accuracy | 0.769 | 0.500 | 53.8 | 4,000 of 4,000 | yes |
| When2Call MCQ | accuracy | 0.594 | 0.250 | 45.8 | 3,652 of 3,652 | yes |
| New Yorker caption matching | accuracy | 0.385 | 0.200 | 23.1 | 528 of 528 | yes |
Speed
On an Apple M4 Mac mini (GPU through MPS, 32-bit floats, one request at a time), over 305 requests of mixed length (median 593 tokens): median 41.5 ms, mean 393 ms, 80th percentile 386 ms, 95th percentile 1,380 ms. Time grows with the length of the request: most requests are short and a few are thousands of tokens long, which is why the mean sits far above the median. Not measured on other hardware. Measured with the weights first published on 17 November 2026; the current ones have the same size and layout.
Architecture
| Part | What it is | Parameters |
|---|---|---|
| Trunk layers | jhu-clsp/ettin-encoder-68m (ModernBERT family), 19 bidirectional layers, width 512 |
42.36M |
| Token table | 46,000 of the trunk's 50,368 entries, width 512; the tokenizer is unchanged | 23.55M |
| Option scorer | MLP over each option, the answer slot and their product | 0.79M |
| Binding codes | 256 codes of width 512, an unused way of passing examples; saved with the model | 0.13M |
| Total | 66.8M |
One question is one token sequence:
[CLS] state [SEP] <type> instructions <ans> [SEP] <opt> option 1 <opt> option 2 ... [SEP]
- The trunk reads the whole sequence in both directions.
- Each option is the mean hidden state over its own tokens, scored against the hidden state at
<ans>. The scorer takes the option state, the answer state and their product. - Scores go through a softmax. There is one temperature per question type; all three are 1.
- A token that was dropped from the table is written as a sequence of kept tokens, so text is still represented exactly, with more pieces for rare words.
- The questions of one request share the state and run as one batch.
Long requests
The base encoder marks positions by rotation, so nothing in its weights ends at a fixed length. What ends is memory: the stock forward pass builds a table of every token against every token.
A question of up to 7,999 tokens takes the stock forward pass. (
config.jsonstill records 7,999, the longest request seen in training; the limit of 32,768 is set by the package.)A longer one, up to 32,768 tokens, is read block by block: the same attention, computed in slices, so memory grows with the length and not with its square. This path is used only when answering, never in training.
Same arithmetic. On 15 requests of up to 5,400 tokens answered both ways, the probabilities differ by at most 2.7e-4 in 32-bit arithmetic and 9.3e-7 in 64-bit: rounding.
Time and memory, measured on an Apple M4 CPU with 3 threads: requests whose longest question has 8,000 to 19,000 tokens took 21 to 108 s each (several questions per request). Peak memory 4.7 GB. Not measured on a GPU at these lengths.
What it does with a long request. The model was not trained on requests of more than 7,999 tokens. In a test with made-up log text holding one fact and one look-alike (6 cases per setting; guessing gets about 1.5):
Request length Fact near the start In the middle Right before the question 3,900 tokens 5 of 6 5 of 6 6 of 6 7,800 3 of 6 5 of 6 6 of 6 11,900 1 of 6 2 of 6 6 of 6 15,900 1 of 6 1 of 6 6 of 6 23,900 0 of 6 0 of 6 5 of 6 31,900 2 of 6 2 of 6 6 of 6 It reads a long request, but it relies on what is close to the question. Put the deciding information within a few thousand tokens of the question.
On benchmark requests beyond 7,999 tokens: measured on 27 requests of the public Decision Index suite only. All 27 were answered, at about chance level. On the two contract-review requests just over 7,999 tokens (34 questions of about 8,300 tokens) accuracy was 0.173, below the chance level of 0.308. On the 23 tool-retrieval requests nDCG@10 was 0.116 (chance 0.127), and on the 2 BRIGHT requests 0.000. Two contract requests are a very small sample, but there is no evidence that answers to requests of this kind beyond 7,999 tokens are better than guessing.
Limitations
- It knows few facts and does not reason in steps. There is no retrieval and no scratchpad. Expect chance-level answers where the answer is not in the request.
- Text only, mostly English.
- Long requests. Up to 32,768 tokens per question are accepted, but the model was not trained on more than 7,999 and uses mainly what is close to the question: a fact far from it is often missed from about 8,000 tokens on, and usually missed beyond 12,000 (see Long requests). Speed and memory at these lengths have been measured on a CPU only.
- Confidence has not been measured on the released weights. On earlier versions it matched accuracy on familiar kinds of task and was too high on unfamiliar ones. Choose a threshold on your own data.
- Benchmark figures are this project's own. In the complete run the model is at or near chance (skill under 5 of 100) on 11 counted benchmarks: Home appliance simulator, POP909-CL, cfcolor, GPQA Diamond, GSM8K, ChessBench, SATA-Bench, CLadder, HLE, MMLU-Pro, RAGTruth response-level hallucination. Every figure is a single run without error bars.
With the Decision Index kit
pip install "./connais-1[kit]"
python connais-1/examples/check.py connais-1 # nine sample requests through the kit's engine interface
decision-index run --engine connais.kit:ConnaisEngine \
--option checkpoint=connais-1 --option max_length=32768 --edition 0.3 --compact --out runs/connais-1
- Settings: 32-bit floats, one process, one request at a time, at most 32,768 tokens per question, no truncation, no option filtering, one rendering for every benchmark.
- The package in this repository is the inference path of the project's development code, under
the name
connais. Checked against that code on 59 requests (1,464 probabilities): both feed the model the same tokens and return the same probabilities, to the last bit, on a CPU and on an Apple GPU. The longest question compared had 18,347 tokens; 1 of the requests went through the block-by-block path. - The development name of the engine,
smolesscion.kit:SmolEngine, is kept as an alias ofconnais.kit:ConnaisEngine. model.safetensorssha256:787355a1f3d230a66484353b7434b20d7922e9be58e2980102ba980d4c6da32c
Versions
- 18 November 2026: the weights were replaced with a set that scores higher on the public Decision Index (25.94 against 25.47 in the same kind of run). Same size and layout; the inference code is unchanged. Every benchmark figure on this page is from the current weights. On the two contract-review requests just beyond 7,999 tokens the first weights did better: 0.517 correct against 0.173.
- 17 November 2026: first weights. They stay available at revision
ff7ae85.
Licence
The weights (model.safetensors, with config.json and vocab.json) are released under the Creative Commons Attribution-NonCommercial 4.0 International licence (CC BY-NC 4.0): you may use, share and adapt them for non-commercial purposes, with credit to Connais. The code in this repository (the connais package, the examples) is released under the Apache License 2.0. The file LICENSE says which text covers what.
The base encoder, jhu-clsp/ettin-encoder-68m, is MIT-licensed (see NOTICE).
Name and contact
Written Connais 1, said "koh-NEH one". French connaître: to know by acquaintance, to recognise. The model recognises which offered option fits; it knows few facts.
Questions and reports: open a discussion on this repository.
- Downloads last month
- 49
Model tree for Connais/connais-1-68m
Base model
jhu-clsp/ettin-encoder-68m