Connais 1

Connais 1

A wooden sorting frame of empty pigeonholes against a grey wall. One compartment holds a single pale envelope.

Connais 1 is a small decision model. It reads a state (text) and one or more typed questions, and returns a probability for every allowed answer in a single encoder pass. It has 66,826,753 parameters, reads text only and generates no text.

Public Decision Index: 25.94, from this project's own complete run of the public suite. It is not a leaderboard entry: the leaderboard's maintainers have not reviewed or rerun it.

Summary

  • What it does. Multiple choice among options you name (choice), yes or no (noul), and an ordered scale (score). Options are free text, set per request, and may carry a few examples.
  • Size. 66,826,753 parameters, 267 MB of weights in 32-bit floats.
  • Context. Accepts up to 32,768 tokens per question; a longer request is refused, never cut. It is most reliable when the deciding information is within a few thousand tokens of the question (see Long requests).
  • Public Decision Index (0.3 suite): 25.94, from this project's own complete run of the public suite. Not a leaderboard entry (see Benchmarks).
  • It knows few facts. It recognises which offered option fits a text. It is not a source of knowledge, and its scores on knowledge benchmarks are low.

Run it

Everything needed is in this repository: the weights, the tokenizer and base-encoder settings, and the inference code. Nothing else is downloaded.

hf download Connais/connais-1-68m --local-dir connais-1
pip install ./connais-1                     # the `connais` package; add "[kit]" for the Decision Index kit
python connais-1/example.py                 # answers examples/request.json
from connais import load_engine

engine = load_engine("connais-1")           # a folder, or "Connais/connais-1-68m"
response, timing = engine(
    "User: is it raining in Zurich?",
    {"tool": {"type": "choice", "instructions": "Which tool?",
              "criteria": {"get_weather": "Current weather for a city",
                           "send_email": "Send an email to a contact",
                           "none": None}}},
)

As a server with the common POST /v1/systemone shape:

python -m connais.server --checkpoint connais-1 --port 8000

requirements.txt lists the exact versions the package was checked with. pip install alone takes compatible ones; transformers and tokenizers are pinned either way.

Request and response

{
  "model": "connais-1",
  "state": "User: is it raining in Zurich?",
  "questions": {
    "tool": {
      "type": "choice",
      "instructions": "Which tool?",
      "criteria": {
        "get_weather": "Current weather for a city",
        "send_email": "Send an email to a contact",
        "none": null
      }
    },
    "urgent": {"type": "noul", "instructions": "Is it urgent?"},
    "tone": {"type": "score", "instructions": "How polite?", "criteria": ["rude", "neutral", "polite"]}
  }
}

The model's answer to that request (captured from these weights on a CPU; the last decimals vary by device):

{
  "model": "connais-1",
  "answers": {
    "tool": {
      "type": "choice",
      "choice": "get_weather",
      "probabilities": {
        "get_weather": 0.970036680744708,
        "send_email": 0.0005388215954726673,
        "none": 0.02942449765981942
      },
      "confidence": 0.970036680744708
    },
    "urgent": {
      "type": "noul",
      "noul": 0.053177199069175146,
      "confidence": 0.9468228009308248
    },
    "tone": {
      "type": "score",
      "score": 1.037020503783832,
      "probabilities": [
        0.014370147023889206,
        0.9342392021683895,
        0.05139065080772126
      ],
      "confidence": 0.9342392021683895
    }
  },
  "usage": {
    "input_tokens": 97
  }
}
  • noul is the probability of yes. score is the expected level (0 is the first level).
  • confidence is the probability of the reported answer. Set a threshold on it to trade coverage for precision.
  • A question may carry examples: a few known cases per option, such as {"billing": ["I was charged twice"], "tech": ["The app crashes on login"]}. They are written into the option they belong to.
  • A choice takes 2 to 255 options, a score 2 to 10 levels.
  • A request that is malformed or too long raises an error (HTTP 422 from the server). Nothing is truncated and no option is dropped.

Intended use

What it is for:

  • choosing a tool, or deciding whether to call one at all, inside an agent;
  • routing a request to an intent or a handler from a list you describe;
  • yes/no checks and judgements about a text you supply;
  • sorting cases into your own categories, with or without a few examples per category.

What it is not for:

  • answering questions from memory (it knows few facts);
  • images, audio or anything but text;
  • multi-step reasoning;
  • decisions with real consequences without a confidence threshold and a fallback: it can be confidently wrong.

Benchmarks

Public Decision Index (0.3 suite): 25.94 (before the correction for chance: 43.82).

This is this project's own complete run of the public suite of the Decision Index, scored with the benchmark's kit. It is not an entry on the leaderboard and not the leaderboard's Full score, which comes from private tests that only its maintainers run. They have not reviewed or rerun this result.

  • 140,178 scored requests, all answered; 0 refused, 0 errors. One run, no error bars.
  • Settings: at most 32,768 tokens per question, 32-bit floats, one request at a time, on an Apple M4 Mac mini. Time per request over the run: median 19.7 ms, mean 145.4 ms, 95th percentile 416.8 ms.

By area (0 is guessing, 100 is perfect):

Area Skill (0 to 100) Raw Coverage
Knowledge & Reasoning 3.8 28.1 100.0%
Language Understanding 23.4 44.0 100.0%
Retrieval & Classification 54.4 65.3 100.0%
Tools & Automation 37.0 43.9 100.0%
Arts & Human Taste 12.5 40.9 100.0%

For comparison, the public index of other small decision models, read from the leaderboard's data on 16 November 2026:

Model Public index
Amber 68M 37.89
Crivo-68M 32.82
Mola-68M 27.27
Connais 1 (this project's own run) 25.94
LiquidAI d1-omni-600M (587M parameters) 17.90
Bekko System One v0 68M 13.01
Every benchmark
Benchmark Metric Score Chance Skill (0 to 100) Answered In the index
BFCL case exact accuracy 0.714 0.259 61.4 1,694 of 1,694 yes
ToolRet nDCG@10 0.308 0.134 20.1 685 of 685 yes
API-Bank accuracy 0.510 0.019 50.0 508 of 508 yes
BANKING77 macro-F1 0.912 0.013 91.1 3,080 of 3,080 yes
CLINC150+OOS macro-F1 0.885 0.006 88.5 5,500 of 5,500 yes
RouterBench selected quality (quality objective) 0.602 0.574 6.6 10,000 of 10,000 shown, not counted
Home appliance simulator case exact accuracy 0.000 0.000 0.0 88 of 88 yes
SGD/SGD-X macro-F1 0.508 0.399 18.2 2,500 of 2,500 shown, not counted
ContractNLI macro-F1 0.459 0.308 21.8 123 of 123 yes
ANLI macro-F1 0.417 0.332 12.7 3,200 of 3,200 yes
BPoMP accuracy 0.677 0.500 33.9 5,000 of 5,000 yes
Humicroedit accuracy 0.528 0.500 5.7 2,628 of 2,628 yes
POP909-CL accuracy 0.005 0.008 0.0 2,000 of 2,000 yes
cfcolor accuracy 0.484 0.500 0.0 5,000 of 5,000 yes
MMLU accuracy 0.297 0.250 6.3 14,033 of 14,033 shown, not counted
GPQA Diamond accuracy 0.219 0.250 0.0 196 of 196 yes
ARC-Easy accuracy 0.555 0.250 40.6 2,376 of 2,376 shown, not counted
ARC-Challenge accuracy 0.361 0.250 14.8 1,172 of 1,172 shown, not counted
WinoGrande accuracy 0.527 0.500 5.4 1,267 of 1,267 yes
HellaSwag accuracy 0.328 0.250 10.4 10,042 of 10,042 yes
GSM8K accuracy 0.172 0.250 0.0 2,638 of 2,638 yes
ChessBench accuracy 0.092 0.082 1.1 5,000 of 5,000 yes
MuSR accuracy 0.480 0.371 17.3 752 of 752 yes
SATA-Bench case exact accuracy 0.049 0.013 3.6 1,650 of 1,650 yes
SimpleBench accuracy 0.200 0.167 4.0 10 of 10 shown, not counted
BRIGHT nDCG@10 0.171 0.116 6.2 220 of 220 yes
Amazon ESCI macro-F1 0.367 0.203 20.6 5,000 of 5,000 yes
ACOS per-review F1 0.273 0.031 25.0 1,565 of 1,565 yes
FinEntity macro-F1 0.804 0.320 71.2 979 of 979 yes
iSarcasmEval Sarcasm F1 · track A, English 0.286 0.223 8.1 4,600 of 4,600 yes
VAST macro-F1 0.678 0.333 51.7 3,006 of 3,006 yes
NLI4CT macro-F1 0.560 0.486 14.3 5,500 of 5,500 yes
CRUXEval accuracy 0.421 0.370 8.1 570 of 570 yes
CLadder accuracy 0.499 0.500 0.0 5,000 of 5,000 yes
HLE accuracy 0.162 0.164 0.0 501 of 501 yes
Habermas Machine accuracy 0.396 0.311 12.4 1,676 of 1,676 yes
PhishNChips phishing decisions accuracy 0.809 0.500 61.9 2,000 of 2,000 yes
MMLU-Pro accuracy 0.131 0.111 2.3 12,032 of 12,032 yes
BBH fixed-option tasks accuracy 0.345 0.310 5.0 5,507 of 5,507 yes
RAGTruth response-level hallucination F1 on hallucinated class 0.181 0.518 0.0 2,700 of 2,700 yes
HoVer claim verification accuracy 0.769 0.500 53.8 4,000 of 4,000 yes
When2Call MCQ accuracy 0.594 0.250 45.8 3,652 of 3,652 yes
New Yorker caption matching accuracy 0.385 0.200 23.1 528 of 528 yes

Speed

On an Apple M4 Mac mini (GPU through MPS, 32-bit floats, one request at a time), over 305 requests of mixed length (median 593 tokens): median 41.5 ms, mean 393 ms, 80th percentile 386 ms, 95th percentile 1,380 ms. Time grows with the length of the request: most requests are short and a few are thousands of tokens long, which is why the mean sits far above the median. Not measured on other hardware. Measured with the weights first published on 17 November 2026; the current ones have the same size and layout.

Architecture

Part What it is Parameters
Trunk layers jhu-clsp/ettin-encoder-68m (ModernBERT family), 19 bidirectional layers, width 512 42.36M
Token table 46,000 of the trunk's 50,368 entries, width 512; the tokenizer is unchanged 23.55M
Option scorer MLP over each option, the answer slot and their product 0.79M
Binding codes 256 codes of width 512, an unused way of passing examples; saved with the model 0.13M
Total 66.8M

One question is one token sequence:

[CLS] state [SEP] <type> instructions <ans> [SEP] <opt> option 1 <opt> option 2 ... [SEP]
  • The trunk reads the whole sequence in both directions.
  • Each option is the mean hidden state over its own tokens, scored against the hidden state at <ans>. The scorer takes the option state, the answer state and their product.
  • Scores go through a softmax. There is one temperature per question type; all three are 1.
  • A token that was dropped from the table is written as a sequence of kept tokens, so text is still represented exactly, with more pieces for rare words.
  • The questions of one request share the state and run as one batch.

Long requests

The base encoder marks positions by rotation, so nothing in its weights ends at a fixed length. What ends is memory: the stock forward pass builds a table of every token against every token.

  • A question of up to 7,999 tokens takes the stock forward pass. (config.json still records 7,999, the longest request seen in training; the limit of 32,768 is set by the package.)

  • A longer one, up to 32,768 tokens, is read block by block: the same attention, computed in slices, so memory grows with the length and not with its square. This path is used only when answering, never in training.

  • Same arithmetic. On 15 requests of up to 5,400 tokens answered both ways, the probabilities differ by at most 2.7e-4 in 32-bit arithmetic and 9.3e-7 in 64-bit: rounding.

  • Time and memory, measured on an Apple M4 CPU with 3 threads: requests whose longest question has 8,000 to 19,000 tokens took 21 to 108 s each (several questions per request). Peak memory 4.7 GB. Not measured on a GPU at these lengths.

  • What it does with a long request. The model was not trained on requests of more than 7,999 tokens. In a test with made-up log text holding one fact and one look-alike (6 cases per setting; guessing gets about 1.5):

    Request length Fact near the start In the middle Right before the question
    3,900 tokens 5 of 6 5 of 6 6 of 6
    7,800 3 of 6 5 of 6 6 of 6
    11,900 1 of 6 2 of 6 6 of 6
    15,900 1 of 6 1 of 6 6 of 6
    23,900 0 of 6 0 of 6 5 of 6
    31,900 2 of 6 2 of 6 6 of 6

    It reads a long request, but it relies on what is close to the question. Put the deciding information within a few thousand tokens of the question.

  • On benchmark requests beyond 7,999 tokens: measured on 27 requests of the public Decision Index suite only. All 27 were answered, at about chance level. On the two contract-review requests just over 7,999 tokens (34 questions of about 8,300 tokens) accuracy was 0.173, below the chance level of 0.308. On the 23 tool-retrieval requests nDCG@10 was 0.116 (chance 0.127), and on the 2 BRIGHT requests 0.000. Two contract requests are a very small sample, but there is no evidence that answers to requests of this kind beyond 7,999 tokens are better than guessing.

Limitations

  • It knows few facts and does not reason in steps. There is no retrieval and no scratchpad. Expect chance-level answers where the answer is not in the request.
  • Text only, mostly English.
  • Long requests. Up to 32,768 tokens per question are accepted, but the model was not trained on more than 7,999 and uses mainly what is close to the question: a fact far from it is often missed from about 8,000 tokens on, and usually missed beyond 12,000 (see Long requests). Speed and memory at these lengths have been measured on a CPU only.
  • Confidence has not been measured on the released weights. On earlier versions it matched accuracy on familiar kinds of task and was too high on unfamiliar ones. Choose a threshold on your own data.
  • Benchmark figures are this project's own. In the complete run the model is at or near chance (skill under 5 of 100) on 11 counted benchmarks: Home appliance simulator, POP909-CL, cfcolor, GPQA Diamond, GSM8K, ChessBench, SATA-Bench, CLadder, HLE, MMLU-Pro, RAGTruth response-level hallucination. Every figure is a single run without error bars.

With the Decision Index kit

pip install "./connais-1[kit]"
python connais-1/examples/check.py connais-1      # nine sample requests through the kit's engine interface
decision-index run --engine connais.kit:ConnaisEngine \
    --option checkpoint=connais-1 --option max_length=32768 --edition 0.3 --compact --out runs/connais-1
  • Settings: 32-bit floats, one process, one request at a time, at most 32,768 tokens per question, no truncation, no option filtering, one rendering for every benchmark.
  • The package in this repository is the inference path of the project's development code, under the name connais. Checked against that code on 59 requests (1,464 probabilities): both feed the model the same tokens and return the same probabilities, to the last bit, on a CPU and on an Apple GPU. The longest question compared had 18,347 tokens; 1 of the requests went through the block-by-block path.
  • The development name of the engine, smolesscion.kit:SmolEngine, is kept as an alias of connais.kit:ConnaisEngine.
  • model.safetensors sha256: 787355a1f3d230a66484353b7434b20d7922e9be58e2980102ba980d4c6da32c

Versions

  • 18 November 2026: the weights were replaced with a set that scores higher on the public Decision Index (25.94 against 25.47 in the same kind of run). Same size and layout; the inference code is unchanged. Every benchmark figure on this page is from the current weights. On the two contract-review requests just beyond 7,999 tokens the first weights did better: 0.517 correct against 0.173.
  • 17 November 2026: first weights. They stay available at revision ff7ae85.

Licence

The weights (model.safetensors, with config.json and vocab.json) are released under the Creative Commons Attribution-NonCommercial 4.0 International licence (CC BY-NC 4.0): you may use, share and adapt them for non-commercial purposes, with credit to Connais. The code in this repository (the connais package, the examples) is released under the Apache License 2.0. The file LICENSE says which text covers what.

The base encoder, jhu-clsp/ettin-encoder-68m, is MIT-licensed (see NOTICE).

Name and contact

Written Connais 1, said "koh-NEH one". French connaître: to know by acquaintance, to recognise. The model recognises which offered option fits; it knows few facts.

Questions and reports: open a discussion on this repository.

Downloads last month
49
Safetensors
Model size
66.8M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 1 Ask for provider support

Model tree for Connais/connais-1-68m

Finetuned
(47)
this model