Instructions to use DreamBlooms/decider-0.8b-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use DreamBlooms/decider-0.8b-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf DreamBlooms/decider-0.8b-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf DreamBlooms/decider-0.8b-GGUF:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf DreamBlooms/decider-0.8b-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf DreamBlooms/decider-0.8b-GGUF:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf DreamBlooms/decider-0.8b-GGUF:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf DreamBlooms/decider-0.8b-GGUF:Q8_0
Use Docker
docker model run hf.co/DreamBlooms/decider-0.8b-GGUF:Q8_0
- Llama new
- LM Studio
- Jan
- Ollama
How to use DreamBlooms/decider-0.8b-GGUF with Ollama:
ollama run hf.co/DreamBlooms/decider-0.8b-GGUF:Q8_0
- Unsloth Desktop
- Pi
How to use DreamBlooms/decider-0.8b-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf DreamBlooms/decider-0.8b-GGUF:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "DreamBlooms/decider-0.8b-GGUF:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use DreamBlooms/decider-0.8b-GGUF with Docker Model Runner:
docker model run hf.co/DreamBlooms/decider-0.8b-GGUF:Q8_0
- Lemonade
How to use DreamBlooms/decider-0.8b-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull DreamBlooms/decider-0.8b-GGUF:Q8_0
Run and chat with the model
lemonade run user.decider-0.8b-GGUF-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use DreamBlooms/decider-0.8b-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf DreamBlooms/decider-0.8b-GGUF:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default DreamBlooms/decider-0.8b-GGUF:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use DreamBlooms/decider-0.8b-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf DreamBlooms/decider-0.8b-GGUF:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "DreamBlooms/decider-0.8b-GGUF:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
decider-0.8b: typed decisions with calibrated probabilities in one forward pass
The smallest decider: a language model that does not generate text. It reads
a state (a string or any JSON value) and a set of typed questions and returns a probability distribution for every question
from a single forward pass: Choice (2-255 options, optionally described), Score (2-10 described levels), Noul (probability of yes).
No decoding, no parsing, no output outside the options you defined. Same code, same wire format (POST /v1/systemone, TypeSafe
Jev's format), same training recipe as the 2B: one epoch of scripts/train.sh full from Qwen/Qwen3.5-0.8B-Base over the full
mixture (1.47M examples, 455M tokens, 4.5 h on one GH200).
Contents: The decider family ยท Usage ยท How it compares with the 2B ยท Limitations ยท Changelog
The decider family
All six repositories share one interface (decider.infer.Decider, POST /v1/systemone in TypeSafe's format) and one
readout: the letter logits at an answer slot, softmaxed over the options. Pick by size and input.
| model | base | weights | use it for | numbers |
|---|---|---|---|---|
| decider-2b v10 | Qwen3.5-2B-Base | 3.5 GB bf16 | the default: routing, classification, judgments, browser agents; 4 ms per request with CUDA graphs on one GPU | regression set 0.805 in-task / 0.755 held-out; live browser 93%; Bespoke suite 0.704 |
| decider-4b v1 | Qwen3.5-4B-Base | 8.4 GB bf16 | the middle point: knowledge and reasoning questions above the 2B in a dense 8.4 GB model; no RL stage | 0.834 / 0.788, above the 2B on 87 of 95 tasks; JevBench hard 0.541; Bespoke 0.757 |
| decider-35b-a3b v1 | Qwen3.5-35B-A3B-Base (3B active) | 65 GB bf16 | when accuracy is worth 3 to 4 times the cost per decision: knowledge and multi-step questions, long policies | 0.855 / 0.810, above the 2B on 93 of 95 tasks; JevBench hard 0.676; Bespoke 0.774; no RL stage |
| decider-35b-a3b-nvfp4 | the 35B in NVFP4 | 19.6 GB | the 35B on Blackwell through vLLM or TensorRT-LLM | 1.0 to 1.5 points under bf16 on the measured fixtures |
| decider-0.8b | Qwen3.5-0.8B-Base | 1.4 GB bf16 | the smallest: routing, yes/no and short-state lookups within 1 to 4 points of the 2B, 1.5x faster | 0.776 / 0.707 on the single-run protocol (2B: 0.809 / 0.739) |
| decider-2b-vision | Qwen3.5-2B vision-language, v5 text weights | 4.1 GB bf16 | decisions from an image plus a question; game frames | Visual7W 0.89; Breakout 41 from pixels |
Code, data registry, training scripts, the changelog and the per-version history: https://github.com/Mapika/decider.
Usage
# pip install git+https://github.com/Mapika/decider
from decider.infer import Decider
d = Decider("Mapika/decider-0.8b")
d.system_one(
{"ticket": "I was charged twice for order A-104. Please refund the duplicate."},
{"team": {"type": "choice", "instructions": "Which team should handle this?",
"criteria": {"billing": "Charges, invoices, refunds", "technical": "Bugs, outages", "other": None}},
"refund_requested": {"type": "noul", "instructions": "Does the customer ask for a refund?"},
"frustration": {"type": "score", "instructions": "How frustrated is the customer?", "criteria": ["calm", "frustrated", "very frustrated"]}})
decide_batch, schema (the cached question set), decider.serve and the TypeSafe request shape work as in the decider-2b
card; decider/ in this repository is the inference subset of the GitHub package.
How it compares with the 2B
Same 93 public tasks, same protocol, one temperature fitted on in-task data (it came out at 1.03 for both: the recipe calibrates at every size). "Held-out" means no example of that dataset was trained on.
| decider-0.8b | decider-2b (same single-run recipe) | |
|---|---|---|
| in-task accuracy / ECE, 69 tasks | 0.776 / 0.032 | 0.809 / 0.030 |
| held-out accuracy / ECE, 24 tasks | 0.707 / 0.096 | 0.739 / 0.086 |
| schema-first layout (the cacheable one), in-task / held-out accuracy | 0.770 / 0.699 | 0.790 / 0.707 |
| teacher-written custom questions, held-out domains: noul / choice / score | 0.94 / 0.95 / 0.81 | 0.98 / 0.97 / 0.83 |
| terse-bucket routing, held-out domains: generic / specific / catch-all | 0.87 / 0.93 / 0.84 | 0.91 / 0.94 / 0.92 |
| JSON state, one of 16 / 64 records named by path (64 with indices written in) | 0.58 / 0.52 (0.62) | 0.59 / 0.53 (0.63) |
| all 64 / 50 / 70 / 219 labels at once: HWU64, TREC-fine, DBpedia L2, L3 (held-out) | 0.81 / 0.63 / 0.66 / 0.83 | 0.85 / 0.76 / 0.72 / 0.86 |
| QuALITY, whole article (5-8k tokens) | 0.63 | 0.68 |
| isolated Score levels vs listwise (teacher-written, held-out domains) | 0.83 vs 0.82, fits sum to 1.02 | 0.84 vs 0.84 |
| hand-written battery: generic option right / catch-all right | 0.95 / 0.95 | 0.90 / 0.85 |
What the smaller model gives up is knowledge, not the decision format: the largest drops are TruthfulQA (0.41 vs 0.55), OpenBookQA (0.67 vs 0.81), HellaSwag (0.76 vs 0.88) and ARC (0.74 vs 0.86), and wide label sets that need fine distinctions (TREC-fine). Routing, classification, yes/no judgments and JSON lookups on short states are within one to four points of the 2B. It is a weaker player: Pong and Breakout stay at the scripted teacher's level, CliffWalking fails (it walks off the cliff), held-out Freeway scores 0. The regression set runs about 1.5x faster than on the 2B; bf16 weights are 1.5 GB.
Limitations
Those of decider-2b, more so: a small model without reasoning; English only; rules written into the question ("fill if empty, otherwise skip") are not followed reliably, so state the decision as a plain question with described options; knowledge-heavy multiple choice is close to the base model; calibration is measured on public datasets and teacher-labelled probes, not on your traffic. The teacher-written training data comes from Qwen3.5-27B and carries its biases.
Changelog
| version | what changed |
|---|---|
| v1 (these weights) | one epoch of scripts/train.sh full from Qwen3.5-0.8B-Base, the single-run form of the supervised recipe that produced decider-2b v8 to v9 |
Every decider release is listed in docs/CHANGELOG.md of the GitHub repository. Code and the training recipe: https://github.com/Mapika/decider.
GGUF
CPU inference is available through
dohnuts.cpp, a native C++ port on
llama.cpp. It runs the model on CPU,
reads the option-letter logits at each Answer: ( slot from the LM head, and
serves the same POST /v1/systemone wire format. No GPU or Python runtime is
needed.
| File | Quantization | Size |
|---|---|---|
decider-0.8b-q8_0.gguf |
Q8_0 | 775 MB |
decider.json |
temperature and flags | 341 B |
This conversion is verifiable and structural; it has not been benchmarked against the PyTorch reference. Q8_0 is the conservative default.
git clone https://github.com/DreamBlooms/dohnuts.cpp
cd dohnuts.cpp
git submodule update --init --depth 1
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON
cmake --build build -j --target dohnuts-cli
build/dohnuts-cli --server --port 8080 --model decider-0.8b-q8_0.gguf --metadata decider.json
Ask one state several questions (same request shape as the PyTorch server):
curl http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' \
-d '{"state":{"ticket":{"messages":[{"from":"customer","text":"I was charged twice. Please refund the duplicate."}]}},
"questions":{"department":{"type":"choice","instructions":"Which team should handle this?",
"criteria":{"billing":null,"technical support":null,"sales":null}},
"refund_requested":{"type":"noul","instructions":"Is a refund requested?"}}}'
The answer keeps the core fields (type, choice, probabilities, noul,
confidence) and adds the model's native confidence and certainty under
native, along with legend / level_fit / fit_mass for isolated Score
levels.
Rebuild the file from the upstream weights with scripts/build_decider_gguf.sh
(convert with --no-mtp; the base config declares an MTP layer but ships no MTP
weights). No retraining or merging is involved.
- Downloads last month
- 59
8-bit