# Neutrino-0.6B — MLX pack (Apple silicon)

> [!IMPORTANT]
> **Neutrino-0.6B is the speculative-decoding draft for Neutrino-8B.** It
> is a base model: not an assistant, no chat, instruction or tool use. For a
> small conversational model use Neutrino-0.6B-Chat.

Run the [Neutrino-0.6B](../README.md) container natively from Python on
Apple silicon with MLX and compiled Metal kernels.

Requirements: a Mac with Apple silicon, macOS 14 or later, Python 3.10 to
3.14 (standard CPython build). `requirements.txt` installs `mlx` 0.32.3,
the version the compiled kernels are built for.

## Install & run

```bash
pip install -r requirements.txt        # mlx, numpy, transformers
python -m fermion_mlx --model ../neutrino-0.6b_v4.bin --mode gen \
    --tokenizer . --prompt "2, 3, 5, 7, 11,"
```

Python API:

```python
import fermion_mlx
from transformers import AutoTokenizer

model = fermion_mlx.load("../neutrino-0.6b_v4.bin")  # memory-maps the container
tok = AutoTokenizer.from_pretrained(".")             # tokenizer files ship here
text, stats = fermion_mlx.generate(model, tok, "2, 3, 5, 7, 11,",
                                   max_tokens=64)
print(text, stats)                                   # {'tok_s': ..., ...}
```

This is a base model: prompt it as a completion engine.

## What runs on the GPU

The model runs through compiled Metal kernels, which ship in `fermion_mlx/`
as Metal libraries with a small compiled extension that loads them; no
Metal compiler is needed on your machine.

## Speed

| path | machine | 512-token greedy decode |
|---|---|---|
| this MLX pack | Apple M5 MacBook, 16 GB | **201 tok/s**, peak MLX memory 0.53 GiB |
| `fermion-run` CPU binary (`../bin/`) | same M5, 9 threads | 225-236 tok/s |

## Drafting for Neutrino-8B

The same package runs `--mode spec`, where this model drafts and the
Neutrino-8B pack verifies; the output is token-identical to plain 8B
greedy decoding. See the Neutrino-8B MLX pack README for the command.

## Files

- `fermion_mlx/`: the runtime package, including the compiled kernels.
- `tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt`,
  `generation_config.json`: Qwen3-0.6B tokenizer.
- `requirements.txt`: `mlx==0.32.3`, numpy, transformers.
- `MANIFEST.json`: size and sha256 of every file in this folder.

Weights are not duplicated in this folder; the container is
`../neutrino-0.6b_v4.bin` (sha256
`ba0e9258d866d5eb309352dd7c1b810c402eac289fb98e871c63b38a3bcd8168`).

## License

Apache-2.0 for the Python code and the weights (derivative of
Qwen/Qwen3-0.6B; see the parent `LICENSE`). The compiled kernels
(`fermion_mlx/*.metallib`, `fermion_mlx/_native*.so`) are free to use with
these weights; no redistribution outside this repo; no reverse engineering.