Text Generation
Transformers
GGUF
trtc_v4
quantization
small-model
speculative-decoding
conversational
Instructions to use FermionResearch/Neutrino-0.6B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FermionResearch/Neutrino-0.6B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="FermionResearch/Neutrino-0.6B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("FermionResearch/Neutrino-0.6B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use FermionResearch/Neutrino-0.6B with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf FermionResearch/Neutrino-0.6B # Run inference directly in the terminal: llama cli -hf FermionResearch/Neutrino-0.6B
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf FermionResearch/Neutrino-0.6B # Run inference directly in the terminal: llama cli -hf FermionResearch/Neutrino-0.6B
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf FermionResearch/Neutrino-0.6B # Run inference directly in the terminal: ./build/bin/llama cli -hf FermionResearch/Neutrino-0.6B
Use Docker
docker model run hf.co/FermionResearch/Neutrino-0.6B
- Llama new
- LM Studio
- Jan
- vLLM
How to use FermionResearch/Neutrino-0.6B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FermionResearch/Neutrino-0.6B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FermionResearch/Neutrino-0.6B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/FermionResearch/Neutrino-0.6B
- SGLang
How to use FermionResearch/Neutrino-0.6B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "FermionResearch/Neutrino-0.6B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FermionResearch/Neutrino-0.6B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "FermionResearch/Neutrino-0.6B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FermionResearch/Neutrino-0.6B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use FermionResearch/Neutrino-0.6B with Ollama:
ollama run hf.co/FermionResearch/Neutrino-0.6B
- Unsloth Desktop
- Pi
How to use FermionResearch/Neutrino-0.6B with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FermionResearch/Neutrino-0.6B
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "FermionResearch/Neutrino-0.6B" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use FermionResearch/Neutrino-0.6B with Docker Model Runner:
docker model run hf.co/FermionResearch/Neutrino-0.6B
- Lemonade
How to use FermionResearch/Neutrino-0.6B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull FermionResearch/Neutrino-0.6B
Run and chat with the model
lemonade run user.Neutrino-0.6B-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use FermionResearch/Neutrino-0.6B with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FermionResearch/Neutrino-0.6B
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default FermionResearch/Neutrino-0.6B
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use FermionResearch/Neutrino-0.6B with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FermionResearch/Neutrino-0.6B
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "FermionResearch/Neutrino-0.6B" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download mlx/MANIFEST.json from FermionResearch/Neutrino-0.6B: direct link, hf CLI and curl.
- Browser
- Download file 4.25 kB
-
https://huggingface.co/FermionResearch/Neutrino-0.6B/resolve/main/mlx/MANIFEST.json
- Command line
-
hf download hf://FermionResearch/Neutrino-0.6B/mlx/MANIFEST.json
-
curl -L -o MANIFEST.json https://huggingface.co/FermionResearch/Neutrino-0.6B/resolve/main/mlx/MANIFEST.json
4.25 kB
| { | |
| "note": "Size and sha256 of every file in this folder. The container is ../neutrino-0.6b_v4.bin.", | |
| "weights_container": { | |
| "file": "neutrino-0.6b_v4.bin", | |
| "bytes": 327719836, | |
| "sha256": "ba0e9258d866d5eb309352dd7c1b810c402eac289fb98e871c63b38a3bcd8168" | |
| }, | |
| "files": { | |
| "README.md": { | |
| "bytes": 3086, | |
| "sha256": "94191e45719e7177cf6ef45f6960b28aeffcd566b88ee19ea1a5c931606dbeb9" | |
| }, | |
| "fermion_mlx/MANIFEST.json": { | |
| "bytes": 1873, | |
| "sha256": "1c32368a2e464b044af7933b409743832598ca7b3846949ac8548e266189a26f" | |
| }, | |
| "fermion_mlx/__init__.py": { | |
| "bytes": 4226, | |
| "sha256": "07cc9e08a642d4641452df7f0f89e83ee3e30e0224ba49b132a6d3053d597f8b" | |
| }, | |
| "fermion_mlx/__main__.py": { | |
| "bytes": 10555, | |
| "sha256": "cafb07360169ab6ec88363263bbad5a02a4bdfb0734dc1b1f96ac9dbbc06c70a" | |
| }, | |
| "fermion_mlx/_dispatch.py": { | |
| "bytes": 1992, | |
| "sha256": "3510bc5846167742eb6f4f271899fa751b65ce6dee0a5300d88c4005e085fb81" | |
| }, | |
| "fermion_mlx/_native.cpython-310-darwin.so": { | |
| "bytes": 169440, | |
| "sha256": "ae70f8733d29b302de6b7cd90ff8825857e047353513e8a1ff0865d00c9f8161" | |
| }, | |
| "fermion_mlx/_native.cpython-311-darwin.so": { | |
| "bytes": 169360, | |
| "sha256": "d05d34d0ebc91cf4ac3a9e9fa80cedc1f5b70b01278c8e1f4e6b41ed51f8d91b" | |
| }, | |
| "fermion_mlx/_native.cpython-312-darwin.so": { | |
| "bytes": 169184, | |
| "sha256": "6061c835fd772640f25cd33c16485234547634238f5b5e41b033e3331d19f653" | |
| }, | |
| "fermion_mlx/_native.cpython-313-darwin.so": { | |
| "bytes": 169328, | |
| "sha256": "41003ea207106d3c4f598d9c6046ca641a2a9ceae9800c1a52503074ed3fdd14" | |
| }, | |
| "fermion_mlx/_native.cpython-314-darwin.so": { | |
| "bytes": 169328, | |
| "sha256": "737d063c11e85448c7b8a10771abf92aaa18a141eb0769335417e02f31d06653" | |
| }, | |
| "fermion_mlx/arch3.py": { | |
| "bytes": 1244, | |
| "sha256": "e4157c5e91c42aaa0837090d91ca9f06a148cbfc52e47dd9c5bdb2647d0e551a" | |
| }, | |
| "fermion_mlx/container.py": { | |
| "bytes": 4494, | |
| "sha256": "7527385c12ac7d3b5a7d991143af705b99d27c766c414009f5acdc612c3dffc7" | |
| }, | |
| "fermion_mlx/drafter.py": { | |
| "bytes": 4279, | |
| "sha256": "a3bce2867d9d81805def7c48070436fe53bc81f92bcd4eb854cf7c5275d7fb71" | |
| }, | |
| "fermion_mlx/flash_sdpa.py": { | |
| "bytes": 985, | |
| "sha256": "1de4dbc5ad4ed2113bec9d324708f73a899920566e47ed398528ef1577b281c6" | |
| }, | |
| "fermion_mlx/gates.py": { | |
| "bytes": 6574, | |
| "sha256": "74be2a0eebbecf3c965b4cd81b8544fc107fab638ec33c7a40550a9251af51e7" | |
| }, | |
| "fermion_mlx/heads.py": { | |
| "bytes": 4070, | |
| "sha256": "7c43d0a95923455afea501629291bc3571c8631fe3f2691fb058222f313c24a1" | |
| }, | |
| "fermion_mlx/kernels_m31.metallib": { | |
| "bytes": 2592052, | |
| "sha256": "bdf2d5a6fa35f975a174a00821cf09c81f5393a48818471b6b7d3411b4c9da32" | |
| }, | |
| "fermion_mlx/kernels_m32.metallib": { | |
| "bytes": 2592676, | |
| "sha256": "4604d08365d5dfc7d7fa732fc28a7cb0dbcc10bbcd29b18f196cc0462a2c7c7d" | |
| }, | |
| "fermion_mlx/kernels_m40.metallib": { | |
| "bytes": 2592676, | |
| "sha256": "86053815d1d7d6fd97b0e7165c00ee0661092724258fb243c9479f2375efbaea" | |
| }, | |
| "fermion_mlx/mbatch.py": { | |
| "bytes": 1735, | |
| "sha256": "bda7a55c9fe9d1a92b47df3a57fadebe1c779184b53150b1d3e789c20cb8f751" | |
| }, | |
| "fermion_mlx/model.py": { | |
| "bytes": 17798, | |
| "sha256": "f5c932a1776d53531c285bee9fd196a3dc890b5d53706d28422883908546a9d3" | |
| }, | |
| "fermion_mlx/spec.py": { | |
| "bytes": 11380, | |
| "sha256": "74b44d232566325325bf2222c528f39adc6d71e85cb40fc16e55b321ba0b8577" | |
| }, | |
| "fermion_mlx/trit_kernels.py": { | |
| "bytes": 1721, | |
| "sha256": "8fe446c9267bb40d3397ab330248ba2916dbfd3523e971046f2bc72716a1f1d3" | |
| }, | |
| "generation_config.json": { | |
| "bytes": 181, | |
| "sha256": "6ca52b0bcb818c9e52db8ea18413494110c5f2def581b769f621dda951a7863b" | |
| }, | |
| "merges.txt": { | |
| "bytes": 1671853, | |
| "sha256": "8831e4f1a044471340f7c0a83d7bd71306a5b867e95fd870f74d0c5308a904d5" | |
| }, | |
| "requirements.txt": { | |
| "bytes": 156, | |
| "sha256": "130c52eb03b687de1795104aff457f69f503137ad0474405d61c7ceb665ac4ec" | |
| }, | |
| "tokenizer.json": { | |
| "bytes": 11422654, | |
| "sha256": "aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4" | |
| }, | |
| "tokenizer_config.json": { | |
| "bytes": 9980, | |
| "sha256": "9f1e48d00384f25decc7f30dc7ec3a76fb3a98b37f5297b8494a134cef786102" | |
| }, | |
| "vocab.json": { | |
| "bytes": 2776833, | |
| "sha256": "ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910" | |
| } | |
| } | |
| } | |