Instructions to use guideboardlabs/SenseNova-Loop-9B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use guideboardlabs/SenseNova-Loop-9B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M
Use Docker
docker model run hf.co/guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use guideboardlabs/SenseNova-Loop-9B-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "guideboardlabs/SenseNova-Loop-9B-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "guideboardlabs/SenseNova-Loop-9B-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M
- Ollama
How to use guideboardlabs/SenseNova-Loop-9B-GGUF with Ollama:
ollama run hf.co/guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use guideboardlabs/SenseNova-Loop-9B-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use guideboardlabs/SenseNova-Loop-9B-GGUF with Docker Model Runner:
docker model run hf.co/guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M
- Lemonade
How to use guideboardlabs/SenseNova-Loop-9B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.SenseNova-Loop-9B-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use guideboardlabs/SenseNova-Loop-9B-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use guideboardlabs/SenseNova-Loop-9B-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "guideboardlabs/SenseNova-Loop-9B-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
SenseNova Loop 9B โ GGUF
Text-only GGUF quants of sensenova/SenseNova-Loop-9B,
the release that supersedes SenseNova-Loop-9B-Preview. Weights, training and the model itself are
SenseNova / OpenSenseNova's work. This repository only converts and quantizes.
Stock llama.cpp cannot load these. The architecture is sensenova_loop, implemented in the runtime
fork https://github.com/GuideboardLabs/llama.cpp-sensenova-loop (branch sensenova-loop, two commits,
patches included). Vision and the MTP head are not part of the export; this is a text decoder.
Files
| file | quant | size |
|---|---|---|
SenseNova-Loop-9B-Q8_0.gguf |
Q8_0 | 9,540,092,256 B |
SenseNova-Loop-9B-Q4_0.gguf |
Q4_0 | 5,325,947,232 B |
SenseNova-Loop-9B-Q4_K_M.gguf |
Q4_K_M | 5,628,723,552 B |
SenseNova-Loop-9B-Q3_K_M.gguf |
Q3_K_M | 4,642,406,752 B |
Q8_0 does not fit an 8 GB card; it is here for the fidelity check and for machines with more VRAM. Every size above is measured from the file, not estimated.
Convolution kernels, norms and the mHC router weights stay FP32; everything else is quantized. The
router weights have to be FP32: ggml_mhc_route asserts GGML_TYPE_F32, and an f16 outtype that leaves
them F16 fails at graph reserve.
The looped block is stored once. The runtime aliases the second visit of those layers onto the first, which is numerically identical and 36 percent smaller than storing the block twice.
What this release changed
Same architecture as the Preview, verified tensor for tensor: 32 stored decoder blocks, 24 of them
executed twice (56 executions per forward pass), two mHC residual streams with a router on every
sublayer inside the repeated range, hybrid linear/full attention on a 3:1 pattern. Exactly three
differences, all in config.json:
| key | Preview | Release |
|---|---|---|
max_position_embeddings |
262144 | 524288 |
rope_parameters.rope_type |
default |
yarn, factor 2.0, original 262144 |
mtp_num_hidden_layers |
1 | 0 (all 16 mtp.* tensors gone) |
All four YaRN keys are carried in the GGUF (rope.scaling.type=yarn, factor=2.0,
original_context_length=262144, context_length=524288) and are exercised by the runtime.
The release checkpoint also carries one extra tensor, model.language_model.loop_gate.logits
(F32 [4096]), which the shipped modeling_sensenova_loop.py โ md5-identical to the Preview's copy โ
never references, while loop_settings.eval_count is still pinned at 2. It is an unused parameter,
most likely the seed of the "Dynamic Loop" work the model card describes as future, and the converter
drops it.
Run
GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1 ./build/bin/llama-server \
-m SenseNova-Loop-9B-Q4_0.gguf -ngl 99 -fa on -c 16384 --port 8080
The environment variable matters on small-BAR cards: without it the recurrent state buffers land in host-visible memory. It only takes effect once the model fits in VRAM.
Sampling from the shipped generation_config.json (temperature 1.0, top_p 0.95) with thinking left
on is how we run it.
Measured
RX 5700 XT (NAVI10), Vulkan, GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1, 6 threads:
| quant | decode | prefill |
|---|---|---|
| Q4_0 | 31.3 t/s | 85.8 t/s (22-token prompt) |
Accuracy
Two checks against the reference transformers implementation (modeling_sensenova_loop.py, bf16,
CPU):
- Tokenization of the development prompt is identical (
[760, 6511, 314, 9338, 369]forThe capital of France is). - Greedy decoding at Q8_0 matches token for token over five tokens (
Paris,.,\n,Paris,is), and the log-probability ordering agrees at every step; the top-1 logprob differs by 0.08 nats at the widest point. That is the check that exercises YaRN: if the runtime ignoredrope.scaling.type=yarn, the first tokens would already diverge. - At Q4_0 the same prompt diverges at step 3, where the reference has
\nat -1.88 andTheat -2.07, a 0.19-nat near-tie. Q4_0 putsThefirst. That is 4-bit noise on a near-tie, not a rope mismatch, which is why the Q8_0 row carries the fidelity claim.
Context was exercised to 16K tokens here, and to 512 positions in the fidelity check; the release advertises 524288 with YaRN, which we have not validated.
- Downloads last month
- -
3-bit
4-bit
8-bit
Model tree for guideboardlabs/SenseNova-Loop-9B-GGUF
Base model
sensenova/SenseNova-Loop-9B