SenseNova Loop 9B โ€” GGUF

Text-only GGUF quants of sensenova/SenseNova-Loop-9B, the release that supersedes SenseNova-Loop-9B-Preview. Weights, training and the model itself are SenseNova / OpenSenseNova's work. This repository only converts and quantizes.

Stock llama.cpp cannot load these. The architecture is sensenova_loop, implemented in the runtime fork https://github.com/GuideboardLabs/llama.cpp-sensenova-loop (branch sensenova-loop, two commits, patches included). Vision and the MTP head are not part of the export; this is a text decoder.

Files

file quant size
SenseNova-Loop-9B-Q8_0.gguf Q8_0 9,540,092,256 B
SenseNova-Loop-9B-Q4_0.gguf Q4_0 5,325,947,232 B
SenseNova-Loop-9B-Q4_K_M.gguf Q4_K_M 5,628,723,552 B
SenseNova-Loop-9B-Q3_K_M.gguf Q3_K_M 4,642,406,752 B

Q8_0 does not fit an 8 GB card; it is here for the fidelity check and for machines with more VRAM. Every size above is measured from the file, not estimated.

Convolution kernels, norms and the mHC router weights stay FP32; everything else is quantized. The router weights have to be FP32: ggml_mhc_route asserts GGML_TYPE_F32, and an f16 outtype that leaves them F16 fails at graph reserve.

The looped block is stored once. The runtime aliases the second visit of those layers onto the first, which is numerically identical and 36 percent smaller than storing the block twice.

What this release changed

Same architecture as the Preview, verified tensor for tensor: 32 stored decoder blocks, 24 of them executed twice (56 executions per forward pass), two mHC residual streams with a router on every sublayer inside the repeated range, hybrid linear/full attention on a 3:1 pattern. Exactly three differences, all in config.json:

key Preview Release
max_position_embeddings 262144 524288
rope_parameters.rope_type default yarn, factor 2.0, original 262144
mtp_num_hidden_layers 1 0 (all 16 mtp.* tensors gone)

All four YaRN keys are carried in the GGUF (rope.scaling.type=yarn, factor=2.0, original_context_length=262144, context_length=524288) and are exercised by the runtime.

The release checkpoint also carries one extra tensor, model.language_model.loop_gate.logits (F32 [4096]), which the shipped modeling_sensenova_loop.py โ€” md5-identical to the Preview's copy โ€” never references, while loop_settings.eval_count is still pinned at 2. It is an unused parameter, most likely the seed of the "Dynamic Loop" work the model card describes as future, and the converter drops it.

Run

GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1 ./build/bin/llama-server \
    -m SenseNova-Loop-9B-Q4_0.gguf -ngl 99 -fa on -c 16384 --port 8080

The environment variable matters on small-BAR cards: without it the recurrent state buffers land in host-visible memory. It only takes effect once the model fits in VRAM.

Sampling from the shipped generation_config.json (temperature 1.0, top_p 0.95) with thinking left on is how we run it.

Measured

RX 5700 XT (NAVI10), Vulkan, GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1, 6 threads:

quant decode prefill
Q4_0 31.3 t/s 85.8 t/s (22-token prompt)

Accuracy

Two checks against the reference transformers implementation (modeling_sensenova_loop.py, bf16, CPU):

  • Tokenization of the development prompt is identical ([760, 6511, 314, 9338, 369] for The capital of France is).
  • Greedy decoding at Q8_0 matches token for token over five tokens ( Paris, ., \n, Paris, is), and the log-probability ordering agrees at every step; the top-1 logprob differs by 0.08 nats at the widest point. That is the check that exercises YaRN: if the runtime ignored rope.scaling.type=yarn, the first tokens would already diverge.
  • At Q4_0 the same prompt diverges at step 3, where the reference has \n at -1.88 and The at -2.07, a 0.19-nat near-tie. Q4_0 puts The first. That is 4-bit noise on a near-tie, not a rope mismatch, which is why the Q8_0 row carries the fidelity claim.

Context was exercised to 16K tokens here, and to 512 positions in the fidelity check; the release advertises 524288 with YaRN, which we have not validated.

Downloads last month
-
GGUF
Model size
9B params
Architecture
sensenova_loop
Hardware compatibility
Log In to add your hardware

3-bit

4-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for guideboardlabs/SenseNova-Loop-9B-GGUF

Quantized
(15)
this model