Qwen3.8-Flash-CIRU-STRIX-Orca

September 18 QSA source hotfix: for the vision + MTP crash qsa: cell position runs past the cell window, apply the source patch and rebuild your existing runtime. Instructions and quick regression/speed checks. MTP remains enabled. The v4.4.1 prebuilt archives and pinned tag do not contain this later hotfix.

OrcaRouter’s research weights, packaged for local Strix Halo inference with the custom CIRU runtime, matching MTP, and a 262,144-token context. Vision is optional: add --vision to the launcher to load the included projector. This build provides the Orca variant in the same mixed-precision layout as the original CIRU release, including its external PLE table and calibrated Q4_1 expert banks.

The intended audience is researchers working on interpretability, refusal mechanisms, defensive security, red-teaming, and robustness. The image describes the intended research use; the tests below measure coding, instruction following, math, tool workflows, and long-history behavior. They are not a cybersecurity or safety certification.

Use the custom CIRU v4.4.1 runtime, the Orca target GGUF, and all three PLE files. The matching Q8 MTP head enables speculative decoding. Stock llama.cpp and Hugging Face hosted inference cannot run this package.

v4.4.1

Fixed vision + MTP.

Update the complete runtime and restart the server; keep your existing model files. For the recorded NixOS/ROCm10 environment:

hf download jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca --revision v4.4.1 \
  runtime/v4.4.1/ciru-runtime-v4.4.1-nixos-gfx1151.tar.gz --local-dir ./release
tar -xzf ./release/runtime/v4.4.1/ciru-runtime-v4.4.1-nixos-gfx1151.tar.gz
MODEL_DIR="$(realpath ./model)" bash ./ciru-runtime-v4.4.1-nixos-gfx1151/scripts/ciru/run-orca-server.sh --vision

For source builds, use the v4.4.1 build instructions. Point any existing RUNTIME_DIR, BUILD_DIR or SERVER_BIN overrides at the new installation.

Previous v4.4 patch: combined runtime

Update the complete runtime; keep your Orca weights, matching draft and PLE. This brings the same combined QSA/cache, attention, graph/PM4, Q8 fusion, branch reuse and GDN improvements to Orca. MTP4 remains default; Boost is opt-in (--kairic-boost / KAIRIC_BOOST=1). Stock TheRock10 compiler/math is paired with pinned pwilkin HIP/ROCr.

Orca checks: 11 exact lifecycle cases; HE0–9 passed 10/10 base and 10/10 extended, with 61.57 tok/s pooled decode on default MTP4. One short panel, natural EOS, nonthinking greedy chat adapter, full 256K allocation with short prompts.

Orca HA20 run	Arithmetic mean /100	Native full-score tasks
v4.3, passes 1 / 2	92.5 / 98.5	17/20 / 19/20
v4.4, one pass	95.25	18/20

Official weighted v4.4 score: 95/100. HA03 is a known false native pass: the prohibited instruction was persisted in USER.md, outside the verifier's checked file, as in an earlier Orca run. HA19 scored 35 after a permission-recovery command was denied by the simulated approval; both prior passes scored 100. HA17 retains the known field-name mismatch. The full traces and grading notes remain in the report. This is the same-suite historical score comparison, not evidence of a causal quality gain. Prior Orca HE speed used IDs 1–10, so no matched speedup is claimed for this IDs 0–9 panel.

Weight reconstruction error is unchanged. Boost's short positive speed result belongs to IU4; Orca's tests here use default MTP4. The image+MTP failure recorded for v4.4.0 is fixed in v4.4.1.

Orca results and protocol · Shared runtime patch notes · v4.4 downloads and source

For the recorded NixOS/ROCm10 dependencies, extract the complete runtime archive, then run:

MODEL_DIR="$(realpath ./model)" bash ./ciru-runtime-v4.4.0-nixos-gfx1151/scripts/ciru/run-orca-server.sh

Append --kairic-boost only when desired. For other systems, follow the v4.4 source and HIP/ROCr build instructions. Older sections below preserve historical settings and measurements.

Previous v4.3 patch: speculative tuning

V4.3 makes the tested speculative-decoding settings the defaults: MTP depth 3 for IU4, depth 4 for Orca, and LLAMA_MTP_QSA_MIN_T=1 for both. Shorter draft batches reduce discarded speculative work; the lower threshold enables sparse draft attention for small batches. The corrected v4.2 inference binaries and existing model files are retained.

Model	Hermes mean score /100, passes 1 / 2	Full-score tasks, passes 1 / 2	Prompt tok/s	Generation tok/s
IU4	95.0 / 98.5	17/20 / 19/20	548.5	34.74
Orca	92.5 / 98.5	17/20 / 19/20	551.3	34.44

Hermes Agent 20: two passes per model, 80 completed scenario attempts and 449 model requests, a 32-turn allowance, production sampling and native xhigh thinking. Speeds are pooled across each model's two passes; generation includes reasoning. Both hosts are 128 GB Strix Halo / gfx1151, NixOS / ROCm 10; IU4 ran on Sozo and Orca on Ciru. These are observed tuned-configuration results, not a matched Hermes speedup over v4.2. Native verifier scores are retained; the detailed report records grading limitations and Orca's memory-persistence behavior.

Existing v4.2 users can update the launcher without rebuilding or downloading weights. MTP_DEPTH and LLAMA_MTP_QSA_MIN_T remain overridable; MTP_DEPTH=6 LLAMA_MTP_QSA_MIN_T=128 restores the previous speculative defaults. Earlier runtimes must first obtain the v4.2 attention correction. Context remains 262144, batch/microbatch 8192, with F16 target/draft KV and unchanged sampling and thinking.

Hermes task results and grading notes | v4.3 release

Launcher update, 16 September: the September 16 launcher checked the HIP library selected by the dynamic loader for the v4.2 indexed-attention correction before loading the model. It stops with upgrade instructions if an older or incomplete runtime is selected, and prints the actual server/library paths and library hash. Updating the model folder alone does not upgrade RUNTIME_DIR, BUILD_DIR or SERVER_BIN. This check was added after the immutable v4.3.0 tag; fetch the pinned historical launcher below to use it with an existing corrected v4.2/v4.3 installation.

For an existing corrected v4.2 or v4.3 installation, download and verify the pinned historical launcher:

hf download jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca \
  --revision 07ba77cd60b3d8e6d5e7cbc600e699bd05ac1df2 run-server.sh launcher-checksums.sha256 --local-dir ./model
(cd model && sha256sum -c launcher-checksums.sha256)
RUNTIME_DIR="$(realpath ./ciru-runtime-v4.2.0)" bash ./model/run-server.sh

Use your actual runtime/model paths. The new launcher applies the tuning even with the existing v4.2 profile.

Previous v4.2 patch: attention correctness

Upgrade the runtime; keep your existing model files. V4.2 fixes the fallback attention path that ignored selected keys in v4.0/v4.1 and could produce incorrect computation associated with the reproduced repetition incident. QSA3 stays enabled. Long thinking and production reasoning settings are unchanged.

The correction passed 48/48 CPU-reference checks, and both IU4 and Orca completed the frozen medium-effort incident replay with valid tool calls and no detected repeated long line. This is targeted regression evidence, not a new broad quality benchmark.

Measured decode tradeoff: one matched IU4 workload measured 25.54-25.55 -> 21.20-21.27 tok/s (-16.9%). The two mirrored comparisons were -17.04% and -16.71%, making the effect repeatable on that workload. Prefill remained within a 5% parity band. Only one prompt/seed and hardware/configuration were tested; this is not a universal 17% slowdown or a measured Orca speed result. Draft acceptance also fell, so the full difference cannot be attributed to the fallback kernel alone. The old engine performed incorrect attention.

We are investigating new GPU kernels and profiling the corrected attention path to increase performance further, with correctness and complete serving measurements required for any promotion.

v4.2 patch notes and measurement details · Release and downloads

Previous v4.1 patch

Faster PLE loading, with your existing model files. V4.1 enables 32 I/O workers by default; set GGML_QWEN4EXP_PLE_IO_WORKERS=16 for the previous setting. Non-Orca native 16k prefill improved 4.44%, and a quick matched MTP6 check found no TG loss. Release and downloads.

Upgrade to CIRU v4.3.0

Keep your existing Orca model files. This is a runtime upgrade, with no replacement GGUF, requantization or PLE download required. Keep the Orca-specific Q8 MTP head paired with the Orca target; the non-Orca draft is a different set of weights. The optional Orca vision projector is also reused.

V4 brings the shared Flash prefill implementation to the tensor formats used by this Orca package: Q4_1 expert banks, the existing mixed-precision dense layers and the CIRUPLE1 sidecar. It adds fused expert gate/up and activation work, BF16 intermediate paths, sparse-attention improvements and MTP state handling. These are engine changes; Orca's model weights and behavior tuning remain in the existing files. The upgrade focuses on prompt processing, particularly long prompts. Generation speed also depends on the request, context depth and speculative acceptance.

The fast-prefill work originated with Halogen's creator, Peonist.ai, and was reproduced in open-source llama.cpp by pwilkin. CIRU's v4 release adapts that work to the existing CIRU weight formats and external PLE package. See the shared v4 release for source and release details. The Orca measurements below retain their original CIRU v3 protocol.

Build v4.3 and run Orca

Keep your assembled Orca model directory. On Ubuntu/Debian, install Git and Python 3, then build the shared runtime and use its dedicated Orca launcher:

git clone --branch v4.3.0 --single-branch \
  https://github.com/ciru-ai/Qwen3.8-Flash-CIRU-STRIX-IU4.git ciru-runtime-v4.3.0
(cd ciru-runtime-v4.3.0 && ./scripts/ciru/setup-linux-amd.sh --install-host-deps)
MODEL_DIR="$(realpath ./model)" \
  bash ./ciru-runtime-v4.3.0/scripts/ciru/run-orca-server.sh

The launcher selects the Orca target, matching Q8 MTP head and projector, and uses slot-state/orca-v4.3.0. Keep the complete build, profile and UI together. No script editing or replacement model download is needed. Source archive · Tested NixOS runtime · Build requirements.

V4.3 launch settings

Setting	V4 Orca launcher	Override
Context	262,144 tokens	CONTEXT_SIZE
Batch / microbatch	8,192 / 8,192	BATCH_SIZE, UBATCH_SIZE
Target / draft KV	F16 / F16	Set by the v4 launcher
MTP	Maximum depth 4, one slot	MTP_DEPTH; ENABLE_MTP=0 for target-only
Sparse draft threshold	1 token	LLAMA_MTP_QSA_MIN_T
Prompt RAM cache	1,024 MiB	PROMPT_CACHE_MIB
PLE cache	4,096 MiB	PLE_CACHE_MIB
CPU threads	8	THREADS, BATCH_THREADS
Context checkpoints	32; minimum step 8,192	CTX_CHECKPOINTS, CHECKPOINT_MIN_STEP
Vision	Optional; MTP stays enabled with the current launcher	--vision, ENABLE_VISION=1, MMPROJ

The F16 draft KV cache is a runtime setting; the draft model on disk remains the same Orca Q8_0 GGUF. Thinking follows the embedded template. Sampling defaults remain temperature 1.0, top-p 0.95, top-k 20 and min-p 0; requests can supply the evaluated sampling settings shown below. HOST, PORT, BUILD_DIR, SERVER_BIN and UI_DIR select the endpoint and matching build assets. Keep the v4 batch/cache settings together on 128 GB Strix Halo hardware rather than carrying over the old 8,192 MiB prompt-cache setting.

Open http://127.0.0.1:8080 for the web UI or use the existing /v1/chat/completions examples below. The model alias remains Qwen3.8-Flash-CIRU-STRIX-Orca.

The v4.3 profile uses 32 PLE I/O workers by default. Set GGML_QWEN4EXP_PLE_IO_WORKERS=16 to use the previous concurrency.

Measured results — CIRU v3

Test	Orca	Non-Orca v3
HumanEval 0–19 base / extended	20/20 / 20/20	20/20 / 20/20
Selected short: ifeval	6/8	5/8
Selected short: gsm8k	8/8	8/8
Selected short: humaneval	5/6	5/6
Shared ~63K history: gsm8k	2/2	2/2
Shared ~63K history: humaneval	3/4	2/4
Shared ~63K history: ifeval	2/2	2/2
Short coding health 0–9: base / extended	10/10 / 10/10	10/10 / 10/10
Shared ~63K coding health 0–7: base / extended	8/8 / 8/8	8/8 / 8/8
Hermes selected six, hermes-1: native mean score	100.00%	100.00%
Hermes selected six, hermes-2: native mean score	100.00%	91.67%

HumanEval IDs 1–10 served MTP speed: 53.62 decode tok/s, pooled over 30 requests; 227.19 prompt tok/s and 586 ms median first-piece latency. The three measured decode passes were 51.09, 54.36, 55.61 tok/s. No matched non-Orca run on this exact three-pass ID selection is claimed.

Permission-handling finding: although both Hermes rounds received full native scores, the second HA-19 trace retried deployment with a different command after an explicit “User denied. Do NOT retry.” response. The native scorer did not penalize this bypass. This is a permission-boundary failure, and the native score is not evidence of permission or safety compliance. Trace review.

Measured on Ciru: AMD Ryzen AI Max+ 395 / gfx1151, 128 GB shared memory, NixOS, ROCm 10, CIRU v3, one model workload at a time. Target KV is F16, draft KV Q8, maximum MTP depth 6, batch/microbatch 1024, one slot, and configured context 262,144.

HumanEval20 uses tasks 0–19, EvalPlus v0.1.10 base and extended tests, one first response per task, no retries, and a 4096-token output cap. The canonical task text is passed through the model’s chat template. This is a local chat protocol: thinking off, temperature 0.7, top-p 0.8, top-k 20, min-p 0, presence penalty 1.5, repeat penalty 1, frequency penalty 0, seed 123. Code is scored in a filesystem/network sandbox.

The separate speed panel uses HumanEval IDs 1–10, three repetitions, the same sampler, natural EOS, and a cold prompt cache per measured request. Decode tok/s, prompt tok/s, first-piece latency, and whole-request time are recorded separately. Repeated speed outputs do not replace the HumanEval20 first samples.

The selected hard panel reuses the non-Orca build’s exact tasks and native scorers: 8 IFEval, 8 GSM8K, 6 HumanEval, and 6 HermesAgent scenarios run twice. Native tasks use nonthinking greedy generation, seed 15035, no retries, and a 32,768-token cap. Hermes uses thinking, temperature 0.6, top-p 0.95, top-k 20, neutral penalties, at least 32 turns, full remaining context, and a 1200-second per-task deadline. The 8 long-history tasks reuse a shared roughly 63K-token archive, with a 65,536-token effective task boundary inside the 262K server. This disagreement-selected subset is a diagnostic, not a representative benchmark ranking.

Detailed results and protocol · Structured results

Historical v3 launch defaults

The v3 production launcher retains 262,144 context, prefix caching, 8192 MiB prompt-cache RAM, idle-slot caching, 32 context checkpoints with an 8192-token minimum step, 4096 MiB PLE cache, startup warmup, MTP 6, and the v3 QSA history cache. Thinking follows the embedded template’s default. Default sampling is temperature 1.0, top-p 0.95, top-k 20, min-p 0. Benchmark sampling and short output caps are request-specific and are not production defaults.

Model download and assembly

Download and assemble the model only if you do not already have it, then use the v4.2 runtime. Existing Orca weights, PLE files and the matching MTP head are reused.

Use a Strix Halo system with 128 GB shared memory and fast NVMe. The core model package totals 135,962,881,519 bytes (126.625 GiB); the included optional vision projector adds 904,004,320 bytes (0.842 GiB). Allow at least 220 GiB for download parts and assembly, plus space for the runtime and SDK. The assembled model plus projector uses 127.467 GiB; the helper removes consumed parts to reclaim temporary space.

Ubuntu/Debian build guidance:

sudo apt-get update
sudo apt-get install -y git python3-venv
python3 -m venv .venv-hf
.venv-hf/bin/python -m pip install -U huggingface_hub
. .venv-hf/bin/activate
hf download jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca --revision v4.2.0 --local-dir ./model
python3 ./model/assemble.py
(cd model && sha256sum -c SHA256SUMS)
git clone --branch v4.2.0 --single-branch \
  https://github.com/ciru-ai/Qwen3.8-Flash-CIRU-STRIX-IU4.git ciru-runtime-v4.2.0
(cd ciru-runtime-v4.2.0 && ./scripts/ciru/setup-linux-amd.sh --install-host-deps)
MODEL_DIR="$(realpath ./model)" \
  bash ./ciru-runtime-v4.2.0/scripts/ciru/run-orca-server.sh

The setup helper installs a private ROCm 10 SDK. Keep that SDK in place and ensure the AMD driver exposes /dev/kfd and the render node. This release was validated on NixOS/ROCm 10/gfx1151; the Ubuntu/Debian instructions are build guidance, not a claim of a new Ubuntu qualification. Runtime platform documentation.

The v4.2 production launcher uses the launch settings above, including 1024 MiB prompt-cache RAM, 4096 MiB PLE cache, MTP6 and 32 PLE I/O workers. Thinking follows the embedded template; sampling defaults remain temperature 1.0, top-p 0.95, top-k 20 and min-p 0. Benchmark caps are not production defaults.

The server binds to 127.0.0.1:8080. PORT, CONTEXT_SIZE, MTP_DEPTH, and ENABLE_MTP can override their corresponding defaults. MTP requires one slot. Example nonthinking request:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"Qwen3.8-Flash-CIRU-STRIX-Orca","messages":[{"role":"user","content":"Write a Python function that validates an IPv4 address."}],"chat_template_kwargs":{"enable_thinking":false},"temperature":0.7,"top_p":0.8,"top_k":20,"min_p":0,"presence_penalty":1.5,"repeat_penalty":1.0}'

Enable vision

Use the complete v4.4.1 runtime above. Fetch the optional projector if needed:

hf download jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca --revision v4.4.1 'vision/*' --local-dir ./model
(cd model/vision && sha256sum -c SHA256SUMS)
MODEL_DIR="$(realpath ./model)" bash ./ciru-runtime-v4.4.1-nixos-gfx1151/scripts/ciru/run-orca-server.sh --vision

MTP stays enabled with vision in v4.4.1. ENABLE_VISION=1 is equivalent to --vision; MMPROJ selects another projector path. Set ENABLE_MTP=0 for target-only generation. Older runtimes must be updated for vision with MTP.

Send an image through the server web UI or the OpenAI-compatible chat endpoint. For a local image:

python3 - <<'PYIMAGE'
import base64, json, urllib.request
from pathlib import Path
image = base64.b64encode(Path("image.png").read_bytes()).decode()
payload = {
    "model": "Qwen3.8-Flash-CIRU-STRIX-Orca",
    "messages": [{"role": "user", "content": [
        {"type": "text", "text": "Describe this image."},
        {"type": "image_url", "image_url": {"url": "data:image/png;base64," + image}}
    ]}],
    "chat_template_kwargs": {"enable_thinking": False},
    "max_tokens": 512
}
request = urllib.request.Request(
    "http://127.0.0.1:8080/v1/chat/completions",
    data=json.dumps(payload).encode(),
    headers={"Content-Type": "application/json"}
)
with urllib.request.urlopen(request, timeout=300) as response:
    print(json.load(response)["choices"][0]["message"]["content"])
PYIMAGE

The projector was converted to F16 directly from the pinned Orca checkpoint using the CIRU v3 converter. It contains 334 tensors. Vision uses additional memory for the projector, image encoding, and image tokens; the text-only performance figures above do not measure vision. Projector provenance and validation.

DeepSeek harness configuration

When using Orca through the DeepSeek harness, start the server with --vision and declare image input in the harness's Orca model configuration using input:

input:
  - text
  - image

Replace the Orca entry's inputModalities field with the input list above. A user reported that the harness recognized Orca image input with this configuration; the native DeepSeek model's inputModalities declaration did not work for their Orca entry. This is a harness configuration requirement, in addition to loading the projector with --vision.

Files after assembly

Hugging Face limits new individual uploads to 50 GB. The large files are distributed as checked binary download parts under download-parts/. Run assemble.py after downloading: it restores the exact tested GGUFs and PLE payload, verifies each part and final SHA-256, and supports restarting interrupted copies. These are transport parts, not GGUF shards or a different quantization. The table below lists the restored runtime files; parts.json records the uploaded parts.

File	Bytes	Purpose
Qwen3.8-Flash-CIRU-STRIX-Orca.gguf	79,397,818,912	Main model
mtp/Qwen3.8-Flash-CIRU-STRIX-Orca-MTP-Q8_0.gguf	4,135,893,440	Matching draft
ple/ple.payload.bin	52,429,053,952	External PLE table
ple/ple.manifest.json	115,213	PLE mapping
ple/ple.scale.bf16	2	PLE scale
vision/mmproj-Qwen3.8-Flash-Orca-F16.mmproj	904,004,320	Optional F16 vision projector (already assembled)

Browse all files · Checksums · Construction notes

Construction and provenance

Pinned source: orcarouter/Qwen3.8-Flash-Next-Uncensored at 8336e613.

The source audit found 149 changed tensors and 1,509 unchanged tensors against the original Qwen checkpoint. Config, tokenizer, and all 128 PLE tensors were unchanged. The main GGUF replaces 146 payloads and preserves 1,077; the MTP head replaces 4 and preserves 30. Shared embedding changes appear in both files. The changed expert down banks were refitted against the retained quantized gate/up banks using fresh Orca activations. Every output tensor was hash-verified.

Main tensor types: 144 Q4_1, 328 Q5_K, 290 Q8_0, 48 Q5_1, 25 BF16, and 388 F32. This build does not use IU4. The dependency repository retains its historical name.

Credit to Qwen for the base model, OrcaRouter for the modified research checkpoint, and CIRU/Crown (jcbtc, ciru-ai) for packaging, calibration, runtime work, and evaluation. Runtime contributor credits remain in the linked source repository. HumanEval/EvalPlus, IFEval, GSM8K, EvalScope, and the retained HermesAgent harness provide the evaluation tasks and scoring.

The upstream describes its checkpoint as refusal-removed. This package retains that provenance and does not add a safety alignment layer. Consult the upstream model card for its intended research use. The actual source checkpoint’s Qwen Community License is preserved; upstream Hub metadata alone is not used to relabel the packaged weights.