agu18dec/mOre-lenses

OLens model sweep: inverters, distill models and RL lenses for six subject models

LoRA checkpoints of OLens (oracle lens) for Gemma-4-31B-it, the Qwen3.5 dense sizes (0.8B / 2B / 4B / 9B) and the Qwen3.6-27B reference, with the exact recipe of every run and all-Opus WorkspaceBench scores. Every adapter here is the one a WorkspaceBench tag actually scored. Each adapter applies to its subject's own base model (the lens reads the model with the model) and inherits that base model's license.

Generated 2026-10-12T20:50:45Z from the training volumes by scripts/ola/port/hf_release_modal.py.

What OLens is

OLens turns a residual-stream activation h (output of decoder block L at one token) into text. It is built in four stages, all LoRA (r16) on the subject model itself:

1. AR (activation reconstructor). Reads a short text span (1-32 tokens) behind a prompt tag [Layer NN] and predicts the subject's residual at that layer through a linear head per layer, trained with a whitened loss. Its reconstructions are the inverter's training inputs, and its embeddings define the RL reward.
2. Inverter. Given an injected vector at a slot token, generates the text span that produced it (prompt continuation_raw). Trained one epoch on (AR reconstruction, span) pairs, k=4 random layers per crop, with the scaled transform: v = s * h with ONE frozen global scale s fitted so the median injected norm is alpha = 16000.
3. Distill. For held-out TRUE residuals, k' = 64 samples are drawn from the subject's OWN inverter (T=1, top-p 0.95), a non-negative OMP over the AR's embeddings of the samples selects the concepts that jointly reconstruct h (prefix candidates 2..32, shrink eps 0.005, min gain 0.005), and the distill model (initialised from the inverter) is SFT'd to emit them as - bullets (prompt concepts_raw, unit transform v = alpha * h/|h|).
4. RL. GRPO (rl_sc) from the distill model for 600 steps; the reward is a joint, whitened, unit-norm non-negative least-squares reconstruction of the true h from the AR embeddings of the generated bullets, contrasted against distractors.

How to read an activation with an inverter

Every checkpoint directory has an olens.json with the base model, the adapter subfolder, the transform, alpha, the frozen scale, the prompt template and the slot token. The injected vector REPLACES the output of the input-embedding module at the slot position (after Gemma's internal sqrt(d) embedding scale).

import json
import torch
from huggingface_hub import hf_hub_download
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "agu18dec/mOre-lenses"
SUB = "qwen3.5-9b/inverter/ao.qwen3.5-9b.s0"  # any inverter / distill / rl directory
card = json.load(open(hf_hub_download(REPO, f"{SUB}/olens.json")))
read = card["read"]
tok = AutoTokenizer.from_pretrained(card["base_model"])
model = AutoModelForCausalLM.from_pretrained(
    card["base_model"], torch_dtype=torch.bfloat16, device_map="auto"
)

# 1) the activation: output of block L at the last token (hidden_states[L + 1]); read it from the
#    BASE model (once the LoRA is attached, wrap reads in `with lens.disable_adapter():`)
L = 20  # must be in read["layers"]
enc = tok("The capital of France is", return_tensors="pt").to(model.device)
with torch.no_grad():
    h = model(**enc, output_hidden_states=True).hidden_states[L + 1][0, -1].float()

# 2) transform -> slot vector
if read["transform"] == "scaled":
    v = read["global_scale"] * h  # frozen global scale (scale_*.json)
else:
    v = read["alpha"] * h / h.norm()  # unit: distill and RL lenses

# 3) prompt with the slot token, inject, generate
lens = PeftModel.from_pretrained(model, REPO, subfolder=card["adapter_subfolder"])
text = read["prompt_template"].format(layer=L, char=read["slot_char"])
ids = tok.apply_chat_template(
    [{"role": "user", "content": text}], add_generation_prompt=True,
    enable_thinking=False, return_tensors="pt", return_dict=True,
)["input_ids"].to(model.device)
slot = (ids[0] == read["slot_id"]).nonzero().item()
emb = lens.get_input_embeddings()(ids)
emb[0, slot] = v.to(emb.dtype)
out = lens.generate(
    inputs_embeds=emb, attention_mask=torch.ones_like(ids), max_new_tokens=64,
    do_sample=True, temperature=1.0, top_p=0.95,
)
print(tok.decode(out[0], skip_special_tokens=True))

Prompts ({layer} = the layer number, {char} = the slot character):

• inverters, continuation_raw: "An activation vector from layer {layer} of a language model is enclosed in activation tags: {char}. Produce the text that immediately follows this activation."
• distill and RL, concepts_raw: "An activation vector from layer {layer} of a language model is enclosed in activation tags: {char}. Produce distinct concepts that encode this activation, each as a '- ' bullet on its own line."

The slot character is the first enclosed ideograph (U+3200-U+33FF) that survives as a single token in the rendered chat prompt. WorkspaceBench sampling of record: T=1.0, top-p 0.95, top-k 64, 256 new tokens, thinking off.

Layout

<subject>/{ar,inverter,distill,rl}/<run>/:

• lora/, lora_hf/ (PEFT adapters; lora_hf is the _orig_mod-free, dropout-0 copy where the trainer wrote one) or iter_000600/ (RL), olens.json
• meta.json (checkpoint meta), run_config.json / run_config.yaml + nla_meta.yaml (RL)
• inverters: scale_<run>.json (the frozen global scale) and heldout_ce.json
• ARs: heads.pt (per-layer linear heads) with lora/. The subject whiteners are not included.

Optimizer / resume state, intermediate steps, the RL KL-reference copy and all logs are excluded.

subject	role	path	WSB tag	uploaded
gemma-4-31b-it	ar	gemma-4-31b-it/ar/ar.gemma4-31b.ptag.unitcos.s0	-	yes
gemma-4-31b-it	ar	gemma-4-31b-it/ar/ar.gemma4-31b.ptag.s0	-	yes
gemma-4-31b-it	inverter	gemma-4-31b-it/inverter/ao.gemma4-31b.unitcos.s0	gemma.inv.unitcos.end	yes
gemma-4-31b-it	inverter	gemma-4-31b-it/inverter/ao.gemma4-31b.base.s0	gemma.inv.base.end	yes
gemma-4-31b-it	inverter	gemma-4-31b-it/inverter/ao.gemma4-31b.unitcos.full.s0	gemma.inv.unitcos_full.end	yes
gemma-4-31b-it	distill	gemma-4-31b-it/distill/ao.gemma4-31b.unitcos.distill.s0	gemma.sft.unitcos	yes
gemma-4-31b-it	rl	gemma-4-31b-it/rl/rl.gemma4-31b.unitcos.ddp600.s0	gemma.rl.unitcos.ddp600.i600	yes
gemma-4-31b-it	inverter	gemma-4-31b-it/inverter/ao.gemma4-31b.gfix.ar.wsd.full	gemma.gfix.ar.wsd.full	yes
gemma-4-31b-it	distill	gemma-4-31b-it/distill/ao.gemma4-31b.gfix.arwsd.distill	gemma.gfix.arwsd.sft	yes
gemma-4-31b-it	rl	gemma-4-31b-it/rl/rl.gemma4-31b.arwsd.ddp600.s0	gemma.gfix.arwsd.rl.i600	yes
qwen3.5-0.8b	ar	qwen3.5-0.8b/ar/ar.qwen3.5-0.8b.ptag.s0	-	yes
qwen3.5-0.8b	inverter	qwen3.5-0.8b/inverter/ao.qwen3.5-0.8b.s0	qwen3.5-0.8b.inv.end	yes
qwen3.5-0.8b	distill	qwen3.5-0.8b/distill/ao.qwen3.5-0.8b.distill.s0	qwen3.5-0.8b.sft	yes
qwen3.5-0.8b	rl	qwen3.5-0.8b/rl/rl.qwen3.5-0.8b.ddp600.s0	qwen3.5-0.8b.rl.i600	yes
qwen3.5-2b	ar	qwen3.5-2b/ar/ar.qwen3.5-2b.ptag.s0	-	yes
qwen3.5-2b	inverter	qwen3.5-2b/inverter/ao.qwen3.5-2b.s0	qwen3.5-2b.inv.end	yes
qwen3.5-2b	distill	qwen3.5-2b/distill/ao.qwen3.5-2b.distill.s0	qwen3.5-2b.sft	yes
qwen3.5-2b	rl	qwen3.5-2b/rl/rl.qwen3.5-2b.ddp600.s0	qwen3.5-2b.rl.i600	yes
qwen3.5-4b	ar	qwen3.5-4b/ar/ar.qwen3.5-4b.ptag.s0	-	yes
qwen3.5-4b	inverter	qwen3.5-4b/inverter/ao.qwen3.5-4b.s0	qwen3.5-4b.inv.end	yes
qwen3.5-4b	inverter	qwen3.5-4b/inverter/ao.qwen3.5-4b.L20.s0	qwen3.5-4b.inv.L20.end	yes
qwen3.5-4b	distill	qwen3.5-4b/distill/ao.qwen3.5-4b.distill.s0	qwen3.5-4b.sft	yes
qwen3.5-4b	rl	qwen3.5-4b/rl/rl.qwen3.5-4b.ddp600.s0	qwen3.5-4b.rl.i600	yes
qwen3.5-9b	ar	qwen3.5-9b/ar/ar.qwen3.5-9b.ptag.s0	-	yes
qwen3.5-9b	inverter	qwen3.5-9b/inverter/ao.qwen3.5-9b.s0	qwen3.5-9b.inv.end	yes
qwen3.5-9b	inverter	qwen3.5-9b/inverter/ao.qwen3.5-9b.L20.s0	qwen3.5-9b.inv.L20.end	yes
qwen3.5-9b	distill	qwen3.5-9b/distill/ao.qwen3.5-9b.distill.s0	qwen3.5-9b.sft	yes
qwen3.5-9b	rl	qwen3.5-9b/rl/rl.qwen3.5-9b.ddp600.s0	qwen3.5-9b.rl.i600	yes
qwen3.6-27b	ar	qwen3.6-27b/ar/ar.ab4.unitcos.ext1	-	yes
qwen3.6-27b	inverter	qwen3.6-27b/inverter/ao.ab4.unitcos.frombase.ext1end_full	inv.unitcos.fullend	yes
qwen3.6-27b	distill	qwen3.6-27b/distill/abwc.unitcos.fullend.own	sft.unitcos.fullend.own	yes
qwen3.6-27b	rl	qwen3.6-27b/rl/rl.abwc.unitcos.fullend.own	rl.unitcos.fullend.own.i600	yes
qwen3.6-27b	rl	qwen3.6-27b/rl/s3d.ddp600.s0	ref-s3d-rl600	yes
qwen3-8b	ar	qwen3-8b/ar/ar.qwen3-8b.ptag.s0	-	yes
qwen3-8b	inverter	qwen3-8b/inverter/ao.qwen3-8b.wsd.s0	qwen3-8b.inv.s6795	yes
qwen3-8b	distill	qwen3-8b/distill/ao.qwen3-8b.wsd.distill.s0	qwen3-8b.sft	yes
qwen3-8b	rl	qwen3-8b/rl/rl.qwen3-8b.ddp600.s0	qwen3-8b.rl.i600	yes
glm-5.3-flash	ar	glm-5.3-flash/ar/ar.glm-5.3-flash.ptag.s1d	-	yes
glm-5.3-flash	inverter	glm-5.3-flash/inverter/ao.glm-5.3-flash.s1	glm.s1.inv.end	yes
glm-5.3-flash	distill	glm-5.3-flash/distill/ao.glm-5.3-flash.distill.s1	glm.s1.sft	yes
glm-5.3-flash	rl	glm-5.3-flash/rl/rl.glm-5.3-flash.ddp600.s1	glm.s1.rl.i300	yes

Recipes

All values below are read from each run's own config / meta. Layer bands come from src/global_workspace/ola/model_spec.py.

subject	base model	blocks / d_model	layers captured	AR band	inverter band	slot char / id
gemma-4-31b-it	google/gemma-4-31B-it	60 / 5376	0,4,8,12,16,20,24,28,32,36,40,44,48,52,56,59	L20-L59 (11)	L20-L56 (10)	㈜ / 246566
qwen3.5-0.8b	Qwen/Qwen3.5-0.8B	24 / 1024	0,2,4,6,8,10,12,14,16,18,20,22,23	L8-L23 (9)	L8-L22 (8)	㈜ / 158983
qwen3.5-2b	Qwen/Qwen3.5-2B	24 / 2048	0,2,4,6,8,10,12,14,16,18,20,22,23	L8-L23 (9)	L8-L22 (8)	㈜ / 158983
qwen3.5-4b	Qwen/Qwen3.5-4B	32 / 2560	0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,31	L10-L31 (12)	L10-L30 (11)	㈜ / 158983
qwen3.5-9b	Qwen/Qwen3.5-9B	32 / 4096	0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,31	L10-L31 (12)	L10-L30 (11)	㈜ / 158983
qwen3.6-27b	Qwen/Qwen3.6-27B	64 / 5120	0,4,8,12,16,20,24,28,32,36,40,44,48,52,56,60,63	L20-L63 (12)	L20-L60 (11)	㈜ / 158983
qwen3-8b	Qwen/Qwen3-8B	36 / 4096	0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,35	L12-L35 (13)	L12-L34 (12)	㈜ / 144490
glm-5.3-flash	zai-org/GLM-5.3-Flash	45 / 4096	0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,44	L15-L44 (11)	L15-L42 (10)	㎡ / 104823

AR (prompt-tag head; loss arm base = raw head output, unitcos = unit-norm head output):

AR run	lr	batch (mb x ga x gpus)	epochs	loss arm	layers	train crops	init from	val wFVE
gemma-4-31b-it ar.gemma4-31b.ptag.unitcos.s0	0.0001 (constant)	24x24 x8 = 576	1	unitcos (whiten loss, head unitnorm)	L20-L59	20,753,852	-	0.181
gemma-4-31b-it ar.gemma4-31b.ptag.s0	0.0001 (constant)	24x24 x8 = 576	1	base (whiten loss, head raw)	L20-L59	20,753,852	-	0.178
qwen3.5-0.8b ar.qwen3.5-0.8b.ptag.s0	0.0001 (constant)	24x24 x8 = 576	1	base (whiten loss, head raw)	L8-L23	23,477,711	-	0.168
qwen3.5-2b ar.qwen3.5-2b.ptag.s0	0.0001 (constant)	24x24 x8 = 576	1	base (whiten loss, head raw)	L8-L23	23,304,683	-	0.183
qwen3.5-4b ar.qwen3.5-4b.ptag.s0	0.0001 (constant)	24x24 x8 = 576	1	base (whiten loss, head raw)	L10-L31	22,543,755	-	0.192
qwen3.5-9b ar.qwen3.5-9b.ptag.s0	0.0001 (constant)	24x24 x8 = 576	1	base (whiten loss, head raw)	L10-L31	23,628,680	-	0.210
qwen3.6-27b ar.ab4.unitcos.ext1	0.0001 (constant)	24x24 x8 = 576	1	unitcos (whiten loss, head unitnorm)	L20-L63	4,075,373	ar.ab4.unitcos.s0	0.231
qwen3-8b ar.qwen3-8b.ptag.s0	0.0001 (constant)	24x24 x8 = 576	1	base (whiten loss, head raw)	L12-L35	21,074,672	-	0.218
glm-5.3-flash ar.glm-5.3-flash.ptag.s1d	0.0001 (constant)	288x2 x2 = 576	1	base (whiten loss, head raw)	L15-L44	8,885,305	ar.glm-5.3-flash.ptag.s0 (continue_from)	0.139

Inverter (scaled transform, frozen global scale, alpha 16000, continuation_raw, k=4 layers per crop, crops of length <= 32):

inverter run	transform / scale	frozen global scale	alpha	prompt	layers/crop	lr	LoRA	batch	epochs	data	layers	AR used
gemma-4-31b-it ao.gemma4-31b.unitcos.s0	scaled / global_frozen	71.416	16000	continuation_raw	k4	0.0003 (constant)	r16 / a32	mb128xga8	1	1,749,402 crops (6,997,608 ex)	L20-L56 (10)	ar.gemma4-31b.ptag.unitcos.s0/end
gemma-4-31b-it ao.gemma4-31b.base.s0	scaled / global_frozen	75.552	16000	continuation_raw	k4	0.0003 (constant)	r16 / a32	mb128xga8	1	1,749,402 crops (6,997,608 ex)	L20-L56 (10)	ar.gemma4-31b.ptag.s0/end
gemma-4-31b-it ao.gemma4-31b.unitcos.full.s0	scaled / global_frozen	57.521	16000	continuation_raw	k4	0.0003 (constant)	r16 / a32	mb128xga8	1	5,704,557 crops (22,818,228 ex)	L20-L56 (10)	ar.gemma4-31b.ptag.unitcos.s0/end
gemma-4-31b-it ao.gemma4-31b.gfix.ar.wsd.full	scaled / global_frozen	75.552	16000	continuation_raw	k4	0.0003 (wsd)	r16 / a32	mb128xga8	1	1,749,402 crops (6,997,608 ex)	L20-L56 (10)	ar.gemma4-31b.ptag.s0/end
qwen3.5-0.8b ao.qwen3.5-0.8b.s0	scaled / global_frozen	458.568	16000	continuation_raw	k4	0.0003 (constant)	r16 / a32	mb128xga8	1	1,749,400 crops (6,997,600 ex)	L8-L22 (8)	ar.qwen3.5-0.8b.ptag.s0/end
qwen3.5-2b ao.qwen3.5-2b.s0	scaled / global_frozen	278.788	16000	continuation_raw	k4	0.0003 (constant)	r16 / a32	mb128xga8	1	1,749,402 crops (6,997,608 ex)	L8-L22 (8)	ar.qwen3.5-2b.ptag.s0/end
qwen3.5-4b ao.qwen3.5-4b.s0	scaled / global_frozen	243.064	16000	continuation_raw	k4	0.0003 (constant)	r16 / a32	mb128xga8	1	1,749,400 crops (6,997,600 ex)	L10-L30 (11)	ar.qwen3.5-4b.ptag.s0/end
qwen3.5-4b ao.qwen3.5-4b.L20.s0	scaled / global_frozen	243.064	16000	continuation_raw	k4	0.0003 (constant)	r16 / a32	mb128xga8	1	1,749,400 crops (6,997,600 ex)	L20-L30 (6)	ar.qwen3.5-4b.ptag.s0/end
qwen3.5-9b ao.qwen3.5-9b.s0	scaled / global_frozen	148.269	16000	continuation_raw	k4	0.0003 (constant)	r16 / a32	mb128xga8	1	1,749,402 crops (6,997,608 ex)	L10-L30 (11)	ar.qwen3.5-9b.ptag.s0/end
qwen3.5-9b ao.qwen3.5-9b.L20.s0	scaled / global_frozen	148.269	16000	continuation_raw	k4	0.0003 (constant)	r16 / a32	mb128xga8	1	1,749,402 crops (6,997,608 ex)	L20-L30 (6)	ar.qwen3.5-9b.ptag.s0/end
qwen3.6-27b ao.ab4.unitcos.frombase.ext1end_full	scaled / global_frozen	192.282	16000	continuation_raw	k4	0.0003 (constant)	r16 / a32	mb128xga8	1	2,722,900 crops (10,891,600 ex)	L20-L60 (11)	ar.ab4.unitcos.ext1/end
qwen3-8b ao.qwen3-8b.wsd.s0	scaled / global_frozen	90.509	16000	continuation_raw	k4	0.0003 (wsd)	r16 / a32	mb128xga8	1	1,749,401 crops (6,997,604 ex)	L12-L34 (12)	ar.qwen3-8b.ptag.s0/end
glm-5.3-flash ao.glm-5.3-flash.s1	scaled / global_frozen	2169.392	16000	continuation_raw	k4	0.0003 (wsd)	r16 / a32	mb256xga4	1	1,747,788 crops (6,991,152 ex)	L15-L42 (10)	ar.glm-5.3-flash.ptag.s1d/end

Distill (own-inverter candidates, k'=64 samples per item, NN-OMP selection with the subject's AR and whitener, unit transform alpha 16000, concepts_raw):

distill run	transform	prompt	variant	init from	lr	epochs	token budget	target sets
gemma-4-31b-it ao.gemma4-31b.unitcos.distill.s0	unit a16000.0	concepts_raw	own.full	ao.gemma4-31b.unitcos.s0/step6799	0.0001	1	16384	16,921
gemma-4-31b-it ao.gemma4-31b.gfix.arwsd.distill	unit a16000.0	concepts_raw	own.full	ao.gemma4-31b.gfix.ar.wsd.full/step6799	0.0001	1	16384	16,740
qwen3.5-0.8b ao.qwen3.5-0.8b.distill.s0	unit a16000.0	concepts_raw	own.full	ao.qwen3.5-0.8b.s0/step6805	0.0001	1	16384	15,669
qwen3.5-2b ao.qwen3.5-2b.distill.s0	unit a16000.0	concepts_raw	own.full	ao.qwen3.5-2b.s0/step6802	0.0001	1	16384	15,581
qwen3.5-4b ao.qwen3.5-4b.distill.s0	unit a16000.0	concepts_raw	own.full	ao.qwen3.5-4b.s0/step6799	0.0001	1	16384	14,502
qwen3.5-9b ao.qwen3.5-9b.distill.s0	unit a16000.0	concepts_raw	own.full	ao.qwen3.5-9b.s0/step6801	0.0001	1	16384	12,238
qwen3.6-27b abwc.unitcos.fullend.own	unit a16000.0	concepts_raw	abwc.own.unitcos.fullend	ao.ab4.unitcos.frombase.ext1end_full/step10603	0.0001	1	16384	11,397
qwen3-8b ao.qwen3-8b.wsd.distill.s0	unit a16000.0	concepts_raw	own.full	ao.qwen3-8b.wsd.s0/step6795	0.0001	1	16384	9,949
glm-5.3-flash ao.glm-5.3-flash.distill.s1	unit a16000.0	concepts_raw	own.full	ao.glm-5.3-flash.s1/step13565	0.0001	1	16384	11,340

RL (GRPO rl_sc, joint whitened unit-norm NNLS reward, 600 steps):

RL run	steps	group	prompts/step	lr	KL beta	reward	init (distill)	AR (reward)
gemma-4-31b-it rl.gemma4-31b.unitcos.ddp600.s0	600	64	14	5e-05	0.02	joint, whiten=True, unit_norm=True, k_max=4, distractors=8	ao.gemma4-31b.unitcos.distill.s0	ar.gemma4-31b.ptag.unitcos.s0/end
gemma-4-31b-it rl.gemma4-31b.arwsd.ddp600.s0	600	64	14	5e-05	0.02	joint, whiten=True, unit_norm=True, k_max=4, distractors=8	ao.gemma4-31b.gfix.arwsd.distill	ar.gemma4-31b.ptag.s0/end
qwen3.5-0.8b rl.qwen3.5-0.8b.ddp600.s0	600	64	14	5e-05	0.02	joint, whiten=True, unit_norm=True, k_max=4, distractors=8	ao.qwen3.5-0.8b.distill.s0	ar.qwen3.5-0.8b.ptag.s0/end
qwen3.5-2b rl.qwen3.5-2b.ddp600.s0	600	64	14	5e-05	0.02	joint, whiten=True, unit_norm=True, k_max=4, distractors=8	ao.qwen3.5-2b.distill.s0	ar.qwen3.5-2b.ptag.s0/end
qwen3.5-4b rl.qwen3.5-4b.ddp600.s0	600	64	14	5e-05	0.02	joint, whiten=True, unit_norm=True, k_max=4, distractors=8	ao.qwen3.5-4b.distill.s0	ar.qwen3.5-4b.ptag.s0/end
qwen3.5-9b rl.qwen3.5-9b.ddp600.s0	600	64	14	5e-05	0.02	joint, whiten=True, unit_norm=True, k_max=4, distractors=8	ao.qwen3.5-9b.distill.s0	ar.qwen3.5-9b.ptag.s0/end
qwen3.6-27b rl.abwc.unitcos.fullend.own	600	64	14	5e-05	0.02	joint, whiten=True, unit_norm=True, k_max=4, distractors=8	abwc.unitcos.fullend.own	ar.ab4.unitcos.ext1/end (mirror abwc/unitcos/fullend.own)
qwen3.6-27b s3d.ddp600.s0	600	?	?	?	0.02	(run.json only) ao.iolens.ptag.s3d.omp4bp.s2/step251 (GRPO 600 steps, beta 0.02, joint bullet-FVE reward on the ptag AR)	-	-
qwen3-8b rl.qwen3-8b.ddp600.s0	600	64	14	5e-05	0.02	joint, whiten=True, unit_norm=True, k_max=4, distractors=8	ao.qwen3-8b.wsd.distill.s0	ar.qwen3-8b.ptag.s0/end
glm-5.3-flash rl.glm-5.3-flash.ddp600.s1	300 (stopped; 600 configured)	64	14	5e-05	0.02	joint, whiten=True, unit_norm=True, k_max=4, distractors=8	ao.glm-5.3-flash.distill.s1	ar.glm-5.3-flash.ptag.s1d/end

Results (all-Opus WorkspaceBench)

headline_23 = mean over 23 WorkspaceBench families (the 24 headline families minus hallucination) with a two-level bootstrap 95% CI. Every LLM-judged family is judged by claude-opus-5-5; the six regex families stay regex. On-policy hallucination = the hallucination family re-run on a bank built from the subject's OWN responses (<tag>.hal_onpolicy; lower is better). Held-out CE = inverter cross-entropy (nats/token) on a held-out pool of the subject's own conversations, 16,384 crops x 4 layers. "pending" = judging not finished at generation time.

subject	stage	WSB tag	checkpoint	headline_23 [95% CI]	on-policy halluc. rate	held-out CE
gemma-4-31b-it	inverter	gemma.inv.unitcos.end	inverter (unitcos AR, 7M ex)	0.453 [0.436, 0.469]	0.157 [0.140, 0.174]	1.456
gemma-4-31b-it	inverter	gemma.inv.base.end	inverter (base AR, 7M ex)	0.492 [0.475, 0.509]	0.192 [0.170, 0.212]	1.469
gemma-4-31b-it	inverter	gemma.inv.unitcos_full.end	inverter (unitcos AR, full pool)	0.463 [0.445, 0.479]	0.177 [0.156, 0.198]	1.275
gemma-4-31b-it	distill	gemma.sft.unitcos	distill	0.514 [0.497, 0.531]	0.177 [0.154, 0.200]	-
gemma-4-31b-it	rl	gemma.rl.unitcos.ddp600.i600	RL i600	0.594 [0.577, 0.610]	0.295 [0.266, 0.322]	-
gemma-4-31b-it	inverter	gemma.gfix.ar.wsd.full	inverter (base AR, WSD decay; fixed)	0.517 [0.500, 0.535]	pending	-
gemma-4-31b-it	distill	gemma.gfix.arwsd.sft	distill (fixed)	0.556 [0.539, 0.573]	0.242 [0.213, 0.271]	-
gemma-4-31b-it	rl	gemma.gfix.arwsd.rl.i600	RL i600 (fixed)	0.614 [0.597, 0.630]	0.320 [0.289, 0.349]	-
qwen3.5-0.8b	inverter	qwen3.5-0.8b.inv.end	inverter	0.300 [0.285, 0.315]	0.136 [0.115, 0.157]	2.295
qwen3.5-0.8b	distill	qwen3.5-0.8b.sft	distill	0.359 [0.344, 0.374]	0.145 [0.122, 0.171]	-
qwen3.5-0.8b	rl	qwen3.5-0.8b.rl.i600	RL i600	0.406 [0.390, 0.422]	0.267 [0.234, 0.302]	-
qwen3.5-2b	inverter	qwen3.5-2b.inv.end	inverter	0.336 [0.320, 0.352]	0.185 [0.161, 0.208]	1.838
qwen3.5-2b	distill	qwen3.5-2b.sft	distill	0.421 [0.406, 0.436]	0.198 [0.172, 0.221]	-
qwen3.5-2b	rl	qwen3.5-2b.rl.i600	RL i600	0.477 [0.462, 0.492]	0.360 [0.323, 0.393]	-
qwen3.5-4b	inverter	qwen3.5-4b.inv.end	inverter	0.439 [0.421, 0.456]	0.227 [0.198, 0.256]	1.552
qwen3.5-4b	distill	qwen3.5-4b.sft	distill	0.517 [0.501, 0.532]	0.249 [0.216, 0.282]	-
qwen3.5-4b	rl	qwen3.5-4b.rl.i600	RL i600	0.595 [0.580, 0.611]	0.412 [0.369, 0.453]	-
qwen3.5-9b	inverter	qwen3.5-9b.inv.end	inverter	0.522 [0.506, 0.539]	0.239 [0.208, 0.268]	1.334
qwen3.5-9b	distill	qwen3.5-9b.sft	distill	0.595 [0.579, 0.610]	0.259 [0.226, 0.291]	-
qwen3.5-9b	rl	qwen3.5-9b.rl.i600	RL i600	0.659 [0.644, 0.673]	0.438 [0.394, 0.480]	-
qwen3.6-27b	inverter	inv.unitcos.fullend	inverter (unitcos AR, full pool)	0.600 [0.583, 0.617]	0.291 [0.254, 0.328]	1.028
qwen3.6-27b	distill	sft.unitcos.fullend.own	distill (own)	0.675 [0.659, 0.692]	0.316 [0.276, 0.354]	-
qwen3.6-27b	rl	rl.unitcos.fullend.own.i600	RL i600 (own)	0.720 [0.703, 0.737]	0.419 [0.372, 0.463]	-
qwen3.6-27b	rl	ref-s3d-rl600	RL i600 (s3d reference of record)	0.726 [0.711, 0.742]	0.399 [0.352, 0.442]	-
qwen3-8b	inverter	qwen3-8b.inv.s6795	inverter (WSD)	0.417 [0.400, 0.433]	0.309 [0.273, 0.344]	1.235
qwen3-8b	distill	qwen3-8b.sft	distill	0.450 [0.434, 0.466]	0.288 [0.255, 0.320]	-
qwen3-8b	rl	qwen3-8b.rl.i600	RL i600	0.537 (provisional: 1 buggy_code cell unjudged; no CI)	0.426 [0.384, 0.466]	-
glm-5.3-flash	inverter	glm.s1.inv.end	inverter (WSD, step13565)	0.649 [0.632, 0.665]	0.284 [0.257, 0.309]	-
glm-5.3-flash	distill	glm.s1.sft	distill	0.712 (provisional: 1 buggy_code cell unjudged; no CI)	0.287 [0.262, 0.312]	-
glm-5.3-flash	rl	glm.s1.rl.i300	RL i300 (stopped at 300 of 600 configured steps)	0.785 [0.771, 0.799]	0.422 [0.390, 0.455]	-

GLM-5.3-Flash (zai-org/GLM-5.3-Flash @ eb9eb208) is model-parallel (320B MoE, mHC residual read as the mean of its 4 streams, slot ㎡) and needs the patched thinking-off chat template shipped as glm-5.3-flash/chat_template.jinja; its RL lens stopped at 300 steps. Details: glm-5.3-flash/README.md.

Machine-readable numbers: results.json in this repo.