SecureCoder — handoff state (2026-10-04)

Everything needed to pick this up later. All code lives in this repo and in C:\Users\Taimwe\.cline\data\workspaces\chat\securecoder-pipeline.

The bug we found and fixed

v1 and v2 both emit literal \n (backslash-n) instead of real newlines, so their Python output does not parse. Measured with the same harness on the same prompts:

Model	literal `\n`	valid Python
base `unsloth/Qwen3-Coder-30B-A3B-Instruct`	—	93.3%
`securecoder-30b-pro` (v1)	14/15	~6.7%
`securecoder-30b-pro-v2`	15/15	0.0%

Root cause: parts of the training data mix store message text JSON-escaped. _unescape_if_needed() in train_securecoder.py now repairs them. Confirmed by --validate-only against 40 rows/source:

Source	repaired/40
`Trendyol/Cybersecurity-Instruction-Tuning`	40 (100%)
`AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1`	40 (100%)
`Humanlearning/CyberSecurity_OWASP-sft-dataset`	40 (100%)
`ukcli/Cybersecurity-Dataset-Heimdall-v1.1`	40 (100%)
`MrClipperz134/CTF-Instruct`	23 (58%)
hermes-fc, xlam, Magicoder, threat-intel, ctf-solver, FineTome	0

Every escaped source is a cybersecurity dataset. That is why tool-calling still scored 100% (clean sources dominate it) while code scored 0% (escaped sources dominate it).

Files in this repo

File	What changed
`train_securecoder.py`	fixed — `_unescape_if_needed()`, `repaired=` counter, BOM stripped so uv PEP 723 deps install
`eval_securecoder.py`	fixed — honours `--max-new-tokens` (was hardcoded 384), excludes truncation from the syntax score, falls back across dataset splits
`pipeline.py`	supervisor: `RESULT` seeded up front (fixes the `UnboundLocalError` that killed every supervised run), retries, `ADAPTER_REPO`/`GGUF_REPO` env overrides
`debug_v2_code.py`	dumps full replies — found the escaped newlines
`bisect_adapter.py`	proved base+adapter (no merge) is also affected ⇒ the adapter, not the merge

Two bugs worth remembering

UTF-8 BOM before # /// script — uv never saw the PEP 723 metadata block, so torch was never installed and every job died in 3–6s with No module named 'torch'. This is almost certainly what the old patch-deps jobs were fighting.
eval_securecoder.py scored truncation as invalid syntax. With max_new_tokens=384 correct code was cut mid-statement and counted as a failure. Now truncated replies are tracked separately (truncated, n_truncated, truncated_rate).

Repo state

Deleted (intentionally, both shipped broken code): securecoder-30b-pro-merged, securecoder-30b-pro-GGUF, securecoder-30b-pro-v2-merged, securecoder-30b-pro-v2-GGUF.
Kept: both adapters (securecoder-30b-pro, securecoder-30b-pro-v2) — re-mergeable.
Eval reports: Taimwe/securecoder-eval-v2, Taimwe/securecoder-eval-base.

How to resume

# 1. retrain with the fixed data  (a100-large, ~2.4h, ~$6)
hf jobs run ghcr.io/astral-sh/uv:python3.12-bookworm `
  uv run --with torch --with transformers --with datasets --with huggingface_hub `
         --with bitsandbytes --with accelerate --with unsloth --with trl --with peft `
  https://huggingface.co/Taimwe/securecoder-scripts/resolve/main/train_securecoder.py `
  --base-model unsloth/Qwen3-Coder-30B-A3B-Instruct `
  --output-repo Taimwe/securecoder-30b-pro-v3 `
  --mix-scale 0.25 --max-steps 480 --max-seq-length 2048 --learning-rate 1e-4 `
  --save-steps 200 --eval-samples 100 --report-to none `
  --flavor a100-large --timeout 10800 --secrets HF_TOKEN --detach

# 2. then merge + quantise + eval

Sizing matters: the full mix is 83,781 rows; at seq 4096 that is 983 steps at 43.5 s/it ≈ 11.8h / $29.50. With --mix-scale 0.25 --max-seq-length 2048 it is 480 steps at 17.7 s/it ≈ 2.4h / $6. Do not omit those two flags.

Also set --timeout generously but below the credit budget you intend to spend.

Still to do (publishing, free)

Fill the _pending_ eval table on the model cards with the eval-14 numbers.
The corrected harness is ready: python publish_cards.py or three hf upload calls.

Money spent this session (~$13 of 20)

Item	~$
two evals (v2 + base, A/B)	2.30
adapter-vs-merge bisect	0.11
two `--validate-only` diagnostics	0.06
cancelled run A (bad scoping: full mix @4096)	7.45
cancelled run B (this one, stopped in error)	2.50

The $7.45 was avoidable — v2's original job command showed --mix-scale 0.25 and --max-seq-length 2048, and I should have read it before launching instead of guessing.

Unrelated scaffold

C:\Users\Taimwe\.cline\data\workspaces\chat\flux2-lora — LoRA fine-tune + FastAPI serving for FLUX.2-klein-4B (Apache-2.0). Not runnable here: HF large-file downloads transfer at 0 MB/min from this machine, and there is no GPU.