Publish Stage A HRM-Text code and tool-use SFT pilot

Final Keras preset, full checkpoint, completed-run telemetry, TensorBoard events, hardware telemetry, and training-loss chart for the Stage A seed-17 pilot.

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-00020000.keras filter=lfs diff=lfs merge=lfs -text
+training-loss.png filter=lfs diff=lfs merge=lfs -text

ARTIFACTS.tsv

@@ -0,0 +1,30 @@
+ARTIFACTS.tsv	0
+LICENSE	561
+README.md	4571
+checkpoint-00020000.keras	11831471907
+preset/assets/tokenizer/merges.txt	635535
+preset/assets/tokenizer/vocabulary.json	1502734
+preset/config.json	1031
+preset/metadata.json	190
+preset/model.weights.h5	2366554504
+preset/preprocessor.json	1746
+preset/task.json	3722
+preset/tokenizer.json	845
+training-loss.png	136398
+training/environment.json	171
+training/gpu.jsonl	947076
+training/history.csv	162
+training/history.json	219
+training/pilot-completed-metrics.jsonl	1735944
+training/pilot-completion.json	878
+training/platform.jsonl	210602
+training/preflight.json	3751
+training/tensorboard/train/events.out.tfevents.1784941536.ccdf07020f90.51.0.v2	932008
+training/tensorboard/validation/events.out.tfevents.1784994496.ccdf07020f90.51.1.v2	78
+training/training-plan.json	355
+path	bytes	integrity
+preset/model.weights.h5	2366554504	sha256:8ddb10cea110edff99380b360ec569e5dbde0d4a233d1c058b0e292700a42b84
+checkpoint-00020000.keras	11831471907	md5:1cbcae0768082e8df30b793c5890bb08
+training/pilot-completed-metrics.jsonl	1735944	sha256:fbbb63a26548fe2b7e1cc49e13b9e1fe3b037f4eb40764d513ff60112ad86f52
+training/pilot-completion.json	878	completion manifest for canonical telemetry
+training-loss.png	136398	chart derived only from canonical telemetry

LICENSE

@@ -0,0 +1,16 @@
+Apache License 2.0
+
+Copyright 2026
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+

README.md

@@ -0,0 +1,137 @@
+---
+language:
+- en
+license: apache-2.0
+base_model: sapientinc/HRM-Text-1B
+library_name: keras
+pipeline_tag: text-generation
+tags:
+- hrm
+- prefix-lm
+- code
+- tool-use
+- keras
+- keras-hub
+- sft
+---
+
+# HRM-Text-1B Code & Tool-Use SFT
+
+This page tracks successive code and tool-use SFT stages of
+[`sapientinc/HRM-Text-1B`](https://huggingface.co/sapientinc/HRM-Text-1B),
+saved as native Keras/KerasHub presets. The current default revision and the
+immutable `stage-a-v1` tag are a full-parameter Stage A pilot. They adapt the
+base PrefixLM to the study's coding and tool-use transcript syntax.
+
+> **Pilot release.** This model has not yet undergone downstream benchmark
+> evaluation. Training loss is not a measure of coding-agent or tool-use task
+> performance. Do not treat it as a production-ready agent.
+
+## Release lineage
+
+| Tag | State | Availability |
+| --- | --- | --- |
+| `stage-a-v1` | Completed 10M-response-token Stage A pilot, seed 17 | Current default revision |
+| `stage-b-v1` | Stage B continuation | Not trained or published yet |
+
+Stage B will be published as `stage-b-v1` on this same page only after its
+training and release checks complete. `stage-a-v1` will remain immutable for
+reproducibility.
+
+## Training result
+
+![Training loss](training-loss.png)
+
+| Metric | Value |
+| --- | ---: |
+| Training budget | 10,000,147 response tokens |
+| Serialized tokens | 19,650,833 |
+| Microbatches / optimizer updates | 38,248 / 4,781 planned |
+| Final training loss | 0.03527 |
+| Validation loss | 0.02808 |
+| Final validation token accuracy | 0.90152 |
+| Hardware | One NVIDIA H100 80 GB |
+| Wall time | 14 h 44 min |
+
+The chart uses the canonical completed-run telemetry segment. The thin trace is
+the loss logged every ten microbatches; the dark trace is a 25-point trailing
+mean.
+
+## Training data and protocol
+
+- Dataset: [`pzarzycki/hrm-text-code-tools-sft`](https://huggingface.co/datasets/pzarzycki/hrm-text-code-tools-sft), canonical v2 Stage A.
+- Source: `nvidia/OpenCodeInstruct` pinned to revision
+  `8f3ba5bafe4d6e8db46082cf7ae6741bc370604d` (CC-BY-4.0).
+- The deterministic pilot selection contains 38,248 rows from the sealed
+  Stage A training split. It is not a full pass over all 1.13M Stage A rows.
+- Context cap: 4,096 serialized tokens. Oversized rows were rejected during
+  data preparation; no example was silently truncated.
+- The only condition used was `direct`.
+
+The learned SFT envelope is:
+
+```text
+<|im_start|><|object_ref_start|>instruction<|im_end|>response<|box_end|>
+```
+
+`<user>`, `<assistant>`, `<tools>`, `<tool_call>`, and `<tool_result>` inside
+the content are ordinary learned transcript markup, not additional pretrained
+HRM control tokens.
+
+## Optimization
+
+Full-parameter BF16 fine-tuning used KerasHub HRM-Text support, AdamW
+(`lr=3e-5`, `beta_1=0.9`, `beta_2=0.95`, `weight_decay=0.1`), global gradient
+clip norm 1.0, EMA (`0.999`), batch size 1 with gradient accumulation 8,
+and 3% warmup. Loss is applied only to response tokens, including
+`<|box_end|>`; prefix tokens receive zero loss weight.
+
+## Files
+
+| Path | Purpose |
+| --- | --- |
+| `preset/` | Final Keras preset: model configuration, tokenizer, preprocessor, and final weights. |
+| `checkpoint-00020000.keras` | Full Keras checkpoint saved at microbatch 20,000. |
+| `training/pilot-completion.json` | Immutable completion manifest and metrics SHA-256. |
+| `training/pilot-completed-metrics.jsonl` | Canonical first-run loss, learning-rate, token, and throughput telemetry. |
+| `training/gpu.jsonl` | GPU utilization, VRAM, power, and temperature telemetry. |
+| `training/platform.jsonl` | RunPod GPU/CPU utilization samples. |
+| `training/history.json` | Final Keras training and validation summary. |
+| `training/tensorboard/` | TensorBoard events for the completed training run. |
+| `ARTIFACTS.tsv` | Sizes and release-integrity checksums for the key artifacts. |
+
+## Loading and prompting
+
+The preset requires a KerasHub build with HRM-Text support. With the study's
+HRM-enabled KerasHub branch installed, load the preset from a local clone or
+using the Hugging Face handle:
+
+```python
+from keras_hub.models import HrmTextCausalLM
+
+model = HrmTextCausalLM.from_preset(
+    "hf://pzarzycki/hrm-text-1b-code-tools-sft"
+)
+prompt = (
+    "<|im_start|><|object_ref_start|>"
+    "Write a Python function that returns the larger of two integers."
+    "<|im_end|>"
+)
+print(model.generate(prompt, max_length=256))
+```
+
+HRM-Text is a PrefixLM base model, not a ChatML/Qwen chat model. Keep the
+`direct` condition and the `<|im_end|>` instruction boundary at inference.
+
+## Limitations and intended use
+
+This is a research artifact for continued evaluation of code generation and a
+fixed tool-transcript protocol. It does not itself execute tools, validate tool
+calls, sandbox generated code, or establish benchmark performance. Use
+independent task-level evaluation before deployment.
+
+## Provenance
+
+The base model is Apache-2.0. Stage A training data is CC-BY-4.0; source pins,
+dataset checksums, serializer details, preflight configuration, and completed
+telemetry are included in this repository's `training/` artifacts.

checkpoint-00020000.keras

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d89e41a7dd481c9d20b5778228961cae38acee09fab3862095f3e4b12b90fbe1
+size 11831471907

preset/config.json

@@ -0,0 +1,36 @@
+{
+    "module": "keras_hub.src.models.hrm_text.hrm_text_backbone",
+    "class_name": "HrmTextBackbone",
+    "config": {
+        "name": "hrm_text_backbone",
+        "trainable": true,
+        "dtype": {
+            "module": "keras",
+            "class_name": "DTypePolicy",
+            "config": {
+                "name": "bfloat16"
+            },
+            "registered_name": null
+        },
+        "vocabulary_size": 65536,
+        "hidden_dim": 1536,
+        "intermediate_dim": 4096,
+        "num_layers_per_stack": 16,
+        "num_attention_heads": 12,
+        "head_dim": 128,
+        "h_cycles": 2,
+        "l_cycles": 3,
+        "max_sequence_length": 4096,
+        "rope_theta": 10000.0,
+        "rms_norm_epsilon": 1e-06,
+        "l_bp_cycles": [
+            0,
+            3
+        ],
+        "initializer_range": 0.025515518153991442,
+        "embedding_scale": 39.191835884530846,
+        "tie_word_embeddings": false,
+        "rematerialization": true
+    },
+    "registered_name": "keras_hub>HrmTextBackbone"
+}

preset/metadata.json

@@ -0,0 +1,9 @@
+{
+    "keras_version": "3.15.0",
+    "keras_hub_version": "0.30.0.dev0",
+    "parameter_count": 1182795264,
+    "date_saved": "2026-07-25@15:49:24",
+    "tasks": [
+        "CausalLM"
+    ]
+}

preset/model.weights.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ddb10cea110edff99380b360ec569e5dbde0d4a233d1c058b0e292700a42b84
+size 2366554504

preset/preprocessor.json

@@ -0,0 +1,51 @@
+{
+    "module": "keras_hub.src.models.hrm_text.hrm_text_causal_lm_preprocessor",
+    "class_name": "HrmTextCausalLMPreprocessor",
+    "config": {
+        "name": "hrm_text_causal_lm_preprocessor",
+        "trainable": true,
+        "dtype": {
+            "module": "keras",
+            "class_name": "DTypePolicy",
+            "config": {
+                "name": "float32"
+            },
+            "registered_name": null
+        },
+        "tokenizer": {
+            "module": "keras_hub.src.models.hrm_text.hrm_text_tokenizer",
+            "class_name": "HrmTextTokenizer",
+            "config": {
+                "name": "hrm_text_tokenizer",
+                "trainable": true,
+                "dtype": {
+                    "module": "keras",
+                    "class_name": "DTypePolicy",
+                    "config": {
+                        "name": "int32"
+                    },
+                    "registered_name": null
+                },
+                "config_file": "tokenizer.json",
+                "sequence_length": null,
+                "add_prefix_space": false,
+                "unsplittable_tokens": [
+                    "<|im_end|>",
+                    "<|quad_end|>",
+                    "<|object_ref_end|>",
+                    "<|object_ref_start|>",
+                    "<|box_end|>",
+                    "<|endoftext|>",
+                    "<|im_start|>",
+                    "<|quad_start|>"
+                ]
+            },
+            "registered_name": "keras_hub>HrmTextTokenizer"
+        },
+        "config_file": "preprocessor.json",
+        "sequence_length": 4096,
+        "add_start_token": true,
+        "add_end_token": true
+    },
+    "registered_name": "keras_hub>HrmTextCausalLMPreprocessor"
+}

preset/task.json

@@ -0,0 +1,95 @@
+{
+    "module": "keras_hub.src.models.hrm_text.hrm_text_causal_lm",
+    "class_name": "HrmTextCausalLM",
+    "config": {
+        "backbone": {
+            "module": "keras_hub.src.models.hrm_text.hrm_text_backbone",
+            "class_name": "HrmTextBackbone",
+            "config": {
+                "name": "hrm_text_backbone",
+                "trainable": true,
+                "dtype": {
+                    "module": "keras",
+                    "class_name": "DTypePolicy",
+                    "config": {
+                        "name": "bfloat16"
+                    },
+                    "registered_name": null
+                },
+                "vocabulary_size": 65536,
+                "hidden_dim": 1536,
+                "intermediate_dim": 4096,
+                "num_layers_per_stack": 16,
+                "num_attention_heads": 12,
+                "head_dim": 128,
+                "h_cycles": 2,
+                "l_cycles": 3,
+                "max_sequence_length": 4096,
+                "rope_theta": 10000.0,
+                "rms_norm_epsilon": 1e-06,
+                "l_bp_cycles": [
+                    0,
+                    3
+                ],
+                "initializer_range": 0.025515518153991442,
+                "embedding_scale": 39.191835884530846,
+                "tie_word_embeddings": false,
+                "rematerialization": true
+            },
+            "registered_name": "keras_hub>HrmTextBackbone"
+        },
+        "preprocessor": {
+            "module": "keras_hub.src.models.hrm_text.hrm_text_causal_lm_preprocessor",
+            "class_name": "HrmTextCausalLMPreprocessor",
+            "config": {
+                "name": "hrm_text_causal_lm_preprocessor",
+                "trainable": true,
+                "dtype": {
+                    "module": "keras",
+                    "class_name": "DTypePolicy",
+                    "config": {
+                        "name": "float32"
+                    },
+                    "registered_name": null
+                },
+                "tokenizer": {
+                    "module": "keras_hub.src.models.hrm_text.hrm_text_tokenizer",
+                    "class_name": "HrmTextTokenizer",
+                    "config": {
+                        "name": "hrm_text_tokenizer",
+                        "trainable": true,
+                        "dtype": {
+                            "module": "keras",
+                            "class_name": "DTypePolicy",
+                            "config": {
+                                "name": "int32"
+                            },
+                            "registered_name": null
+                        },
+                        "config_file": "tokenizer.json",
+                        "sequence_length": null,
+                        "add_prefix_space": false,
+                        "unsplittable_tokens": [
+                            "<|im_end|>",
+                            "<|quad_end|>",
+                            "<|object_ref_end|>",
+                            "<|object_ref_start|>",
+                            "<|box_end|>",
+                            "<|endoftext|>",
+                            "<|im_start|>",
+                            "<|quad_start|>"
+                        ]
+                    },
+                    "registered_name": "keras_hub>HrmTextTokenizer"
+                },
+                "config_file": "preprocessor.json",
+                "sequence_length": 4096,
+                "add_start_token": true,
+                "add_end_token": true
+            },
+            "registered_name": "keras_hub>HrmTextCausalLMPreprocessor"
+        },
+        "name": "hrm_text_causal_lm"
+    },
+    "registered_name": "keras_hub>HrmTextCausalLM"
+}

preset/tokenizer.json

@@ -0,0 +1,30 @@
+{
+    "module": "keras_hub.src.models.hrm_text.hrm_text_tokenizer",
+    "class_name": "HrmTextTokenizer",
+    "config": {
+        "name": "hrm_text_tokenizer",
+        "trainable": true,
+        "dtype": {
+            "module": "keras",
+            "class_name": "DTypePolicy",
+            "config": {
+                "name": "int32"
+            },
+            "registered_name": null
+        },
+        "config_file": "tokenizer.json",
+        "sequence_length": null,
+        "add_prefix_space": false,
+        "unsplittable_tokens": [
+            "<|im_end|>",
+            "<|quad_end|>",
+            "<|object_ref_end|>",
+            "<|object_ref_start|>",
+            "<|box_end|>",
+            "<|endoftext|>",
+            "<|im_start|>",
+            "<|quad_start|>"
+        ]
+    },
+    "registered_name": "keras_hub>HrmTextTokenizer"
+}

training/environment.json

@@ -0,0 +1 @@
+{"keras_backend": "torch", "pid": 51, "platform": "Linux-6.17.0-1008-nvidia-x86_64-with-glibc2.39", "python": "3.12.3", "recorded_at": "2026-07-25T15:50:59.589577+00:00"}

training/history.csv

@@ -0,0 +1,2 @@
+epoch,loss,sparse_categorical_accuracy,val_loss,val_sparse_categorical_accuracy
+0,0.03527289628982544,0.879867672920227,0.028078513219952583,0.9015151262283325

training/history.json

@@ -0,0 +1,14 @@
+{
+  "loss": [
+    0.03527289628982544
+  ],
+  "sparse_categorical_accuracy": [
+    0.879867672920227
+  ],
+  "val_loss": [
+    0.028078513219952583
+  ],
+  "val_sparse_categorical_accuracy": [
+    0.9015151262283325
+  ]
+}

training/pilot-completion.json

@@ -0,0 +1,20 @@
+{
+  "examples_seen": 38248,
+  "final_preset": "preset/",
+  "final_training_loss": 0.03527289628982544,
+  "finished_at": "2026-07-25T15:49:24.079371+00:00",
+  "format": "hrm-sft-pilot-completion-v1",
+  "metrics_segment": "pilot-completed-metrics.jsonl",
+  "metrics_segment_lines": 3826,
+  "metrics_segment_sha256": "fbbb63a26548fe2b7e1cc49e13b9e1fe3b037f4eb40764d513ff60112ad86f52",
+  "microbatch_steps": 38248,
+  "mid_run_checkpoint": "checkpoint-00020000.keras",
+  "post_completion_stream_present": true,
+  "post_completion_stream_reason": "container restarted after successful completion; pod stopped before second cycle was used",
+  "raw_metrics_stream": "metrics.jsonl",
+  "response_tokens_seen": 10000147,
+  "run_id": "hrm-stage-a-pilot-20260724-restart1",
+  "serialized_tokens_seen": 19650833,
+  "started_at": "2026-07-25T01:05:36.032965+00:00",
+  "status": "completed"
+}

training/preflight.json

@@ -0,0 +1,112 @@
+{
+  "config": {
+    "data": {
+      "context_length": 4096,
+      "manifest": "/workspace/sft/stage-a/manifest.json",
+      "response_only_loss": true,
+      "tokenizer_json": "/workspace/models/hrm-text-1b/tokenizer.reference.json"
+    },
+    "framework": "keras",
+    "model": {
+      "family": "hrm-text",
+      "preset": "/workspace/models/hrm-text-1b",
+      "rematerialization": true,
+      "source_lock": "keras-hub.lock"
+    },
+    "run_name": "hrm-stage-a-pilot",
+    "training": {
+      "backup_save_freq": 500,
+      "beta_1": 0.9,
+      "beta_2": 0.95,
+      "checkpoint_save_freq": 20000,
+      "clipnorm": 1.0,
+      "cosine_min_lr_ratio": 0.1,
+      "ema_momentum": 0.999,
+      "gradient_accumulation_steps": 8,
+      "learning_rate": 3e-05,
+      "micro_batch_size": 1,
+      "observability": {
+        "every_n_batches": 10,
+        "tensorboard_every_n_batches": 10
+      },
+      "output_dir": "artifacts/hrm-stage-a-pilot-seed17",
+      "precision": "bfloat16",
+      "seed": 17,
+      "target_response_tokens": 10000000,
+      "use_ema": true,
+      "validation_steps": 128,
+      "warmup_ratio": 0.03,
+      "weight_decay": 0.1
+    }
+  },
+  "config_sha256": "3e535c768bd399c8dbdbbb54d13ad03e1fc56d83bf6171cf3ed44a090f80327c",
+  "manifest": {
+    "active_token_ids": {
+      "<|box_end|>": 11,
+      "<|endoftext|>": 5,
+      "<|im_end|>": 7,
+      "<|im_start|>": 6,
+      "<|object_ref_end|>": 9,
+      "<|object_ref_start|>": 8,
+      "<|quad_end|>": 13,
+      "<|quad_start|>": 12
+    },
+    "condition_distribution": {
+      "direct": 1157064
+    },
+    "data_schema_version": 2,
+    "declared_license": "CC-BY-4.0",
+    "decontamination": "complete",
+    "decontamination_report": "decontamination.json",
+    "decontamination_report_sha256": "3c4ace8d0ca0a668db2ebcea4ef6874499b6aefbb359d324dbfc694ac48f35ba",
+    "deduplication": {
+      "exact": "complete"
+    },
+    "format": "hrm-code-tools-sft-v2",
+    "migration": {
+      "input_manifest_sha256": "18c43a551e6c62adf600deb9614d3057ebb1db6dd1ecff5e76b76f26ee375881",
+      "input_train_sha256": "5d5b4a8f0fd37d4efb43db433c5d715f0f7ded228a07406e294e8cfed63f752a",
+      "input_validation_sha256": "3101880dc4b750499617f24312984c05d23081ae68e37739366f9ffebf0cd0d0",
+      "method": "v1 curated rows -> canonical v2 fields; re-split, deduplicate, and re-cap"
+    },
+    "records": {
+      "accepted": 1157064,
+      "by_source": {
+        "nvidia/OpenCodeInstruct": 1157064
+      },
+      "input_rows": 1157064,
+      "rejected_context_limit": 0,
+      "rejected_duplicate_id": 0,
+      "rejected_exact_duplicate": 0,
+      "response_tokens": 300785226,
+      "serialized_tokens": 590957065,
+      "train": 1133817,
+      "validation": 23247
+    },
+    "selection": {
+      "input_was_full_stage": true,
+      "max_serialized_sequence_length": 4096
+    },
+    "serializer_version": "hrm-prefixlm-sft-v1",
+    "sources": {
+      "nvidia/OpenCodeInstruct": {
+        "license": "CC-BY-4.0",
+        "revision": "8f3ba5bafe4d6e8db46082cf7ae6741bc370604d"
+      }
+    },
+    "split": {
+      "method": "sha256(source-id)%100",
+      "validation_percent": 2
+    },
+    "stage": "stage-a",
+    "tokenizer_json": "/data1/contributions/keras/hrm-text-1b-reference/tokenizer.json",
+    "tokenizer_revision": "d08ed34cc79b3c788d7b5d81399511439a0e39af",
+    "tokenizer_sha256": "ba520107dc9b27d25643e8f43fc1d9921038c86d3ada9dcaabf18526c5836d50",
+    "train_jsonl": "train.jsonl",
+    "train_sha256": "b6e340ea64568570b24e2d5f08c506a6f3221482d9fca6778a49c5577832171c",
+    "transform_version": "hrm-sft-v2",
+    "validation_jsonl": "validation.jsonl",
+    "validation_sha256": "f8c52067e80f5a7edb8be22a08cbc20de421ae6f368878f9bab64abe88a1bcc4"
+  },
+  "status": "preflight-passed"
+}

training/tensorboard/train/events.out.tfevents.1784941536.ccdf07020f90.51.0.v2

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:524387a608d8a3f2e29f1262f5a8dcc6063d6e3bc6dc956aa4f84f6d0f33ee12
+size 932008

training/tensorboard/validation/events.out.tfevents.1784994496.ccdf07020f90.51.1.v2

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f3c28f06dc827ca97d96dd544759c46d1ad6ad53b6fa99a17286a80246f2223
+size 78

training/training-plan.json

@@ -0,0 +1,14 @@
+{
+  "gradient_accumulation_steps": 8,
+  "micro_batch_size": 1,
+  "microbatch_steps": 38248,
+  "optimizer_steps": 4781,
+  "train": {
+    "requested_response_tokens": 10000000,
+    "selected_response_tokens": 10000147,
+    "selected_rows": 38248,
+    "selected_serialized_tokens": 19650833,
+    "shuffle_cycles_touched": 1,
+    "source_rows": 1133817
+  }
+}