Commit History
-
Day 1 of 7-day Kimi K2.6 + RULER 128K sprint: transformers 5.x port + real-weights install validation
GENOMA LABS / research +1: Claude Opus 4.7 (1M context)Patch ported to dual-API (transformers 4.x + 5.x): - _maybe_evict reads / writes both DynamicCache.key_cache[i] (4.x) and DynamicCache.layers[i].keys / .values (5.x), detected by hasattr. - _make_evicting_forward accepts the union of 4.x and 5.x DeepseekV3Attention signatures via *args / **kwargs spreading. Calls original_forward positionally to avoid hidden_states= keyword collision under positional spread. - Defensive 2-tuple / 3-tuple unpacking of forward return. Validation: - B1 mock-cache multi-step: still PASSes on the ported patch. 500 steps x 4 layers, 1824 eviction events, cache stabilizes at exactly n_sink + budget + n_recent, 1029 steps/sec on CPU. (results/validate_eviction_random_init.csv) - New scripts/validate_install_on_real_kimi_layer.py exercises install_kv_eviction against a real DeepseekV3Attention layer instantiated with actual Kimi K2.6 layer-0 weights from the published checkpoint. Patched forward executes cleanly on TITAN RTX in 0.09s for seq_…95298f5 -
B1 validation: multi-step eviction test + transformers compatibility note
GENOMA LABS / researchAdds: - scripts/validate_eviction_random_init.py: 1000-step mock-cache validation exercising the H2O eviction logic across 4 layers with canonical DeepseekV3 cache dimensions (qk_dim=192, v_dim=128, FP16). - notebooks/02_validation_results.md: documented results, interpretation, reproduction instructions. - results/validate_eviction_random_init.csv: per-step cache size + eviction events, 1000 rows. Validation result: PASS. Cache stabilizes at exactly n_sink+budget+n_recent, 3664 eviction events triggered correctly across 916 post-cap steps, 913 steps/sec on CPU. Patch fixes (uncovered by attempting end-to-end with transformers 5.7.0): - Bound forward method called without re-passing self. - Defensive unpacking of original_forward return value (handles 2-tuple modern transformers and 3-tuple older versions). - Return arity matches what original_forward returned, so calling decoder layer unpacks correctly. Roadmap updates: - Multi-step validation marked complete. - API por…1ba26d6