CMF: new, memory-mappable format for quantized LLMs

Beginners
1

CMF: a single-file, memory-mappable format for quantized LLMs

Hi all,

I’ve been building CMF (Cortiq Model Format) single-file container for quantized models with a Rust runtime. Most of the converted models live on the Hub, so I wanted to share it here.

What one .cmf file holds

  • a fixed 128-byte envelope that addresses every section
  • header JSON: architecture, quantization defaults, chat template, provenance
  • a binary tensor directory with dtype, shape, offsets and a 64-bit hash per tensor
  • a page-aligned weight blob you can mmap and read in place
  • optional extras: tokenizer, task masks, and skill overlays (replacement tensors on top of one backbone)

Quantization is chosen per tensor: f16 / q8 / q8_2f for sensitive tensors, q4tp / q2tp for the large matrices. cortiq verify checks every section’s bounds and every tensor hash, and there is a small independent

Try it

cargo install cortiq-cli
cortiq convert --model Qwen/Qwen3-0.6B --quant q8 --output qwen.cmf
cortiq run qwen.cmf --prompt "What is the capital of France?" --greedy --no-think

Already have a GGUF: cortiq import-gguf model.gguf --output model.cmf. cortiq serve exposes an OpenAI-compatible API. It runs on CPU, and on GPU through Vulkan / DX12 (wgpu) or Metal.

On the Hub
53 converted models tagged cortiq: infosave (Oleg Kirichenko)

I’d welcome feedback on the format itself, especially from people who have written safetensors or GGUF loaders: what would you want in the header or tensor directory that isn’t there?

GitHub: GitHub - infosave2007/cmf: One backbone, many specialists. CMF is a self-describing, memory-mappable single-file format and a dependency-free runtime for quantized LLMs — zero-copy execution on CPU or GPU (Vulkan · Metal · DX12) that overlays task-specialized skills onto one shared model, with no per-model duplication. · GitHub