Instructions to use AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16")
model = AutoModelForMultimodalLM.from_pretrained("AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16

SGLang

How to use AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16 with Docker Model Runner:
```
docker model run hf.co/AutomatosX/AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16
```

AX-Clef-Flash-9B-CUDA-AXQ-NVFP4-W4A16

Native AXQuant RTN NVFP4 W4A16 development checkpoint. Eligible language attention and MLP matrices use E2M1 weights, E4M3FN block scales of 16, and an FP32 inverse global scale. Activations stay BF16. There is no AWQ and no activation quantization. The checkpoint uses the public compressed-tensors NVFP4 layout.

Source: Cloudflare/clef-flash at 8b2e5fd17c09fd49bc2880b805d5323ec7ff4ff3. Serialization backend: torch-cuda.

Decision model

Clef is not a chat model. A request is a state plus a schema of typed questions (noul, choice, score). One prefill returns a probability for every allowed option. joint_head.safetensors is the original BF16 joint schema head, copied unchanged, with joint_schema_model.py and joint_head_config.json. Text generation does not run that head.

Precision

Vision, norms, token embeddings, and the LM head stay at source precision. The joint schema head is not part of the language index and is not quantized. This package is development evidence. It is not a Decision Index certificate and it does not claim parity with the upstream BF16 checkpoint.

The upstream license is Apache-2.0. See LICENSE and NOTICE.