Instructions to use wowbager/granite-cermat-cz-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use wowbager/granite-cermat-cz-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="wowbager/granite-cermat-cz-7b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("wowbager/granite-cermat-cz-7b")
model = AutoModelForCausalLM.from_pretrained("wowbager/granite-cermat-cz-7b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use wowbager/granite-cermat-cz-7b with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf wowbager/granite-cermat-cz-7b:F16
# Run inference directly in the terminal:
llama cli -hf wowbager/granite-cermat-cz-7b:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf wowbager/granite-cermat-cz-7b:F16
# Run inference directly in the terminal:
llama cli -hf wowbager/granite-cermat-cz-7b:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf wowbager/granite-cermat-cz-7b:F16
# Run inference directly in the terminal:
./llama-cli -hf wowbager/granite-cermat-cz-7b:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf wowbager/granite-cermat-cz-7b:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf wowbager/granite-cermat-cz-7b:F16

Use Docker

docker model run hf.co/wowbager/granite-cermat-cz-7b:F16

LM Studio
Jan

vLLM

How to use wowbager/granite-cermat-cz-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "wowbager/granite-cermat-cz-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wowbager/granite-cermat-cz-7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/wowbager/granite-cermat-cz-7b:F16

SGLang

How to use wowbager/granite-cermat-cz-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "wowbager/granite-cermat-cz-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wowbager/granite-cermat-cz-7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "wowbager/granite-cermat-cz-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wowbager/granite-cermat-cz-7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use wowbager/granite-cermat-cz-7b with Ollama:
```
ollama run hf.co/wowbager/granite-cermat-cz-7b:F16
```
Unsloth Desktop

How to use wowbager/granite-cermat-cz-7b with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf wowbager/granite-cermat-cz-7b:F16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "wowbager/granite-cermat-cz-7b:F16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use wowbager/granite-cermat-cz-7b with Docker Model Runner:
```
docker model run hf.co/wowbager/granite-cermat-cz-7b:F16
```

Lemonade

How to use wowbager/granite-cermat-cz-7b with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull wowbager/granite-cermat-cz-7b:F16

Run and chat with the model

lemonade run user.granite-cermat-cz-7b-F16

List all available models

lemonade list

Hermes Agent

How to use wowbager/granite-cermat-cz-7b with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf wowbager/granite-cermat-cz-7b:F16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default wowbager/granite-cermat-cz-7b:F16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use wowbager/granite-cermat-cz-7b with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf wowbager/granite-cermat-cz-7b:F16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "wowbager/granite-cermat-cz-7b:F16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

See axolotl config

axolotl version: 0.13.0.dev0

base_model: ibm-granite/granite-4.0-tiny-preview
trust_remote_code: true

datasets:
  - path: cermat_all.jsonl # combination of all three czech cermat datasets 
    type: alpaca

dataset_prepared_path: last_run_prepared
val_set_size: 0.05

output_dir: ./outputs/granite-cermat-7b

sequence_len: 4096
sample_packing: true

micro_batch_size: 1
gradient_accumulation_steps: 8
num_epochs: 1

optimizer: adamw_bnb_8bit
learning_rate: 2e-4
lr_scheduler: cosine
warmup_ratio: 0.05

bf16: auto
tf32: false

gradient_checkpointing: true
flash_attention: true

logging_steps: 10
evals_per_epoch: 1
saves_per_epoch: 1

resume_from_checkpoint:

DO NOT USE THIS MODEL FOR OTHER PURPOSES THAN EXPERIMENTAL TESTING

granite-cermat-cz-7b

TL;DR
Experimental full-parameter fine-tune of IBM Granite (7B, “Tiny” preview variant) on Czech CERMAT exam data (open + MC + TF). Shows strong format adherence only when prompted exactly like during training; otherwise overfits heavily. Not recommended for general use (or any use at all).

Quick Benchmark (unseen CERMAT test, 48 questions)

The benchmark ran each question ten times with the original prompt (used during training) and five times with five different prompt formulations.

Strict Format = exact match of the expected answer schema (no extra text or deviations).

Original prompt performance

Model	Strict Format	Exact Accuracy
granite-cermat-7b	62.5%	37.5%
granite-base	18.8%	45.8%

Varied prompt performance

Model	Strict Format	Exact Accuracy
granite-cermat-7b	0.0%	33.8%
granite-base	40.0%	40.0%

Conclusion: The model learned the short-answer format well for the trained prompt style but lost generalization and some accuracy due to full fine-tuning on limited data. Base Granite performs better overall.

For more reliable Czech tasks, use the original Granite model.

Model description

This model is a full-parameter fine-tune of IBM Granite (7B) on Czech CERMAT exam data.
It strongly overfits to the training prompt structure and should be viewed as an experiment demonstrating prompt-specific format learning rather than a generally useful language model.

Intended uses & limitations

Intended use

Experimental analysis of full fine-tuning effects on small Czech datasets
Studying prompt overfitting and format memorization

Not intended for

General Czech language tasks
Instruction-following or chat
Any production or evaluation use

The model is extremely sensitive to prompt wording and fails to generalize beyond the training format.

Training and evaluation data

Training data consists of combined Czech CERMAT datasets (open-ended, multiple choice, and true/false) from CZLC/cermat_czech_*.

Evaluation was performed on an unseen CERMAT test set of 48 questions, using both the original training prompt and multiple alternative prompt formulations to measure robustness.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 1
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 8
total_train_batch_size: 8
optimizer: AdamW (bitsandbytes), betas=(0.9, 0.999), epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 2
training_steps: 38

Framework versions

Transformers 4.57.1
PyTorch 2.8.0+cu128
Datasets 4.4.1
Tokenizers 0.22.1

Downloads last month: 38

Safetensors

Model size

7B params

Tensor type

BF16

Model tree for wowbager/granite-cermat-cz-7b

Base model

ibm-granite/granite-4.0-tiny-base-preview

Finetuned

ibm-granite/granite-4.0-tiny-preview

Quantized

(10)

this model

Quantizations

3 models

wowbager
/

granite-cermat-cz-7b

granite-cermat-cz-7b

Quick Benchmark (unseen CERMAT test, 48 questions)

Original prompt performance

Varied prompt performance

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Framework versions

Model tree for wowbager/granite-cermat-cz-7b

Datasets used to train wowbager/granite-cermat-cz-7b