metadata
base_model: google/gemma-4-2b-it
language:
- sq
- en
license: apache-2.0
datasets:
- klei1/bleta-sq-dataset-v1
tags:
- text-generation
- gemma
- albanian
- language-model
- fine-tuned
model_name: Bleta SQ 2B
pipeline_tag: text-generation
Bleta SQ 2B — Albanian Language Model
Bleta is a fine-tuned Gemma-4 2B model specialized for the Albanian language (Shqip). Built to understand and generate natural, grammatically correct Albanian text.
Bleta (🐝) means "bee" in Albanian — a symbol of diligence and precision.
Model Details
| Property | Value |
|---|---|
| Base Model | google/gemma-4-2b-it |
| Architecture | Gemma4ForConditionalGeneration |
| Parameters | ~2 Billion |
| Fine-tuning Method | LoRA → merged into full weights |
| Language | Albanian (sq), English (en) |
| License | Apache 2.0 |
Training Dataset
Fine-tuned on klei1/bleta-sq-dataset-v1 — a curated Albanian language instruction dataset covering conversation, grammar, reasoning, and general knowledge.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "klei1/bleta-sq-2b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
messages = [
{"role": "user", "content": "Cila eshte kryeqyteti i Shqiperise?"}
]
inputs = tokenizer.apply_chat_template(
messages,
return_tensors="pt",
add_generation_prompt=True
).to(model.device)
outputs = model.generate(
inputs,
max_new_tokens=512,
temperature=0.7,
top_p=0.95,
repetition_penalty=1.2,
do_sample=True,
)
response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
print(response)
Recommended Generation Parameters
| Parameter | Value | Notes |
|---|---|---|
temperature |
0.7 | Balanced creativity |
max_new_tokens |
400–512 | Prevents loops |
repetition_penalty |
1.2 | Reduces repetition |
top_p |
0.95 | Nucleus sampling |
Capabilities
- Albanian conversational AI
- Grammar correction and explanation
- Albanian text generation and creative writing
- Translation (Albanian ↔ English)
- General knowledge in Albanian
- Question answering
Limitations
- 2B parameter model — complex reasoning may be limited
- Primarily trained on Albanian; performance varies by topic
- May occasionally produce grammatically imperfect outputs
Bleta Model Family
| Model | Params | Focus |
|---|---|---|
| bleta-sq-2b | 2B | Albanian general |
| bleta-meditor-27b | 27B | Medical + specialized |
| bleta-logjike-27b | 27B | Logic + reasoning |
| bleta-1B | 1B | Lightweight |
Citation
@model{bleta_sq_2b_2026,
title = {Bleta SQ 2B: Gemma-4 Fine-tuned for Albanian Language},
author = {klei1},
year = {2026},
url = {https://huggingface.co/klei1/bleta-sq-2b}
}
License
This model is released under the Apache 2.0 License.