LahjaMT — Context-Aware English → Dialectal-Arabic MT (LoRA experts)

LahjaMT translates English dialogue into thirteen country-level Arabic varieties (EG, JO, LB, LY, MA, MR, OM, PS, SA, SD, SY, TN, YE). It adapts UBC-NLP/NileChat-3B-Base (a Qwen2.5-3B continuation) with LoRA, conditions on dialogue history and metadata, and routes each dialect to its best checkpoint–prompt expert.

This repository hosts the trained LoRA adapters, the routing table, the prompt template, and a minimal inference script. It is the model release accompanying our AlexandriaX-2026 shared-task paper.

You only choose a dialect. Each dialect is already mapped to its best expert (this is the full-power system, including the Libyan/Sudanese specialists) — there is no "track" to pick.

Fastest way to try it: open LahjaMT_demo.ipynb in Colab (GPU runtime) and call translate(text, dialect).

• 💻 Code / experiments: https://github.com/m-abdallah98/LahjaMT
• 📄 Paper: LahjaMT at AlexandriaX-2026: A Context-Aware English-to-Dialectal Arabic MT System with Lightweight Routing of LoRA Experts

What's in this repo

adapter_config.json     # default adapter at the repo ROOT (= stage2_step5200); makes
adapter_model.safetensors # `PeftModel.from_pretrained(base, "MohamedAbdallah98/LahjaMT")` and the
                          # "Use this model" button work out of the box (general checkpoint)
adapters/
  stage1_parent/       # Stage-1 parent adapter (best dev checkpoint)
  stage2_step1600/     # Stage-2 continuation checkpoints (the retained experts)
  stage2_step2000/
  stage2_step4900/
  stage2_step5200/     # default checkpoint for most dialects
  stage2_step6400/
  stage2_step7600/
  specialist_LY/       # Libyan SMOL specialist  (best LY route)
  specialist_SD/       # Sudanese SMOL specialist (best SD route)
routing.json           # per-dialect route: which adapter + which prompt config
prompt_template.txt    # the instruction block used for every example
inference_example.py   # translate(text, dialect) helper (PEFT over NileChat-3B-Base)
LahjaMT_demo.ipynb     # one-click Colab demo
tokenizer.json + config # shared tokenizer (carries the training-style marker token)

Each adapter is LoRA with r=16, α=32, dropout 0.05, applied to all seven projection modules of every Transformer block (~29.9M params, ~114 MB). The repo root holds a copy of the default stage2_step5200 adapter so generic loaders and the HF "Use this model" snippet resolve to a working general checkpoint; for the best per-dialect quality, select the routed adapter via routing.json.

Dialect routing (best expert per dialect)

At inference, the target dialect label deterministically selects a (checkpoint, prompt) expert from routing.json. Scores are held-out spBLEU / chrF++.

Variety	Adapter	Prompt	spBLEU	chrF++
EG	stage2_step5200	P3	31.88	45.60
JO	stage2_step5200	P1	35.50	49.12
LB	stage2_step5200	P1	32.29	46.00
LY	specialist_LY	P3	23.86	39.14
MA	stage2_step5200	P2	23.31	39.77
MR	stage2_step2000	P1	17.94	34.34
OM	stage2_step4900	P1	32.57	47.11
PS	stage2_step5200	P1	34.26	48.26
SA	stage2_step1600	P2	35.25	49.78
SD	specialist_SD	P1	27.43	42.06
SY	stage2_step5200	P3	39.38	53.19
TN	stage2_step7600	P1	35.23	47.61
YE	stage2_step4900	P1	25.23	41.71
Macro			30.32	44.90

Prompt configurations:

• P1 — two deterministic same-country / same-domain demonstrations (matches the fine-tuning format).
• P2 — two semantically retrieved demonstrations (all-MiniLM-L6-v2 over the English source).
• P3 — retrieval plus persona / participant-role metadata.

P2 and P3 require a demonstration pool (the official training/dev/continuation splits) and the MiniLM encoder for retrieval; those are not shipped here. Use P1 for a self-contained run, or the code repo for the full retrieval pipeline.

Quick start

The base model is gated — accept the terms once at UBC-NLP/NileChat-3B-Base and log in (huggingface-cli login) before running.

import json, torch
from huggingface_hub import hf_hub_download
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

REPO, BASE = "MohamedAbdallah98/LahjaMT", "UBC-NLP/NileChat-3B-Base"
TARGET = "EG"   # any of: EG JO LB LY MA MR OM PS SA SD SY TN YE

routing = json.load(open(hf_hub_download(REPO, "routing.json")))
route, gen = routing["routes"][TARGET], routing["generation"]

tok = AutoTokenizer.from_pretrained(REPO)   # shipped tokenizer (with training marker)
base = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, REPO, subfolder=f"adapters/{route['adapter']}").eval()

prompt = "...instruction block for your turn (see inference_example.py / prompt_template.txt)..."
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, num_beams=gen["num_beams"], do_sample=False,
                     length_penalty=gen["length_penalty"], repetition_penalty=gen["repetition_penalty"],
                     max_new_tokens=gen["max_new_tokens"], early_stopping=True)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

See inference_example.py for a runnable end-to-end example including the prompt builder and the Latin-script post-filter.

Shared-task results

On the AlexandriaX-2026 private test (macro-averaged over 13 dialects), LahjaMT ranked 2nd in the constrained track (28.30 spBLEU / 43.81 chrF++) and 3rd in the unconstrained track (28.54 / 44.02). The model published here is the full-power system (the best expert per dialect, i.e. the unconstrained configuration), and outperformed our own inference-only setups built on the far larger gpt-oss-20b and gpt-oss-120b.

Reproducibility. To reproduce the paper's data-constrained submission (no external SMOL data), override the two specialist routes in routing.json: LY → stage2_step6400 + P3 and SD → stage2_step5200 + P1. Everything else is identical.

Training summary

• Backbone: NileChat-3B-Base (Qwen2.5-3B continuation), frozen.
• LoRA: r=16, α=32, dropout 0.05, all 7 projections; BF16 on a single NVIDIA RTX 5090.
• Stage 1: 66,480 turns, 3 epochs, LR 2e-4, cosine, warm-up 0.03, weight decay 0.01, eff. batch 8, max len 2,048.
• Stage 2: continue from Stage-1 parent on 20,920 turns (dev + 60% public-test), 3 epochs, LR 2e-5; six checkpoints retained.
• Specialists (LY/SD): SMOL ayl/apd pairs + replayed in-domain turns, 2 epochs, LR 5e-6.

Intended use & limitations

Research use, consistent with the base model's license. Built for English→dialectal-Arabic dialogue translation with conversational context; not evaluated for other language pairs, long documents, or production safety-critical settings. Quality varies by dialect (see the table); low-resource varieties (MR, MA, LY) score lowest.

License

The adapters are released under the base model's terms: Qwen Research License (qwen-research), via NileChat-3B-Base. This is a non-commercial research license — review it before use. You must comply with the licenses of NileChat-3B-Base and Qwen2.5-3B when loading the base weights.

Citation

@inproceedings{abdallah2026lahjamt,
  title     = {{LahjaMT} at {AlexandriaX-2026}: A Context-Aware English-to-Dialectal Arabic Machine Translation System with Lightweight Routing of {LoRA} Experts},
  author    = {Abdallah, Mohamed A. and El-Beltagy, Samhaa R.},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference: Shared Tasks},
  year      = {2026},
  address    = {Budapest, Hungary},
  publisher = {Association for Computational Linguistics}
}

Acknowledgments

We gratefully acknowledge the Nawy AI lab for the computational resources that supported this work.