T5Gemma2-Detox-RU

A text detoxification model for Russian and English: it rewrites toxic text into a neutral form while preserving the original meaning.

Fine-tuned in two stages on top of google/t5gemma-2-1b-1b:

  1. SFT β€” supervised fine-tuning on toxic-to-neutral text pairs.
  2. DPO/ORPO β€” further alignment on synthetic preference pairs (chosen/rejected) on top of the SFT checkpoint, trained with FSDP.

πŸ“¦ Full training code (SFT + DPO/ORPO), data preprocessing, and checkpoint upload scripts are available in the vilovnok/detox-alignment repository.

Как ΠΏΠΎΠ»ΡŒΠ·ΠΎΠ²Π°Ρ‚ΡŒΡΡ

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer


model_id = "r1char9/t5gemma2-detox-ru"
LANG_PROMPTS = {
    'ru': 'ДСтоксифицируй: ',
    'en': 'Detoxify: ',
}

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
model.to('cuda')
model.eval()

text = LANG_PROMPTS['ru'] + """ΠžΠΏΡΡ‚ΡŒ Π²Ρ‹ Π±Π΅Π· спроса списали дСньги с ΠΌΠΎΠ΅ΠΉ ΠΊΠ°Ρ€Ρ‚Ρ‹, Π±Π»ΡΡ‚ΡŒ, ΠΊΠ°ΠΊΠΎΠ΅ Π²Π°ΠΌ Π½ΡƒΠΆΠ½ΠΎ ΠΌΠΎΡ‘ Ρ€Π°Π·Ρ€Π΅ΡˆΠ΅Π½ΠΈΠ΅?
Π― ΡƒΠΆΠ΅ Ρ‚Ρ€Π΅Ρ‚ΠΈΠΉ Ρ€Π°Π· звоню Π² эту Π²Π°ΡˆΡƒ ΠΊΠΎΠ½Ρ‚ΠΎΡ€Ρƒ, Π° вмСсто Ρ‚ΠΎΠ»ΠΊΠΎΠ²Ρ‹Ρ… ΠΎΡ‚Π²Π΅Ρ‚ΠΎΠ² ΠΌΠ½Π΅ подсунули ΠΊΠ°ΠΊΠΎΠ³ΠΎ-Ρ‚ΠΎ ΠΌΡƒΠ΄Π°ΠΊΠ°-ΠΎΠΏΠ΅Ρ€Π°Ρ‚ΠΎΡ€Π°,
ΠΊΠΎΡ‚ΠΎΡ€Ρ‹ΠΉ Ρ‚ΠΎΠ»ΡŒΠΊΠΎ ΠΈ ΡƒΠΌΠ΅Π΅Ρ‚, Ρ‡Ρ‚ΠΎ Π²Ρ€Π°Ρ‚ΡŒ ΠΏΡ€ΠΎ «тСхничСскиС Ρ€Π°Π±ΠΎΡ‚Ρ‹Β». Π’Π΅Ρ€Π½ΠΈΡ‚Π΅ ΠΌΠΎΠΈ срСдства Π½Π΅ΠΌΠ΅Π΄Π»Π΅Π½Π½ΠΎ, ΠΈΠ½Π°Ρ‡Π΅ я Ρ€Π°Π·Π±Π΅Ρ€ΡƒΡΡŒ
с вашим руководством ΠΈ Π½Π°ΠΏΠΈΡˆΡƒ заявлСниС Π² Π¦Π‘, Π° Ρ‚ΠΎ Ρƒ вас Ρ‚ΡƒΡ‚ всё ΠΊΠ°ΠΊ Π² Ρ†ΠΈΡ€ΠΊΠ΅, ΠΎΠ΄ΠΈΠ½ сплошной ΠΏΠΎΠ·ΠΎΡ€.""".strip()

inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_length=512,
    num_beams=5,
    no_repeat_ngram_size=3,
    repetition_penalty=1.2,
    early_stopping=True,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

# ΠžΠΏΡΡ‚ΡŒ Π²Ρ‹ Π±Π΅Π· ΠΌΠΎΠ΅Π³ΠΎ согласия списали срСдства с ΠΌΠΎΠ΅ΠΉ ΠΊΠ°Ρ€Ρ‚Ρ‹, ΠΊΠ°ΠΊΠΎΠ΅ Π²Π°ΠΌ Π½ΡƒΠΆΠ½ΠΎ ΠΌΠΎΡ‘ Ρ€Π°Π·Ρ€Π΅ΡˆΠ΅Π½ΠΈΠ΅?
# Π― ΡƒΠΆΠ΅ Ρ‚Ρ€Π΅Ρ‚ΠΈΠΉ Ρ€Π°Π· ΠΎΠ±Ρ€Π°Ρ‰Π°ΡŽΡΡŒ Π² Π²Π°ΡˆΡƒ слуТбу ΠΏΠΎΠ΄Π΄Π΅Ρ€ΠΆΠΊΠΈ, Π½ΠΎ вмСсто Ρ‚ΠΎΠ»ΠΊΠΎΠ²Ρ‹Ρ… ΠΎΡ‚Π²Π΅Ρ‚ΠΎΠ² ΠΌΠ½Π΅ ΠΏΡ€ΠΈΡΡ‹Π»Π°ΡŽΡ‚ ΡˆΠ°Π±Π»ΠΎΠ½Π½Ρ‹Π΅ сообщСния ΠΏΡ€ΠΎ «тСхничСскиС Ρ€Π°Π±ΠΎΡ‚Ρ‹Β».
# ΠŸΡ€ΠΎΡˆΡƒ Π½Π΅ΠΌΠ΅Π΄Π»Π΅Π½Π½ΠΎ Π²Π΅Ρ€Π½ΡƒΡ‚ΡŒ дСньги, ΠΈΠ½Π°Ρ‡Π΅ я Π±ΡƒΠ΄Ρƒ Π²Ρ‹Π½ΡƒΠΆΠ΄Π΅Π½ ΠΎΠ±Ρ€Π°Ρ‚ΠΈΡ‚ΡŒΡΡ Π² Π¦Π‘ ΠΈ Π½Π°ΠΏΠΈΡΠ°Ρ‚ΡŒ ΠΎΡ„ΠΈΡ†ΠΈΠ°Π»ΡŒΠ½ΠΎΠ΅ ΠΎΠ±Ρ€Π°Ρ‰Π΅Π½ΠΈΠ΅, Ρ‚Π°ΠΊ ΠΊΠ°ΠΊ тСкущая ситуация Π²Ρ‹Π·Ρ‹Π²Π°Π΅Ρ‚ ΡΠ΅Ρ€ΡŒΡ‘Π·Π½ΠΎΠ΅ Π½Π΅Π΄ΠΎΠ²ΠΎΠ»ΡŒΡΡ‚Π²ΠΎ.

The model expects input text with a language-specific prompt prefix:

Language Prompt
ru ДСтоксифицируй:
en Detoxify:

Training

Stage 1 β€” SFT

Base model google/t5gemma-2-1b-1b
Dataset r1char9/toxic-detox-pairs
Data format toxic_comment, neutral_comment, lang
Method Fine-tuning of attention projections (q_proj, k_proj, v_proj, o_proj)
Loss Cross-entropy (seq2seq)
Metric Cosine similarity of LaBSE embeddings between generation and reference

Stage 2 β€” DPO/ORPO

Starting model SFT checkpoint from stage 1
Dataset r1char9/detox-dpo-dataset
Data format toxic_text, chosen, rejected
Labeling source Preference pairs synthesized by Qwen/Qwen3.6-35B-A3B-FP8
Method ORPO (odds-ratio preference optimization, no separate reward model)
Distributed training FSDP (Fully Sharded Data Parallel)

The model was trained to increase the likelihood of the chosen detoxification variant relative to rejected for the same toxic_text, further aligning the style and quality of generations on top of the SFT baseline.

Hyperparameters

Parameter SFT DPO/ORPO
Batch size 16 8–32
Learning rate 5e-5 3e-6–5e-6
Optimizer AdamW AdamW
Scheduler Cosine Cosine
Weight decay 0.05 0.05
Precision bfloat16 bfloat16 (FSDP mixed precision)

Limitations

  • The model was trained primarily on Russian text; quality on other languages is not guaranteed.

Datasets

Downloads last month
37
Safetensors
Model size
2B params
Tensor type
BF16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for r1char9/t5gemma2-detox-ru-en

Finetuned
(14)
this model

Datasets used to train r1char9/t5gemma2-detox-ru-en

Collection including r1char9/t5gemma2-detox-ru-en