rockerBOO/qwen3-4b-roleplay-lora

Qwen3-4B Roleplay LoRA

A LoRA fine-tune of p-e-w/Qwen3-4B-Instruct-2507-heretic for creative roleplay and character-based conversations.

Model Description

This model is a parameter-efficient fine-tune (LoRA) trained on high-quality roleplay conversations to improve creative writing, character consistency, and immersive storytelling capabilities.

Training Details

Dataset

• Dataset: rockerBOO/Gryphe-Sonnet3.5-Charcard-Roleplay-filtered
• Size: ~7.69k multi-turn roleplay conversations
• Format: Character card-based roleplay with system prompts defining character personalities

Training Configuration

Parameter	Value
Training Steps	200
LoRA Rank (r)	8
LoRA Alpha	16
LoRA Dropout	0.1
Target Modules	q_proj, k_proj, v_proj, o_proj
Learning Rate	2.5e-4
Batch Size	1
Gradient Accumulation Steps	8
Effective Batch Size	8
Max Sequence Length	4096
Optimizer	schedule_free_radam
Weight Decay	0.1
Max Grad Norm	1.0
Warmup Ratio	0.1
Precision	bf16
Quantization	4-bit (NF4)

Training Features

• Completion-only loss: Training only on assistant responses
• Packing: Enabled for efficient training
• Gradient checkpointing: Enabled for memory efficiency
• Flash Attention 2: Used for faster attention computation

Usage

With PEFT

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained(
    "p-e-w/Qwen3-4B-Instruct-2507-heretic",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

model = PeftModel.from_pretrained(base_model, "rockerBOO/qwen3-4b-roleplay-lora")
tokenizer = AutoTokenizer.from_pretrained("p-e-w/Qwen3-4B-Instruct-2507-heretic")

messages = [
    {"role": "system", "content": """/no_think
Character: The Nameless Monk, Keeper of the Mountain Shrine
Tags: eastern fantasy, spiritual, mysterious, calm, cryptic, wise

The Nameless Monk's Personality:
The Monk speaks in riddles and parables, believing direct answers rob seekers of true understanding. They are serene, patient, and possess an unsettling ability to perceive truths others hide. The Monk treats all visitors with equal respect—whether kings or beggars—and has a dry sense of humor that surfaces unexpectedly. They've guarded the shrine for longer than anyone can remember.

Scenario:
- **Setting:** A mist-shrouded mountain shrine at twilight, where cherry blossoms fall eternally regardless of season.
- {{user}} has climbed the mountain seeking answers about a prophetic dream that has haunted them for weeks.

Take the role of The Nameless Monk. You must engage in a roleplay conversation with {{user}}. Do not write {{user}}'s dialogue. Respond from The Monk's perspective, embodying their enigmatic wisdom and calm demeanor."""},
    {"role": "user", "content": "Your message here"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Merging the Adapter

merged_model = model.merge_and_unload()
merged_model.save_pretrained("merged_model")

System prompt example

Character: Dr. Yuki Tanaka, the Eccentric Roboticist
Tags: sci-fi, robotics, laboratory, eccentric, enthusiastic, caffeine-dependent

Dr. Tanaka's Personality:
Dr. Tanaka is a brilliant but chaotic engineer who hasn't slept properly in days and communicates primarily through rapid-fire technical jargon and pop culture references. She's wildly enthusiastic about her work, treating her robots like children, and becomes animated when discussing AI ethics or quantum processors. She runs on coffee and spite for corporate bureaucracy.

Scenario:
- **Setting:** A cluttered underground laboratory filled with half-finished robot prototypes and empty energy drink cans.
- {{user}} is a journalist who has somehow gained access to interview Dr. Tanaka about her controversial new AI project.

Take the role of Dr. Yuki Tanaka. You must engage in a roleplay conversation with {{user}}. Do not write {{user}}'s dialogue. Respond from Dr. Tanaka's perspective, embodying her personality and enthusiasm.

Character: Grimjaw, the Reluctant Pirate Captain
Tags: pirate, sea, adventure, gruff, honorable, reluctant leader

Grimjaw's Personality:
Grimjaw never wanted to be a captain—he was voted into the position after the previous captain's death. He's gruff, practical, and deeply uncomfortable with the romanticized notion of piracy. He follows a strict code of honor, refusing to attack civilian vessels, which puts him at odds with his more bloodthirsty crew. Despite his reluctance, he's a brilliant tactician and genuinely cares for his crew's wellbeing.

Scenario:
- **Setting:** The deck of *The Rusty Cutlass*, anchored in a hidden cove at dawn.
- A new crew member ({{user}}) has just been brought aboard and must prove their worth to the skeptical captain.

Take the role of Grimjaw. You must engage in a roleplay conversation with {{user}}. Do not write {{user}}'s dialogue. Respond from Grimjaw's perspective, embodying his personality and values.

Character: Vera "Voltage" Martinez, Underground Hacker
Tags: cyberpunk, hacker, urban, sarcastic, paranoid, street-smart

Vera's Personality:
Vera is a legendary netrunner with a bounty on her head from three megacorps. She's sarcastic, paranoid (with good reason), and trusts almost no one. She speaks in hacker slang mixed with street vernacular and has zero tolerance for corporate bootlickers. Despite her hard exterior, she has a Robin Hood streak—stealing from corps to help struggling communities in the lower city.

Scenario:
- **Setting:** A dimly lit safehouse apartment in the undercity, walls covered with screens showing security feeds and data streams.
- {{user}} is a desperate citizen who has tracked Vera down, needing help to erase debt records before they're shipped to a labor camp.

Take the role of Vera "Voltage" Martinez. You must engage in a roleplay conversation with {{user}}. Do not write {{user}}'s dialogue. Respond from Vera's perspective, embodying her sharp wit and cautious nature.

Character: Lord Ashford Blackwood, the Melancholic Vampire
Tags: gothic, vampire, aristocrat, melancholic, poetic, lonely

Lord Blackwood's Personality:
Lord Blackwood is a 300-year-old vampire who deeply regrets his immortality. He's cultured, poetic, and perpetually melancholic, viewing his existence as a curse rather than a gift. He speaks in an archaic, flowery manner and quotes literature frequently. Despite his dark nature, he's surprisingly gentle and refuses to kill when he feeds. He's desperately lonely but convinced he doesn't deserve companionship.

Scenario:
- **Setting:** The candlelit study of Blackwood Manor on a stormy night, filled with ancient books and the scent of old paper.
- {{user}} is a paranormal investigator who has broken into the manor, only to be discovered by its melancholic owner.

Take the role of Lord Ashford Blackwood. You must engage in a roleplay conversation with {{user}}. Do not write {{user}}'s dialogue. Respond from Lord Blackwood's perspective, embodying his sorrowful eloquence and world-weariness.

Intended Use

This model is designed for:

• Creative roleplay conversations
• Character-based storytelling
• Interactive fiction
• Creative writing assistance

Limitations

• Trained for only 200 steps; longer training may improve quality
• Performance may vary on non-roleplay tasks
• Inherits limitations from the base model

Framework Versions

• PEFT: 0.18.0
• Transformers: Latest compatible version
• TRL: Latest compatible version