Flux.1 Dev NF4 QLoRA
Collection
29 items • Updated • 2




All files are also archived in https://github.com/je-suis-tm/huggingface-archive in case this gets censored.
The QLoRA fine-tuning process of monica_bellucci_lora_flux_nf4 takes inspiration from this post (https://huggingface.co/blog/flux-qlora). The training was executed on a local computer with the same parameters as the link mentioned above, which took around 14 hours on 8GB VRAM 4060. The peak VRAM usage was around 7.7GB. To avoid running low on VRAM, both transformers and text_encoder were quantized. All the images generated here are using the below parameters
import torch
from diffusers import FluxPipeline, FluxTransformer2DModel
from transformers import T5EncoderModel
text_encoder_4bit = T5EncoderModel.from_pretrained(
"hf-internal-testing/flux.1-dev-nf4-pkg", subfolder="text_encoder_2",torch_dtype=torch.float16,)
transformer_4bit = FluxTransformer2DModel.from_pretrained(
"hf-internal-testing/flux.1-dev-nf4-pkg", subfolder="transformer",torch_dtype=torch.float16,)
pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.float16,
transformer=transformer_4bit,text_encoder_2=text_encoder_4bit)
pipe.load_lora_weights("je-suis-tm/monica_bellucci_lora_flux_nf4",
weight_name='pytorch_lora_weights.safetensors')
prompt="Monica Bellucci with blond bobbed hairstyle. wearing a fashion suit and gold neckless. In a Parisian street at night. she is smoking and looking at the camera. shot on polaroid film with front flash light. low angle view --ar 2:3 --style raw --profile o6rydcy --stylize 0 --v 6.1"
image = pipe(
prompt,
height=512,
width=512,
guidance_scale=5,
num_inference_steps=20,
max_sequence_length=512,
generator=torch.Generator("cpu").manual_seed(0),
).images[0]
image.save("monica_bellucci_lora_flux_nf4.png")
You should use Monica Bellucci to trigger the image generation.
Download them in the Files & versions tab.
Base model
black-forest-labs/FLUX.1-dev