Adnane10/NovaAdsAI

# Model Card for Falcon-RW-1B Fine-Tuned Model

This model is a fine-tuned version of tiiuae/falcon-rw-1b trained on an advertising-related dataset to generate ad text based on prompts.

Model Details

Model Description

This model is a fine-tuned version of the Falcon-RW-1B model, specifically adapted for generating advertising content. The fine-tuning process utilized a dataset containing ad-related text, formatted as structured prompt-response pairs.

• Developed by: Adnane Touiyate
• Funded by [optional]: Adnane10
• Shared by [optional]: Adnane10
• Model type: Falcon-RW-1B (Causal Language Model)
• Language(s) (NLP): English
• License: MIT
• Finetuned from model [optional]: tiiuae/falcon-rw-1b

Uses

Direct Use

This model can be used for generating advertising content based on structured prompts. It is useful for marketers and advertisers who need AI-generated ad copies.

Downstream Use [optional]

The model can be further fine-tuned for specific ad categories or integrated into larger marketing automation workflows.

Out-of-Scope Use

This model is not intended for generating non-advertising-related content, and its performance may be suboptimal in general text generation tasks beyond its training scope.

Bias, Risks, and Limitations

Since the model has been fine-tuned on advertising content, it may inherit biases present in the dataset. Users should be cautious when generating ads to ensure they meet ethical and regulatory standards.

Recommendations

Users should validate the generated content for appropriateness, compliance, and factual accuracy before using it in real-world applications.

How to Get Started with the Model

Use the code below to load and use the model:

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-rw-1b")
model = AutoModelForCausalLM.from_pretrained("path_to_finetuned_model")

def generate_ad(prompt):
    inputs = tokenizer(prompt, return_tensors="pt").to('cuda')
    outputs = model.generate(**inputs, max_length=100)
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

print(generate_ad("Introducing our latest product: "))

Training Details

Training Data

The model was trained on fixed_ads_list.json, a dataset containing structured ad-related prompts and responses.

Training Procedure

• Preprocessing: Tokenized text in the format ### Prompt: [User Input] ### Response: [Ad Text]
• Quantization: Used 4-bit quantization (NF4) with bitsandbytes for efficiency.
• Fine-tuning method: LoRA (Low-Rank Adaptation) for efficient adaptation.
• Hardware: GPU-accelerated training.

Training Hyperparameters

• Learning Rate: 1e-4
• Batch Size: 2 (per device)
• Gradient Accumulation: 8 steps
• Epochs: 6
• Precision: BF16
• Evaluation Strategy: Epoch-based
• Early Stopping: Enabled after 2 epochs without improvement

Evaluation

Testing Data, Factors & Metrics

• Metrics: BLEU and ROUGE scores
• Results: Sample evaluation showed:

Environmental Impact

• Hardware Type: NVIDIA P100 GPU
• Hours used: ~54 minutes
• Cloud Provider: Kaggle

Model Architecture and Objective

The Falcon-RW-1B model is a causal language model optimized for text generation.

Compute Infrastructure

Hardware

• GPUs (NVIDIA P100)
• Used bitsandbytes for memory-efficient training

Software

• transformers
• datasets
• peft
• torch
• accelerate
• bitsandbytes

Model Card Authors

Adnane Touiyate (@Adnane10)

Contact

For questions or collaborations, reach out via LinkedIn or email: adnanetouiayte11@gmail.com