Model Card for Sandeep4235/llama3-8b-Wisdom-gguf

Model Details

Model Description

This model, named llama3-8b-Wisdom-gguf, is a fine-tuned version of the Llama-3 8B base model (specifically unsloth/llama-3-8b-bnb-4bit) using the Unsloth framework. Its core purpose is to provide insightful and reflective responses rooted in ancient spiritual, mystic, and yogic traditions.

This iteration has undergone extended fine-tuning for 251 steps and incorporates advanced inference parameters (temperature=0.85, repetition_penalty=1.3, top_p=0.95, do_sample=True) along with robust post-processing. These enhancements aim to deliver more concise, context-specific, and non-repetitive answers, directly addressing previous output quality concerns such as excessive verbosity and unintended conversational loops. The model is designed to bridge deep existential insights with modern conceptual understanding, offering clear guidance on spiritual pathways, mindfulness, and inner inquiry.

• Developed by: Sandeep Hipparagi
• Model type: Causal Language Model (Llama-3 8B, GGUF Quantized)
• Language(s) (NLP): English
• License: Llama-3.1
• Finetuned from model: unsloth/llama-3-8b-bnb-4bit

Model Sources

• Repository: https://huggingface.co/Sandeep4235/llama3-8b-Wisdom-gguf

Uses

This model is primarily intended for applications requiring philosophical exploration, spiritual guidance, contextual insight generation, and reflective conversational AI. It excels in queries related to spiritual wisdom, yogic science, and inner inquiry.

Direct Use

For optimal performance and to receive the most accurate and context-specific responses, users must adhere to the following instruction-based prompt format:

### Instruction: {Your Instruction/Question Here}

### [Internal Thought]: Analyzing intent.

### [Author Response]:

Out-of-Scope Use

This model is not designed or validated for, and should not be used for:

• Providing medical, psychiatric, legal, or crisis intervention advice.
• Generating factual information where absolute accuracy is critical without external verification (e.g., precise historical dates, scientific facts, financial advice).
• Generating content that could be interpreted as harmful, discriminatory, or unethical, or that justifies such actions or beliefs.

Bias, Risks, and Limitations

Interpretive Scope

The model's responses are derived from its training on philosophical and contemplative literature. Its interpretations are inherently limited by this data and may not encompass the full breadth of human thought, cultural nuances, or scientific understanding. It should serve as an informational and reflective tool, not a definitive authority or replacement for professional expertise.

Factual Verification

As with all large language models, the potential for generating plausible but incorrect or hallucinated information exists. Users are strongly advised to verify any specific historical dates, scriptural citations, technical terminology, or other factual claims when accuracy is paramount.

Recommendations

Users are encouraged to approach the model's outputs with critical thinking. Responses should be seen as a starting point for reflection or further research rather than definitive statements. Continuous monitoring of model outputs is recommended to identify and mitigate any emerging biases, misinterpretations, or safety concerns.

How to Get Started with the Model

To interact with the fine-tuned model, you can use llama-cpp-python or Hugging Face transformers library. Ensure you load the model with the adapter and apply the specified inference parameters and the strict prompt template.

Using llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="Sandeep4235/llama3-8b-Wisdom-gguf",
    filename="llama-3-8b.Q4_K_M.gguf", # Or other quantization if available
    n_ctx=4096, # Adjust context window as needed
    # ... other Llama.cpp parameters
)

# Construct the prompt as specified in 'Direct Use' section
formatted_prompt = (
    "### Instruction: How can one maintain internal clarity amidst daily stress?\n\n"
    "### [Internal Thought]: Analyzing intent.\n\n"
    "### [Author Response]:"
)

response = llm.create_chat_completion(
    messages=[
        {
            "role": "system",
            "content": "You are a wise guide versed in ancient philosophy, yogic sciences, and inner inquiry."
        },
        {
            "role": "user",
            "content": formatted_prompt
        }
    ],
    temperature=0.85,
    repetition_penalty=1.3,
    top_p=0.95,
    do_sample=True,
    max_tokens=200 # Limit output length
)

print(response["choices"][0]["message"]["content"])

Using Hugging Face transformers and Unsloth (if continuing fine-tuning or specialized inference)

from unsloth import FastLanguageModel
from transformers import pipeline

# Load model and tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/llama-3-8b-bnb-4bit", # Or your preferred base model
    max_seq_length = 2048,
    load_in_4bit = True,
)

# Load the fine-tuned adapter weights
model.load_adapter("Sandeep4235/llama3-8b-Wisdom-gguf") # If pushed as an adapter
# If you pushed the merged GGUF, load it directly via Llama.cpp as above.

# Create a pipeline for text generation
pip = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.float16, # Adjust based on your GPU and model precision
    device_map="auto",
)

# Construct the prompt as specified in 'Direct Use' section
formatted_prompt = (
    "### Instruction: What is the significance of self-awareness in daily life?\n\n"
    "### [Internal Thought]: Analyzing intent.\n\n"
    "### [Author Response]:"
)

result = pip(formatted_prompt,
             max_new_tokens=200,
             temperature=0.85,
             repetition_penalty=1.3,
             top_p=0.95,
             do_sample=True,
             return_full_text=False # Only return the generated text
            )

print(result[0]['generated_text'])

Training Details

Training Data

The model was fine-tuned on a custom dataset generated by cleaning and chunking.

The raw text underwent several preprocessing steps:

• Ligature Error Correction: Specific extraction errors like <e, <is, <ere were corrected.
• Whitespace Standardization: Multiple whitespace characters were reduced to single spaces.
• Sliding Window Chunking: Text was chunked into samples of approximately 500 words with a 100-word overlap, formatted into the instruction-based prompt structure. Only chunks with more than 150 words were retained.

The resulting dataset (training.jsonl) contained some samples. This was then split into a training set (train_final.jsonl with some samples) and a validation set (eval_final.jsonl with some samples).

Training Procedure

Training Hyperparameters

• Training regime: bf16 mixed precision (if supported by hardware, otherwise fp16)
• Optimizer: adamw_8bit
• Learning Rate: 2e-4
• LoRA Rank (r): 32
• LoRA Alpha: 64
• LoRA Dropout: 0
• Bias: none
• Batch Size (per device): 2
• Gradient Accumulation Steps: 4
• Warmup Steps: 10
• Maximum Training Steps: 251
• Weight Decay: 0.01
• LR Scheduler Type: linear
• Seed: 3407
• Gradient Checkpointing: unsloth

Evaluation

Testing Data, Factors & Metrics

Testing Data

The model's performance was qualitatively evaluated against eval_final.jsonl, a validation set of samples. While no formal quantitative metrics (e.g., perplexity, ROUGE scores) were explicitly tracked, iterative qualitative assessment guided the fine-tuning process to ensure desired output behavior.

Results

Initial fine-tuning aimed to address issues such as context-agnostic answers, self-generation of questions/conversational loops, and high token consumption. Post-training, the model demonstrates significantly improved adherence to the intended 'Author Response' format, reduced verbosity, and more focused outputs, largely due to the extended training steps and the integration of refined inference parameters and robust post-processing.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: Tesla T4 GPU
• Hours used: Approximately 4 hours
• Cloud Provider: Google Colab

Technical Specifications

Compute Infrastructure

Hardware

Training and inference for this model were conducted primarily on a Tesla T4 GPU, as provided by the Google Colab environment.

Software

• Fine-tuning Framework: Unsloth (version 2026.8.5)
• Core Libraries: transformers (version 5.5.0), trl, peft (version 0.19.1), accelerate, bitsandbytes.
• Data Preprocessing: pymupdf
• Deep Learning Framework: torch (version 2.11.0+cu128)

Model Card Contact

Sandeep Hipparagi

AI Developer from India and Co-Founder of BluePatterns AI, focused on leveraging machine learning to address the gap between AI frontier models and open-source models, challenge of information inequality by creating tools that provide reliable and accessible information to diverse communities. My work is driven by a belief in democratizing access to intelligence and empowering individuals through technology.