raditotev/ai-radipro-chatbot

RadiPro Chatbot - Llama 3.2 3B Instruct

Model Description

RadiPro Chatbot is a fine-tuned version of Meta's Llama-3.2-3B-Instruct model, specifically optimized to serve as a conversational AI assistant for RadiPro AI agency. This model has been trained to provide helpful, accurate, and contextually appropriate responses regarding the company's services. Since RadiPro AI agency is a rather small company with limited number of services the chatbot's main purpose is to demonstrate clients what potential implementation on their platform might look.

Model Format: GGUF (Q8_0 quantization)

Model Details

• Base Model: meta-llama/Llama-3.2-3B-Instruct
• Architecture: Llama 3.2 3B Instruct
• Quantization: Q8_0 (8-bit quantization)
• Model Size: ~3B parameters
• Format: GGUF

Intended Use

This model is designed for:

• Primary Use Case: Conversational AI chatbot for AI agency services
• Applications:
  • Customer support and engagement
  • Professional consultation assistance
  • Information retrieval and Q&A
  • General conversational tasks in agency contexts

Training

Base Model

• Model: meta-llama/Llama-3.2-3B-Instruct
• Architecture: Transformer-based language model

Fine-tuning

This model has been trained using mlx_lm on a completion dataset consisting of 40 entries in train.jsonl and 4 entries in valid.jsonl.

Usage

Using with llama.cpp

# Download the model
# Run inference
./llama-cli -m radipro-chatbot-llama.Q8_0.gguf -p "Your prompt here"

Using with Python (llama-cpp-python)

from llama_cpp import Llama

# Load the model
llm = Llama(model_path="radipro-chatbot-llama.Q8_0.gguf")

# Generate response
response = llm(
    "User: Hello, how can you help me?\nAssistant:",
    max_tokens=512,
    temperature=0.7,
    top_p=0.9,
    echo=False
)

print(response['choices'][0]['text'])

Performance

This model maintains the strong performance characteristics of the base Llama-3.2-3B-Instruct model while being optimized for conversational AI agency use cases. The Q8_0 quantization provides a good balance between model size and performance.

Limitations

• The model may occasionally generate incorrect or nonsensical responses
• Responses are based on training data and may not reflect the most current information
• The model may not always follow complex multi-step instructions perfectly
• Context window limitations may affect very long conversations
• As a quantized model, there may be slight quality trade-offs compared to the full-precision version
• This model is trained to answer only questions regarding the agency and the provided services

License

This model is released under the same license as the base model (Llama 3.2 Community License). Please review the Llama 3.2 Community License for details on usage terms and restrictions.

Acknowledgments

• Meta AI for the base Llama-3.2-3B-Instruct model
• The open-source community for tools and frameworks that made this fine-tuning possible

Contact

For questions regarding this model, please open an issue in the repository.