| --- |
| tags: |
| - gguf |
| - llama.cpp |
| - unsloth |
| - mistral |
| - python |
| base_model: |
| - mistralai/Mistral-7B-Instruct-v0.3 |
| --- |
| |
| # mistral-7b-python-gguf |
|
|
| Conversational Python fine-tune of Mistral 7B exported to GGUF format for local inference. |
|
|
| - Base model: Mistral 7B |
| - Fine-tuning framework: Unsloth |
| - Format: GGUF |
| - Author: AntoineChatry |
|
|
| --- |
|
|
| # ⚠️ Disclaimer |
|
|
| This is an **early experimental fine-tune**. |
|
|
| It is **not production-ready**, not fully aligned, and not optimized for reliability or long-form reasoning. |
| This project was created primarily for learning and experimentation. |
|
|
| Please do not expect state-of-the-art coding performance. |
|
|
| --- |
|
|
| # Model Overview |
|
|
| This model is a conversational fine-tune of Mistral 7B trained primarily on: |
|
|
| - ShareGPT-style conversations |
| - Python-focused discussions |
| - Coding Q&A format |
|
|
| The objective was to: |
|
|
| - Experiment with fine-tuning |
| - Build a conversational Python model |
| - Export to GGUF for llama.cpp compatibility |
| - Test local inference workflows |
|
|
| No RLHF or advanced alignment was applied beyond the base model. |
|
|
| --- |
|
|
| # Known Limitations |
|
|
| ## Repetition Issues |
|
|
| - Frequently repeats phrases like: |
| > "Here's the code:" |
| - Can loop or restate similar sentences |
| - Overuses patterns learned from dataset formatting |
|
|
| ## Weak Long-Form Explanations |
|
|
| - Struggles with multi-paragraph structured reasoning |
| - May repeat itself when asked for detailed explanations |
| - Limited depth on conceptual explanations |
|
|
| ## Instruction Following |
|
|
| - Not fully aligned |
| - May ignore strict formatting constraints |
| - Tends to prioritize generating code over detailed explanations |
|
|
| ## Dataset Bias |
|
|
| - Strong ShareGPT conversational tone |
| - Python-heavy bias |
| - Some templated response structure |
|
|
| --- |
|
|
| # What Works Reasonably Well |
|
|
| - Short Python snippets |
| - Basic debugging help |
| - Simple function generation |
| - Conversational coding prompts |
|
|
| Best performance is observed when: |
|
|
| - Prompts are clear and direct |
| - Expected output is short |
| - Tasks are code-focused |
|
|
| --- |
|
|
| # Training Details |
|
|
| - Base: Mistral 7B |
| - Dataset format: ShareGPT-style conversational dataset (Python-oriented) |
| - Fine-tuned using Unsloth notebooks |
| - Converted to GGUF for llama.cpp compatibility |
| - Quantized version included (Q4_K_M) |
|
|
| No additional safety tuning or post-training optimization was applied. |
|
|
| --- |
|
|
| # Example Usage |
|
|
| This model was finetuned and converted to GGUF format using Unsloth. |
|
|
| ## llama.cpp |
|
|
| For text-only LLMs: |
|
|
| ```bash |
| llama-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja |
| ``` |
|
|
| For multimodal models: |
|
|
| ```bash |
| llama-mtmd-cli -hf AntoineChatry/mistral-7b-python-gguf --jinja |
| ``` |
|
|
| --- |
| ## Available Model files: |
| - `mistral-7b-instruct-v0.3.Q4_K_M.gguf` |
| --- |
| # Ollama |
|
|
| An Ollama Modelfile is included for easy deployment. |
|
|
| Example: |
|
|
| ```bash |
| ollama create mistral-python -f Modelfile |
| ollama run mistral-python |
| ``` |
|
|
| --- |
|
|
| # Why This Model Is Public |
|
|
| This model represents a learning milestone. |
|
|
| Sharing imperfect models helps: |
|
|
| - Document fine-tuning progress |
| - Enable experimentation |
| - Collect feedback |
| - Iterate toward better versions |
|
|
| This is not a finished product. |
|
|
| --- |
|
|
| # Unsloth |
|
|
| This model was trained 2x faster using Unsloth. |
|
|
| https://github.com/unslothai/unsloth |
|
|
| <img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/> |
|
|
| --- |
|
|
| # License |
|
|
| Please refer to the original Mistral 7B license from Mistral AI. |
