| license: apache-2.0 | |
| language: | |
| en | |
| tags: | |
| text-generation | |
| fine-tuning | |
| bangkit | |
| agrease | |
| metrics: | |
| loss | |
| base_model: mistralai/Mistral-7B-Instruct-v0.3 | |
| datasets: | |
| custom-scraped | |
| model-index: | |
| name: agrease-mistral-finetune | |
| results: | |
| task: | |
| type: text-generation | |
| dataset: | |
| type: custom-scraped | |
| name: Agrease Application Data | |
| metrics: | |
| type: loss | |
| value: 0.11 | |
| name: Fine-Tuning Loss | |
| Fine-Tuning Mistral for the Agrease Application | |
| Author: Benito Yvan Deva Putra Arung Dirgantara | |
| Contact: benitodeva84@gmail.com | |
| Project: Bangkit Academy 2024 - Machine Learning Path | |
| 1. Project Overview | |
| This project focuses on the fine-tuning of the Mistral v3 Large Language Model (LLM) to create a specialized model for the "Agrease" application. The primary objective was to adapt the general capabilities of the Mistral LLM to understand and process domain-specific data relevant to Agrease, enhancing its performance for tasks such as recommendation and data interpretation. | |
| This was developed as a capstone project during my participation in the Bangkit Academy 2024 Batch 2 program, under the Machine Learning learning path. | |
| 2. Methodology | |
| The project followed a structured machine learning workflow, from data acquisition to model evaluation. | |
| 2.1. Data Collection | |
| To build a relevant dataset for fine-tuning, web scraping techniques were employed. | |
| Tools: BeautifulSoup and Scrapy were used to gather application data from various online marketplaces and sources. | |
| Process: The scrapers were designed to extract specific information required to train the model effectively for the Agrease application's context. | |
| 2.2. Model Fine-Tuning | |
| The core of this project involved the fine-tuning process. | |
| Base Model: We used the pre-trained Mistral v3 as our foundation model. | |
| Frameworks: The fine-tuning process was implemented using Python, with primary libraries being PyTorch and TensorFlow. | |
| Objective: The goal was to train the model on our custom-scraped dataset, adjusting its weights to specialize its responses and understanding, while minimizing the training loss. | |
| 3. Results | |
| The fine-tuning process yielded significant improvements in the model's performance on domain-specific tasks. | |
| Fine-Tuned LLM: Achieved a final loss rate of 11%, indicating a successful adaptation of the model to the new data. | |
| Recommendation Model: As part of the broader Agrease application, a recommendation model was also developed, which achieved a 10% loss rate. | |
| These results demonstrate the model's strong capability to serve the specific needs of the Agrease application. | |
| 4. Technical Stack | |
| Programming Language: Python | |
| ML/DL Frameworks: TensorFlow, PyTorch | |
| Data Scraping: BeautifulSoup, Scrapy | |
| Base LLM: Mistral v3 | |
| 5. Acknowledgements | |
| I would like to thank Google, GoTo, Traveloka, and the Ministry of Education, Culture, Research, and Technology of the Republic of Indonesia for the opportunity to participate in the Bangkit Academy program. The skills and experience gained were invaluable in the successful completion of this project. |