license: apache-2.0 language:
en tags:
text-generation
fine-tuning
bangkit
agrease metrics:
loss base_model: mistralai/Mistral-7B-Instruct-v0.3 datasets:
custom-scraped model-index:
name: agrease-mistral-finetune results:
task: type: text-generation dataset: type: custom-scraped name: Agrease Application Data metrics:
type: loss value: 0.11 name: Fine-Tuning Loss
Fine-Tuning Mistral for the Agrease Application Author: Benito Yvan Deva Putra Arung Dirgantara Contact: benitodeva84@gmail.com Project: Bangkit Academy 2024 - Machine Learning Path
- Project Overview This project focuses on the fine-tuning of the Mistral v3 Large Language Model (LLM) to create a specialized model for the "Agrease" application. The primary objective was to adapt the general capabilities of the Mistral LLM to understand and process domain-specific data relevant to Agrease, enhancing its performance for tasks such as recommendation and data interpretation.
This was developed as a capstone project during my participation in the Bangkit Academy 2024 Batch 2 program, under the Machine Learning learning path.
- Methodology The project followed a structured machine learning workflow, from data acquisition to model evaluation.
2.1. Data Collection To build a relevant dataset for fine-tuning, web scraping techniques were employed.
Tools: BeautifulSoup and Scrapy were used to gather application data from various online marketplaces and sources.
Process: The scrapers were designed to extract specific information required to train the model effectively for the Agrease application's context.
2.2. Model Fine-Tuning The core of this project involved the fine-tuning process.
Base Model: We used the pre-trained Mistral v3 as our foundation model.
Frameworks: The fine-tuning process was implemented using Python, with primary libraries being PyTorch and TensorFlow.
Objective: The goal was to train the model on our custom-scraped dataset, adjusting its weights to specialize its responses and understanding, while minimizing the training loss.
- Results The fine-tuning process yielded significant improvements in the model's performance on domain-specific tasks.
Fine-Tuned LLM: Achieved a final loss rate of 11%, indicating a successful adaptation of the model to the new data.
Recommendation Model: As part of the broader Agrease application, a recommendation model was also developed, which achieved a 10% loss rate.
These results demonstrate the model's strong capability to serve the specific needs of the Agrease application.
- Technical Stack Programming Language: Python
ML/DL Frameworks: TensorFlow, PyTorch
Data Scraping: BeautifulSoup, Scrapy
Base LLM: Mistral v3
- Acknowledgements I would like to thank Google, GoTo, Traveloka, and the Ministry of Education, Culture, Research, and Technology of the Republic of Indonesia for the opportunity to participate in the Bangkit Academy program. The skills and experience gained were invaluable in the successful completion of this project.