victordiao
commited on
Commit
·
54c5660
1
Parent(s):
703c219
Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
inference: false
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
# Robin Model Card
|
| 6 |
+
|
| 7 |
+
## Model Details
|
| 8 |
+
|
| 9 |
+
Robin is a series of models finetuned from LLaMA on several high-quality data.
|
| 10 |
+
|
| 11 |
+
- **Developed by:** [LMFlow](https://github.com/OptimalScale/LMFlow/)
|
| 12 |
+
- **Model type:** An auto-regressive language model based on the transformer architecture.
|
| 13 |
+
- **License:** Non-commercial license
|
| 14 |
+
- **Finetuned from model:** [LLaMA](https://arxiv.org/abs/2302.13971).
|
| 15 |
+
|
| 16 |
+
### Model Sources
|
| 17 |
+
|
| 18 |
+
- **Repository:** https://github.com/OptimalScale/LMFlow/
|
| 19 |
+
- **Blog:** https://medium.com/@hkust.ml/robin-v2-launches-achieves-unparalleled-performance-on-openllm-4f6886e822c1
|
| 20 |
+
- **Paper:** https://arxiv.org/abs/2306.12420
|
| 21 |
+
- **Demo:** https://lmflow.com/
|
| 22 |
+
|
| 23 |
+
## Uses
|
| 24 |
+
|
| 25 |
+
Robin is primarily utilized for conducting research on extensive language models and chatbots, catering to users specializing in natural language processing, machine learning, and artificial intelligence research.
|
| 26 |
+
|
| 27 |
+
## How to Get Started with the Model
|
| 28 |
+
|
| 29 |
+
We provide four kinds of demos including:
|
| 30 |
+
|
| 31 |
+
- Online Service: If you don't want to run any code and just want to try our models, we deploy our instruction-tuned LLaMA you to have a try.
|
| 32 |
+
- Colab Chatbot (shell): An interactive shell-based chatbot for you to easily deploy a chatbot on colab.
|
| 33 |
+
- Colab Chatbot (web): An interactive web-based chatbot for you to easily deploy your own chatbot on colab.
|
| 34 |
+
- Local Deploy: We also provide a way for you to deploy your model/chatbot locally, which means you can deploy much larger model than previous three methods if you have enough resource.
|
| 35 |
+
|
| 36 |
+
Please refer to https://github.com/OptimalScale/LMFlow#demos
|
| 37 |
+
|
| 38 |
+
## Training Details
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
Expanding upon the initial idea of self-instruct techniques, we incorporated several different data sources and build a new dataset called [LMFlow Dataset](http://lmflow.org:5000/lmflow_data.tar.gz).
|
| 42 |
+
The new training split is created by merging the following datasets:
|
| 43 |
+
- ShareGPT: randomly sample 50K English data and 10K Chinese data from ShareGPT.
|
| 44 |
+
- GPT-4-LLM: 52K English data from GPT-4-LLM.
|
| 45 |
+
- BELLE: randomly sample 80K Chinese data from BELLE.
|
| 46 |
+
|
| 47 |
+
See more details in the "Instruction Tuning" section in our [paper](https://arxiv.org/pdf/2306.12420.pdf).
|
| 48 |
+
|
| 49 |
+
## Evaluation
|
| 50 |
+
|
| 51 |
+
Robin is evaluated with [LMFlow Benchmark](https://blog.gopenai.com/lmflow-benchmark-an-automatic-evaluation-framework-for-open-source-llms-ef5c6f142418).
|
| 52 |
+
See more details in this [paper](https://arxiv.org/pdf/2306.12420.pdf).
|
| 53 |
+
|
| 54 |
+
## Citation
|
| 55 |
+
If you find this repository useful, please consider giving ⭐ and citing our [paper](https://arxiv.org/abs/2306.12420):
|
| 56 |
+
|
| 57 |
+
```
|
| 58 |
+
@misc{lmflow,
|
| 59 |
+
author = {Shizhe Diao and Rui Pan and Hanze Dong and KaShun Shum and Jipeng Zhang and Wei Xiong and Tong Zhang},
|
| 60 |
+
title = {LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models},
|
| 61 |
+
year = {2023},
|
| 62 |
+
publisher = {GitHub},
|
| 63 |
+
journal = {GitHub repository},
|
| 64 |
+
howpublished = {\url{https://optimalscale.github.io/LMFlow/}},
|
| 65 |
+
}
|
| 66 |
+
```
|