| --- |
| license: apache-2.0 |
| --- |
| |
|
|
| <style> |
| img{ |
| user-select: none; |
| transition: all 0.2s ease; |
| border-radius: .5rem; |
| } |
| img:hover{ |
| transform: rotate(2deg); |
| filter: invert(100%); |
| } |
| @import url('https://fonts.googleapis.com/css2?family=Vollkorn:ital,wght@0,400..900;1,400..900&display=swap'); |
| </style> |
| |
| <div style="background-color: transparent; border-radius: .5rem; padding: 2rem; font-family: monospace; font-size: .85rem; text-align: justify;"> |
| |
|  |
|
|
| arco consistently outperforms every sota model below 600m parameters as well as some 1b base models on average on 5 core benchmarks and is competitive with the best 0.7b-1b llms. arco is a merge of multiple internal models fine-tuned on a diverse set of styles and finally merged with the several models (including palmer-004-turbo and danube3-chat), followed by a merge with base model to preserve knowledge. |
|
|
| #### prompt |
|
|
| there is no prompt intentionally set but this one worked really good for me: |
|
|
| ``` |
| The following is a conversation between a super smart AI assistant and an user. |
| |
| user: <your question> |
| |
| assistant: |
| ``` |
|
|
| #### benchmarks |
|
|
| zero-shot evaluations performed on current sota ~0.5b models. |
|
|
| | Parameters | Model | MMLU | ARC-C | HellaSwag | PIQA | Winogrande | Average | |
| | -----------|--------------------------------|-------|-------|-----------|--------|------------|---------| |
| | 0.5b | qwen2 |**44.13**| 28.92| 49.05 | 69.31 | 56.99 | 49.68 | |
| | 1.1b | tinyllama | 25.77| 30.29| 59.35 | 73.29 | 59.59 | 49.66 | |
| | 0.5b | danube3-base | 24.81| 36.18| 60.46| 73.78 | 61.01 | 51.25 | |
| | 0.5b | danube3-chat | 25.54 | 36.26 | 60.72 | 74.32 | 61.40 | 51.64 | |
| | 0.5b | palmer-004-turbo | 27.36|35.58|61.79|73.67 | 61.17 |51.91| |
| | 0.5b | arco |26.17|**37.29**|**62.88**|**74.37**|**62.27**|**52.60**| |
| #### supporters |
|
|
| <a href="https://ko-fi.com/appvoid" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 34px !important; margin-top: -4px;width: 128px !important; filter: contrast(2) grayscale(100%) brightness(100%);" ></a> |
|
|
| ### trivia |
|
|
| arco comes from spanish word "bow" which is always associated with arrows and hence, speed. |
| </div> |
