Hoglet-33/3.33BT-Pre-Training-Mix-Three-Of-Three
Viewer • Updated • 3.9M • 192
🔄 In a Training Loop
Ash W.
Hoglet-33
AI & ML interests
Open source AI, datasets, parameter efficiency, SLMs, AI for the betterment of humanity. Contact at ash@basicallyai.co
Recent Activity
reacted to theirpost with 👀 about 11 hours ago
Pebble 10M and Pebble 10M Chat are now released!
Both models use our Mamba/Transformer 3:1 hybrid architecture and were pretrained on 25 billion tokens.
Pebble 10M Chat was additionally fine-tuned on 250 million tokens of Smol-SmolTalk to improve its conversational capabilities.
You can find them here:
- https://huggingface.co/basically-ai/Pebble-10M
- https://huggingface.co/basically-ai/Pebble-10M-Chat
We hope you enjoy using them. The rest of the Pebble family will be released soon.
Follow for more:
@Hoglet-33
https://huggingface.co/basically-ai reacted to theirpost with 👍 about 11 hours ago
Pebble 10M and Pebble 10M Chat are now released!
Both models use our Mamba/Transformer 3:1 hybrid architecture and were pretrained on 25 billion tokens.
Pebble 10M Chat was additionally fine-tuned on 250 million tokens of Smol-SmolTalk to improve its conversational capabilities.
You can find them here:
- https://huggingface.co/basically-ai/Pebble-10M
- https://huggingface.co/basically-ai/Pebble-10M-Chat
We hope you enjoy using them. The rest of the Pebble family will be released soon.
Follow for more:
@Hoglet-33
https://huggingface.co/basically-ai reacted to theirpost with 😎 about 11 hours ago
Pebble 10M and Pebble 10M Chat are now released!
Both models use our Mamba/Transformer 3:1 hybrid architecture and were pretrained on 25 billion tokens.
Pebble 10M Chat was additionally fine-tuned on 250 million tokens of Smol-SmolTalk to improve its conversational capabilities.
You can find them here:
- https://huggingface.co/basically-ai/Pebble-10M
- https://huggingface.co/basically-ai/Pebble-10M-Chat
We hope you enjoy using them. The rest of the Pebble family will be released soon.
Follow for more:
@Hoglet-33
https://huggingface.co/basically-aiOrganizations
