Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
CodeSoft 
posted an update 1 day ago
Banaxi-Tech
•
edited about 2 hours ago

@CodeSoft Very nice first model, you still got a lot to improve on (since 26% on Hellaswag) but you could definitely get competitive!

·
Banaxi-Tech

My tips would be
Use more tokens even 10B tokens could get competitive!
Replace Infiniwebmath to FineMAth 4 PLus
Add some Fineweb HQ

But thats just my opinion.
In this post