Qwen/Qwen3.8-27B
Image-Text-to-Text • 28B • Updated • 666k • • 11.1k
Robert Agee
RobAgrees
AI & ML interests
None yet
Recent Activity
new activity 4 days ago
Blackfrost-AI/Qwen3.8-27B-ABLITERATED-GGUF:Center your div liked a model 5 days ago
Qwen/Qwen3.8-27B liked a model 17 days ago
deepseek-ai/DeepSeek-V4-Flash-0731Organizations
None yet
Center your div
5
#1 opened 4 days ago
by RobAgrees
liked a
model 5 days ago
liked a
model 17 days ago
liked a
model 22 days ago
Model leaking its reasoning in the response?
#2 opened 29 days ago
by RobAgrees
Q3_K_M - Results so far
👍 2
3
#3 opened about 1 month ago
by RobAgrees
need higer quant
👍 2
6
#1 opened about 1 month ago
by 
gopi87
huggingface_hub is out-of-date
#5 opened 6 months ago
by RobAgrees
This one is another banger
👍 1
#1 opened 7 months ago
by RobAgrees
updated a
model 8 months ago
published a
model 8 months ago
liked a
model 9 months ago
This model is special
❤️ 2
1
#1 opened 9 months ago
by RobAgrees
72GB VRAM Users - What size quants do you want (70B models) ?
6
#2 opened about 1 year ago
by 
FrenzyBiscuit
[Feedback] Hardware Compatibility for AMD Ryzen AI Max+ 395 aka Strix Halo
➕👍 33
7
#73 opened about 1 year ago
by RobAgrees
liked a
Space about 1 year ago
reacted to ProCreations's post with 🚀 about 1 year ago
Post
2943
Eyyyy 50 followers 🤯
- 1 reply
Driver Code or GemmaCPP support?
🤗 8
4
#7 opened about 1 year ago
by 
yoyou446
Adding Evaluation Results
1
#4 opened almost 2 years ago
by 
leaderboard-pr-bot
reacted to codys12's post with 🚀 over 1 year ago
Post
2537
Introducing bitnet-r1-llama-8b and bitnet-r1-qwen-32b preview! These models are the first successful sub 1-billion-token finetune to BitNet architecture. We discovered that by adding an aditional input RMSNorm to each linear, you can finetune directly to BitNet with fast convergence to original model performance!
We are working on a pull request to use this extra RMS for any model.
To test these models now, install this fork of transformers:
Then load the models and test:
bitnet-r1-llama-8b and bitnet-r1-llama-32b were trained on ~ 300M and 200M tokens of the open-thoughts/OpenThoughts-114k dataset respectively, and were still significantly improving at the end of training. This preview simply demonstrates that the concept works, for future training runs we will leave the lm_head unquantized and align the last hidden state with the original model.
Huge thanks to the team that made this possible:
Gavin Childress, Aaron Herbst, Gavin Jones, Jasdeep Singh, Eli Vang, and Keagan Weinstock from the MSOE AI Club.
We are working on a pull request to use this extra RMS for any model.
To test these models now, install this fork of transformers:
pip install git+https://github.com/Codys12/transformers.gitThen load the models and test:
from transformers import (AutoModelForCausalLM, AutoTokenizer)
model_id = "codys12/bitnet-r1-qwen-32b"
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="cuda",
)
tokenizer = AutoTokenizer.from_pretrained(model_id, padding_side="left")bitnet-r1-llama-8b and bitnet-r1-llama-32b were trained on ~ 300M and 200M tokens of the open-thoughts/OpenThoughts-114k dataset respectively, and were still significantly improving at the end of training. This preview simply demonstrates that the concept works, for future training runs we will leave the lm_head unquantized and align the last hidden state with the original model.
Huge thanks to the team that made this possible:
Gavin Childress, Aaron Herbst, Gavin Jones, Jasdeep Singh, Eli Vang, and Keagan Weinstock from the MSOE AI Club.
- 1 reply