60 1 45

Gonzales

Fernanda24

AI & ML interests

None yet

Recent Activity

new activity 30 minutes ago

eousphoros/DeepSeek-V3.2-NVFP4:Is it possible to make smaller NVFP4 quant at 340-360GB to fit in 4x96gb?

new activity about 5 hours ago

Kebob/DeepSeek-V3.2-CPU-NUMA2-AMXINT4:attention backend

new activity 1 day ago

QuantTrio/DeepSeek-V3.2-AWQ:Aww Man!

View all activity

Organizations

None yet

New activity in eousphoros/DeepSeek-V3.2-NVFP4 30 minutes ago

Is it possible to make smaller NVFP4 quant at 340-360GB to fit in 4x96gb?

👍 1

#1 opened 5 days ago by

Fernanda24

New activity in Kebob/DeepSeek-V3.2-CPU-NUMA2-AMXINT4 about 5 hours ago

attention backend

#1 opened about 5 hours ago by

Fernanda24

New activity in QuantTrio/DeepSeek-V3.2-AWQ 1 day ago

Aww Man!

#1 opened 5 days ago by

mtcl

New activity in Intel/DeepSeek-V3.1-Terminus-int4-mixed-AutoRound 2 days ago

Question will it work in vllm or sglang with rtx 6000 blackwells? cuda arch sm120

#1 opened about 2 months ago by

Fernanda24

New activity in QuantTrio/DeepSeek-V3.1-AWQ-Lite 2 days ago

ooof this fits in 4x96gb can we get this for the new 3.2 Speciale ase well please :)

#2 opened 6 days ago by

Fernanda24

New activity in tencent/DeepSeek-V3.1-Terminus-W4AFP8 3 days ago

4 x RTX PRO 6000

👍 1

#1 opened 6 days ago by

willfalco

New activity in eousphoros/DeepSeek-V3.2-NVFP4 3 days ago

How did you bypass deepseek-v32 not recognized in Tranformers?

#3 opened 4 days ago by

Fernanda24

liked a model 4 days ago

QuantTrio/DeepSeek-V3.2-Speciale-AWQ

Text Generation • 685B • Updated 4 days ago • 46 • 4

New activity in deepseek-ai/DeepSeek-V3.2 4 days ago

model type `deepseek_v32` not found

#16 opened 6 days ago by

harryhan618

reacted to csabakecskemeti's post with 👀 4 days ago

Post

2007

Looking for some help to test an INT8 Deepseek 3.2:
SGLang supports Channel wise INT8 quants on CPUs with AMX instructions (Xeon 5 and above AFAIK)
https://lmsys.org/blog/2025-07-14-intel-xeon-optimization/

Currently uploading an INT8 version of Deepseek 3.2 Speciale:
DevQuasar/deepseek-ai.DeepSeek-V3.2-Speciale-Channel-INT8

I cannot test this I'm on AMD
"AssertionError: W8A8Int8LinearMethod on CPU requires that CPU has AMX support"
(I assumed it can fall back to some non optimized kernel but seems not)

If anyone with the required resources (Intel Xeon 5/6 + ~768-1TB ram) can help to test this that would be awesome.

If you have hints how to make this work on AMD Threadripper 7000 Pro series please guide me.

Thanks all!