gn00029914's picture

gn00029914

gn00029914

·

AI & ML interests

None yet

Recent Activity

liked a Space about 10 hours ago

OpenHands/openhands-index

reacted to grimjim's post with 🚀 about 12 hours ago

After tinkering with Gemma Scope 2, I now have an mechanistic explanation of why Winsorization was as effective as it was in my ablation experiments on Gemma 3 12B Instruct. In short, the activation for the BOS token overwhelms everything else. Gemma Scope 2 deliberately did not train on the BOS token. Winsorization capped the magnitude of the BOS token, allowing the activations of other tokens to be compared. https://huggingface.co/google/gemma-scope-2-12b-it

upvoted an article about 12 hours ago

Norm-Preserving Biprojected Abliteration

View all activity

Organizations

models 0

None public yet

datasets 0

None public yet