🚩 Report: Spam
#3
by
Enderchef
- opened
Is this a real distill?
This is a completely different model from BasedBase's and produces very distinct outputs (gibberish).
It's what BasedBase's script would have achieved if it wasn't a non-functional vibecoded mess.
I don't think it's spam at all as while the model is not functional it's a valuable research artifact to the community as it shows that (backprop free) svd distillation doesn't seem to transfer well to transformer architectures.