Qwen3 1.7B sparse autoencoder
A 32,768-feature Top-K SAE trained on the layer 14 residual stream of Qwen3 1.7B Base using 500M tokens from FineWeb-Edu. Includes checkpoint, 100M tokens of recorded activations, and feature interpretations.
Explore
Browse the features at nanointerpret.pages.dev, or run the visualizer locally:
git clone https://github.com/Belluxx/nanointerpret
cd nanointerpret
pip install -r requirements.txt
hf download Belluxx/nanointerpret-qwen3 --local-dir artifacts/nanointerpret-qwen3
python visualize.py --activations artifacts/nanointerpret-qwen3/activations
Training code and details: Belluxx/nanointerpret.
Metrics
| Width / K | 32,768 / 16 |
| Explained variance | 71.88% |
| Validation MSE | 0.1558 |
| Downstream KL | 0.2048 |
| Dead features | 1.74% |
- Downloads last month
- 368
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support