username-generator
Byte-level GPTs that dream up login-style usernames (dark_phoenix, Cargan66), trained on the
ks46/usernames corpus. Every folder is one trained model: its checkpoint (ckpt.pt), the export
for the single-threaded x86-64 batch kernel ndgen (model-int7.ndq, model.ndq), the kernel's
golden reference NLLs, eval.json, manifest.json, samples.txt and a card with the details.
Vocabulary is 256 bytes with byte 0 as the stop token; the held-out split is
xxh3_64(name) % 64 == 0, so bits/char are comparable only between models trained on the same corpus.
| model | layers × width | params | iters | held-out bits/char | novel samples |
|---|---|---|---|---|---|
| teacher3-16x1024 | 16×1024 | 205.9M | 50,000 | 3.2067 | 90% |
| g3-6x640-a09 | 6×640 | 30.7M | 40,000 | 3.2513 | 90% |
| g3-12x256-a09 | 12×256 | 9.7M | 40,000 | 3.2891 | 91% |
| g3-6x384-a09 | 6×384 | 10.7M | 40,000 | 3.2958 | 91% |
| g3-8x256-a09 | 8×256 | 6.5M | 40,000 | 3.3178 | 92% |
| g3-12x192-a09 | 12×192 | 5.5M | 40,000 | 3.3239 | 91% |
| g3-4x384-a09 | 4×384 | 7.2M | 40,000 | 3.3369 | 91% |
| g3-6x256-a09 | 6×256 | 4.9M | 40,000 | 3.3448 | 91% |
| g3-8x192-a09 | 8×192 | 3.7M | 40,000 | 3.3586 | 92% |
| teacher2-16x1024 | 16×1024 | 205.9M | 25,000 | 3.4010 | 91% |
| g2-6x640-a09 | 6×640 | 30.7M | 40,000 | 3.4378 | 91% |
| g2-6x512-a09 | 6×512 | 19.4M | 40,000 | 3.4537 | 91% |
| g2-8x384-a09 | 8×384 | 14.3M | 40,000 | 3.4602 | 91% |
| g2-12x256-a09 | 12×256 | 9.7M | 40,000 | 3.4739 | 92% |
| g2-6x384-a09 | 6×384 | 10.7M | 40,000 | 3.4810 | 91% |
| g2-8x256-a09 | 8×256 | 6.5M | 40,000 | 3.5020 | 92% |
| g2-12x192-a09 | 12×192 | 5.5M | 40,000 | 3.5074 | 92% |
| g2-4x384-a09 | 4×384 | 7.2M | 40,000 | 3.5193 | 92% |
| g3-4x128-a09 | 4×128 | 0.8M | 40,000 | 3.5269 | 93% |
| g2-6x256-a09 | 6×256 | 4.9M | 40,000 | 3.5280 | 92% |
| g3-6x96-a09 | 6×96 | 0.7M | 40,000 | 3.5305 | 94% |
| g2-8x192-a09 | 8×192 | 3.7M | 40,000 | 3.5397 | 93% |
| prod-8x768-phase2 | 8×768 | 56.9M | 60,000 | 3.5619 | 93% |
| g2-12x128-a09 | 12×128 | 2.5M | 40,000 | 3.5705 | 93% |
| g3-3x128-a09 | 3×128 | 0.6M | 40,000 | 3.5738 | 93% |
| g3-4x96-a09 | 4×96 | 0.5M | 40,000 | 3.5920 | 94% |
| g3-8x64-a09 | 8×64 | 0.4M | 40,000 | 3.5923 | 93% |
| teacher-16x1024 | 16×1024 | 205.9M | 9,000 | 3.6068 | 94% |
| pre-6x640-a09 | 6×640 | 30.7M | 20,000 | 3.6137 | 94% |
| pre-8x512-a09 | 8×512 | 25.9M | 20,000 | 3.6138 | 95% |
| g2-8x128-a09 | 8×128 | 1.6M | 40,000 | 3.6138 | 94% |
| pre-16x320-a09 | 16×320 | 20.4M | 20,000 | 3.6145 | 95% |
| pre-12x384-a09 | 12×384 | 21.4M | 20,000 | 3.6147 | 95% |
| prod-fast-4x768-phase2 | 4×768 | 28.6M | 30,000 | 3.6154 | 94% |
| pre-10x384-a09 | 10×384 | 17.8M | 20,000 | 3.6197 | 95% |
| pre-5x640-a09 | 5×640 | 25.6M | 20,000 | 3.6206 | 94% |
| pre-6x512-a09 | 6×512 | 19.4M | 20,000 | 3.6232 | 94% |
| pre-4x768-a09 | 4×768 | 28.6M | 20,000 | 3.6247 | 94% |
| pre-8x384-a09 | 8×384 | 14.3M | 20,000 | 3.6279 | 94% |
| pre-10x320-a09 | 10×320 | 12.8M | 20,000 | 3.6295 | 95% |
| g3-6x64-a09 | 6×64 | 0.3M | 40,000 | 3.6299 | 94% |
| pre-6x448-a09 | 6×448 | 14.8M | 20,000 | 3.6312 | 95% |
| pre-12x256-a09 | 12×256 | 9.7M | 20,000 | 3.6380 | 95% |
| pre-6x384-a09 | 6×384 | 10.7M | 20,000 | 3.6423 | 95% |
| pre-4x512-a09 | 4×512 | 13.0M | 20,000 | 3.6481 | 95% |
| g3-2x128-a09 | 2×128 | 0.4M | 40,000 | 3.6572 | 93% |
| pre-8x256-a09 | 8×256 | 6.5M | 20,000 | 3.6588 | 95% |
| pre-4x384-a09 | 4×384 | 7.2M | 20,000 | 3.6730 | 95% |
| g3-4x64-a09 | 4×64 | 0.2M | 40,000 | 3.6984 | 94% |
| g2-4x128-a09 | 4×128 | 0.8M | 40,000 | 3.7052 | 93% |
| g2-8x64-a09 | 8×64 | 0.4M | 40,000 | 3.7648 | 94% |
| g3-8x32-a09 | 8×32 | 0.1M | 40,000 | 3.8129 | 95% |
| g3-2x64-a09 | 2×64 | 0.1M | 40,000 | 3.8361 | 94% |
| g3-4x32-a09 | 4×32 | 0.1M | 40,000 | 3.9393 | 95% |
| g3-1x64-a09 | 1×64 | 0.1M | 40,000 | 4.0124 | 95% |
| g3-2x32-a09 | 2×32 | 0.0M | 40,000 | 4.0682 | 96% |
Sampling
uv run python -m training.sample --ckpt <model>/ckpt.pt -n 20 --prefix dark --temperature 0.9
./ndgen gen --model <model>/model-int7.ndq -n 1000 --batch 64 --temperature 0.9 --prefix dark