exp182 / exp183 — 309M ChessTransformer (A40 run)
Models
Pretrain (latest.pt / pretrain_step10000.pt)
- 309M deep-narrow ChessTransformer (96L × 512d, 8 heads)
- StrengthenedBoardEncoder + SwiGLU + chess relative bias
- Muon pretrain to step 10,000 / 50k target (batch=192, bf16)
- Repo file: uploaded as
pretrain_step10000.pt and latest_pretrain.pt
RL (rl_iter010.pt)
- Expert-iteration RL from the step-10k pretrain
- Soft MCTS visit targets + hard Stockfish moves (full strength, depth 8)
- Checkpoint after iteration 10
Usage
from huggingface_hub import hf_hub_download
from chess_inference import load_checkpoint
path = hf_hub_download("avewright/exp182-pretrain-309m", "rl_iter010.pt")
model = load_checkpoint(path, device="cuda")
Code: https://github.com/avewright/transform
