chesSLM model

Chess Small Language Model: a decoder-only language model that plays chess, trained from scratch on chesSLM-dataset. Architecture:

Component	Value
Parameters	101,307,136
Layers	10
Hidden size	896
Attention heads	14 (head size 64)
Feed-forward	GELU, 4 × hidden size
Normalization	Pre-LayerNorm
Positions	Rotary (RoPE)
Embeddings	tied input and output
Context length	704 moves
Vocabulary	one token per move, plus <pad> and <bos>
Level	Stockfish ~1650 ELO at temperature 0.0

The model predicts the next move of a game written in Standard Algebraic Notation, one token per move. It is trained in two stages: first on all games, then on games between players rated 2500 or more. At play time a legal-move mask from python-chess guarantees rule-compliant moves, and the engine speaks UCI.

Files in this repository:

File	Content
chesslm_model.pt	the model weights
chesslm_vocab.json	token to id, with <pad> = 0 and <bos> = 1

git clone https://github.com/antoniopelusi/chesSLM
cd chesSLM
make setup
./chesSLM.py --getmodel

chesSLM speaks UCI: add ./chesSLM.py as an engine in any UCI chess GUI, such as CuteChess, to play against it, with the repository directory as working directory.

Training code and all details: GitHub. The complete training log (training.log) is available in the GitHub repository.