idle-intelligence/stencil-life

Stencil Life

The smallest networks we could train to be Conway's Game of Life, exactly: a 1,442-parameter MLP, a 3,490-parameter transformer encoder, and a 3,329-parameter stencil attention model that learned the rule at 16x16 and is exact with the same weights at 256x256.

Try the demo or the compare page.

The idea

We're spending more and more compute to do the same thing. A stupid idea: LLM of Life, where an LLM gets the rules and state of the current cell, and predicts if it lives or dies. Now that it's done, let's roll with the stupid idea. A model of the shelf is pretty bad at this. Can we improve it?

Life is a good place to ask that question, because the rule is tiny: a cell's next state depends on its 8 neighbours and itself, and the whole rule fits in 512 table entries (9 raw cells, or a Moore neighbourhood, in classical terms).

Qwen2.5-0.5B-Instruct, with the rules stated in the prompt and no fine-tuning, gets 215 of 512 neighbourhoods right. That is, from a random start, it doesn't destroy everything, but it gets much of the rules wrong. Trying to make it reproduce a glider fails 100% of the time. The full grid version immediately kills all cells at iteration 1. A LoRA adapter of about a million parameters gets 509 of 512 with the rules in the prompt, and all 512 without them (llm-of-life-lora, base-model numbers in docs/runs/2026-09-20-base-vs-finetuned.md). One forward pass per cell: 2.5 s per generation at 16x16 on an RTX 3080, 41 s at 64x64, and about 40 s per 16x16 generation in a browser tab on an M2.

Deciding alive or dead is a classification, so the next model here is a one-layer, one-head BERT-style encoder that reads the 9 cells as tokens: 3,490 parameters, exact on all 512 neighbourhoods, 1.2 ms per generation at 16x16 on the same GPU.

Dropping the language model and keeping the same approach gives a 2-layer MLP over the 9 raw cell values: 1,442 parameters, also exact on all 512 neighbourhoods.

The last model here treats the grid as a grid rather than 256 independent cells: one attention layer over the whole grid, masked so each cell only attends to its 3x3 stencil, the same locality a finite-difference scheme assumes. One forward pass gives the next grid. 3,329 parameters, trained on random 16x16 grids. Because the mask encodes the rule and not the grid size, the same weights run at 32x32, 64x64, 128x128 and 256x256 with every generation exact (docs/runs/2026-09-22-sizes.md). It is not a faster Life: the exact rule on one CPU core beats this model at every grid size measured, up to 1024x1024 on an RTX 3080 and 2048x2048 on an M2 (docs/runs/2026-09-22-stencil-vs-cpu.md).

That is the ladder llm-life runs in the browser, from a hand-written lookup table up to the LLM, scored on the same grids. Timings are from docs/runs/2026-09-20-bert.md, docs/runs/2026-09-20-vector.md, docs/runs/2026-09-20-a-rollout.md and docs/runs/2026-09-20-tab.md in that repo.

Files

File	Size	Method	Config
bert-d16-L1.bin	14.2 KB	Transformer encoder over the 9 neighbourhood cells as tokens, no LLM	d_model=16, layers=1, heads=1, 3490 parameters
mlp2-32-lrfix.bin	5.9 KB	2-layer MLP over the raw neighbourhood floats, no tokens, no LLM	9->32->32->2, 1442 parameters
stencil-d16-L1.bin	13.5 KB	Stencil-masked attention over the whole grid in one pass, no LLM	d_model=16, layers=1, heads=1, 3329 parameters

The files are the engine's own checkpoint format for these model classes, not GGUF or safetensors.

Usage

Loaded by llm-life's demo page (web/index.html, web/compare/index.html) via the Burn/WebGPU engine from llm-web (llm_wasm), compiled to WASM.

• bert-d16-L1.bin: "BERT of Life" mode.
• mlp2-32-lrfix.bin: "9 numbers to centre" mode.
• stencil-d16-L1.bin: "grid to grid, one channel" mode.

Training

• bert-d16-L1.bin: exhaustive 512-case neighbourhood split (448 train / 64 held out), 105 steps to exact on all 512 cases (docs/runs/2026-09-20-bert.md).
• mlp2-32-lrfix.bin: same 512-case split, cosine lr decay 2e-3 -> 2e-4 over 2000 steps, exact on all 512 cases by step 665 (docs/runs/2026-09-20-vector.md).
• stencil-d16-L1.bin: density sweep of random 16x16 grids (0.10-0.50 density, 9 densities x 8 seeds), 80 steps until fewer than one cell in a thousand was wrong on the training grids (docs/runs/2026-09-20-vector.md).

Results

Held-out results from the run docs above: neighbourhoods or cells correct, and whether every generation of a rollout came out exact.

File	Held-out set	Result
bert-d16-L1.bin	64 held-out neighbourhoods of the 512, then rollouts at 16, 64, 128 and 256, 5 generations, 3 grids each	all correct, every generation exact (docs/runs/2026-09-20-bert.md, docs/runs/2026-09-22-sizes.md)
mlp2-32-lrfix.bin	same 64 held-out neighbourhoods, then rollouts at 16, 64, 128 and 256, 5 generations, 3 grids each	all correct, every generation exact (docs/runs/2026-09-20-vector.md, docs/runs/2026-09-22-sizes.md)
stencil-d16-L1.bin	rollouts at 16x16 (trained size) and at 32, 64, 128 and 256 (never seen), 5 generations, 3 grids each	every generation exact at every size (docs/runs/2026-09-20-vector.md, docs/runs/2026-09-22-sizes.md)

Model Details

• BERT of Life: transformer encoder, d_model=16, 1 layer, 1 head, 3490 parameters
• 9 numbers to centre: 2-layer MLP, 9->32->32->2, 1442 parameters
• Grid to grid, one channel: stencil-masked attention, d_model=16, 1 layer, 1 head, 3329 parameters
• Engine: llm-web's Burn 0.20 + custom WGSL WebGPU backend
• License: MIT

Disclaimer

This is independent work by ilnmtlbnm@idle-intelligence.