Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior
Paper • 2609.39827 • Published • 12
Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior
This repository contains a model described in the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior. The model is a SmolLM3-based causal language model trained as part of the experiments in the paper.
For full details, please refer to the GitHub repository and the project page.