ho22joshua/hep-posttraining

HEP Posttraining

This repository contains the training, data preparation, evaluation, and plotting setup used for HEP-focused post-training experiments.

Main Workflows

• Generate and validate signature-background datasets.
• Prepare VERL SFT and RL parquet data.
• Launch Qwen2.5 SFT and RL/GRPO experiments.
• Export VERL checkpoints to Hugging Face format.
• Evaluate checkpoint losses and decoded validation generations.

Published Artifacts

• Model: ho22joshua/hep-qwen2.5-7b-lora16-sigbg-irred-red-step2200
• Datasets: ho22joshua/hep-signature-backgrounds and ho22joshua/hep-config-sft

Train either published SFT dataset with VERL

Clone this repository, start the supplied VERL container, and bootstrap the pinned VERL revision. setup.sh installs VERL under .deps/ and needs to be sourced once per container session.

git clone git@hf.co:ho22joshua/hep-posttraining
cd hep-posttraining
source container.sh
source setup.sh

Then either command downloads the required Parquet split automatically on its first run, selects the appropriate VERL loader, and starts FSDP/LoRA SFT:

# Direct question-answer SFT: dominant irreducible/reducible backgrounds.
DATASET=hep-signature-backgrounds \\
MODEL_PATH=Qwen/Qwen2.5-7B-Instruct \\
NPROC_PER_NODE=4 \\
bash run_verl_sft.sh

# Multi-turn tool-use SFT: repair and edit toy HEP analysis configurations.
DATASET=hep-config-sft \\
MODEL_PATH=Qwen/Qwen2.5-Coder-1.5B-Instruct \\
NPROC_PER_NODE=4 \\
bash run_verl_sft.sh

The signature-background dataset follows the standard Hub prompt + completion convention; its bundled adapter joins those fields for VERL. The configuration dataset has a portable JSON tools column; its adapter decodes those schemas before VERL applies the model chat template. Use a model that supports assistant tool calls and tool-result messages for the latter. Only its messages and tools fields are given to the model: expected_config remains an evaluation oracle.

Useful overrides include TRAIN_FILE, VAL_FILE, SAVE_DIR, MAX_LENGTH, TOTAL_EPOCHS, LR, and any additional VERL/Hydra settings appended to the launcher command. Local downloads, VERL source, checkpoints, and logs are ignored by Git.

Notes

Large generated files such as checkpoints, local datasets, downloaded PDFs, caches, and logs are intentionally excluded from this code repository.