XLOverflow/qwen3-eagle3-prm-g1.0

PRM (γ=1.0) — EAGLE3 Draft Model for Qwen3-8B

Our method. Per-step loss weighted by a Process Reward Model score: w_s = 1 + γ · PRM', γ=1.0.

Part of a course project evaluating per-step weighted loss functions for training EAGLE3 draft models. Full pipeline and source: https://github.com/XLOverflow/anlp_course_project

Collection: Qwen3 EAGLE3 — Weighted Loss Variants

Training

• Framework: SpecForge (our fork: https://github.com/XLOverflow/SpecForge)
• Target model: Qwen/Qwen3-8B
• Draft init: AngelSlim/Qwen3-8B_eagle3
• Data: ShareGPT-style reasoning traces (see scripts/data/ in project repo)
• Loss weight: w_s = 1 + γ · PRM', γ=1.0
• PRM: Qwen/Qwen2.5-Math-PRM-7B (see scripts/data/prm_score.py)
• Initialized from: baseline-uniform/epoch_4_step_82000
• Additional epochs: 1

Evaluation (Qwen3-8B target)

Dataset	τ (accept. length)	Speedup	Accuracy
GSM8K	7.354	4.473×	95.38%
MATH500	7.319	4.612×	94.40%

Baselines for reference: Vanilla ≈ 1× speedup, EAGLE-orig ≈ 2× speedup.

Files

• model.safetensors — draft model weights (~763 MB)
• config.json — model config
• Corresponds to: outputs/eagle3-prm-g1.0/epoch_0_step_17026 in the original training output

Optimizer state (~3 GB) is not uploaded — use the project repo's training scripts to resume from scratch if needed.

Usage

from huggingface_hub import snapshot_download
draft_path = snapshot_download(repo_id="XLOverflow/qwen3-eagle3-prm-g1.0")
# Then load with EAGLE's EaModel — see scripts/eval/eval_combined.py in the project repo.