patrickbdevaney/GLM-5.3-Flash-REAP50-FP8

GLM-5.3-Flash REAP-50 (FP8)

Corrected 2026-08-28 — healing scalar re-fit

This checkpoint has been revised. The original upload applied a first-moment healing gain (median 0.6933) to every retained expert's output. That estimator ignores that norm_topk_prob renormalises the surviving top-8, so the router already returns essentially all the pruned gate mass by itself — measured: gate mass 2.5000 before pruning and 2.5000 after. The derivation therefore prescribed a ~39% shrink where the true output inflation is ~10%.

Re-measured by replaying post-prune routing from a cached router-score trace over 2.3M tokens, the correct gain is median 0.9111 (per-layer 0.846–0.950). Every layer's block scales have been multiplied by measured / shipped (median 1.3121, range 1.0719–1.5260), correcting a systematic under-scaling of the entire MoE pathway relative to attention, the shared experts and the residual stream.

The fix is exact and lossless: it scales the F32 weight_scale_inv tensors, never the FP8 values, so no weight was requantised. 6,048 block scales were corrected — exactly 42 layers x 144 retained experts.

If you downloaded the earlier revision, re-pull. The previous state remains in this repo's git history, and is recoverable by dividing the block scales by the factors above.

Not yet propagated to the NVFP4 sibling repo, which still carries the original gain.

50% of routed experts removed with REAP (Router-weighted Expert Activation Pruning, arXiv:2510.13999), calibrated on a permissively-licensed multi-domain corpus that includes real image-text pairs.

Base	zai-org/GLM-5.3-Flash (MIT, FP8 E4M3, 128x128 block scales)
Experts	288 -> 144 per layer, top-8 routing unchanged
Size	157.0 GiB (FP8)
Healed	yes — re-fit 2026-08-28, measured gain 0.9111
MTP block	excluded (see below)

Why FP8 and not BF16

The upstream release is FP8, not BF16. Routed experts are stored per-expert with their own weight_scale_inv block scales, so pruning is deleting whole tensors - lossless on every retained weight. The 642 GB BF16 repo elsewhere on the Hub is a dequantised upcast carrying no additional information.

Calibration

Mixture weighted for a coding/agentic model that stays empirically grounded: agentic 24%, code 21%, math 15%, multimodal 15%, science+bio 10%, finance 8%, ballast 7%. Permissive licences only, so this checkpoint keeps the base model's MIT lineage.

Vision is first-class: the vision tower contains no MoE and is untouched, but image tokens route through the same expert pool as text, so text-only calibration would have deleted vision-serving experts with certainty. Real image-text pairs were asserted present.

Evaluation status: NONE

This checkpoint has not been evaluated. No benchmark has been run against it - not coding, not agentic, not vision, not knowledge. What has been verified is structural: expert counts match the config, routers are sliced to the retained set, every tensor loads, the vision tower is untouched, and the MTP block is cleanly absent.

The pruning itself measured 1.29x better than random at retaining expert output contribution (saliency mass 0.643 against 0.50 for random pruning at the same ratio). That says the criterion selected well. It does not say the model is good.

Treat this as a research artifact pending evaluation, not a drop-in replacement.

Known limitations

• The MTP (multi-token-prediction) block at layer 45 is excluded. transformers' Glm5NextForConditionalGeneration does not instantiate it, so the pruning path cannot see it. Its original tensors are archived unmodified rather than inconsistently pruned.
• REAP has no published data above 50% compression; this checkpoint sits at the validated ceiling, not beyond it.
• Expect factual-recall regression before reasoning or coding regression. That is the measured failure mode of expert pruning on this architecture family: the closest published analogue (cerebras/Kimi-Linear-REAP-35B-A3B, same KDA + full-attention stack) loses 3.4 points on FRAMES at only 30% pruning while code and maths hold flat.
• Healing is a first-moment output-scale correction derived from the calibration saliency (median gain 0.696, applied exactly to the F32 block scales). It is not distillation and does not attempt to recover lost knowledge.
• Routing is disrupted more than expert count suggests: the retained experts carry ~0.90x the routing mass an average expert would, because REAP preserves rare-but-strong experts over common-but-weak ones.

Serving on Jetson Thor

Use the cutlass fused-MoE backend (the Marlin FP4 MoE kernel faults at >=256 experts) and TRITON_MLA for the 11 MLA+DSA layers (FLASHINFER is invalid for MLA).