Laguna-XS-2.1 J-lens: bf16 vs NVFP4 A/B
Two Jacobian lenses of poolside/Laguna-XS-2.1 (~33B fine-grained MoE, 40 layers, d=2048, 256 experts/8 active) fitted on byte-identical prompts (200 Γ wikitext-103, seq 128, all 39 source layers, target 39) β one from the bf16 checkpoint, one from the official NVFP4 twin. The twins' non-expert tensors are bit-identical (verified: lm_head, final norm, embeddings, attention, router), so every lens difference is attributable to NVFP4 quantization of expert weights (plus one ~0.6% export-time perturbation of the 3 layer-0 dense-MLP tensors, documented in the report).
Files
laguna_bf16_jacobian_lens.pt,laguna_nvfp4_jacobian_lens.ptβ merged n=200 lenses (fp16 payload; fp16 rounding is irrelevant at 1βCKA β 2e-9)shards_{bf16,nvfp4}/β 8 per-GPU shard lenses per arm (fp32, 25 prompts each; shard i = same prompts in both arms)convergence_{bf16,nvfp4}/β per-shard convergence CSVsresults/βab_summary.json, per-analysis npz, figures,REPORT.md
Probe: P = W_U[ids]Β·Ξ³ (plain-Ξ³ RMSNorm, no softcap), ids = 4096 frozen seed-0 over
vocab 100352; identical across arms. Fit code: open-jlens-data repo
(code/moe/fit_moe.py laguna dispatch, nvfp4_experts.py, analyze_laguna_ab.py).
Fitted + analyzed 2026-07-17. Context: this A/B calibrates the NVFP4 caveat of the
Inkling ~950B lens (PrimeIntellect/inkling-jlens) β same modelopt recipe family;
transfer caveat: Laguna experts are 512-dim vs Inkling 3072-dim.
xs2/ β Laguna-XS.2 version-drift extension (E9)
xs2/ holds the same fit for poolside/Laguna-XS.2 (bf16, identical 200 prompts/probe/tokenizer)
plus the version comparison vs XS-2.1. Verdict: version drift is a regime change β same-layer
CKA median 0.638 = 43Γ the NVFP4 effect and 54Γ prompt noise β while the depth architecture
survives (both versions k=2 at layer 27/28, cross-matrix depth correlation 0.990). The versions
share no token basis (same-token unembedding rows near-orthogonal), so each is read through its
own probe. Perturbation ladder on identical prompts (1βCKA): noise 0.007 < NVFP4 0.008 βͺ version 0.362.
