RLinf PPO beta=0 on LIBERO-Spatial
RLinf PPO fine-tuning of the RLinf pi0.5 LIBERO SFT base, seed 42. Training ran for 100 optimizer steps and saved full and distributed-resume checkpoints every 10 steps.
Checkpoint selection uses the exact logged rollout reward at saved steps. Step 100 is the reward-best saved PPO checkpoint (reward 0.008400225).
Step-100 evaluation (generation horizon 10, denoising steps 3):
| Suite | Execute | Success once | Success at end |
|---|---|---|---|
| LIBERO-Spatial | 5 | 0.874 | 0.652 |
| LIBERO-Spatial | 10 | 0.920 | 0.706 |
| LIBERO-Plus Spatial | 5 | 0.572856 | 0.389675 |
| LIBERO-Plus Spatial | 10 | 0.618651 | 0.482515 |
Each global_step_* directory contains portable full_weights.pt weights and DCP shards for resuming. Raw training and evaluation logs are included under training/ and evaluation/.
Final Spatial PPO continuation: step 136
global_step_136 contains the final Spatial-trained plain PPO checkpoint, including
full model weights and distributed optimizer/resume state. Continued from step 100
and stopped after step 136; CSD disabled. Action expert and value head trained,
VLM frozen. Training: generation 10, execution 5, denoise 3, Flow-SDE noise 0.5,
episode horizon 240, AdamW epsilon 1e-8.
Evaluation (September 14, 2026): standard LIBERO-Spatial, 500 fixed trials (10 tasks x 50), gen10/act5, 3 ODE denoising steps, horizon240. Success-once: 91.6% (458/500). Success-at-end: 66.2% (331/500). Single evaluation, not a multi-seed mean. Environment seed0; rollout seed42 with rank offsets. Evaluated in 50-environment batches over 10 rollout epochs, strict checkpoint loading, local-only logging.
This is the Spatial-trained policy, not the separately released Object-trained PPO. Raw metrics: step136 metrics.