Recharge23/FastWAM-single

Fast-WAM GELLO task specialists

The complete real-world fine-tuning and inference code is also available in JAM-realrobot. See GitHub code and integration guide for the workflow and client interface.

This repository contains five independently fine-tuned Fast-WAM adapters for Franka GELLO tasks 4, 6, 7, 8, and 9. Each task has its own weights, normalization, frozen prompt embedding, and inference report. Each adapter starts independently from the same original Fast-WAM Base checkpoint. The delivered adapters require the original base weights and VAE; they are not standalone models.

The implementation uses video and action supervision during training. At inference, it consumes the current external RGB image and measured robot state and predicts an action chunk without generating future observations.

Checkpoints

Folder	Task	Adaptation
task_4	4	Independent fine-tuning
task_6	6	Independent fine-tuning
task_7	7	Independent fine-tuning
task_8	8	Independent fine-tuning
task_9	9	Independent fine-tuning

See INFERENCE.md for installation and the GELLO client interface. Use the exact prompt.txt from the selected task folder.

Base model and adaptation

• Base: yuanty/fastwam, revision 8eaceeb24c3cc92ff2a9c9a9d266a4941b836705, file libero_uncond_2cam224.pt. This is the LIBERO Fast-WAM Base checkpoint.
• Original model source: 7faa71108368fbb3b6885649f112af607427a2d4.
• VAE: Wan-AI/Wan2.2-TI2V-5B, revision 921dbaf3f1674a56f47e83fb80a34bac8a8f203e, file Wan2.2_VAE.pth.
• The original model is loaded strictly before replacing its seven-dimensional action and eight-dimensional proprioception interfaces with freshly initialized pose10 projections. These replacements are trained and included in each adapter.
• FP32 master LoRA parameters use rank 32, alpha 32, and no dropout. There are 611 adapted linear layers and 124,050,442 trainable parameters across 1,228 tensors, including the new projections.
• The native video/action flow objectives and attention masks are retained. VAE preprocessing runs eagerly with BF16 inputs outside autocast; this avoids a CUDA-graph tracing issue without changing the VAE computation.

Data and optimization

Each task uses 50 demonstrations split into 45 training and five held-out episodes. Tasks 6–9 use dataset revision 0685cc7194a817d22d958e494cd35120aa7effa3; Task 4 uses 83b5cbeec3e156494bd42fb9211789bde0bb7909.

The JAM-matched elements are the admitted episodes, train/validation split, task prompt, action windows, normalization, target definition, global batch of eight, and seed 42. Every normalized action target was checked against the corresponding JAM target. Architecture, initialization, and adaptation parameters differ, so this is not a controlled comparison at equal compute.

Training uses microbatch four with two accumulation passes, AdamW with betas (0.9, 0.95), weight decay 0.01, gradient clipping at 1.0, and constant learning rates of 3e-6 for the video-side LoRA parameters and 1e-5 for action-side parameters and new projections. Complete optimizer-state save/restore was checked before continuation, and the final adapter was checked against the saved training state.

The training clip contains nine frames at source indices 0, 4, …, 32, encoded afresh per window. Actions use 32 targets at 15 Hz. Inference uses only the current image. Dataset admission and final inference checks compare this image's latent with the first latent in the corresponding training clip.

Action and observation contract

The camera input is one external RGB view, with no wrist view. A native 480 × 640 image is padded to 512 × 640 with black rows. Preprocessing removes that padding, resizes to 224 × 320 with bilinear antialiasing, and maps RGB values to [-1, 1].

State and action use [x, y, z, rotation_column_1(3), rotation_column_2(3), aperture]. Poses are absolute base-frame flange poses at panda_link8, not joint positions, TCP poses, deltas, or Euler angles. Position is in metres. Aperture is zero for closed and one for open. Targets use the next achieved pose and the commanded binary aperture. The server applies and reverses the task's frozen normalization and projects predicted rotation columns once to an orthonormal rotation.

Validation and limitations

The Task 4 and Task 9 specialists have been updated with verified continued adapters. Their task folders contain refreshed final-weight inference reports; the existing inference interface is unchanged. Task 9's final held-out action flow loss is above its lowest intermediate value, so this release is not presented as a validation-selected best checkpoint. Recorded prediction diagnostics and physical task success are different measurements.

Each task's INFERENCE_REPORT.json contains diagnostics for all 40 fixed held-out windows across five episodes, selected before training. It reports position error, rotation error, aperture classification agreement, and inference latency. Final-weight tests also verify deterministic inference, the GELLO query HTTP interface, and exact native-versus-portable output agreement on a fixed recorded input in the same tested environment.

These are software and recorded-data checks. They do not measure closed-loop task success, robustness to a changed camera setup, or performance on the physical robot. Target spacing is 15 Hz; it does not imply the inference service runs at 15 requests per second. Hardware control and camera calibration remain the responsibility of the deployment environment.

PUBLIC_SHA256.json and per-task BUNDLE_MANIFEST.json provide file integrity checks. Raw demonstrations, optimizer states, and credentials are not part of this release. Original source attribution and its MIT license are retained under inference/source/; original model assets retain their upstream terms.

Recorded validation summary

Task	Windows	XYZ L2 (m)	First-eight XYZ L2 (m)	Rotation (degrees)	Aperture agreement
4	40	0.1153	0.0756	8.87	0.646
6	40	0.0560	0.0409	6.28	0.802
7	40	0.0759	0.0574	7.12	0.737
8	40	0.0594	0.0444	5.89	1.000
9	40	0.1221	0.0883	7.65	0.695

Fine-tuning source and workflow

See FINETUNE.md for the complete portable workflow and finetune/ for the code, frozen data recipes, asset downloader, training/resume controller and inference exporter.