mlboydaisuke/xfeat-litert

XFeat (Accelerated Features) — LiteRT (CompiledModel GPU)

XFeat (Apache-2.0, ~1.5M, a lightweight pure-CNN local feature extractor for image matching — SLAM / AR / image registration) re-authored to a GPU-native LiteRT .tflite via litert_torch. FP16, 1.4 MB, input [1, 480, 640, 1] NHWC normalized grayscale.

Verified on a Pixel 8a: full LITERT_CL residency (72/72 nodes, 1 partition), ~0.4 ms, GPU output matches CPU/PyTorch (corr 0.9999).

I/O

• Input [1, 480, 640, 1] NHWC, grayscale, per-image InstanceNorm applied host-side ((g - mean)/sqrt(var+1e-5) over the image).
• Outputs (all at H/8 × W/8 = 60×80): feats [1,64,60,80] dense descriptors; keypoints [1,65,60,80] keypoint logits; heatmap [1,1,60,80] reliability. Keypoint NMS, descriptor bilinear-sampling, and mutual-nearest-neighbor matching run host-side.

GPU-clean re-authoring

• Input gray + InstanceNorm moved host-side (its spatial reduction over H·W would overflow fp16 on the delegate).
• _unfold2d(x, 8) (space-to-depth via unfold → >4-D / GATHER_ND) → a one-hot Conv2d(1,64,k=8,s=8) (exact, single CONV_2D). Result: zero GATHER/SELECT/TopK/Cast, no >4-D — full GPU residency.

Training data & PII

XFeat is trained on public correspondence data (MegaDepth + synthetic homographies). It outputs geometric keypoints/descriptors only — no faces, identities, or personal attributes. Official weights; only the op graph was re-authored for GPU.

Sample app + conversion script

https://github.com/google-ai-edge/litert-samples (compiled_model_api, two-image matching).