hudini model checkpoints
A HUD/UI parser for the da Vinci Xi system overlay.
This repository holds the model weights for hudini. The da Vinci Xi system overlays status information onto the endoscopic view, and hudini reads that overlay out of a recording. Each checkpoint below reads one kind of overlay element.
Checkpoints
| File | What it reads | Architecture | Input |
|---|---|---|---|
camera_state_cnn.pt |
active or inactive state of the camera status pod | MobileNetV4-conv-small | 32x192 pod crop |
arm_digit_cnn.pt |
arm digit 1-4 in the status pod circle | small VGG-style CNN, ~6k parameters | 24x24 circle crop |
offscreen_state_rfdetr.pt |
off-screen indicator bars at the frame edge, active or inactive | RF-DETR-nano | active region at 704 px |
offscreen_digit_cnn.pt |
arm digit of a detected off-screen bar | MobileNetV4-conv-small, four sigmoid heads | 96x96 bar-end crops |
tool_association_rfdetr.pt |
per-arm tool association badges, digit 1-4 | RF-DETR-nano | active region at 576 px |
All files are PyTorch checkpoints. Each one loads with torch.load(..., weights_only=True) and contains a state dict plus metadata. The RF-DETR checkpoints state their own model class, resolution, and class names.
Training data
The models were trained on frames from surgical video datasets that show the Da Vinci Xi user interface. The CNN classifiers saw interface crops only. The RF-DETR detectors saw full video frames.
Intended use
Research on surgical video analysis. The models read user-interface elements. They make no clinical judgment and are not a medical device.