Instructions to use voxide/voxide-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sam2
How to use voxide/voxide-models with sam2:
# Use SAM2 with images import torch from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained("voxide/voxide-models") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): predictor.set_image(<your_image>) masks, _, _ = predictor.predict(<input_prompts>)# Use SAM2 with videos import torch from sam2.sam2_video_predictor import SAM2VideoPredictor predictor = SAM2VideoPredictor.from_pretrained("voxide/voxide-models") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): state = predictor.init_state(<your_video>) # add new prompts and instantly get the output on the same frame frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>) # propagate the prompts to get masklets throughout the video for frame_idx, object_ids, masks in predictor.propagate_in_video(state): ... - Notebooks
- Google Colab
- Kaggle
Voxide segmentation models (ONNX)
ONNX exports of SAM 2.1 and SAM-Med3D (turbo) used by Voxide, a GPU volume viewer for microscopy, for interactive segmentation. Each model is an encoder/decoder pair run on the CPU with ONNX Runtime.
Voxide ships the SAM 2.1 Tiny pair. It downloads the others on request from this repository at a
pinned revision and checks every file against the SHA-256 in SHA256SUMS, so a file
that does not match is never loaded.
It also downloads six network-feature models for its pixel classifier (a DINOv2 ViT-S/14, a DINOv2 ViT-B/14, the first two VGG16 blocks, the Cellpose cyto3 backbone and micro-sam's ViT-T light-microscopy and electron-microscopy encoders), pinned and checked the same way: see Network-feature models for their files, provenance and licences.
And it downloads one spot-detection model for finding dots: Spotiflow's hybiss model, see
Spot-detection model.
And one nucleus-segmentation model for finding nuclei: StarDist's 2D_versatile_fluo, see
Nucleus-segmentation model.
Files
| Model | Encoder | Decoder | Use in Voxide |
|---|---|---|---|
| SAM 2.1 Hiera Tiny | sam2.1_hiera_tiny_encoder.onnx (110 MB) |
sam2.1_hiera_tiny_decoder.onnx (17 MB) |
2D slices (default, fastest) |
| SAM 2.1 Hiera Small | sam2.1_hiera_small_encoder.onnx (139 MB) |
sam2.1_hiera_small_decoder.onnx (17 MB) |
2D slices |
| SAM 2.1 Hiera Base+ | sam2.1_hiera_base_plus_encoder.onnx (278 MB) |
sam2.1_hiera_base_plus_decoder.onnx (17 MB) |
2D slices |
| SAM 2.1 Hiera Large | sam2.1_hiera_large_encoder.onnx (853 MB) |
sam2.1_hiera_large_decoder.onnx (17 MB) |
2D slices (best masks, slowest) |
| SAM-Med3D turbo | sammed3d_turbo_encoder.onnx (373 MB) |
sammed3d_turbo_decoder.onnx (31 MB) |
3D volumes |
Using them without the in-app download
Download the files you need and either use Models → Import model… in Voxide or copy them
into Voxide's model folder. The file names must stay as they are: Voxide finds each model by its
<name>_encoder.onnx / <name>_decoder.onnx pair.
pip install -U huggingface_hub
hf download voxide/voxide-models --include "sam2.1_hiera_small_*" --local-dir voxide-models
How they were made
Unmodified inference exports of the upstream checkpoints, produced with PyTorch 2.9.0 (ONNX opset 18, IR version 8). No fine-tuning, quantization or other change to the weights.
License and attribution
Both upstream model families are released under the Apache License 2.0 (see LICENSE),
and so are these exports.
- SAM 2.1 © Meta Platforms, Inc. — https://github.com/facebookresearch/sam2
- SAM-Med3D © the SAM-Med3D authors — https://github.com/uni-medical/SAM-Med3D
The network-feature models, the spot-detection model and the nucleus-segmentation models carry their
own licences, so this card's licence is per file (license: other): Apache-2.0, BSD-3-Clause or
CC-BY-4.0 (micro-sam's ViT-T encoder), as listed for each file in
their section, in
the spot-detection section and in
the nucleus-segmentation section, with the attribution
each licence requires under Licences and attribution,
Spotiflow's licence and
StarDist's licence and
InstanSeg's licence.
If you use these models in published work, please cite the original papers:
@article{ravi2024sam2,
title = {SAM 2: Segment Anything in Images and Videos},
author = {Ravi, Nikhila and others},
journal = {arXiv preprint arXiv:2408.00714},
year = {2024}
}
@article{wang2023sammed3d,
title = {SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Images},
author = {Wang, Haoyu and others},
journal = {arXiv preprint arXiv:2310.15161},
year = {2023}
}
Network-feature models (pixel classifier)
Feature extractors for Voxide's pixel classifier: each turns an image tile into a per-pixel feature map that the classifier trains on. Single files, run on the CPU with ONNX Runtime; Voxide finds each by its exact file name, so keep the names as they are when importing them by hand.
| Model | File | Size | SHA-256 | Licence |
|---|---|---|---|---|
| DINOv2 ViT-S/14 with registers | dinov2-with-registers-small.onnx |
88 522 426 bytes (89 MB) | 815e440d222e60294bb5b165491c4760e7f08dc7ecd2488a5450e93d50887efe |
Apache-2.0 |
| VGG16 blocks 1–2 | vgg16-blocks12.onnx |
1 042 858 bytes (1 MB) | 230fb1431be9f9b6088fc68c24d4ff8c4894e7ef8ae3b16e14e0116b351f1b45 |
BSD-3-Clause |
| Cellpose cyto3 backbone | cellpose-cyto3-backbone.onnx |
26 483 022 bytes (26 MB) | e689706a9709ba7fcf70480019bdd1a6a4a94eff05277dfd64d024f2d015195d |
BSD-3-Clause |
| DINOv2 ViT-B/14 with registers | dinov2-with-registers-base.onnx |
346 622 747 bytes (347 MB) | 419f8937888b0d4cd607dccaed37b790b48d79cd1aad65a66bf96c7769e3b48b |
Apache-2.0 |
| micro-sam ViT-T (light microscopy) | microsam-vit-t-lm-encoder.onnx |
27 983 702 bytes (28 MB) | f4ee03d6c4e5286ee2a4caf92212ce2cc90349f9f26b0d45c3490e989dc814b3 |
CC-BY-4.0 |
| micro-sam ViT-T (electron microscopy) | microsam-vit-t-em-encoder.onnx |
27 983 702 bytes (28 MB) | 6479517ccbfe9e5174e00c2235d28ce0f33405698cfdc0764256cbdfe81b1ad9 |
CC-BY-4.0 |
hf download voxide/voxide-models --include "dinov2-*" "vgg16-*" "cellpose-*" --local-dir voxide-models
hf download voxide/voxide-models --include "microsam-*" --local-dir voxide-models
Provenance
- DINOv2 ViT-S/14 with registers — Meta AI's
facebook/dinov2-with-registers-small, in the fp32 ONNX export by onnx-community (onnx-community/dinov2-with-registers-small, fileonnx/model.onnxat revisionec7abea1a8757ec4f9f7b26399d31234ffaa6e6a), copied here unchanged under a new name. Inputpixel_values[N, 3, H, W](H and W multiples of 14), outputlast_hidden_state; opset 14. - DINOv2 ViT-B/14 with registers — Meta AI's
facebook/dinov2-with-registers-base, in the fp32 ONNX export by onnx-community (onnx-community/dinov2-with-registers-base, fileonnx/model.onnxat revisionfcd050203baf48e192055356f379b5949a111255), copied here unchanged under a new name. Inputpixel_values[N, 3, H, W](H and W multiples of 14), outputlast_hidden_state; opset 14. Voxide reads 768 features per 14 × 14 patch (768 × 32 × 32 for a 448 × 448 tile). - VGG16 blocks 1–2 — torchvision's ImageNet VGG16 (
VGG16_Weights.IMAGENET1K_V1,vgg16-397923af.pth), cut after the second convolution block: the ReLU outputs of conv1_2 (64 channels) and of conv2_2 (128 channels, upsampled ×2 inside the graph), concatenated to a 192-channel stride-1 map. Exported byscripts/export_vgg16_onnx.pyin the Voxide repository (pinned torch 2.9, torchvision 0.24, onnx 1.22; opset 17). The weights are unmodified. - Cellpose cyto3 backbone — Cellpose 3's built-in
cyto3model (cellpose 3.1.1), the U-Net's 32-channel stride-1 decoder output before the final 1×1 output convolution (no flows or cell probability). Inputpixel_values[1, 2, H, W](H and W multiples of 16; channel 0 the 1st–99th-percentile-normalised plane, channel 1 the nuclei channel or zeros). Exported byscripts/export_cellpose_onnx.pyin the Voxide repository (pinned cellpose 3.1, onnx 1.22; opset 17). The weights are unmodified. - micro-sam ViT-T (light microscopy) — the image encoder of micro-sam's SAM LM Generalist
ViT-T (
vit_t_lm), from the BioImage Model Zoo recordfaithful-chickenv1.1, filevit_t.pt(SHA-25632b92ab64959088878a1cc3e52b9b64aaa0ba4b24c98e9959746d7390bec7b7a). Exported byscripts/export_microsam_vit_t_onnx.pyin the Voxide repository (pinned torch 2.14, onnx 1.22; opset 17): the image encoder alone (no prompt encoder or mask decoder), with a bilinear resize of the 448 × 448 input to 1024 × 1024 baked in front of it and its attention biases set to zero, as micro-sam's runtime has them. Inputpixel_values[1, 3, 448, 448](fixed; ImageNet-normalised RGB), outputfeatures[1, 256, 64, 64]: one 256-vector per 7 × 7 input pixels. - micro-sam ViT-T (electron microscopy) — the image encoder of micro-sam's SAM EM Organelle
Generalist ViT-T (
vit_t_em_organelles), from the BioImage Model Zoo recordgreedy-whalev1.3, filevit_t.pt(SHA-256d8422c7a2d84ac338476ae59f4c5b56ee44860f7b9fae9269a1270131a496c10; itsmodel_state). Exported by the same script (--model em) into the same graph as the light-microscopy encoder, with the trained attention biases kept, as the Zoo record's own test embedding was computed. Same input and output.
Licences and attribution
- DINOv2 © Meta Platforms, Inc., Apache License 2.0 (see
LICENSE) — https://github.com/facebookresearch/dinov2, https://huggingface.co/facebook/dinov2-with-registers-small; ONNX export by onnx-community — https://huggingface.co/onnx-community/dinov2-with-registers-small. - DINOv2 ViT-B/14 © Meta Platforms, Inc., Apache License 2.0 (see
LICENSE) — https://github.com/facebookresearch/dinov2, https://huggingface.co/facebook/dinov2-with-registers-base; ONNX export by onnx-community — https://huggingface.co/onnx-community/dinov2-with-registers-base. - VGG16 weights from torchvision — https://github.com/pytorch/vision, BSD-3-Clause (https://github.com/pytorch/vision/blob/main/LICENSE).
- Cellpose © Howard Hughes Medical Institute (HHMI) / MouseLand — https://github.com/MouseLand/cellpose,
BSD-3-Clause (https://github.com/MouseLand/cellpose/blob/main/LICENSE). The same cyto3 weights
are redistributed by the BioImage Model Zoo as
famous-fish("CellPose(cyto3)", https://bioimage.io/#/?id=famous-fish) under BSD-3-Clause. Note: per the Cellpose README, the datasets the built-in models were trained on include images under CC-BY-NC licences; check that your use fits those terms. Use of the built-in models in Voxide was confirmed with the Cellpose authors on 2026-10-24. - micro-sam ViT-T (CC-BY-4.0) — SAM LM Generalist (ViT-T) image encoder from micro-sam, by
Anwai Archit and Constantin Pape (University of Göttingen;
https://github.com/computational-cell-analytics/micro-sam), BioImage Model Zoo
faithful-chickenv1.1 (https://bioimage.io/#/?id=faithful-chicken), licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Changed: the image encoder alone, converted to ONNX, with a bilinear resize of a 448 × 448 input to 1024 × 1024 in front of it and its attention biases set to zero as micro-sam's runtime has them. Provided as is, without warranties. Please cite: Archit, A., Freckmann, L., Nair, S. et al. Segment Anything for Microscopy. Nat Methods 22, 579–591 (2025). https://doi.org/10.1038/s41592-024-02580-4. Architecture: MobileSAM / TinyViT (Apache-2.0, https://github.com/ChaoningZhang/MobileSAM), built on Segment Anything (© Meta, Apache-2.0). - micro-sam ViT-T EM (CC-BY-4.0) — SAM EM Organelle Generalist (ViT-T) image encoder from
micro-sam, by Anwai Archit and Constantin Pape (University of Göttingen;
https://github.com/computational-cell-analytics/micro-sam), BioImage Model Zoo
greedy-whalev1.3 (https://bioimage.io/#/?id=greedy-whale), licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Changed: the image encoder alone, converted to ONNX, with a bilinear resize of a 448 × 448 input to 1024 × 1024 in front of it. Provided as is, without warranties. Please cite: Archit, A., Freckmann, L., Nair, S. et al. Segment Anything for Microscopy. Nat Methods 22, 579–591 (2025). https://doi.org/10.1038/s41592-024-02580-4. Architecture: MobileSAM / TinyViT (Apache-2.0), built on Segment Anything (© Meta, Apache-2.0).
If you use these models in published work, please cite the original papers:
@article{oquab2023dinov2,
title = {DINOv2: Learning Robust Visual Features without Supervision},
author = {Oquab, Maxime and others},
journal = {arXiv preprint arXiv:2304.07193},
year = {2023}
}
@article{darcet2023registers,
title = {Vision Transformers Need Registers},
author = {Darcet, Timoth{\'e}e and Oquab, Maxime and Mairal, Julien and Bojanowski, Piotr},
journal = {arXiv preprint arXiv:2309.16588},
year = {2023}
}
@article{simonyan2014vgg,
title = {Very Deep Convolutional Networks for Large-Scale Image Recognition},
author = {Simonyan, Karen and Zisserman, Andrew},
journal = {arXiv preprint arXiv:1409.1556},
year = {2014}
}
@article{stringer2021cellpose,
title = {Cellpose: a generalist algorithm for cellular segmentation},
author = {Stringer, Carsen and Wang, Tim and Michaelos, Michalis and Pachitariu, Marius},
journal = {Nature Methods},
volume = {18},
pages = {100--106},
year = {2021}
}
@article{stringer2025cellpose3,
title = {Cellpose3: one-click image restoration for improved cellular segmentation},
author = {Stringer, Carsen and Pachitariu, Marius},
journal = {Nature Methods},
year = {2025}
}
@article{archit2025microsam,
title = {Segment Anything for Microscopy},
author = {Archit, Anwai and Freckmann, Luca and Nair, Sushmita and others},
journal = {Nature Methods},
volume = {22},
pages = {579--591},
year = {2025},
doi = {10.1038/s41592-024-02580-4}
}
Spot-detection model (dots)
Voxide's Find dots can use Spotiflow instead of its built-in method, per channel, on single planes. One file, run with ONNX Runtime on the CPU or, in a build with CUDA, on the graphics card; Voxide finds it by its exact file name.
| Model | File | Size | SHA-256 | Licence |
|---|---|---|---|---|
Spotiflow hybiss (2D) |
spotiflow-hybiss.onnx |
37 903 970 bytes (38 MB) | 90d6eabdd7eb00c2564af54d83e1e237cb132f9f564a405d18dbe2bb54663689 |
BSD-3-Clause |
hf download voxide/voxide-models --include "spotiflow-*" --local-dir voxide-models
Provenance
Spotiflow's pretrained hybiss model (trained on HybISS images), from the spotiflow-models
release 0.6.0 (hybiss.zip, MD5 254afa97c137d0bd74fd9c1827f0e323, checkpoint best.pt, SHA-256
fa5d5cb313bcfd75527f3d0962ccc3654bc7aa33359097d16f3b0835b0103bd1). Exported by
scripts/export_spotiflow_onnx.py in the Voxide repository (pinned spotiflow 0.6.5, torch 2.15.0
CPU, onnx 1.23.2; opset 17): the network alone, unmodified weights. Input image [1, 1, H, W]
(the plane scaled by its 1st and 99.8th percentiles, as Spotiflow does; H and W multiples of 16),
outputs prob [1, 1, H, W] (the spot probability) and flow [1, 3, H, W] (the stereographic
flow Voxide refines positions with). The export matches PyTorch within 1e-4, and Voxide's peak
finding and refinement reproduce Spotiflow.predict's points.
Spotiflow's licence and attribution
- Spotiflow (code) © 2023, Albert Dominguez Mantes, Martin Weigert, BSD-3-Clause — https://github.com/weigertlab/spotiflow.
- Spotiflow pretrained models (weights) © 2025, weigertlab, BSD-3-Clause — https://github.com/weigertlab/spotiflow-models.
Redistribution of the weights in ONNX form keeps the BSD-3-Clause notice above; the licence text is in the Spotiflow repositories linked. Please cite:
@article{dominguezmantes2025spotiflow,
title = {Spotiflow: accurate and efficient spot detection for fluorescence microscopy with deep stereographic flow regression},
author = {Dominguez Mantes, Albert and Herrera, Antonio and Khven, Irina and Schlaeppi, Anjalie and Kyriacou, Eftychia and Tsissios, Georgios and Skoufa, Evangelia and Santangeli, Luca and Buglakova, Elena and Durmus, Emine Berna and Manley, Suliana and Kreshuk, Anna and Arendt, Detlev and Aztekin, Can and Lingner, Joachim and La Manno, Gioele and Weigert, Martin},
journal = {Nature Methods},
year = {2025},
doi = {10.1038/s41592-025-02662-x}
}
Nucleus-segmentation models (Find nuclei)
Voxide's Find nuclei tool outlines nuclei with StarDist or InstanSeg, plane by plane, and joins the planes of a Z-stack into 3D objects. The files run with ONNX Runtime on the CPU or, in a build with CUDA, on the graphics card; Voxide finds them by their exact file names.
| Model | File | Size | SHA-256 | Licence |
|---|---|---|---|---|
StarDist 2D_versatile_fluo |
stardist-2d-versatile-fluo.onnx |
5 715 272 bytes (6 MB) | 285e52c264fef4ee81b501793c80637136bb82d142e5ecadc40cf1e4051234f6 |
BSD-3-Clause |
InstanSeg single_channel_nuclei 0.1.2 (network) |
instanseg-single-channel-nuclei.onnx |
15 286 204 bytes (15 MB) | d58e25c563adbdb8d6b4f187d186f0500a9b0e4e07f946cfca6d9134a6acbbae |
Apache-2.0 |
InstanSeg single_channel_nuclei 0.1.2 (per-seed classifier) |
instanseg-single-channel-nuclei-seeds.onnx |
994 bytes | de91e9755111c884cfe13a420a952c48c59196909a5e2ab38569f0eab3060b41 |
Apache-2.0 |
hf download voxide/voxide-models --include "stardist-*" "instanseg-*" --local-dir voxide-models
Provenance
StarDist's pretrained 2D_versatile_fluo model (trained on a subset of the DSB 2018 nuclei
images, fluorescent nuclear stains), from the stardist-models release v0.1
(python_2D_versatile_fluo.zip, SHA-256
4ad678d0758eed6e55625f1b5ae30771e59adb79f1239e09b9772eac8846c3dd). Exported by
scripts/export_stardist_onnx.py in the Voxide repository (pinned stardist 0.9.2, TensorFlow
2.15.1, tf2onnx 1.17.0, onnx 1.19.0; opset 17): the network alone, unmodified weights. Input
image [1, H, W, 1] (the plane scaled by its 1st and 99.8th percentiles, as StarDist does; H and
W multiples of 16), outputs prob [1, H/2, W/2, 1] (the object probability) and dist
[1, H/2, W/2, 32] (the 32 ray distances of the star-convex polygons). The export matches
TensorFlow within 3e-5, and Voxide's polygon and non-maximum-suppression steps reproduce
StarDist2D.predict_instances' labels on the test crops pixel for pixel.
StarDist's licence and attribution
- StarDist (code and pretrained models) © 2018-2025, Uwe Schmidt, Martin Weigert, BSD-3-Clause — https://github.com/stardist/stardist, https://github.com/stardist/stardist-models.
Redistribution of the weights in ONNX form keeps the BSD-3-Clause notice above; the licence text is
in LICENSE-stardist.txt. Please cite:
@inproceedings{schmidt2018stardist,
title = {Cell Detection with Star-Convex Polygons},
author = {Schmidt, Uwe and Weigert, Martin and Broaddus, Coleman and Myers, Gene},
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2018},
pages = {265--273},
year = {2018},
doi = {10.1007/978-3-030-00934-2_30}
}
InstanSeg's provenance
InstanSeg's single_channel_nuclei model 0.1.2 (fluorescent nuclei, one channel), from the
instanseg release instanseg_models_v0.1.2 (single_channel_nuclei.zip, SHA-256
6bc2e4cd8acd9bea64a04e926a74a53f922d76fd6c1dc633ca0bca335df4d4d7; instanseg.pt SHA-256
118066231fb7753ffb048bcd2164186397bcc5c8fc4ad7f826efe928b515794c). Exported by
scripts/export_instanseg_onnx.py in the Voxide repository (instanseg-torch 0.1.2, onnx 1.23.2;
opset 17), unmodified weights, as two files:
instanseg-single-channel-nuclei.onnx: the network, with the model's input clamp to [-2, 3] in front. Inputimage[1, 1, H, W](the plane scaled by its 0.1st and 99.9th percentiles; H and W multiples of 16, reflect-padded at the bottom and right), outputprediction[1, 5, H, W](pixel embeddings, seed map and mask logits).instanseg-single-channel-nuclei-seeds.onnx: the per-seed pixel classifier (61 parameters), with its sigmoid behind. Inputfeatures[N, 4], outputprobability[N, 1].
The exports match PyTorch within 1.3e-4, and Voxide's seed, window and merge steps reproduce
model(x)'s labels on the test crops pixel for pixel (minimum size 20, merging on the overlap over
the smaller area).
InstanSeg's licence and attribution
- InstanSeg (code and pretrained models) © Thibaut Goldsborough, University of Edinburgh, Apache-2.0 — https://github.com/instanseg/instanseg.
Redistribution in ONNX form keeps the Apache-2.0 notice above; the licence text is in
LICENSE-instanseg.txt. The modifications are the split into two files,
the input clamp and the sigmoid named above. Please cite:
@article{goldsborough2024instanseg,
title = {InstanSeg: an embedding-based instance segmentation algorithm optimized for accurate,
efficient and portable cell segmentation},
author = {Goldsborough, Thibaut and others},
journal = {arXiv preprint arXiv:2408.15954},
year = {2024}
}
@article{goldsborough2024channelinvariant,
title = {A novel channel invariant architecture for the segmentation of cells and nuclei in
multiplexed images using InstanSeg},
author = {Goldsborough, Thibaut and others},
journal = {bioRxiv},
year = {2024},
doi = {10.1101/2024.09.04.611150}
}
Model tree for voxide/voxide-models
Base model
facebook/dinov2-with-registers-base