Hugging Face's logo

whilethis
/
NEMESIS

medical-imaging
self-supervised-learning
masked-autoencoder
3d-ct
pretraining
Model card Files
xet
 Community
NEMESIS
File size: 2,233 Bytes
1c09125
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
---
license: mit
tags:
  - medical-imaging
  - self-supervised-learning
  - masked-autoencoder
  - 3d-ct
  - pretraining
---

# NEMESIS

**Superpatch-based 3D Medical Image Self-Supervised Pretraining via Noise-Enhanced Dual-Masking**

> IEEE AICAS 2026

## Overview

NEMESIS is a self-supervised pretraining framework for 3D CT volumes using:
- **Superpatch processing** (128³ sub-volumes) — memory-efficient ViT pretraining
- **Dual-masking (MATB)** — plane-wise (xy) + axis-wise (z) masking, exploiting CT anisotropy
- **NEMESIS Tokens (NTs)** — learnable tokens summarising visible patches via cross-attention
- **Noise-enhanced reconstruction** — Gaussian noise injection for regularisation

### Key result (BTCV organ classification, frozen linear probe)

| Method | AUROC |
|---|---|
| **NEMESIS (frozen)** | **0.9633** |
| SuPreM (fine-tuned) | 0.9493 |
| VoCo (fine-tuned) | 0.9387 |

## Checkpoints

| File | embed_dim | depth | mask_ratio |
|---|---|---|---|
| `MAE_768_0.5.pt` | 768 | 6 | 0.5 | ← **main model (paper)** |
| `MAE_768_0.25.pt` | 768 | 6 | 0.25 | ablation |
| `MAE_768_0.75.pt` | 768 | 6 | 0.75 | ablation |
| `MAE_576_0.5.pt` | 576 | 6 | 0.5 | ablation |
| `MAE_384_0.5.pt` | 384 | 6 | 0.5 | ablation |
| (others) | | | | embed_dim × mask_ratio ablations |

## Usage

```bash
pip install huggingface_hub
huggingface-cli download whilethis/NEMESIS MAE_768_0.5.pt --local-dir pretrained/
```

```python
import torch
from nemesis.models.mae import MAEgic3DMAE

ckpt = torch.load("pretrained/MAE_768_0.5.pt", map_location="cpu")
model = MAEgic3DMAE(
    embed_dim=768, depth=6, num_heads=8,
    decoder_embed_dim=128, decoder_depth=3,
    num_maegic_tokens=8,
)
model.load_state_dict(ckpt["model_state_dict"])
encoder = model.encoder
```

## Code

[https://github.com/whilethis00/NEMESIS-public](https://github.com/whilethis00/NEMESIS-public)

## Citation

```bibtex
@inproceedings{jung2026nemesis,
  title     = {{NEMESIS}: Superpatch-based 3{D} Medical Image Self-Supervised Pretraining via Noise-Enhanced Dual-Masking},
  author    = {Jung, Hyeonseok and others},
  booktitle = {IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS)},
  year      = {2026},
}
```