| | --- |
| | {} |
| | --- |
| | # AM-RADIO: Reduce All Domains Into One |
| |
|
| | Mike Ranzinger, Greg Heinrich, Jan Kautz, Pavlo Molchanov |
| |
|
| | [NVIDIA Research](https://www.nvidia.com/en-us/research/) |
| |
|
| | \[[Paper](https://arxiv.org/abs/2312.06709)\]\[[BibTex](#citing-radio)\] |
| |
|
| | ## Pretrained Models |
| |
|
| |
|
| | ### HuggingFace Hub |
| |
|
| | Pull the E-RADIO model from a Python script: |
| |
|
| | ```Python |
| | from transformers import AutoModel |
| | model = AutoModel.from_pretrained("nvidia/E-RADIO", trust_remote_code=True) |
| | ``` |
| |
|
| | ### Usage |
| |
|
| | E-RADIO will return a tuple with two tensors. |
| | The `summary` is similar to the `cls_token` in ViT and is meant to represent the general concept of the entire image. |
| | It has shape $(B,C)$ with $B$ being the batch dimension, and $C$ being some number of channels. |
| | The `spatial_features` represent more localized content which should be suitable for dense tasks such as semantic segmentation, or for integration into an LLM. |
| | Spatial features have shape $(B,H,W,D)$ with $H$ being the height, and $W$ being the width of the spatial features. |
| |
|
| | ## Training |
| |
|
| | _Coming Soon_ |
| |
|
| | ## License |
| |
|
| | RADIO code and weights are released under the [NSCLv1 License](LICENSE). |
| |
|
| | ## Citing RADIO |
| |
|
| | If you find this repository useful, please consider giving a star and citation: |
| | ``` |
| | @misc{ranzinger2023amradio, |
| | title={AM-RADIO: Agglomerative Model -- Reduce All Domains Into One}, |
| | author={Mike Ranzinger and Greg Heinrich and Jan Kautz and Pavlo Molchanov}, |
| | year={2023}, |
| | eprint={2312.06709}, |
| | archivePrefix={arXiv}, |
| | primaryClass={cs.CV} |
| | } |
| | ``` |
| |
|
