README.md

metadata

license: apache-2.0
language:
  - en
library_name: transformers
pipeline_tag: feature-extraction
tags:
  - medical
  - cardiovascular
  - ecg
  - ecg-text representation learning
  - ecg-foundation-model
  - pytorch

Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners

(ICML 2025)

Hung Manh Pham   Aaqib Saeed   Dong Ma  

Load with transformers==4.36.2

from transformers import AutoModel
import torch

model = AutoModel.from_pretrained("Manhph2211/D-BETA", trust_remote_code=True)
model.eval()

ecgs = torch.randn(2, 12, 5000) # [batch, leads, length]
with torch.no_grad():
    output = model(ecgs)

ecg_features = output.pooler_output
print(ecg_features.shape)  # (2, 768)

Load with the GitHub repo

Clone the project and prepare the environment:

git clone https://github.com/manhph2211/D-BETA.git && cd D-BETA
conda create -n dbeta python=3.9
conda activate dbeta
pip install -r requirements.txt

import torch
from models.processor import get_model, get_ecg_feats

model = get_model(config_path='configs/config.json', checkpoint_path='checkpoints/pytorch_model.bin')
ecgs = torch.randn(2, 12, 5000)  # [batch, leads, length]
ecg_features = get_ecg_feats(model, ecgs)
print(ecg_features.shape)  # (2, 768)

Citation

If you find this work useful, please consider citing our paper:

@inproceedings{
  hung2025boosting,
  title={Boosting Masked {ECG}-Text Auto-Encoders as Discriminative Learners},
  author={Manh Pham Hung and Aaqib Saeed and Dong Ma},
  booktitle={Forty-second International Conference on Machine Learning},
  year={2025},
}