skv03/ner-span-glirel

GLiREL Multi-Domain Zero-Shot Relation Extraction

This model is a fine-tuned version of jackboyla/glirel-large-v0 for multi-domain zero-shot relation extraction.

Model Description

GLiREL (Generalist and Lightweight model for Relation Extraction) is a state-of-the-art model for zero-shot relation extraction. This version has been specifically fine-tuned on multi-domain data to improve performance across diverse domains in zero-shot scenarios.

Training Data

The model was trained on a multi-domain dataset with domain-based splits to ensure true zero-shot evaluation:

• Training Examples: N/A
• Training Domains: N/A
• Relation Types: N/A
• Entity Types: N/A

Key Features

• Zero-shot relation extraction: Can extract relations for unseen relation types
• Multi-domain capability: Trained on diverse domains for better generalization
• Domain-based splitting: Training and evaluation use different domains for true zero-shot evaluation
• Lightweight: Efficient inference while maintaining high performance

Usage

from glirel import GLiREL

# Load the model
model = GLiREL.from_pretrained("skv03/ner-span-glirel")

# Example usage
text = "John works at OpenAI in San Francisco."
labels = ["works_at", "located_in", "founded_by"]

# Extract relations
relations = model.predict_relations(text, labels)
print(relations)

Training Configuration

• Base Model: jackboyla/glirel-large-v0
• Training Steps: 15,000
• Batch Size: 6
• Learning Rate (Encoder): 1e-5
• Learning Rate (Others): 5e-5
• Max Length: 512
• Evaluation Strategy: Every 4,000 steps
• Zero-shot Setup: Domain-based splits (no domain overlap between train/test)

Model Architecture

• Label Embedding Strategy: both (label + entity token)
• Loss Function: Binary Cross Entropy
• Scheduler: Cosine with Warmup
• Dropout: 0.1
• Max Types per Batch: 50

Performance

This model is designed for zero-shot relation extraction across multiple domains. Performance metrics will vary depending on the specific domains and relation types in your use case.

Limitations

• Performance may vary significantly across different domains
• Best suited for English text
• Requires entity spans to be provided for relation extraction

Citation

If you use this model, please cite the original GLiREL paper:

@misc{boylan2025glirelgeneralistmodel,
      title={GLiREL -- Generalist Model for Zero-Shot Relation Extraction}, 
      author={Jack Boylan and Chris Hokamp and Demian Gholipour Ghalandari},
      year={2025},
      eprint={2501.03172},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2501.03172},
}

Model Card Authors

Created by the GLiREL fine-tuning team.