ppsingh/iki_sector_setfit

SetFit with ppsingh/SECTOR-multilabel-mpnet_w

This is a SetFit model that can be used for Text Classification. This SetFit model uses ppsingh/SECTOR-multilabel-mpnet_w as the Sentence Transformer embedding model. A SetFitHead instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: ppsingh/SECTOR-multilabel-mpnet_w
• Classification head: a SetFitHead instance
• Maximum Sequence Length: 512 tokens
• Number of Classes: 4 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("ppsingh/iki_sector_setfit")
# Run inference
preds = model("In the shipping and aviation sectors, emission reduction efforts will be focused on distributing eco-friendly ships and enhancing the operational efficiency of aircraft. Agriculture, livestock farming and fisheries: The Republic Korea is introducing various options to accelerate low-carbon farming, for instance, improving irrigation techniques in rice paddies and adopting low-input systems for nitrogen fertilizers.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	35	76.164	170

• Training Dataset: 250

  Class	Positive Count of Class
  Economy-wide	88
  Energy	63
  Other Sector	64
  Transport	139

• Validation Dataset: 42

  Class	Positive Count of Class
  Economy-wide	15
  Energy	11
  Other Sector	11
  Transport	24

Training Hyperparameters

• batch_size: (16, 2)
• num_epochs: (1, 10)
• max_steps: -1
• sampling_strategy: oversampling
• body_learning_rate: (2e-05, 1e-05)
• head_learning_rate: 0.01
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.01
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0005	1	0.2029	-
0.0993	200	0.0111	0.1124
0.1985	400	0.0063	0.111
0.2978	600	0.0183	0.1214
0.3970	800	0.0197	0.1248
0.4963	1000	0.0387	0.1339
0.5955	1200	0.0026	0.1181
0.6948	1400	0.0378	0.1208
0.7940	1600	0.0285	0.1267
0.8933	1800	0.0129	0.1254
0.9926	2000	0.0341	0.1271

Classifier Training Results

Epoch	Training F1-micro	Training F1-Samples	Training F1-weighted	Validation F1-micro	Validation F1-samples	Validation F1-weighted
0	0.954	0.972	0.945	0.824	0.819	0.813
1	0.994	0.996	0.994	0.850	0.832	0.852
2	0.981	0.989	0.979	0.850	0.843	0.852
3	0.995	0.997	0.995	0.852	0.843	0.858
4	0.994	0.996	0.994	0.852	0.843	0.858
5	0.995	0.997	0.995	0.859	0.848	0.863

label	precision	recall	f1-score	support
Economy-wide	0.857	0.800	0.827	15.0
Energy	1.00	0.818	0.900	11.0
Other Sector	0.615	0.727	0.667	11.0
Transport	0.958	0.958	0.958	24.0

• Micro Avg: Precision = 0.866, Recall = 0.852, F1 = 0.859504
• Samples Avg: Precision = 0.869, Recall = 0.861, F1 = 0.848

Environmental Impact

Carbon emissions were measured using CodeCarbon.

• Carbon Emitted: 0.026 kg of CO2
• Hours Used: 0.622 hours

Training Hardware

• On Cloud: No
• GPU Model: 1 x Tesla T4
• CPU Model: Intel(R) Xeon(R) CPU @ 2.00GHz
• RAM Size: 12.67 GB

Framework Versions

• Python: 3.10.12
• SetFit: 1.0.3
• Sentence Transformers: 2.3.1
• Transformers: 4.35.2
• PyTorch: 2.1.0+cu121
• Datasets: 2.3.0
• Tokenizers: 0.15.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}