Overwatch886/bge-base-agri

SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Represent this sentence for searching relevant passages: What fertiliser should I use for rice?',
    'Crop Rice | Country Tanzania | Title Fertiliser recommendation for Rice | Text Fertiliser programme for rice (Oryza sativa). Apply phosphorus and potassium basally and split nitrogen into three: at transplanting/establishment, active tillering, and panicle initiation. Avoid blanket high nitrogen, which worsens blast and blight; aim for balanced ~100-120 kg N/ha by leaf-colour chart. | Source CGIAR',
    'Crop Cowpea | Country Ghana | Title Fertiliser recommendation for Cowpea (Humid forest) | Text Fertiliser programme for cowpea in the humid forest zone, where heavy rainfall leaches and acidifies soils and fungal pressure is high. As a nitrogen-fixing legume, cowpea needs little nitrogen; apply a phosphorus-rich basal (SSP or DAP) to feed nodulation and add potassium on depleted soils. In the humid forest, correct soil acidity with lime, watch for rapid fungal spread under the canopy, and improve drainage on the heavy leached soils. | Source FAO',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8315, 0.4254],
#         [0.8315, 1.0000, 0.5164],
#         [0.4254, 0.5164, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 742 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 742 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 17 tokens mean: 22.11 tokens max: 29 tokens	min: 53 tokens mean: 117.47 tokens max: 275 tokens	min: 52 tokens mean: 113.78 tokens max: 269 tokens

• Samples:

  anchor	positive	negative
  Represent this sentence for searching relevant passages: How do I cope with flooding and excess rain on my farm?	Crop (general) | Country Ghana | Title Adapting to flooding and excess rain (Highlands) | Text Adapting to flooding and excess rain in the cool highlands, where heavy dew and mild temperatures favour foliar fungal diseases. Plant on ridges or raised beds, open drainage channels, choose flood-tolerant varieties, and stagger planting to spread risk. In the highlands, the long dewy nights drive foliar disease, so prioritise resistant varieties, wider spacing for airflow, and early scouting. | Source AGRA	Crop (general) | Country Tanzania | Title Managing waterlogging (Sahel) | Text Managing waterlogging in the hot semi-arid Sahel, where rains are short and erratic and sandy soils leach nutrients quickly. Cut drainage channels, plant on ridges or raised beds, improve structure with organic matter, and choose tolerant crops/varieties. Because sandy Sahelian soils leach fast, split applications finely, micro-dose into the planting hole, and conserve every drop of moisture with mulch and tied ridges. | Source Plantwise
  Represent this sentence for searching relevant passages: How do I cope with flooding and excess rain on my farm?	Crop (general) | Country Nigeria | Title Adapting to flooding and excess rain (Humid forest) | Text Adapting to flooding and excess rain in the humid forest zone, where heavy rainfall leaches and acidifies soils and fungal pressure is high. Plant on ridges or raised beds, open drainage channels, choose flood-tolerant varieties, and stagger planting to spread risk. In the humid forest, correct soil acidity with lime, watch for rapid fungal spread under the canopy, and improve drainage on the heavy leached soils. | Source FAO	Crop (general) | Country Nigeria | Title Managing waterlogging (Humid forest) | Text Managing waterlogging in the humid forest zone, where heavy rainfall leaches and acidifies soils and fungal pressure is high. Cut drainage channels, plant on ridges or raised beds, improve structure with organic matter, and choose tolerant crops/varieties. In the humid forest, correct soil acidity with lime, watch for rapid fungal spread under the canopy, and improve drainage on the heavy leached soils. | Source CGIAR
  Represent this sentence for searching relevant passages: How do I cope with flooding and excess rain on my farm?	Crop (general) | Country Tanzania | Title Adapting to flooding and excess rain (Sahel) | Text Adapting to flooding and excess rain in the hot semi-arid Sahel, where rains are short and erratic and sandy soils leach nutrients quickly. Plant on ridges or raised beds, open drainage channels, choose flood-tolerant varieties, and stagger planting to spread risk. Because sandy Sahelian soils leach fast, split applications finely, micro-dose into the planting hole, and conserve every drop of moisture with mulch and tied ridges. | Source FAO	Crop (general) | Country Tanzania | Title Managing waterlogging (Guinea savanna) | Text Managing waterlogging in the Guinea savanna, with one long rainy season and moderately fertile soils. Cut drainage channels, plant on ridges or raised beds, improve structure with organic matter, and choose tolerant crops/varieties. In the Guinea savanna, time operations to the single rains, top-dress at the onset of steady rainfall, and use the long season for a full-duration variety. | Source ICRISAT

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• learning_rate: 2e-05
• weight_decay: 0.01
• warmup_steps: 0.1
• fp16: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• do_predict: False
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 8
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: None
• warmup_steps: 0.1
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 42
• data_seed: None
• bf16: False
• fp16: True
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.8333	10	1.3753
1.6667	20	0.6970
2.5	30	0.6573

Training Time

• Training: 1.9 minutes

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.4.1
• Transformers: 5.0.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0
• Datasets: 5.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}