alantang2025/reranker-MiniLM-L12-H384-uncased-msmarco-margin-mse

CrossEncoder based on microsoft/MiniLM-L12-H384-uncased

This is a Cross Encoder model finetuned from microsoft/MiniLM-L12-H384-uncased on the msmarco dataset using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: microsoft/MiniLM-L12-H384-uncased
• Maximum Sequence Length: 512 tokens
• Number of Output Labels: 1 label
• Training Dataset:
  • msmarco
• Language: en

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("alantang2025/reranker-MiniLM-L12-H384-uncased-msmarco-margin-mse")
# Get scores for pairs of texts
pairs = [
    ['what is mcafee sidewinder control center', 'McAfee Firewall Enterprise Control Center. McAfee Firewall Enterprise Control Center (CommandCenterâ\x84¢) provides a central interface for simplifying the management of multiple McAfee Firewall Enterprise (Sidewinder®) appliances.'],
    ['where is cork ie', 'Cork, Irish Corcaigh (â\x80\x9cMarshâ\x80\x9d), seaport and seat of County Cork, in the province of Munster, Ireland. It is located at the head of Cork Harbour on the River Lee. Cork is, after Dublin, the Irish republicâ\x80\x99s second largest conurbation. The city is administratively independent of the county.'],
    ['what is a embedded computer system', 'Embedded Computer Systems. An embedded system is a special-purpose system in which the computer is completely encapsulated by the device it controls. Unlike a general-purpose computer, such as a personal computer, an embedded system performs pre-defined tasks, usually with very specific requirements.'],
    ['who is kennedy space center named after', "Cecil replies: Sure, but let's get our facts straight: they didn't change the name of the space center, they changed the name of the cape â\x80\x94 i.e., the land under the space center (or under part of it, anyway). The NASA launch facility continues to be known as the John F. Kennedy Space Center. The whole confusing business got started back on November 27, 1963, shortly after JFK's assassination, when Lyndon Johnson was casting about for a suitable memorial for the slain president."],
    ['how to calculate protein requirement for a horse', 'Although protein is listed as a percentage on feed tags, the National Research Councilâ\x80\x99s (NRC) latest recommendation for horses lists the protein requirement unit in grams. To understand how much protein to feed a horse, we need to do some math and further reading.Many people think that a 30 percent protein feed is way too much for a horse to handle. If one looks a little closer, the tag of Hubbard Lifeâ\x80\x99s 30% Supplement indicates that a 1,100-pound horse should consume one pound per day.Thirty percent of one pound is 136 grams of protein.verage mixed hay is about 17 percent protein. Feeding 15 pounds of hay alone can deliver 1,157 grams of protein to a horse. Understanding the protein content and feeding rates of all the feed sources a horse has access to is key to achieving the proper protein ratio.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'what is mcafee sidewinder control center',
    [
        'McAfee Firewall Enterprise Control Center. McAfee Firewall Enterprise Control Center (CommandCenterâ\x84¢) provides a central interface for simplifying the management of multiple McAfee Firewall Enterprise (Sidewinder®) appliances.',
        'Cork, Irish Corcaigh (â\x80\x9cMarshâ\x80\x9d), seaport and seat of County Cork, in the province of Munster, Ireland. It is located at the head of Cork Harbour on the River Lee. Cork is, after Dublin, the Irish republicâ\x80\x99s second largest conurbation. The city is administratively independent of the county.',
        'Embedded Computer Systems. An embedded system is a special-purpose system in which the computer is completely encapsulated by the device it controls. Unlike a general-purpose computer, such as a personal computer, an embedded system performs pre-defined tasks, usually with very specific requirements.',
        "Cecil replies: Sure, but let's get our facts straight: they didn't change the name of the space center, they changed the name of the cape â\x80\x94 i.e., the land under the space center (or under part of it, anyway). The NASA launch facility continues to be known as the John F. Kennedy Space Center. The whole confusing business got started back on November 27, 1963, shortly after JFK's assassination, when Lyndon Johnson was casting about for a suitable memorial for the slain president.",
        'Although protein is listed as a percentage on feed tags, the National Research Councilâ\x80\x99s (NRC) latest recommendation for horses lists the protein requirement unit in grams. To understand how much protein to feed a horse, we need to do some math and further reading.Many people think that a 30 percent protein feed is way too much for a horse to handle. If one looks a little closer, the tag of Hubbard Lifeâ\x80\x99s 30% Supplement indicates that a 1,100-pound horse should consume one pound per day.Thirty percent of one pound is 136 grams of protein.verage mixed hay is about 17 percent protein. Feeding 15 pounds of hay alone can deliver 1,157 grams of protein to a horse. Understanding the protein content and feeding rates of all the feed sources a horse has access to is key to achieving the proper protein ratio.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Reranking

• Datasets: NanoMSMARCO_R100, NanoNFCorpus_R100 and NanoNQ_R100
• Evaluated with CrossEncoderRerankingEvaluator with these parameters:

  {
      "at_k": 10,
      "always_rerank_positives": true
  }
  

Metric	NanoMSMARCO_R100	NanoNFCorpus_R100	NanoNQ_R100
map	0.5676 (+0.0780)	0.3529 (+0.0919)	0.7124 (+0.2928)
mrr@10	0.5587 (+0.0812)	0.6198 (+0.1200)	0.7270 (+0.3003)
ndcg@10	0.6364 (+0.0960)	0.4158 (+0.0907)	0.7557 (+0.2551)

Cross Encoder Nano BEIR

• Dataset: NanoBEIR_R100_mean
• Evaluated with CrossEncoderNanoBEIREvaluator with these parameters:

  {
      "dataset_names": [
          "msmarco",
          "nfcorpus",
          "nq"
      ],
      "rerank_k": 100,
      "at_k": 10,
      "always_rerank_positives": true
  }
  

Metric	Value
map	0.5443 (+0.1542)
mrr@10	0.6352 (+0.1672)
ndcg@10	0.6026 (+0.1473)

Training Details

Training Dataset

msmarco

• Dataset: msmarco at 9e329ed
• Size: 1,990,000 training samples
• Columns: score, query, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	score	query	positive	negative
  type	float	string	string	string
  details	min: -1.47 mean: 13.31 max: 22.71	min: 10 characters mean: 33.44 characters max: 105 characters	min: 75 characters mean: 355.82 characters max: 952 characters	min: 85 characters mean: 338.69 characters max: 1197 characters

• Samples:

  score	query	positive	negative
  20.983789285024006	what is a french bulldog	The French Bulldog is a small breed of domestic dog. Frenchies were the result in the 1800s of a cross between bulldog ancestors imported from England and local ratters in Paris, France.In the UK, they moved up to become the fourth most popular registered dog by 2014.he modern French Bulldog breed descends directly from the dogs of the Molossians, an ancient Greek tribe.	The many faces of Duke Dog: Bulldog, human mascot, statue and cartoon. In all likelihood, James Madison University is the only college or university in the country whose athletic teams draw their nickname from the name of the school's president.
  20.930927753448486	difference between hostname and domain name	hostname is the name given to the end-point (the machine in question) and will be used to identify it over DNS if that is configured. domain is the name given to the 'network' it will be required to reach the network from an external point (like the Internet)	Determining the domain of a function. Determine the domains of functions according to various considerations. 1 How to determine the domain of a radical function (example) (Video). 2 How to determine the domain of algebraic functions (examples) (Video). 3 Domain of algebraic functions (Exercise).
  12.55810809135437	will sod come back if overwatered	Root Death. Pull up pieces of the sod in different areas after the sod has established for a few weeks. Healthy sod that gets the right amount of water will resist the pull because the roots are healthy and growing. Overwatered sod will come right up because its roots are dying or dead.f the soil is wet and mushy, you're watering too much. Let the sod dry out until the top 2 inches of soil are dry and crumbly. Washington State University recommends early morning watering, which allows soil and grass blades to dry out thoroughly during the day.	THE ADVANTAGES OF HYDRO SEEDING! Hydroseeding is a fast, cost effective way to have a new lawn that will turn your neighbors green with envy. Hydroseeding costs only a little more than old fashioned methods using dry seeding techniques combined with a messy straw mulch.ydro Seeding mulch adds to the humus content of a lawn as it decomposes. The bacterial action of straw will leach nitrogen from the soil as it decomposes. ADVANTAGES OF HYDROSEEDING OVER SOD Sod is a good solution to the need for a new lawn. It is expensive. Sod generally costs 3 to 5 times more than hydroseeding.

• Loss: MarginMSELoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity"
  }
  

Evaluation Dataset

msmarco

• Dataset: msmarco at 9e329ed
• Size: 10,000 evaluation samples
• Columns: score, query, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	score	query	positive	negative
  type	float	string	string	string
  details	min: -1.29 mean: 13.64 max: 22.17	min: 11 characters mean: 33.82 characters max: 98 characters	min: 55 characters mean: 360.68 characters max: 990 characters	min: 57 characters mean: 340.77 characters max: 964 characters

• Samples:

  score	query	positive	negative
  18.650261163711548	what is mcafee sidewinder control center	McAfee Firewall Enterprise Control Center. McAfee Firewall Enterprise Control Center (CommandCenter™) provides a central interface for simplifying the management of multiple McAfee Firewall Enterprise (Sidewinder®) appliances.	JSC's Mission Control — now formally known as the Christopher C. Kraft, Jr. Mission Control Center — has helped plan, support and operate every NASA human spaceflight mission since 1965.
  14.571923971176147	where is cork ie	Cork, Irish Corcaigh (“Marsh”), seaport and seat of County Cork, in the province of Munster, Ireland. It is located at the head of Cork Harbour on the River Lee. Cork is, after Dublin, the Irish republic’s second largest conurbation. The city is administratively independent of the county.	suberin. (biochemistry). A fatty substance found in many plant cell walls, especially cork.
  18.947856426239014	what is a embedded computer system	Embedded Computer Systems. An embedded system is a special-purpose system in which the computer is completely encapsulated by the device it controls. Unlike a general-purpose computer, such as a personal computer, an embedded system performs pre-defined tasks, usually with very specific requirements.	Confidence votes 13.4K. It means that an application (a software, a computer program) is not compatible (it doesn't work well or doesn't work at all)with the operating system running on a computer. Applications are software programs that you can download onto your computer and some mobile phones. A computer game, for example, is an application. An MP3 player system … on your computer or phone is also an application.. + 12 others found this useful. Ashley Reeves. Answered.

• Loss: MarginMSELoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 8e-06
• num_train_epochs: 1
• warmup_ratio: 0.1
• seed: 12
• bf16: True
• dataloader_num_workers: 4
• load_best_model_at_end: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 8e-06
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 12
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 4
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	NanoMSMARCO_R100_ndcg@10	NanoNFCorpus_R100_ndcg@10	NanoNQ_R100_ndcg@10	NanoBEIR_R100_mean_ndcg@10
-1	-1	-	-	0.0185 (-0.5220)	0.1835 (-0.1415)	0.0374 (-0.4632)	0.0798 (-0.3756)
0.0000	1	183.8208	-	-	-	-	-
0.0322	4000	148.9381	-	-	-	-	-
0.0643	8000	27.4436	-	-	-	-	-
0.0965	12000	8.3597	-	-	-	-	-
0.1286	16000	6.374	-	-	-	-	-
0.1608	20000	5.4584	4.7464	0.6619 (+0.1215)	0.3947 (+0.0696)	0.7165 (+0.2158)	0.5910 (+0.1356)
0.1930	24000	4.9963	-	-	-	-	-
0.2251	28000	4.6626	-	-	-	-	-
0.2573	32000	4.3576	-	-	-	-	-
0.2894	36000	4.1884	-	-	-	-	-
0.3216	40000	4.0283	4.4428	0.6594 (+0.1190)	0.3927 (+0.0676)	0.7435 (+0.2428)	0.5985 (+0.1432)
0.3538	44000	3.8617	-	-	-	-	-
0.3859	48000	3.734	-	-	-	-	-
0.4181	52000	3.5985	-	-	-	-	-
0.4503	56000	3.6228	-	-	-	-	-
0.4824	60000	3.4607	3.4983	0.6285 (+0.0881)	0.4059 (+0.0809)	0.7429 (+0.2423)	0.5924 (+0.1371)
0.5146	64000	3.4429	-	-	-	-	-
0.5467	68000	3.3256	-	-	-	-	-
0.5789	72000	3.2512	-	-	-	-	-
0.6111	76000	3.2302	-	-	-	-	-
0.6432	80000	3.1449	3.0647	0.6427 (+0.1023)	0.4070 (+0.0820)	0.7428 (+0.2421)	0.5975 (+0.1421)
0.6754	84000	3.1304	-	-	-	-	-
0.7075	88000	3.0615	-	-	-	-	-
0.7397	92000	3.0513	-	-	-	-	-
0.7719	96000	3.0657	-	-	-	-	-
0.804	100000	2.9726	3.0527	0.6364 (+0.0960)	0.4158 (+0.0907)	0.7557 (+0.2551)	0.6026 (+0.1473)
0.8362	104000	2.9537	-	-	-	-	-
0.8683	108000	2.9453	-	-	-	-	-
0.9005	112000	2.8856	-	-	-	-	-
0.9327	116000	2.9301	-	-	-	-	-
0.9648	120000	2.8464	2.9493	0.6475 (+0.1070)	0.4105 (+0.0855)	0.7406 (+0.2399)	0.5995 (+0.1442)
0.9970	124000	2.8968	-	-	-	-	-
-1	-1	-	-	0.6364 (+0.0960)	0.4158 (+0.0907)	0.7557 (+0.2551)	0.6026 (+0.1473)

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.9
• Sentence Transformers: 5.1.2
• Transformers: 4.51.3
• PyTorch: 2.6.0
• Accelerate: 1.11.0
• Datasets: 3.6.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MarginMSELoss

@misc{hofstätter2021improving,
    title={Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation},
    author={Sebastian Hofstätter and Sophia Althammer and Michael Schröder and Mete Sertkan and Allan Hanbury},
    year={2021},
    eprint={2010.02666},
    archivePrefix={arXiv},
    primaryClass={cs.IR}
}