mentee-embed-v2 (125M) — trilingual embeddings trained from scratch

A compact 125M-parameter, 768-dim text embedding model for Arabic, English and Urdu, trained entirely from scratch (no pretrained base) in two stages on a single consumer GPU.

Recipe

1. Stage A — Masked language modeling on ~31M tokens across the three languages
2. Stage B — Contrastive fine-tuning (symmetric InfoNCE, in-batch negatives + hard-negative column, temperature 0.02) on 1797858 triplets incl. 300K English↔Urdu parallel pairs (OPUS-100)

Results (Protocol B — 15K-doc corpus-pool retrieval, higher is better)

| 1 | paraphrase-multilingual-mpnet-base-v2 | MRR 0.904 · R@5 0.960 · R@100 0.997 | MRR 0.677 · R@5 0.763 · R@100 0.917 | MRR 0.556 · R@5 0.623 · R@100 0.847 | 0.712 | | 2 | paraphrase-multilingual-MiniLM-L12-v2 | MRR 0.896 · R@5 0.943 · R@100 0.993 | MRR 0.659 · R@5 0.730 · R@100 0.930 | MRR 0.499 · R@5 0.577 · R@100 0.800 | 0.685 | | 3 | all-MiniLM-L6-v2 | MRR 0.916 · R@5 0.967 · R@100 1.000 | MRR 0.100 · R@5 0.000 · R@100 0.000 | MRR 0.100 · R@5 0.000 · R@100 0.007 | 0.372 | | 4 | mentee-embed-v2 (ours) ⭐ ours | MRR 0.236 · R@5 0.280 · R@100 0.747 | MRR 0.116 · R@5 0.040 · R@100 0.330 | MRR 0.124 · R@5 0.050 · R@100 0.400 | 0.159 |

Our model reaches 0.159 vs leader paraphrase-multilingual-mpnet-base-v2 at 0.712 on this suite. Protocol details: queries ranked against the full pool; baselines run with their recommended usage (E5 query/passage prefixes applied).

Usage

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("<your-hf-user>/mentee-embed-v1")
emb = model.encode(["مرحبا بالعالم", "hello world", "دنیا میں خوش آمدید"])

Limitations

NLI+parallel derived training data; small-pool protocol is easier than full MIRACL; expect open-domain gaps versus web-scale models.