Great point—recall@20 after the 32× stage would clearly show what is lost during early filtering. We’ll include it alongside final nDCG and latency. 😏
🔄 In a Training Loop
Xinping Zhao
Yuki131
AI & ML interests
LLMs, RAG, Embedding, Reranker——A Pokémon Trainer on a journey to become a Pokémon Master.
Recent Activity
liked a model 16 minutes ago
opensearch-project/opensearch-neural-sparse-encoding-doc-v3-gte liked a model about 2 hours ago
Qwen/Qwen3.8-2.4T-A95BOrganizations
replied to their post 3 days ago
reacted to their post with 👍🔥 4 days ago
Post
2142
Test-Time Scaling for Rerankers?
Can rerankers scale at test time—not by generating longer reasoning traces, but by selectively using richer document representations?
KaLM-Reranker-V1 supports Matryoshka compression from 1× to 32×, which suggests a progressive multi-fidelity pipeline:
- Embedding retrieval → Top-100
- KaLM-Reranker @ 32× compression → Top-20
- The same reranker @ 2× compression → final ranking
The intuition is simple: cheaply screen many candidates, then allocate higher-fidelity cross-attention only to the most promising ones.
For 100@32× → 20@2×, the passage-token interaction budget is roughly 31.8% of directly running 100@2×, before fixed model overheads. The key question is whether it can retain nearly the same ranking quality.
We’re considering evaluating nDCG–latency Pareto curves.
Would you consider this a useful form of test-time scaling for retrieval?
KaLM-Embedding/KaLM-Reranker-V1-Nano
KaLM-Embedding/KaLM-Reranker-V1-Small
KaLM-Embedding/KaLM-Reranker-V1-Large
KaLM-Reranker-V1: Fast but Not Late Interaction for Compressed Document Reranking (2606.22807)
https://huggingface.co/collections/KaLM-Embedding/lychee-kalm-reranker
KaLM-Embedding
Can rerankers scale at test time—not by generating longer reasoning traces, but by selectively using richer document representations?
KaLM-Reranker-V1 supports Matryoshka compression from 1× to 32×, which suggests a progressive multi-fidelity pipeline:
- Embedding retrieval → Top-100
- KaLM-Reranker @ 32× compression → Top-20
- The same reranker @ 2× compression → final ranking
The intuition is simple: cheaply screen many candidates, then allocate higher-fidelity cross-attention only to the most promising ones.
For 100@32× → 20@2×, the passage-token interaction budget is roughly 31.8% of directly running 100@2×, before fixed model overheads. The key question is whether it can retain nearly the same ranking quality.
We’re considering evaluating nDCG–latency Pareto curves.
Would you consider this a useful form of test-time scaling for retrieval?
KaLM-Embedding/KaLM-Reranker-V1-Nano
KaLM-Embedding/KaLM-Reranker-V1-Small
KaLM-Embedding/KaLM-Reranker-V1-Large
KaLM-Reranker-V1: Fast but Not Late Interaction for Compressed Document Reranking (2606.22807)
https://huggingface.co/collections/KaLM-Embedding/lychee-kalm-reranker
- 2 replies
posted an update 4 days ago
Post
2142
Test-Time Scaling for Rerankers?
Can rerankers scale at test time—not by generating longer reasoning traces, but by selectively using richer document representations?
KaLM-Reranker-V1 supports Matryoshka compression from 1× to 32×, which suggests a progressive multi-fidelity pipeline:
- Embedding retrieval → Top-100
- KaLM-Reranker @ 32× compression → Top-20
- The same reranker @ 2× compression → final ranking
The intuition is simple: cheaply screen many candidates, then allocate higher-fidelity cross-attention only to the most promising ones.
For 100@32× → 20@2×, the passage-token interaction budget is roughly 31.8% of directly running 100@2×, before fixed model overheads. The key question is whether it can retain nearly the same ranking quality.
We’re considering evaluating nDCG–latency Pareto curves.
Would you consider this a useful form of test-time scaling for retrieval?
KaLM-Embedding/KaLM-Reranker-V1-Nano
KaLM-Embedding/KaLM-Reranker-V1-Small
KaLM-Embedding/KaLM-Reranker-V1-Large
KaLM-Reranker-V1: Fast but Not Late Interaction for Compressed Document Reranking (2606.22807)
https://huggingface.co/collections/KaLM-Embedding/lychee-kalm-reranker
KaLM-Embedding
Can rerankers scale at test time—not by generating longer reasoning traces, but by selectively using richer document representations?
KaLM-Reranker-V1 supports Matryoshka compression from 1× to 32×, which suggests a progressive multi-fidelity pipeline:
- Embedding retrieval → Top-100
- KaLM-Reranker @ 32× compression → Top-20
- The same reranker @ 2× compression → final ranking
The intuition is simple: cheaply screen many candidates, then allocate higher-fidelity cross-attention only to the most promising ones.
For 100@32× → 20@2×, the passage-token interaction budget is roughly 31.8% of directly running 100@2×, before fixed model overheads. The key question is whether it can retain nearly the same ranking quality.
We’re considering evaluating nDCG–latency Pareto curves.
Would you consider this a useful form of test-time scaling for retrieval?
KaLM-Embedding/KaLM-Reranker-V1-Nano
KaLM-Embedding/KaLM-Reranker-V1-Small
KaLM-Embedding/KaLM-Reranker-V1-Large
KaLM-Reranker-V1: Fast but Not Late Interaction for Compressed Document Reranking (2606.22807)
https://huggingface.co/collections/KaLM-Embedding/lychee-kalm-reranker
- 2 replies
reacted to Banaxi-Tech's post with 👀 about 1 month ago
Post
10762
A new model is coming!
Its going to take a long time on my 5070 Ti so expect a release in ~1 month.
We think this model is going to be SOTA For its size.
Our Mini Version will be 25M Parameters and Pro with 140M.
The Pro version has a 3072 Context Window (Extensible to up to 6K with RoPE) And the Mini version has a context window of 4096 (Up to 8K with RoPE)
Meanwhile we are currently working on a Instruct Version of our BananaMind 1.5 Base.
The training will start this weekend
We are very exited to release it when its done!
Its going to take a long time on my 5070 Ti so expect a release in ~1 month.
We think this model is going to be SOTA For its size.
Our Mini Version will be 25M Parameters and Pro with 140M.
The Pro version has a 3072 Context Window (Extensible to up to 6K with RoPE) And the Mini version has a context window of 4096 (Up to 8K with RoPE)
Meanwhile we are currently working on a Instruct Version of our BananaMind 1.5 Base.
The training will start this weekend
We are very exited to release it when its done!
- 12 replies
reacted to tomaarsen's post with 🔥 about 1 month ago
Post
1580
🤗 Announcing the Ettin Reranker family: six new state-of-the-art CrossEncoder rerankers for search from 17M to 1B parameters, plus the full training data and the ~150-line recipe. Built on the Ettin ModernBERT encoders, Apache 2.0. Details:
All six were trained with the same single-stage pointwise MSE distillation recipe, with mixedbread-ai/mxbai-rerank-large-v2 (1.54B) as the teacher. Only the learning rate and per-device batch size change between sizes. The 1B student matches the teacher within 0.0001 NDCG@10 on MTEB(eng, v2) Retrieval, the 150M is the strongest reranker I tested in the under-600M range, and the 17M beats the 33M ms-marco-MiniLM-L12-v2 by +0.051 NDCG@10 at roughly half the parameter count.
Speed matters as much as quality for a reranker, since it determines whether the model fits the latency budget between retrieval and showing results. Our 17M is the fastest reranker in the whole comparison at 7517 pairs/sec on an H100. Our 150M runs 2.3x faster than the two other 150M ModernBERT-base rerankers (gte-reranker-modernbert-base and granite-embedding-reranker-english-r2) because the modular Transformer module propagates unpadded inputs through every layer rather than just the FA2 attention kernel. And our 1B is 2.4x faster than its 1.5B teacher while matching it on quality.
I bootstrapped the training recipe with the new train-sentence-transformers Agent Skill shipped in Sentence Transformers v5.5.0. Install it with
I wrote a blog post walking through usage, results across six embedder pairings, the speed story, and the complete training script. Check it out, or just point your Agent to the URL:
https://huggingface.co/blog/ettin-reranker
Collection: https://huggingface.co/collections/cross-encoder/ettin-rerankers
All six were trained with the same single-stage pointwise MSE distillation recipe, with mixedbread-ai/mxbai-rerank-large-v2 (1.54B) as the teacher. Only the learning rate and per-device batch size change between sizes. The 1B student matches the teacher within 0.0001 NDCG@10 on MTEB(eng, v2) Retrieval, the 150M is the strongest reranker I tested in the under-600M range, and the 17M beats the 33M ms-marco-MiniLM-L12-v2 by +0.051 NDCG@10 at roughly half the parameter count.
Speed matters as much as quality for a reranker, since it determines whether the model fits the latency budget between retrieval and showing results. Our 17M is the fastest reranker in the whole comparison at 7517 pairs/sec on an H100. Our 150M runs 2.3x faster than the two other 150M ModernBERT-base rerankers (gte-reranker-modernbert-base and granite-embedding-reranker-english-r2) because the modular Transformer module propagates unpadded inputs through every layer rather than just the FA2 attention kernel. And our 1B is 2.4x faster than its 1.5B teacher while matching it on quality.
I bootstrapped the training recipe with the new train-sentence-transformers Agent Skill shipped in Sentence Transformers v5.5.0. Install it with
hf skills add train-sentence-transformers --claude and ask Claude Code (or Codex / Cursor / Gemini CLI) to fine-tune a SentenceTransformer, CrossEncoder, or SparseEncoder model on your data.I wrote a blog post walking through usage, results across six embedder pairings, the speed story, and the complete training script. Check it out, or just point your Agent to the URL:
https://huggingface.co/blog/ettin-reranker
Collection: https://huggingface.co/collections/cross-encoder/ettin-rerankers
reacted to mlabonne's post with 🚀🔥🔥 10 months ago
Post
8506
LiquidAI/LFM2-8B-A1B just dropped!
8.3B params with only 1.5B active/token 🚀
> Quality ≈ 3–4B dense, yet faster than Qwen3-1.7B
> MoE designed to run on phones/laptops (llama.cpp / vLLM)
> Pre-trained on 12T tokens → strong math/code/IF
8.3B params with only 1.5B active/token 🚀
> Quality ≈ 3–4B dense, yet faster than Qwen3-1.7B
> MoE designed to run on phones/laptops (llama.cpp / vLLM)
> Pre-trained on 12T tokens → strong math/code/IF
- 1 reply