GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval
Paper • 2112.07577 • Published
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
This is the zero-shot baseline model in the paper "GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval"
The training setup:
- Start from
distilbert-base-uncased; - Mine 50 hard negatives for each query on MS MARCO with
sentence-transformers/msmarco-distilbert-base-v3andsentence-transformers/msmarco-MiniLM-L-6-v3; - Do Margin-MSE training on the tuples (including queries, gold relevant, and hard negatives) with the teacher model
cross-encoder/ms-marco-MiniLM-L-6-v2for 70K steps with batch size 75, max. sequence-length 350.
- Downloads last month
- 96