Update README.md
Browse files
README.md
CHANGED
|
@@ -12,13 +12,13 @@ datasets:
|
|
| 12 |
- truehealth/medqa
|
| 13 |
metrics:
|
| 14 |
- accuracy
|
| 15 |
-
base_model:
|
| 16 |
-
- Qwen/Qwen3-Embedding-0.6B
|
| 17 |
pipeline_tag: text-ranking
|
| 18 |
language:
|
| 19 |
- zh
|
| 20 |
- en
|
| 21 |
library_name: transformers
|
|
|
|
|
|
|
| 22 |
---
|
| 23 |
# Diver-Retriever-0.6B
|
| 24 |
|
|
@@ -38,7 +38,7 @@ as well as the Mteb-Medical Benchmark.
|
|
| 38 |
|
| 39 |
- **Model type:** Text Embedding
|
| 40 |
- **Language(s) (NLP):** Bilingual (Chinese & English)
|
| 41 |
-
- **Context Length:**
|
| 42 |
- **Number of Paramaters:** 0.6B
|
| 43 |
|
| 44 |
For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our GitHub (https://github.com/AQ-MedAI/Diver).
|
|
@@ -213,7 +213,23 @@ For more details, including benchmark evaluation, hardware requirements, and inf
|
|
| 213 |
<td style="text-align:right">30.5</td>
|
| 214 |
</tr>
|
| 215 |
<tr>
|
| 216 |
-
<td>DIVER-Retriever</td>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 217 |
<td style="text-align:right"><strong>28.9</strong></td>
|
| 218 |
<td style="text-align:right"><strong>41.8</strong></td>
|
| 219 |
<td style="text-align:right">43.7</td>
|
|
@@ -360,7 +376,7 @@ For more details, including benchmark evaluation, hardware requirements, and inf
|
|
| 360 |
<td style="text-align:right"><strong>43.4</strong></td>
|
| 361 |
</tr>
|
| 362 |
<tr>
|
| 363 |
-
<td>DIVER-Retriever-
|
| 364 |
<td style="text-align:right"><strong>32.1</strong></td>
|
| 365 |
<td style="text-align:right">51.9</td>
|
| 366 |
<td style="text-align:right">53.5</td>
|
|
@@ -411,7 +427,7 @@ For more details, including benchmark evaluation, hardware requirements, and inf
|
|
| 411 |
<td style="text-align:right">36.8</td>
|
| 412 |
</tr>
|
| 413 |
<tr>
|
| 414 |
-
<td>DIVER-Retriever
|
| 415 |
<td style="text-align:right"><strong>33.9</strong></td>
|
| 416 |
<td style="text-align:right">54.5</td>
|
| 417 |
<td style="text-align:right">52.7</td>
|
|
@@ -553,7 +569,7 @@ print(scores.tolist())
|
|
| 553 |
|
| 554 |
|
| 555 |
### Finetuning
|
| 556 |
-
We recommend you to use [swift](https://github.com/modelscope/ms-swift) to finetune our DIVER-Retriever-
|
| 557 |
|
| 558 |
Before starting training, please ensure your environment is properly configured.
|
| 559 |
|
|
@@ -578,7 +594,7 @@ Using infonce loss as an example, the complete training command is as follows:
|
|
| 578 |
nproc_per_node=8
|
| 579 |
NPROC_PER_NODE=$nproc_per_node \
|
| 580 |
swift sft \
|
| 581 |
-
--model
|
| 582 |
--task_type embedding \
|
| 583 |
--model_type qwen3_emb \
|
| 584 |
--train_type full \
|
|
|
|
| 12 |
- truehealth/medqa
|
| 13 |
metrics:
|
| 14 |
- accuracy
|
|
|
|
|
|
|
| 15 |
pipeline_tag: text-ranking
|
| 16 |
language:
|
| 17 |
- zh
|
| 18 |
- en
|
| 19 |
library_name: transformers
|
| 20 |
+
base_model:
|
| 21 |
+
- Qwen/Qwen3-Embedding-0.6B
|
| 22 |
---
|
| 23 |
# Diver-Retriever-0.6B
|
| 24 |
|
|
|
|
| 38 |
|
| 39 |
- **Model type:** Text Embedding
|
| 40 |
- **Language(s) (NLP):** Bilingual (Chinese & English)
|
| 41 |
+
- **Context Length:** 32k
|
| 42 |
- **Number of Paramaters:** 0.6B
|
| 43 |
|
| 44 |
For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our GitHub (https://github.com/AQ-MedAI/Diver).
|
|
|
|
| 213 |
<td style="text-align:right">30.5</td>
|
| 214 |
</tr>
|
| 215 |
<tr>
|
| 216 |
+
<td>DIVER-Retriever-0.6B</td>
|
| 217 |
+
<td style="text-align:right">25.2</td>
|
| 218 |
+
<td style="text-align:right">36.4</td>
|
| 219 |
+
<td style="text-align:right">41.9</td>
|
| 220 |
+
<td style="text-align:right">29.0</td>
|
| 221 |
+
<td style="text-align:right">31.0</td>
|
| 222 |
+
<td style="text-align:right">21.2</td>
|
| 223 |
+
<td style="text-align:right">24.6</td>
|
| 224 |
+
<td style="text-align:right">23.2</td>
|
| 225 |
+
<td style="text-align:right">15.6</td>
|
| 226 |
+
<td style="text-align:right">6.8</td>
|
| 227 |
+
<td style="text-align:right">8.4</td>
|
| 228 |
+
<td style="text-align:right">33.2</td>
|
| 229 |
+
<td style="text-align:right">31.7</td>
|
| 230 |
+
</tr>
|
| 231 |
+
<tr>
|
| 232 |
+
<td>DIVER-Retriever-4B</td>
|
| 233 |
<td style="text-align:right"><strong>28.9</strong></td>
|
| 234 |
<td style="text-align:right"><strong>41.8</strong></td>
|
| 235 |
<td style="text-align:right">43.7</td>
|
|
|
|
| 376 |
<td style="text-align:right"><strong>43.4</strong></td>
|
| 377 |
</tr>
|
| 378 |
<tr>
|
| 379 |
+
<td>DIVER-Retriever-4B</td>
|
| 380 |
<td style="text-align:right"><strong>32.1</strong></td>
|
| 381 |
<td style="text-align:right">51.9</td>
|
| 382 |
<td style="text-align:right">53.5</td>
|
|
|
|
| 427 |
<td style="text-align:right">36.8</td>
|
| 428 |
</tr>
|
| 429 |
<tr>
|
| 430 |
+
<td>DIVER-Retriever</td>
|
| 431 |
<td style="text-align:right"><strong>33.9</strong></td>
|
| 432 |
<td style="text-align:right">54.5</td>
|
| 433 |
<td style="text-align:right">52.7</td>
|
|
|
|
| 569 |
|
| 570 |
|
| 571 |
### Finetuning
|
| 572 |
+
We recommend you to use [swift](https://github.com/modelscope/ms-swift) to finetune our DIVER-Retriever-0.6B with infonce.
|
| 573 |
|
| 574 |
Before starting training, please ensure your environment is properly configured.
|
| 575 |
|
|
|
|
| 594 |
nproc_per_node=8
|
| 595 |
NPROC_PER_NODE=$nproc_per_node \
|
| 596 |
swift sft \
|
| 597 |
+
--model AQ-MedAI/Diver-Retriever-0.6B \
|
| 598 |
--task_type embedding \
|
| 599 |
--model_type qwen3_emb \
|
| 600 |
--train_type full \
|