Instructions to use papr-ai/papr-embed-code-1-0.6b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use papr-ai/papr-embed-code-1-0.6b with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("papr-ai/papr-embed-code-1-0.6b") sentences = [ "That is a happy person", "That is a happy dog", "That is a very happy person", "Today is a sunny day" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
papr-embed-code-1-0.6b
An open-weights text embedding model tuned for code retrieval. It is Qwen/Qwen3-Embedding-0.6B further trained on code-retrieval data; it takes text and returns one dense vector. It does not take a schema (for schema-conditioned embeddings see papr-embed-schema-1).
| Parameters | 0.6B |
| Embedding dimension | 1024 |
| Max sequence length | 8192 tokens |
| Pooling / normalisation | last-token, L2 |
| Reference precision | fp32 |
| License | Apache 2.0 |
What it is good at, and what it is not
- Good at: code retrieval (natural-language to code, code to code, SQL). See CoIR below.
- Not a general-purpose upgrade. On non-code retrieval it is flat to worse than the Qwen/Qwen3-Embedding-0.6B it starts from, with a clear drop on TREC-COVID (table below). If your workload is mostly prose, use the base model.
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("papr-ai/papr-embed-code-1-0.6b") # loads in fp32
q = model.encode(["Instruct: Given a question, retrieve the code that answers it\nQuery:how do I reverse a list in python"], normalize_embeddings=True)
d = model.encode(["def rev(xs): return xs[::-1]"], normalize_embeddings=True)
print(q @ d.T)
Queries take an instruction in the form Instruct: {instruction}\nQuery:{query}; documents are embedded bare. The instructions behind every number below are in task_prompts.json.
Hosted: POST https://memory.papr.ai/v1/embeddings with "model": "papr-embed-code-1-0.6b".
No PyTorch? The ONNX build runs in FastEmbed and Qdrant; see ONNX, FastEmbed and Qdrant. On a Mac, the CoreML build runs on the Neural Engine at about 17 ms per query: papr-ai/papr-embed-code-1-0.6b-CoreML.
ONNX, FastEmbed and Qdrant
This repo includes an ONNX build of the same weights, so the model runs in FastEmbed and plain onnxruntime with no PyTorch.
| File | Precision | Size | CoIR-10 mean | Cosine vs PyTorch (min) |
|---|---|---|---|---|
model.safetensors (reference) |
fp32 | 2.38 GB | 77.19 | 1 |
onnx/model.onnx + onnx/model.onnx_data |
fp32 | 2.38 GB | 76.85 | 1.0000 |
onnx/model_quantized.onnx |
int8 weights, fp32 activations | 1.12 GB | 76.57 | 0.9997 |
The fp32 export returns the same vectors as the PyTorch model. Its CoIR mean is lower only because of two tie-sensitive tasks, CosQA and SyntheticText2SQL, where the order of the batches decides which of several near-identical documents ranks first (see the table below and the note on SyntheticText2SQL under Evaluation). On the other eight CoIR tasks, PyTorch, ONNX fp32 and ONNX int8 all average 82.43.
The graph takes input_ids and attention_mask and returns last_hidden_state. Pooling and normalization happen outside it: take the last non-padding token and L2-normalize (FastEmbed's PoolingType.LAST_TOKEN). The int8 file stores 8-bit MatMulNBits weights (block size 32), which need onnxruntime>=1.23. Don't embed empty strings with it: an empty document was the one input where int8 drifted (cosine 0.948 to the reference).
FastEmbed + Qdrant:
from fastembed import TextEmbedding
from fastembed.common.model_description import ModelSource, PoolingType
from qdrant_client import QdrantClient, models
TextEmbedding.add_custom_model(
model="papr-ai/papr-embed-code-1-0.6b", pooling=PoolingType.LAST_TOKEN, normalization=True,
sources=ModelSource(hf="papr-ai/papr-embed-code-1-0.6b"), dim=1024,
model_file="onnx/model.onnx", additional_files=["onnx/model.onnx_data"], # int8: model_file="onnx/model_quantized.onnx", no additional_files
)
emb = TextEmbedding(model_name="papr-ai/papr-embed-code-1-0.6b")
client = QdrantClient(":memory:") # or QdrantClient(url=..., api_key=...) for Qdrant Cloud
client.create_collection("code", vectors_config=models.VectorParams(size=1024, distance=models.Distance.COSINE))
docs = ["def rev(xs): return xs[::-1]", "SELECT name FROM users WHERE age > 30;"]
client.upsert("code", points=[models.PointStruct(id=i, vector=v.tolist(), payload={"code": d})
for i, (d, v) in enumerate(zip(docs, emb.embed(docs)))]) # documents: bare
q = "Instruct: Given a question about coding, retrieval code or passage that can solve user's question\nQuery:how do I reverse a list in python"
print(client.query_points("code", query=next(emb.query_embed([q])).tolist(), limit=1).points[0].payload) # queries: instruction + text
A complete example is in examples/fastembed_qdrant.py. Two things matter: use last-token pooling, and put the Instruct: โฆ\nQuery: prefix on queries only. FastEmbed's default for custom models is mean pooling, which returns vectors that look normal but retrieve poorly.
How the ONNX numbers were measured. Same harness as the PyTorch numbers: mteb 2.10.0, task_prompts.json, max length 8192. The only change is the encoder, swapped for onnxruntime-gpu 1.23.2 on H100 with TF32 off. Batches are length-sorted under a token budget rather than fixed at 32. Parity was checked on 17 texts against SentenceTransformer in fp32, through both onnxruntime and FastEmbed: 8 instructed queries, plus 9 documents including an empty one, a non-English one, and a 3k-token one. Every query's top-1 result matched the reference for both builds. Scripts: onnx/export_onnx.py, onnx/quantize_int8.py, onnx/parity.py, onnx/onnx_mteb.py. Per-task results: results/onnx_coir.json.
| Task | PyTorch fp32 | ONNX fp32 | ONNX int8 |
|---|---|---|---|
| Apps | 79.31 | 79.31 | 79.27 |
| CodeSearchNet | 86.19 | 86.19 | 86.18 |
| CodeFeedback-MT | 94.22 | 94.22 | 94.25 |
| CodeFeedback-ST | 88.52 | 88.54 | 88.56 |
| CodeSearchNet-CC | 92.23 | 92.23 | 92.23 |
| CodeTransOcean-Contest | 90.93 | 90.93 | 90.92 |
| CodeTransOcean-DL | 34.47 | 34.47 | 34.46 |
| StackOverflowQA | 93.57 | 93.57 | 93.60 |
| CosQA (tie-sensitive) | 40.42 | 41.43 | 40.77 |
| SyntheticText2SQL (tie-sensitive) | 72.04 | 67.64 | 65.50 |
| CoIR-10 mean | 77.19 | 76.85 | 76.57 |
| CoIR mean without the two tie-sensitive tasks | 82.43 | 82.43 | 82.43 |
| SciFact (in-domain) | 82.42 | 82.42 | 82.18 |
CoreML and the Apple Neural Engine
Also published as its own repo with full docs: papr-ai/papr-embed-code-1-0.6b-CoreML. The coreml/ folder here holds the same files.
coreml/papr-embed-code-1-0.6b.mlpackage runs the same weights on the Apple Neural Engine (Apple Silicon, macOS 15+). It is one multifunction model with three fixed input lengths, len_64, len_256 and len_1024. Pooling and L2 normalization happen inside the model, so it returns the final 1024-d unit vector.
| Function | Use | Latency p50 (M2 Pro, ANE) | Cosine vs PyTorch fp32 (min) |
|---|---|---|---|
len_64 |
queries | 16.7 ms | 0.9998 |
len_256 |
short files | 45 ms | 0.9997 |
len_1024 |
files up to 1k tokens | 306 ms | 0.9997 |
fp16, 1.2 GB. Inputs: input_ids int32 [1, L] and last_index int32 [1]. Output: embedding [1, 1024]. Pick the smallest L that fits.
How to use it:
- Tokenize with
tokenizer.json. - Append
<|endoftext|>(id 151643). - Right-pad with zeros to
L. - Pass the index of that last real token as
last_index.
Queries take the same Instruct: โฆ\nQuery: prefix as the PyTorch model; documents are bare.
Three things that break it:
- The end token. Use
<|endoftext|>, nottokenizer.eos_token. With recenttransformers,eos_tokenresolves to<|im_end|>(151645). The vectors still look normal but drift to cosine 0.5โ0.8 from the reference. - The Hugging Face cache. Download with
local_dir=...(orhf download ... --local-dir). CoreML can't compile through the symlinks in the default cache and fails with "weight.bin doesn't exist". - Which length you load. The first load compiles for the Neural Engine, about 30โ60 s per length. macOS caches the compiled model, so later loads take seconds. Load only the lengths you need.
Python:
import numpy as np, coremltools as ct
from huggingface_hub import snapshot_download
from tokenizers import Tokenizer
d = snapshot_download("papr-ai/papr-embed-code-1-0.6b", allow_patterns=["coreml/*", "tokenizer.json"], local_dir="papr-embed-code-1-0.6b")
tok = Tokenizer.from_file(f"{d}/tokenizer.json")
m = ct.models.MLModel(f"{d}/coreml/papr-embed-code-1-0.6b.mlpackage", compute_units=ct.ComputeUnit.CPU_AND_NE, function_name="len_64")
ids = tok.encode("Instruct: Given a question about coding, retrieval code or passage that can solve user's question\nQuery:reverse a list in python", add_special_tokens=False).ids + [151643]
x = np.zeros((1, 64), np.int32); x[0, :len(ids)] = ids
v = m.predict({"input_ids": x, "last_index": np.array([len(ids) - 1], np.int32)})["embedding"][0] # unit vector
A full example that handles all three lengths is in examples/coreml_embed.py. From Swift, set MLModelConfiguration.functionName = "len_64" and computeUnits = .cpuAndNeuralEngine.
How it was converted. coreml/convert_coreml.py (coremltools 9). Three changes make the Neural Engine run it correctly:
- Fixed input shapes. With enumerated or flexible shapes, CoreML placed every op on the CPU. With fixed shapes, 2,079 of 2,083 ops run on the ANE.
- RMSNorm rewritten as
layer_norm([x, -x]). The two are mathematically identical, because the input has zero mean. Qwen3 activations reach about 6,000, soxยฒoverflows fp16 inside a plain RMSNorm, and the output cosine fell to 0.05. - A pre-scaled final L2 norm, so the sum of squares stays in fp16 range.
An 8-bit-weight build was tested and not shipped. It was faster (10.7 ms per query) and half the size, but its minimum cosine was 0.9986. Measurements: results/coreml_parity.json.
Evaluation
Run with stock mteb 2.10.0, fp32, max length 8192, on 8x H100 80GB. The full protocol, including every deviation and when it was decided, is in PROTOCOL.md in this repo.
CoIR (nDCG@10). Mean over the 10 tasks: 77.19
| Task | nDCG@10 | Batch size |
|---|---|---|
| Apps | 79.31 | 16 |
| CodeSearchNet | 86.19 | 16 |
| CodeFeedback-MT | 94.22 | 16 |
| CodeFeedback-ST | 88.52 | 16 |
| CodeSearchNet-CC | 92.23 | 16 |
| CodeTransOcean-Contest | 90.93 | 32 |
| CodeTransOcean-DL | 34.47 | 32 |
| CosQA | 40.42 | 32 |
| StackOverflowQA | 93.57 | 16 |
| SyntheticText2SQL | 72.04 | 32 |
Batch size is mteb's default (32) wherever the task fit in 80 GB in fp32; tasks that ran out of memory were rerun at 16. Scores on most tasks barely move with batch size. SyntheticText2SQL does: the same weights scored between 65 and 72 depending on batching and hardware, because the corpus contains near-identical SQL answers and which one ranks first comes down to floating-point ties. Treat differences of a few points on that task as noise. For reference, stock Qwen3-Embedding-4B scores 68.44 on it in this same environment (its leaderboard entry, from an older mteb, is 78.21).
Held out from training (nDCG@10, fp32).
| Task | papr-embed-code-1-0.6b | Qwen3-Embedding-0.6B | ฮ |
|---|---|---|---|
| ArguAna | 66.48 | 69.34 | -2.86 |
| NFCorpus | 37.55 | 36.32 | +1.23 |
| SciDocs | 24.03 | 24.39 | -0.36 |
| FiQA2018 | 47.76 | 47.25 | +0.51 |
| TRECCOVID | 82.70 | 89.30 | -6.60 |
| CodeEditSearch | 66.84 | 64.25 | +2.59 |
| CodeSearchNet | 90.97 | 90.77 | +0.20 |
SciFact: 82.42 vs 69.72 for the base. We trained on the SciFact train split, so this is an in-domain result, not zero-shot.
Training data and contamination
No test split was used anywhere; only train splits entered training. The model saw the train splits of all 10 CoIR tasks, plus the SciFact train split, and its MTEB metadata declares these as training data. CoIR is therefore mostly an in-domain result for this model, not a zero-shot one. In the held-out table, MTEB's CodeSearchNet shares its source with CoIR's CodeSearchNet, whose train split we did use, so read that row as near-domain. CodeEditSearch was not confirmed absent from our training mix either, so read it as indicative.
Limitations
- Regresses on non-code retrieval (above). - Running in fp16 costs about 0.8 points. - English and code only; not evaluated on other languages.
Reproduce
replicate_mteb.py in this repo is the script behind every number above. With mteb==2.10.0, torch==2.10.0, transformers==4.57.6, sentence-transformers==5.2.3:
hf download papr-ai/papr-embed-code-1-0.6b --local-dir papr-embed-code-1-0.6b
python papr-embed-code-1-0.6b/replicate_mteb.py --model papr-embed-code-1-0.6b --tasks coir --out results --batch 32 --efficient-attn
--efficient-attn only changes which attention kernel runs (same math, ~1e-7 relative). On an 80 GB GPU in fp32, Apps, StackOverflowQA, CodeFeedback and both CodeSearchNet tasks need --batch 16.
- Downloads last month
- 261