papr-embed-code-1-0.6b

An open-weights text embedding model tuned for code retrieval. It is Qwen/Qwen3-Embedding-0.6B further trained on code-retrieval data; it takes text and returns one dense vector. It does not take a schema (for schema-conditioned embeddings see papr-embed-schema-1).

Parameters 0.6B
Embedding dimension 1024
Max sequence length 8192 tokens
Pooling / normalisation last-token, L2
Reference precision fp32
License Apache 2.0

What it is good at, and what it is not

  • Good at: code retrieval (natural-language to code, code to code, SQL). See CoIR below.
  • Not a general-purpose upgrade. On non-code retrieval it is flat to worse than the Qwen/Qwen3-Embedding-0.6B it starts from, with a clear drop on TREC-COVID (table below). If your workload is mostly prose, use the base model.

Usage

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("papr-ai/papr-embed-code-1-0.6b")  # loads in fp32
q = model.encode(["Instruct: Given a question, retrieve the code that answers it\nQuery:how do I reverse a list in python"], normalize_embeddings=True)
d = model.encode(["def rev(xs): return xs[::-1]"], normalize_embeddings=True)
print(q @ d.T)

Queries take an instruction in the form Instruct: {instruction}\nQuery:{query}; documents are embedded bare. The instructions behind every number below are in task_prompts.json.

Hosted: POST https://memory.papr.ai/v1/embeddings with "model": "papr-embed-code-1-0.6b".

No PyTorch? The ONNX build runs in FastEmbed and Qdrant; see ONNX, FastEmbed and Qdrant. On a Mac, the CoreML build runs on the Neural Engine at about 17 ms per query: papr-ai/papr-embed-code-1-0.6b-CoreML.

ONNX, FastEmbed and Qdrant

This repo includes an ONNX build of the same weights, so the model runs in FastEmbed and plain onnxruntime with no PyTorch.

File Precision Size CoIR-10 mean Cosine vs PyTorch (min)
model.safetensors (reference) fp32 2.38 GB 77.19 1
onnx/model.onnx + onnx/model.onnx_data fp32 2.38 GB 76.85 1.0000
onnx/model_quantized.onnx int8 weights, fp32 activations 1.12 GB 76.57 0.9997

The fp32 export returns the same vectors as the PyTorch model. Its CoIR mean is lower only because of two tie-sensitive tasks, CosQA and SyntheticText2SQL, where the order of the batches decides which of several near-identical documents ranks first (see the table below and the note on SyntheticText2SQL under Evaluation). On the other eight CoIR tasks, PyTorch, ONNX fp32 and ONNX int8 all average 82.43.

The graph takes input_ids and attention_mask and returns last_hidden_state. Pooling and normalization happen outside it: take the last non-padding token and L2-normalize (FastEmbed's PoolingType.LAST_TOKEN). The int8 file stores 8-bit MatMulNBits weights (block size 32), which need onnxruntime>=1.23. Don't embed empty strings with it: an empty document was the one input where int8 drifted (cosine 0.948 to the reference).

FastEmbed + Qdrant:

from fastembed import TextEmbedding
from fastembed.common.model_description import ModelSource, PoolingType
from qdrant_client import QdrantClient, models

TextEmbedding.add_custom_model(
    model="papr-ai/papr-embed-code-1-0.6b", pooling=PoolingType.LAST_TOKEN, normalization=True,
    sources=ModelSource(hf="papr-ai/papr-embed-code-1-0.6b"), dim=1024,
    model_file="onnx/model.onnx", additional_files=["onnx/model.onnx_data"],  # int8: model_file="onnx/model_quantized.onnx", no additional_files
)
emb = TextEmbedding(model_name="papr-ai/papr-embed-code-1-0.6b")

client = QdrantClient(":memory:")  # or QdrantClient(url=..., api_key=...) for Qdrant Cloud
client.create_collection("code", vectors_config=models.VectorParams(size=1024, distance=models.Distance.COSINE))
docs = ["def rev(xs): return xs[::-1]", "SELECT name FROM users WHERE age > 30;"]
client.upsert("code", points=[models.PointStruct(id=i, vector=v.tolist(), payload={"code": d})
                              for i, (d, v) in enumerate(zip(docs, emb.embed(docs)))])           # documents: bare

q = "Instruct: Given a question about coding, retrieval code or passage that can solve user's question\nQuery:how do I reverse a list in python"
print(client.query_points("code", query=next(emb.query_embed([q])).tolist(), limit=1).points[0].payload)  # queries: instruction + text

A complete example is in examples/fastembed_qdrant.py. Two things matter: use last-token pooling, and put the Instruct: โ€ฆ\nQuery: prefix on queries only. FastEmbed's default for custom models is mean pooling, which returns vectors that look normal but retrieve poorly.

How the ONNX numbers were measured. Same harness as the PyTorch numbers: mteb 2.10.0, task_prompts.json, max length 8192. The only change is the encoder, swapped for onnxruntime-gpu 1.23.2 on H100 with TF32 off. Batches are length-sorted under a token budget rather than fixed at 32. Parity was checked on 17 texts against SentenceTransformer in fp32, through both onnxruntime and FastEmbed: 8 instructed queries, plus 9 documents including an empty one, a non-English one, and a 3k-token one. Every query's top-1 result matched the reference for both builds. Scripts: onnx/export_onnx.py, onnx/quantize_int8.py, onnx/parity.py, onnx/onnx_mteb.py. Per-task results: results/onnx_coir.json.

Task PyTorch fp32 ONNX fp32 ONNX int8
Apps 79.31 79.31 79.27
CodeSearchNet 86.19 86.19 86.18
CodeFeedback-MT 94.22 94.22 94.25
CodeFeedback-ST 88.52 88.54 88.56
CodeSearchNet-CC 92.23 92.23 92.23
CodeTransOcean-Contest 90.93 90.93 90.92
CodeTransOcean-DL 34.47 34.47 34.46
StackOverflowQA 93.57 93.57 93.60
CosQA (tie-sensitive) 40.42 41.43 40.77
SyntheticText2SQL (tie-sensitive) 72.04 67.64 65.50
CoIR-10 mean 77.19 76.85 76.57
CoIR mean without the two tie-sensitive tasks 82.43 82.43 82.43
SciFact (in-domain) 82.42 82.42 82.18

CoreML and the Apple Neural Engine

Also published as its own repo with full docs: papr-ai/papr-embed-code-1-0.6b-CoreML. The coreml/ folder here holds the same files.

coreml/papr-embed-code-1-0.6b.mlpackage runs the same weights on the Apple Neural Engine (Apple Silicon, macOS 15+). It is one multifunction model with three fixed input lengths, len_64, len_256 and len_1024. Pooling and L2 normalization happen inside the model, so it returns the final 1024-d unit vector.

Function Use Latency p50 (M2 Pro, ANE) Cosine vs PyTorch fp32 (min)
len_64 queries 16.7 ms 0.9998
len_256 short files 45 ms 0.9997
len_1024 files up to 1k tokens 306 ms 0.9997

fp16, 1.2 GB. Inputs: input_ids int32 [1, L] and last_index int32 [1]. Output: embedding [1, 1024]. Pick the smallest L that fits.

How to use it:

  1. Tokenize with tokenizer.json.
  2. Append <|endoftext|> (id 151643).
  3. Right-pad with zeros to L.
  4. Pass the index of that last real token as last_index.

Queries take the same Instruct: โ€ฆ\nQuery: prefix as the PyTorch model; documents are bare.

Three things that break it:

  • The end token. Use <|endoftext|>, not tokenizer.eos_token. With recent transformers, eos_token resolves to <|im_end|> (151645). The vectors still look normal but drift to cosine 0.5โ€“0.8 from the reference.
  • The Hugging Face cache. Download with local_dir=... (or hf download ... --local-dir). CoreML can't compile through the symlinks in the default cache and fails with "weight.bin doesn't exist".
  • Which length you load. The first load compiles for the Neural Engine, about 30โ€“60 s per length. macOS caches the compiled model, so later loads take seconds. Load only the lengths you need.

Python:

import numpy as np, coremltools as ct
from huggingface_hub import snapshot_download
from tokenizers import Tokenizer
d = snapshot_download("papr-ai/papr-embed-code-1-0.6b", allow_patterns=["coreml/*", "tokenizer.json"], local_dir="papr-embed-code-1-0.6b")
tok = Tokenizer.from_file(f"{d}/tokenizer.json")
m = ct.models.MLModel(f"{d}/coreml/papr-embed-code-1-0.6b.mlpackage", compute_units=ct.ComputeUnit.CPU_AND_NE, function_name="len_64")
ids = tok.encode("Instruct: Given a question about coding, retrieval code or passage that can solve user's question\nQuery:reverse a list in python", add_special_tokens=False).ids + [151643]
x = np.zeros((1, 64), np.int32); x[0, :len(ids)] = ids
v = m.predict({"input_ids": x, "last_index": np.array([len(ids) - 1], np.int32)})["embedding"][0]  # unit vector

A full example that handles all three lengths is in examples/coreml_embed.py. From Swift, set MLModelConfiguration.functionName = "len_64" and computeUnits = .cpuAndNeuralEngine.

How it was converted. coreml/convert_coreml.py (coremltools 9). Three changes make the Neural Engine run it correctly:

  • Fixed input shapes. With enumerated or flexible shapes, CoreML placed every op on the CPU. With fixed shapes, 2,079 of 2,083 ops run on the ANE.
  • RMSNorm rewritten as layer_norm([x, -x]). The two are mathematically identical, because the input has zero mean. Qwen3 activations reach about 6,000, so xยฒ overflows fp16 inside a plain RMSNorm, and the output cosine fell to 0.05.
  • A pre-scaled final L2 norm, so the sum of squares stays in fp16 range.

An 8-bit-weight build was tested and not shipped. It was faster (10.7 ms per query) and half the size, but its minimum cosine was 0.9986. Measurements: results/coreml_parity.json.

Evaluation

Run with stock mteb 2.10.0, fp32, max length 8192, on 8x H100 80GB. The full protocol, including every deviation and when it was decided, is in PROTOCOL.md in this repo.

CoIR (nDCG@10). Mean over the 10 tasks: 77.19

Task nDCG@10 Batch size
Apps 79.31 16
CodeSearchNet 86.19 16
CodeFeedback-MT 94.22 16
CodeFeedback-ST 88.52 16
CodeSearchNet-CC 92.23 16
CodeTransOcean-Contest 90.93 32
CodeTransOcean-DL 34.47 32
CosQA 40.42 32
StackOverflowQA 93.57 16
SyntheticText2SQL 72.04 32

Batch size is mteb's default (32) wherever the task fit in 80 GB in fp32; tasks that ran out of memory were rerun at 16. Scores on most tasks barely move with batch size. SyntheticText2SQL does: the same weights scored between 65 and 72 depending on batching and hardware, because the corpus contains near-identical SQL answers and which one ranks first comes down to floating-point ties. Treat differences of a few points on that task as noise. For reference, stock Qwen3-Embedding-4B scores 68.44 on it in this same environment (its leaderboard entry, from an older mteb, is 78.21).

Held out from training (nDCG@10, fp32).

Task papr-embed-code-1-0.6b Qwen3-Embedding-0.6B ฮ”
ArguAna 66.48 69.34 -2.86
NFCorpus 37.55 36.32 +1.23
SciDocs 24.03 24.39 -0.36
FiQA2018 47.76 47.25 +0.51
TRECCOVID 82.70 89.30 -6.60
CodeEditSearch 66.84 64.25 +2.59
CodeSearchNet 90.97 90.77 +0.20

SciFact: 82.42 vs 69.72 for the base. We trained on the SciFact train split, so this is an in-domain result, not zero-shot.

Training data and contamination

No test split was used anywhere; only train splits entered training. The model saw the train splits of all 10 CoIR tasks, plus the SciFact train split, and its MTEB metadata declares these as training data. CoIR is therefore mostly an in-domain result for this model, not a zero-shot one. In the held-out table, MTEB's CodeSearchNet shares its source with CoIR's CodeSearchNet, whose train split we did use, so read that row as near-domain. CodeEditSearch was not confirmed absent from our training mix either, so read it as indicative.

Limitations

  • Regresses on non-code retrieval (above). - Running in fp16 costs about 0.8 points. - English and code only; not evaluated on other languages.

Reproduce

replicate_mteb.py in this repo is the script behind every number above. With mteb==2.10.0, torch==2.10.0, transformers==4.57.6, sentence-transformers==5.2.3:

hf download papr-ai/papr-embed-code-1-0.6b --local-dir papr-embed-code-1-0.6b
python papr-embed-code-1-0.6b/replicate_mteb.py --model papr-embed-code-1-0.6b --tasks coir --out results --batch 32 --efficient-attn

--efficient-attn only changes which attention kernel runs (same math, ~1e-7 relative). On an 80 GB GPU in fp32, Apps, StackOverflowQA, CodeFeedback and both CodeSearchNet tasks need --batch 16.

Downloads last month
261
Safetensors
Model size
0.6B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for papr-ai/papr-embed-code-1-0.6b

Quantized
(288)
this model
Quantizations
1 model