delavalom/embeddinggemma-2-coreml

EmbeddingGemma 2 for Core ML

This is google/embeddinggemma-2 (Google DeepMind's text and image embedding model, one shared 768-d space) converted to multifunction Core ML models that run on the Apple Neural Engine, one per image budget. Every stage of the conversion was checked against the original PyTorch model. It is an unofficial conversion. It is not made or endorsed by Google.

• Base model: google/embeddinggemma-2 at revision 914f7f89142e33e77833254d9c9b90c3cef7303b (model.safetensors sha256 197a32965d4b1105faf060417baa899e193fb73cd401f42ec9295234d5553d79).
• Modalities: text and images. Audio and video are not converted.
• Variant shipped: int8_table4: int8 per-channel conv weights, int4 per-block (32) table; tables and norms FP16.
• Layout: one folder per image budget, b70/ and b280/. Each holds the text functions and that budget's image functions; a device uses one budget only, so it downloads one folder's archive.
• Size: apps download one file, b<budget>/EmbeddingGemma2.aar (an LZFSE Apple Archive of the package and the tokenizer: 343.2 MB for budget 70, 352.9 MB for budget 280; the s4 gate is 360 MB for the larger), described by b<budget>/manifest.json (size, SHA-256 and image budget). Unpacked it is 435.0 / 454.9 MB; the package alone is 402.8 / 422.8 MB. The same files are also in each folder unpacked, each listed with its size and SHA-256 in b<budget>/files.json.
• Runs on: iOS 18+ and macOS 15+ (the model uses ios18 ops). It is built for .cpuAndNeuralEngine.
• Swift package: EmbeddingGemma (download, verification, tokenizer, image preprocessing, plus an on-disk vector index). See "Use from Swift" below.

License and terms

• The weights in this repo are a modified version of google/embeddinggemma-2, under the Apache License, Version 2.0. This is the license the upstream model card declares at the pinned revision (license: apache-2.0, linking Google's Gemma 4 license, which is the Apache 2.0 text). The original model is by Google DeepMind. See LICENSE (the full Apache 2.0 text) and NOTICE.
• Changes made (Apache 2.0, section 4(b)): the original weights were re-implemented in PyTorch, converted to Core ML, cast to FP16, and compressed (int8 per-channel convolution weights, int4 per-block embedding table). The vectors are close to the original's but not identical (see "Accuracy").
• Upstream use policy, passed through: the upstream model card says "Deployments must adhere to the Gemma Prohibited Use Policy." That requirement comes from the upstream model, and it applies to this conversion too. The upstream card also says developers are responsible for application-level safeguards (retrieval filtering, fairness testing) suited to their use case. Read Google's model card for its intended uses, limitations and ethical considerations. They all apply here.
• The tokenizer files (tokenizer/tokenizer.json, tokenizer/tokenizer_config.json) are unmodified copies from the upstream revision above.
• The conversion code and the Swift package are Apache-2.0 too (the GitHub repo's LICENSE and NOTICE).

Functions

One .mlpackage per image budget, b70/EmbeddingGemma2.mlpackage and b280/EmbeddingGemma2.mlpackage, each with nine functions with fixed shapes: the four text functions (the same in both) and the five image functions of its budget. Both were split from one 14-function package, and every function gives the same output bytes as there. Every function returns embedding, float32 [1, 768], L2-normalized.

function	input	tokens
embed_text_16, _32, _64, _128	input_ids int32 [1, L], attention_mask int32 [1, L] (1 = token, 0 = padding)	L = 16 / 32 / 64 / 128
embed_image_70_{tall,portrait,square,landscape,wide} (in b70/)	pixels [1, 3, H, W], RGB in 0-1	60-70 image tokens
embed_image_280_{tall,portrait,square,landscape,wide} (in b280/)	pixels [1, 3, H, W], RGB in 0-1	256-280 image tokens

Image shapes (H x W). The bucket comes from the aspect ratio r = width / height: tall r < 0.6, portrait r < 0.83, square r <= 1.2, landscape r <= 1.67, wide above that.

budget	tall	portrait	square	landscape	wide
70	576 x 240	480 x 336	384 x 384	336 x 480	240 x 576
280	1152 x 528	960 x 672	768 x 768	672 x 960	528 x 1152

• Text is <bos> + tokens(prompt + text) + <eos>, padded with <pad> (id 0) to the smallest length that fits. Longer inputs are cut to 128 tokens, keeping <bos> and <eos> (the upstream model allows up to 8K tokens). Use the upstream prompts: task: search result | query: for queries, title: none | text: for documents (also task: classification | query: and task: clustering | query: ).
• Images are resized straight to the bucket size (no crop or padding). Use 8-bit sRGB, antialiased bicubic resampling, and scale by 1/255 with no mean or std normalization, as the reference processor does. The Swift package does this.
• Matryoshka: you can truncate the 768-d output to 512, 256 or 128 dims and re-normalize. The Swift package defaults to 256. All accuracy numbers below are at 768 dims.

Accuracy against the original model

Measured by the pipeline on 200 hand-written texts (120 of them search queries) and 200 images (CC0 and public domain photos from Wikimedia Commons, plus synthetic screenshots, memes and receipts; 14 categories), at both image budgets. Cosine values are the minimum over all inputs, not the mean.

stage	what is compared	min cosine, text	min cosine, image
s1	our PyTorch rewrite (FP32) vs the official sentence-transformers model	1 - 1.4e-12	1 - 1.0e-08
s2	Core ML FP32 vs the official model	1 - 1.2e-12	1 - 1.9e-08
s3	Core ML FP16 on the Neural Engine vs the official model (at least 99.86% of ops on the Neural Engine)	0.99996	0.99404 (1st percentile 0.99848)
s4	shipped int8_table4 on CPU_AND_NE vs the official model	0.99791	0.99133 (1st percentile 0.99539)
s5	the Swift package end to end on a Mac (tokenizer + preprocessing + model) vs the official model on the same decoded pixels; vs Python Core ML on identical inputs: 1 - 1.0e-15	0.99017 (text and image)

Retrieval with the shipped variant (the 120 queries ranking the 200 images, budget 70 / 280):

metric	budget 70	budget 280
text-to-image recall@10 vs the FP reference on the same bucketed images	0.962	0.966
5th-percentile Spearman of the top 50, same reference	0.966	0.972
text-to-image recall@10 vs the exact-aspect FP model (recorded, not gated)	0.844	0.874

For 98.3% of queries (worst budget), the exact-aspect model's top-1 image is in the shipped model's top 10. Image-to-image precision@10 by category label is at most 0.012 below the exact-aspect FP model. Swift tokenization matches the reference ids for every fixture text (True). The 99th-percentile pixel difference against the reference preprocessing is 0.000000 (0-1 units).

Caveats:

• The retrieval gates use a bucketed FP reference. The upstream processor resizes each image to its own aspect-preserving size. This model has five fixed shapes per budget, so even with no compression, the bucketed vector differs from the exact-aspect one (min cosine 0.916 at budget 70, 0.893 at 280). That is why recall@10 against the exact-aspect model is lower than recall@10 against the bucketed reference. The compression and FP16 gates (min cosine >= 0.99, recall@10 >= 0.95, Spearman >= 0.9) are therefore measured against the FP model on the same bucketed images. Bucketing gets its own task-level gates (top-1 in top 10

  = 0.95, category precision@10 drop <= 0.03). Measured by category labels, bucketing costs no retrieval quality. Details: ADR-0007 in the GitHub repo.

• Two images fall below the FP16 bar on the Neural Engine. In FP16 on the Neural Engine, fixture images i141 and i161 reach 0.99404 / 0.99405 cosine. The other 398 are at 0.998 or above. On the GPU these two images pass, but the GPU is 2.8x slower at budget 70. They change the top 10 of only a few queries: with only those two vectors swapped in, recall@10 stays at 0.9975 (budget 70) and 0.9992 (budget 280). They were accepted at task level, and images stay on the Neural Engine. The lowest image cosines of the shipped variant are i161 0.99133 (embed_image_280_landscape), i141 0.99322 (embed_image_70_wide), i165 0.99336 (embed_image_280_portrait). Details: ADR-0006.
• The test set is small: 200 texts, 200 images. Multilingual text, code and long documents were not tested separately. For quality on standard benchmarks, see Google's model card. This conversion only claims closeness to the original.

Speed and memory

Mac (s4, CPU_AND_NE, median of 20 runs after warm-up). These were measured on the build Mac; the v1.0.0 build Mac is an M1 Pro with macOS 26. embed_text_16 takes 4.4 ms and embed_image_70_square takes 43.8 ms. In every function, at least 99.81% of the ops are placed on the Neural Engine (MLComputePlan, minimum over the 14 functions, measured on the package the two budget packages were split from).

iPhone (s6, PhotoSearchDemo's benchmark: each iPhone runs its own budget, ImageBudget.auto, 280 on A17 Pro and newer and 70 otherwise: the newest 1,000 photos at 70 or 200 at 280, and 20 queries; times cover preprocessing + prediction):

device	iOS	ms per image, 70 (median)	ms per image, 280 (median)	ms per query (median)	peak memory, MB
iPhone14,3	27.0.1	64.0	not run	6.7	630

Worst device: 64.0 ms per image at 70, not measured (no A17 Pro or newer iPhone ran s6 yet) at 280, 6.7 ms per query, peak 630 MB (the gate is 800 MB; the target for budget 70 is 60 ms, which is recorded, not gated).

Storage: the Swift package downloads the archive of the device's image budget, b<budget>/EmbeddingGemma2.aar (343.2 MB at 70, 352.9 MB at 280, resumable), checks its size and SHA-256 against b<budget>/manifest.json, unpacks it (435.0 / 454.9 MB) with the AppleArchive framework and compiles the package. It needs free space for the archive, the unpacked files and the compiled model while it installs; afterwards it keeps the compiled model and the tokenizer. The s4 gate measures the larger archive, 352.9 MB (limit 360 MB).

Use from Swift

// Package.swift: .package(url: "https://github.com/Delavalom/embeddinggemma-coreml", from: "1.0.0")
import EmbeddingGemma
import EmbeddingIndex

let model = try await EmbeddingGemma.load(.huggingFace(revision: EmbeddingGemma.defaultRevision))
let photo = try await model.embed(image: cgImage, orientation: .up)  // [Float], 256-d by default, normalized
let query = try await model.embed(query: "receipts from last week")

The package pins this repo (delavalom/embeddinggemma-2-coreml) at a commit and the SHA-256 of each budget's manifest.json, which pins its archive's size and SHA-256. budget: .auto picks 280 tokens on A17 Pro and newer iPhones and on Macs, and 70 on other devices, and loads that budget's folder (b280/ or b70/) only.

Use from Python (coremltools, macOS 15+)

import numpy as np, coremltools as ct
from tokenizers import Tokenizer
from torchvision.transforms.functional import pil_to_tensor, resize, InterpolationMode
from PIL import Image

BUDGET = 280  # the folder you downloaded: b280/ or b70/; its text functions are the same in both
PKG = f"b{BUDGET}/EmbeddingGemma2.mlpackage"
tok = Tokenizer.from_file(f"b{BUDGET}/tokenizer/tokenizer.json")
# self-check: the layout the model was traced with is <bos> ... <eos>
assert tok.encode("task: search result | query: screenshots of tweets").ids == \
    [2, 8071, 236787, 3927, 1354, 1109, 7609, 236787, 75490, 529, 46331, 1]

def fn(name):
    return ct.models.MLModel(PKG, function_name=name, compute_units=ct.ComputeUnit.CPU_AND_NE)

def embed_text(text, prompt="task: search result | query: "):
    ids = tok.encode(prompt + text).ids
    ids = ids if len(ids) <= 128 else ids[:127] + [ids[-1]]
    n = next(L for L in (16, 32, 64, 128) if L >= len(ids))
    x = np.zeros((1, n), np.int32); x[0, :len(ids)] = ids
    m = np.zeros((1, n), np.int32); m[0, :len(ids)] = 1
    return fn(f"embed_text_{n}").predict({"input_ids": x, "attention_mask": m})["embedding"][0]

SHAPES = {70: {"tall": (576, 240), "portrait": (480, 336), "square": (384, 384), "landscape": (336, 480),
               "wide": (240, 576)},
          280: {"tall": (1152, 528), "portrait": (960, 672), "square": (768, 768), "landscape": (672, 960),
                "wide": (528, 1152)}}

def bucket(r):  # r = width / height
    return ("tall" if r < 0.6 else "portrait" if r < 0.83 else "square" if r <= 1.2
            else "landscape" if r <= 1.67 else "wide")

def embed_image(path):
    img = Image.open(path).convert("RGB")  # like the reference: 8-bit RGB, ICC profile ignored
    name = bucket(img.width / img.height)
    px = resize(pil_to_tensor(img), list(SHAPES[BUDGET][name]), InterpolationMode.BICUBIC, antialias=True)
    px = px.float() / 255
    return fn(f"embed_image_{BUDGET}_{name}").predict({"pixels": px[None].numpy()})["embedding"][0]

def truncate(v, dims=256):  # Matryoshka
    v = v[:dims]; return v / np.linalg.norm(v)

The model is compiled each time MLModel is created. To make many calls, keep one instance per function. The Python path is not gated; the gated path is the Swift package (s5).

Reproducibility

The model was built by an 8-stage pipeline in the GitHub repo. Each stage writes artifacts/<stage>/report.json, which is checked against gates.toml. A stage counts only if its report passed and is current. Each report records git_sha and code_hash (sha256 over the stage's code, the lock file and its gate table), and inputs_sha256 chains it to the reports it read. The reports behind this release:

stage	status	git_sha	code_hash
s0_reference	passed	59b9bf9863cb	9a6a53c2abe21038
s1_rewrite	passed	7645eff4d951	e98a54bc6bbc6abf
s2_coreml_fp32	passed	82b02a4da1ec	21bb94d9d33d0230
s3_coreml_ane	passed	8e33af79825f	edb80276335be2a4
s4_compress	passed	b9064e9abb2b	e68c9ca0822062d0
s5_swift	passed	90679438068a	464d01a178d66eb5
s6_device	passed	d5b2e6edc33e	70b253e29d57b0ce

Reference environment: transformers 5.20.2, sentence-transformers 6.2.0; conversion: coremltools 9.0. To rebuild: just setup, then just stage 0 through just stage 5 (just stage 6 needs an iPhone). Numbers measured on the CPU and the Neural Engine can vary slightly from run to run.

Results

Generated by python -m scripts.gen_results from artifacts/*/report.json and docs/results/*.json. Do not edit by hand.

Model: google/embeddinggemma-2 at revision 914f7f89142e33e77833254d9c9b90c3cef7303b.

Gates

stage	status	metric	value	gate	ok
s0_reference	passed	license_apache	true	== true	yes
s0_reference	passed	key_inventory_written	true	== true	yes
s0_reference	passed	deterministic	true	== true	yes
s1_rewrite	passed	strict_load	true	== true	yes
s1_rewrite	passed	min_cosine_text	1 - 1.4e-12	>= 0.9999	yes
s1_rewrite	passed	min_cosine_image	1 - 1e-08	>= 0.9999	yes
s2_coreml_fp32	passed	min_cosine_text	1 - 1.2e-12	>= 0.9999	yes
s2_coreml_fp32	passed	min_cosine_image	1 - 1.9e-08	>= 0.9999	yes
s3_coreml_ane	passed	min_cosine_text	0.999965	>= 0.995	yes
s3_coreml_ane	passed	min_cosine_image	0.994037	>= 0.99	yes
s3_coreml_ane	passed	min_p1_cosine_image	0.998482	>= 0.995	yes
s3_coreml_ane	passed	min_recall_at_10	0.9675	>= 0.95	yes
s3_coreml_ane	passed	min_p5_spearman_top50	0.987025	>= 0.9	yes
s3_coreml_ane	passed	min_ane_fraction	0.99864	>= 0.95	yes
s4_compress	passed	min_cosine	0.991329	>= 0.99	yes
s4_compress	passed	min_recall_at_10	0.961667	>= 0.95	yes
s4_compress	passed	min_p5_spearman_top50	0.966043	>= 0.9	yes
s4_compress	passed	min_t2i_top1_in_top10	0.983333	>= 0.95	yes
s4_compress	passed	max_i2i_label_p10_drop	0.012	<= 0.03	yes
s4_compress	passed	max_download_mb	352.911	<= 360	yes
s5_swift	passed	token_ids_identical	true	== true	yes
s5_swift	passed	max_pixel_abs_diff	0	<= 0.003922	yes
s5_swift	passed	min_cosine_runtime	1 - 1e-15	>= 0.9999	yes
s5_swift	passed	min_cosine	0.990173	>= 0.99	yes
s5_swift	passed	min_recall_at_10	0.9575	>= 0.95	yes
s5_swift	passed	min_p5_spearman_top50	0.96449	>= 0.9	yes
s5_swift	passed	min_t2i_top1_in_top10	0.983333	>= 0.95	yes
s5_swift	passed	max_i2i_label_p10_drop	0.0065	<= 0.03	yes
s6_device	passed	max_peak_rss_mb	630.065	<= 800	yes
s6_device	passed	max_cold_compile_peak_mb	1517.66	<= 2100	yes
s6_device	passed	min_devices	1	>= 1	yes
s7_publish	not run	-	-	-	-

s4 variants

Chosen: int8_table4 (winner: int8_table4). Variants are tried smallest first; the first that passes every gate wins, so later ones may not have run. Download limit: 360 MB.

variant	download MB	package MB	min cosine	recall@10	p5 Spearman top-50	exact-aspect top-1 in top 10	i2i p@10 drop	min ANE fraction	recall@10 vs exact-aspect	passes	description
int8_table4	358.669	452.4	0.991329	0.961667	0.966043	0.983333	0.012	0.998131	0.844167	all	int8 per-channel conv weights, int4 per-block (32) table; tables and norms FP16

Shipped per image budget (download MB = the device archive; max_download_mb is the larger): budget 70 EmbeddingGemma2-b70.aar 343.178 MB (9 functions), budget 280 EmbeddingGemma2-b280.aar 352.911 MB (9 functions). Same outputs as the 14-function package (CPU_ONLY, bitwise): True.

s5 Swift parity

Pixels (0-1 units; PNG and synthetic are gated, JPEG includes the decoder and is recorded only, ADR-0004):

inputs	p99	max	bit-exact
png	0	0	60/60
synthetic	0	0	20/20
jpeg	0.00784314	0.121569	0/340

comparison	min cosine	recall@10 (worst budget)	p5 Spearman top-50
Swift vs Python Core ML, same inputs (gated)	1 - 1e-15	-	-
Swift vs FP reference on Swift's pixels (gated)	0.990173	0.9575	0.96449
Swift vs s0 (recorded)	0.971191	0.946667	0.952845
FP on Swift's pixels vs s0: the decoder (recorded)	0.971614	0.968333	0.969143

Against s0 (recorded only; on JPEGs it includes the decoder, ADR-0008):

vectors	n	min cosine	mean cosine
text query	120	0.998874	0.99951
text document	80	0.997913	0.999262
image 70_jpeg	170	0.978874	0.999002
image 70_png	30	0.995402	0.99898
image 280_jpeg	170	0.971191	0.998763
image 280_png	30	0.999682	0.999764

Device benchmarks (s6)

source	device	status	key metrics
artifacts/s6_device/report.json	iPhone14,3	passed	max_peak_rss_mb=630.065, max_cold_compile_peak_mb=1517.66, min_devices=1
docs/results/s6_iPhone14,3.json	iPhone14,3	passed	max_peak_rss_mb=630.065, max_cold_compile_peak_mb=1517.66, min_devices=1

Limitations

• Text and images only. Text is limited to 128 tokens; images are limited to budgets of 70 or 280 tokens and five fixed aspect buckets each.
• Results are close to the original model, not bit-identical (see "Accuracy"). Do not mix vectors from this model with vectors from the PyTorch model or another conversion in the same index. The Swift index stores the model revision for that reason.
• Intel Macs have no Neural Engine; they run on the CPU/GPU and are slower. The fastest settings are on the Neural Engine with .cpuAndNeuralEngine.
• The first load compiles the model on the device, which takes a while. The Swift package caches the compiled model.
• Everything in the upstream card's "Limitations" and "Ethical Considerations and Risks" applies.

Citation

Please cite the original model as Google DeepMind's model card asks (google/embeddinggemma-2). For this conversion:

@software{embeddinggemma_coreml,
  title  = {EmbeddingGemma 2 for Core ML},
  author = {Delavalom},
  year   = {2026},
  url    = {https://huggingface.co/delavalom/embeddinggemma-2-coreml},
  note   = {Core ML conversion of google/embeddinggemma-2}
}