spectral-embed-v2-140m

A second-generation, closed-form, CPU-only linear-spectral embedding model.
Features a multi-scale coordinate featurizer, title-aware document ingestion, folded metric whitening, and pre-fused subspace calibration.
53.21 Mean across the full 40-task MTEB v2 suite and 64.73 on the core 11-task subset, operating at sub-millisecond CPU latency.

spectral-embed-v2-140m replaces heavy neural transformer backbones with a deterministic sparse featurizer and a single learned linear operator ($z = X W^T + b$), followed by unit L2 normalization.

No GPU, no PyTorch, no ONNX Runtime, no tokenizers, no learned vocabulary. The entire inference path requires only standard Python, NumPy, and SciPy.

---

What’s New in V2?

Dimension	spectral-embed-v1-140m	spectral-embed-v2-140m	Technical Impact
MTEB (40-Task Benchmark)	—	53.21 (Mean Task) / 52.53 (Mean Type)	Complete, validated general English benchmark
MTEB (11-Task Core Subset)	57.47	64.73	+7.26 pts overall improvement
Title-Aware Ingestion	Body text only	Structured title + " " + text	Massive breakthrough on Retrieval & QA
Retrieval Category (Core)	25.86	41.33	+15.47 pts jump from title preservation
STS Category Avg (Full 40)	—	72.65	Near-transformer parity on semantic similarity
Metric Whitening	Runtime diagonal scaling	Folded directly into $W, b$	Zero runtime overhead
Subspace Calibration	Base SVD projection	Fused Calibration Adapter	Maximized class separability
Inference Path	Dense GEMM	Sparse CSR GEMM ($X W^T + b$)	3–4× faster batch throughput

---

Title-Aware Document Ingestion for Retrieval

In modern retrieval benchmarks (e.g. HotpotQA, FEVER, CQADupStack, Touche2020, ArguAna), documents provide structured "title" and "text" fields. Because transformer models typically concatenate these into a single context prompt, linear models that only embed body text lose high-salience topical keywords concentrated in headings.

In spectral-embed-v2-140m, the evaluation protocol and native encoder ingest both fields:

# Canonical document preparation
if "title" in doc and doc["title"]:
    full_text = f"{doc['title']} {doc['text']}".strip()
else:
    full_text = doc["text"].strip()

Why Title Preservation Matters for Linear Spectral Models

Unlike deep multi-head attention layers that can infer document topics from peripheral phrasing, a linear-spectral projection ($z = X W^T + b$) calculates coordinate activations directly from token and $n$-gram frequencies. Prepending the document title ensures that the primary semantic anchor is present in the sparse feature vector $X$, allowing the retrieval projection to match query concepts directly with zero semantic dilution.

This single structural fix propelled FEVER to 28.11, HotpotQA to 38.33, Touche2020 to 40.21, and TRECCOVID to 54.09.

---

Full MTEB (eng, v2) Benchmark Results

Evaluated under mteb v2.26.14 with batch size 32. All 40 evaluated tasks are listed below:

Category Summary

Category	Tasks Completed	Category Mean	Star Tasks
Semantic Textual Similarity (STS)	9 / 9	72.65	STS17 (83.89), STS15 (81.34), STSBenchmark (76.36)
Pair Classification	3 / 3	76.12	SprintDuplicateQuestions (86.91), TwitterURLCorpus (82.19)
Classification	8 / 8	69.25	MTOPDomain (89.54), AmazonCounterfactual (74.88), Banking77 (73.94)
Reranking	1 / 1	53.20	AskUbuntuDupQuestions (53.20)
Retrieval	10 / 10	30.53	TRECCOVID (54.09), ArguAna (40.64), Touche2020 (40.21)
Clustering	8 / 8	38.23	ArXivHierarchicalP2P (52.05), ArXivHierarchicalS2S (50.51)
Summarization	1 / 1	27.73	SummEvalSummarization (27.73)
MEAN (TASK)	40 / 40	53.21	(MindSmallReranking skipped)
MEAN (TASK TYPE)	7 / 7	52.53	—

---

Detailed 40-Task Score Breakdown

Click to expand full 40-task benchmark breakdown

Type	Task	Score (x100)	Evaluation Time
Classification	AmazonCounterfactualClassification	74.88	15s
Classification	Banking77Classification	73.94	25s
Classification	ImdbClassification	63.68	3m 55s
Classification	MTOPDomainClassification	89.54	30s
Classification	MassiveIntentClassification	60.83	25s
Classification	MassiveScenarioClassification	69.64	25s
Classification	ToxicConversationsClassification	63.79	1m 40s
Classification	TweetSentimentExtractionClassification	57.73	35s
Clustering	ArXivHierarchicalClusteringP2P	52.05	30s
Clustering	ArXivHierarchicalClusteringS2S	50.51	25s
Clustering	BiorxivClusteringP2P.v2	34.79	25s
Clustering	MedrxivClusteringP2P.v2	33.90	30s
Clustering	MedrxivClusteringS2S.v2	29.06	20s
Clustering	StackExchangeClustering.v2	38.48	35s
Clustering	StackExchangeClusteringP2P.v2	34.23	40s
Clustering	TwentyNewsgroupsClustering.v2	32.80	20s
PairClassification	SprintDuplicateQuestions	86.91	1m 25s
PairClassification	TwitterSemEval2015	59.25	25s
PairClassification	TwitterURLCorpus	82.19	1m 00s
Reranking	AskUbuntuDupQuestions	53.20	25s
Retrieval	ArguAna	40.64	1m 05s
Retrieval	CQADupstackGamingRetrieval	39.55	2m 00s
Retrieval	CQADupstackUnixRetrieval	19.78	3m 10s
Retrieval	ClimateFEVERHardNegatives	16.79	3m 45s
Retrieval	FEVERHardNegatives	28.11	8m 30s
Retrieval	FiQA2018	14.37	3m 15s
Retrieval	HotpotQAHardNegatives	38.33	7m 40s
Retrieval	SCIDOCS	13.40	2m 10s
Retrieval	TRECCOVID	54.09	12m 20s
Retrieval	Touche2020Retrieval.v3	40.21	31m 27s
STS	BIOSSES	75.53	10s
STS	SICK-R	64.69	20s
STS	STS12	66.04	15s
STS	STS13	74.76	15s
STS	STS14	71.26	15s
STS	STS15	81.34	15s
STS	STS17	83.89	15s
STS	STS22.v2	59.96	20s
STS	STSBenchmark	76.36	15s
Summarization	SummEvalSummarization.v2	27.73	20s

---

11-Task Core Comparison (V1 vs V2)

For direct comparison against spectral-embed-v1-140m on the original 11-task evaluation subset:

Task	Category	Metric	V1 Score	V2 Score	Delta
STS12	STS	Spearman (cosine)	61.31	66.04	+4.73
STS13	STS	Spearman (cosine)	65.44	74.76	+9.32
STS14	STS	Spearman (cosine)	69.79	71.26	+1.47
STS15	STS	Spearman (cosine)	77.76	81.34	+3.58
STS16	STS	Spearman (cosine)	70.88	76.00	+5.12
STSBenchmark	STS	Spearman (cosine)	70.13	76.36	+6.23
Banking77Classification	Classification	Accuracy	69.45	73.94	+4.49
AmazonCounterfactualClassification	Classification	Accuracy	69.80	74.88	+5.08
ArguAna	Retrieval	nDCG@10	30.92	40.64	+9.72
NFCorpus	Retrieval	nDCG@10	14.59	26.00	+11.41
SciFact	Retrieval	nDCG@10	32.06	53.00	+20.94
Core Average	—	—	57.47	64.73	+7.26

Note on training disclosure: NFCorpus was present in the multi-domain teacher distillation pool; its score is reported for regression monitoring and is declared in the MTEB metadata.

---

Quickstart

1. Standalone Python (NumPy + SciPy Only)

pip install numpy scipy huggingface_hub nltk

import numpy as np
from huggingface_hub import hf_hub_download

# 1. Download model artifact
npz_file = hf_hub_download(
    repo_id="JMullings/spectral-embed-v2-140m",
    filename="spectral-embed-v2-140m.npz"
)

# 2. Load model tensors
with np.load(npz_file, allow_pickle=True) as d:
    W_T = np.ascontiguousarray(d["W"].T, dtype=np.float32)  # (65536, 2048)
    b = np.asarray(d["b"], dtype=np.float32)                # (2048,)
    D_prior = np.asarray(d["idf"], dtype=np.float32)        # (65536,)
    dim_h = int(d["H"])

# 3. Import canonical featurizer (or use MTEB model loader)
from BUILD_CONFIG import SpectralFeaturizer, load_stemmer
stem = load_stemmer()[0] if isinstance(load_stemmer(), tuple) else load_stemmer()
featurizer = SpectralFeaturizer(H=dim_h, seed=42, avgdl=166.7, k1=1.5, b=0.75, stem=stem)

# 4. Handle title-aware inputs for retrieval or plain text for sentences
documents = [
    {"title": "Account Security", "text": "Steps to change login credentials and regain account access."},
    {"title": "France Tourism", "text": "The capital of France is Paris with historic architectural landmarks."}
]

# Concatenate title and text for retrieval documents
texts = [f"{d['title']} {d['text']}".strip() if d.get("title") else d["text"] for d in documents]

# 5. Embed via sparse linear projection
X = featurizer.transform(texts, idf=D_prior)            # Sparse CSR (N, 65536)
Z = np.asarray(X @ W_T, dtype=np.float32) + b           # Linear projection (N, 2048)
embeddings = Z / np.maximum(np.linalg.norm(Z, axis=1, keepdims=True), 1e-12)

# 6. Compute query similarity
query = "How do I reset my password?"
X_q = featurizer.transform([query], idf=D_prior)
Z_q = np.asarray(X_q @ W_T, dtype=np.float32) + b
q_emb = Z_q / np.maximum(np.linalg.norm(Z_q, axis=1, keepdims=True), 1e-12)

scores = (embeddings @ q_emb.T).ravel()
print(f"Match Score (Doc 1 - Account): {scores[0]:.4f}")  # High (~0.62)
print(f"Match Score (Doc 2 - Paris)  : {scores[1]:.4f}")  # Low (~0.11)

2. Ollama-Compatible Local HTTP Server

Using the repository runner on port 7860:

python app.py --model spectral-embed-v2-140m.npz --port 7860

Query with curl:

curl http://localhost:7860/api/embed \
  -H "Content-Type: application/json" \
  -d '{"model": "spectral-embed-v2-140m", "input": "Steps to restore database backup"}'

---

Technical Specifications

Field	Value
Architecture	Closed-form linear spectral operator with Sparse CSR GEMM ($z = X W^T + b$)
Parameters	139,471,360 (~139.5M) — $W \in \mathbb{R}^{2048 \times 65536}$ + $b \in \mathbb{R}^{2048}$
Output Dimension	2048
Serving Dtype	float32
Featurizer	spectral_v6 (Blake2b-64, partitioned multi-scale unigrams/bigrams/char-grams, sub-linear saturation)
Latency	~0.35 ms / sentence (single CPU core); >50,000 texts/sec batch throughput
RAM Footprint	~560 MB active memory
Hardware Requirements	Any commodity x86_64 or ARM64 CPU (Apple Silicon, AWS Graviton, Raspberry Pi)
License	Spectral Embeddings Community License v1.0 (Free up to $14M ARR)

---

Citation

@misc{spectral_embed_v2_140m,
  author = {Mullings, J.},
  title  = {spectral-embed-v2-140m: A Second-Generation Closed-Form Linear Spectral Embedding Model},
  year   = {2026},
  url    = {https://huggingface.co/JMullings/spectral-embed-v2-140m}
}