Loom Weave 3

Loom Weave 3

31.5M parameters Β· 16 layers Β· 1024 context Β· Textile Labs

The first Loom that reads real prose. Successor to Loom Weave 2 β€” and deliberately smaller than it (Weave 2 was 59.65M). Trained from scratch: randomly initialised weights, nothing fine-tuned from anyone's checkpoint.

It decides when to look something up, writes the query, reads what comes back from live Wikipedia, and answers from it β€” saying it checked rather than knew. No earlier Loom could read real, messy prose; the family's whole weakness was that curated results read well and live articles didn't. This is the model that closes that gap.

you            who wrote dracula
Loom Weave 3   <lookup>dracula</lookup>
harness        ← Dracula is an 1897 Gothic horror novel by Irish author Bram Stoker.
Loom Weave 3   I think Irish author Bram Stoker. Worth saying I checked rather than knew it.

Why it's smaller than Weave 2

Weave 2 (59.65M) failed for method, not size β€” a thin instruction veneer over a pretraining register, train and eval written by the same hand. Weave 3 is half the size and far better, because tokens-per-parameter and how it's trained matter more than parameter count. That has precedent: Loom Spark 3 (12.2M) succeeded and beat Spark 2 (19.9M). Bigger is not the goal; behaviour is.

What changed

The reading skill is a real circuit, and it only formed once two things were right:

lever what went wrong before fix
learning rate on the attention matrices trained 8Γ— too low, so the copy/reading circuit never formed β€” the model was fluent but couldn't pull "Paris" out of a sentence containing "Paris" Muon LR set to the family standard (0.025); reading appeared
what the loss grades it graded every token equally, spending most of its effort on raw web text loss is masked to the reply tokens; whole conversations packed into each block
how long it trained over-trained a tiny corpus into memorised, garbled output short training, kept the best-validation checkpoint
admitting personal limits learned "I can't know your name" but not "I can't know what you did" β€” so it hallucinated on other personal questions added varied personal-unknowable examples; it now declines all of them

The search harness

The model decides a search is needed and writes the query. harness.py does the rest:

  • searches the model's query and the subject it can see in your question
  • prefers the real article over lists, films, albums and disambiguation pages
  • reads the intro first, further only when the intro has no answer of the right kind
  • strips brackets and pronunciation guides, so real text looks like training text
  • hands back one sentence, not a paragraph β€” a model this size misreads paragraphs

Measured behaviour

Same harness and settings a user gets. Run on 2026-09-20.

The acceptance battery, row by row (total ties the best in the Loom family):

row Loom Weave 3
A Β· says its own name 10/12
B Β· its own name under rough typing 10/12
C Β· 5-turn conversation stays on thread 5/5
D Β· answers from a search result 5/5
E Β· follow-up answered from the same result 2/5
F Β· says it looked, after a lookup 5/5
G Β· never claims a lookup it didn't make 16/16
H Β· admits what it can't know about you 8/8
I Β· says when a result doesn't contain the answer 0/5
J Β· never leaks a search tag with tools off 28/28
K Β· stops on its own 12/12
L Β· searches when it should, not for your private things 19/20
total 120/133

End to end, everyday questions it had never seen, live Wikipedia, the model writing its own query, scored on the final answer:

set decided to search wrote its own query answer reached the model answered right
tuning 20/20 20/20 14/20 11/20
held-out 20/20 20/20 12/20 6/20

It is the first Loom to read live prose above a coin-flip on the tuning set β€” but read the held-out number honestly: on unfamiliar, messy passages it is right about a third of the time.

Read this before you use it

Every point here was measured.

  • Held-out live reading is ~30%. "I looked that up" means it searched β€” not that it read the result correctly. Run harness.py --show and trust the sentence it read.
  • It is not a calculator. Small sums are out of scope (0/6). Use a tool for arithmetic.
  • Follow-up questions from the same result are weaker (~40%).
  • It never says a result doesn't contain the answer β€” it answers from whatever it read.
  • It rarely asks a clarifying question on an ambiguous request.
  • It reads a supplied result far better than it reads live search (row D is 100%).
  • Harness search is Wikipedia only β€” time, weather, news and prices can't be answered.
  • It is a small reader/assistant, not a chat stylist. Voice is plain and brief by design.

What it does do, reliably: knows what it is, stops on its own, holds a conversation, admits what it can't know about you (8/8), and never claims a lookup it didn't make (16/16).

Usage β€” the harness

python3 harness.py "who wrote dracula"
python3 harness.py                          # interactive
python3 harness.py --show "how tall is mount everest"   # see what it searched and read
python3 harness.py --no-tools "who are you"

Stdlib + certifi. Wikipedia needs no API key. Never feed a failed lookup back as a result β€” the model will answer from the error text; harness.py fails loudly instead.

Usage β€” Ollama

ollama run hf.co/textilelabs/Loom-Weave-3 "who are you"

template and params are read automatically.

Usage β€” transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Weave-3")
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Weave-3").eval()
eot = tok.convert_tokens_to_ids("<|eot|>")

def ask(message, tools=False):
    p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"
    ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids
    with torch.no_grad():
        out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=eot,
                             pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]
    return tok.decode(out[ids.shape[1]:], skip_special_tokens=False).replace("<|eot|>", "").strip()

Prompt format is exact: <tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n.

How it was built

architecture Llama β€” 16 layers Γ— 384d, GQA (6 heads / 2 KV), SwiGLU, RoPE, tied embeddings
parameters 31,469,952
context 1,024
vocabulary 16,384 custom BPE
optimiser Muon (LR 0.025) on the 2D hidden matrices, AdamW on embeddings and norms
loss masked to the reply tokens; whole conversations packed per block (FFD)
schedule warmup β†’ stable β†’ cosine decay (WSD); best-validation checkpoint kept
corpus 173,043 conversations; grounded-reading rows carry real encyclopedic prose
training ~1,280 steps Β· ~336M tokens seen Β· 10 epochs Β· from random init
hardware Kaggle dual T4 (GPU) Β· ~2 hours

Files

config.json / model.safetensors           the model
tokenizer.json / tokenizer_config.json    custom BPE tokenizer, 16,384 tokens
loom-weave-3-f16.gguf                      for Ollama / llama.cpp (lookup tags USER_DEFINED)
harness.py                                 runnable search harness
template / params                          read automatically by `ollama run hf.co/...`
Modelfile                                  for building locally
ATTRIBUTION.md                             required credits for the training corpora

Training data

slice source
grounded reading of real prose SQuAD 2.0 (CC BY-SA 4.0) Β· Wikipedia (CC BY-SA)
when to reach for a tool MASSIVE (CC BY 4.0) Β· CLINC150 (CC BY 3.0)
instruction following databricks-dolly-15k (CC BY-SA 3.0)
multi-turn dialogue structure OpenAssistant OASST1 (Apache 2.0)
tokenizer coverage of real prose HuggingFaceFW/fineweb-edu (ODC-By 1.0)
identity, limits, warmth, attribution Textile Labs β€” written for Loom

No language model wrote any training query, and nothing is fine-tuned from anyone's checkpoint. No real user data was used. See ATTRIBUTION.md.

License

Model: MIT. Training data retains its original licences and attribution.

Downloads last month
111
Safetensors
Model size
31.5M params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Collection including textilelabs/Loom-Weave-3