Qevi-2B

A full fine-tune of Qwen3-VL-2B-Instruct that answers typed, closed questions about images by reading the model's own logits instead of generating text.

It is not a chat model. Ask it "is there a ladder in this image?" and it returns P(Yes) = 0.97, not a sentence. In exchange for giving up open-ended answers you get substantially better accuracy on closed questions, calibrated probabilities you can threshold on, and up to ~21x faster inference when asking many questions about one image.

	Base Qwen3-VL-2B	Qevi-2B
Accuracy, in-domain	0.855	0.977
Accuracy, held-out domains	0.745	0.889
Expected Calibration Error, held-out	0.160	0.054

---

New to this? EXPLANATION.md is a short, plain-language walkthrough of how it works and why it is fast.

Read this before you use it

This model must be used through a logit readout, not .generate(). All the numbers above are measured by reading the LM-head logits at the position where the model would begin its reply, restricted to the allowed answer tokens. If you call .generate() and parse the text, you will not reproduce them, because that is not the path the model was fine-tuned on.

The readout is ~30 lines of ordinary transformers code and needs no trust_remote_code. A self-contained version is below; the qevi engine bundled in this repo adds question packing.

---

Usage

Minimal, dependency-free (one question)

import torch
from PIL import Image
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

REPO = "MeerDevelopment/Qevi-2B"
model     = Qwen3VLForConditionalGeneration.from_pretrained(REPO, dtype=torch.bfloat16).cuda().eval()
processor = AutoProcessor.from_pretrained(REPO)
tok       = processor.tokenizer

image     = Image.open("photo.jpg").convert("RGB")
statement = "There is a ladder in this image."
answers   = ["Yes", "No"]

# <|image_pad|> is the placeholder the processor expands into the image tokens.
# Omit it and you get: "Image features and image tokens do not match".
prompt = (
    "<|im_start|>user\n<|vision_start|><|image_pad|><|vision_end|>"
    f"Statement: {statement}\n"
    "Is this statement true of the image? Answer Yes or No."
    "<|im_end|>\n<|im_start|>assistant\n"
)

inputs = processor(text=[prompt], images=[image], return_tensors="pt").to(model.device)
with torch.no_grad():
    logits = model(**inputs).logits[0, -1]          # scores at the sentinel position

cand = [tok.encode(a, add_special_tokens=False)[0] for a in answers]
probs = torch.softmax(logits[cand].float(), dim=-1)          # plain softmax: no temperature needed
print(dict(zip(answers, probs.tolist())))           # {'Yes': 0.97, 'No': 0.03}

Many questions, one forward pass (using the bundled engine)

from qevi import build_readouts, forward_logits, softmax, noul_question, choice_question_generic

specs = [
    (noul_question("There is a ladder in this image."),                    ["Yes", "No"]),
    (noul_question("There is a bicycle in this image."),                   ["Yes", "No"]),
    (choice_question_generic("Which is shown?", ["ladder", "bicycle",
                                                 "car", "person"]),        ["A", "B", "C", "D"]),
]
readouts = build_readouts(tok, specs)
logit_sets = forward_logits(model, processor, image, readouts)   # ONE forward pass

for lg in logit_sets:
    print([round(p, 4) for p in softmax(lg)])

All three questions share a single encoding of the image and are isolated from each other by a block-diagonal attention mask, so the answers are identical to asking them separately (verified bitwise, see Order invariance).

---

Question types

Type	Answer alphabet	Temperature	Trained on?
noul	["Yes", "No"]	1.0	yes, 57,000 questions
choice	["A", "B", ...], up to 26 options	1.0	yes, 28,500 questions
score	ordered level words, e.g. ["Poor","Fair","Good","Excellent"]	unvalidated	no — see below

Use plain softmax (T = 1.0). This model needs no temperature scaling: label smoothing during fine-tuning already removed the base model's overconfidence, and every calibration figure reported below is measured at T = 1.0.

Applying temperature scaling on top makes calibration worse, because it over-corrects a model that is no longer overconfident. Measured on the held-out set:

Temperature	ECE (held-out)	ECE (in-domain)
1.0	0.054	0.025
1.5	0.051	0.091
2.45	0.160	0.224

At T = 2.45 the held-out ECE (0.160) is essentially the base model's (0.160) — the entire calibration gain is cancelled out. Earlier versions of this card recommended 2.45 / 1.9 / 3.0; those values were fitted against the base model before fine-tuning and should not be used here.

On score: the training corpus contains only noul and choice questions. The engine supports score and the base model answers such questions zero-shot, but this fine-tune never saw one, so score accuracy and calibration are unmeasured. Treat it as untested.

---

Evaluation

Scored against base Qwen3-VL-2B-Instruct run through the identical readout path, on the same images, in the same session. Both models therefore differ only in their weights, which is what isolates the effect of fine-tuning from the effect of the readout. Neither model saw any of these images during training.

In-domain (17,100 questions / 5,700 images, domains seen in training)

Metric	Base 2B	Qevi-2B	Δ
Accuracy	0.855	0.977	+0.122
Brier (lower better)	0.231	0.039	−0.192
ECE (lower better)	0.088	0.025	−0.063
Overconfidence gap	+0.088	−0.015	−0.103

Held-out (10,800 questions / 3,600 images, 12 domains never trained on)

Metric	Base 2B	Qevi-2B	Δ
Accuracy	0.745	0.889	+0.144
Brier	0.407	0.183	−0.224
ECE	0.160	0.054	−0.106
Overconfidence gap	+0.160	+0.054	−0.106

The model improved more on unseen domains (+14.4) than on trained ones (+12.2), which is the main evidence that it learned a transferable skill rather than memorising the corpus.

A note on the baseline. The corpus stores teacher_probs recorded when it was built, and an earlier draft of this card used those as the baseline. Three of the 31 groups (food101, flowers102, and the content-safety set) had been labelled by the larger Qwen3-VL-8B, so that baseline was a mixture of two models rather than the 2B this model was fine-tuned from. The tables above instead come from re-running base 2B over both eval sets directly. The difference is small (in-domain 0.855 vs the mixed 0.863; held-out 0.745 vs 0.741) and moves the headline slightly in this model's favour, but the comparison is now exactly what it claims to be.

Generalisation to unseen corruptions

Training included only clean, contrast, gaussian_noise and impulse_noise. The other four corruptions were held out entirely, and gained just as much:

Satellite imagery	Seen in training?	Base	Qevi-2B
clean / contrast / gaussian / impulse	yes	0.54–0.72	0.96–0.98
jpeg / motion blur / pixelate / spatter	no	0.45–0.61	0.89–0.97

Known regressions

Two of 31 domains got worse. Stated plainly because they bound what this model is good for:

• German traffic signs (GTSRB): 0.834 vs 0.879 base. The only true regression. No traffic-sign data was in the training corpus, so nothing anchored this domain against weight drift. If signs matter to you, use the base model or fine-tune further with sign data included.
• Pixelated car models: 0.474 vs 0.501 base. Both models are near the floor. Distinguishing 196 car models after heavy pixelation is close to impossible; the information is not in the image.

Speed

Packed inference vs. asking the same questions one at a time. Both paths read logits rather than generating, so this measures the benefit of packing alone and is not flattered by comparing against text generation. RTX 3090, bfloat16, CUDA-synchronised, median of 3 runs after warm-up:

Questions per image	1	5	10	20	30
Packed (ms)	504	509	518	577	618
One at a time (ms)	501	2,504	5,002	9,988	15,013
Speedup	0.99x	4.9x	9.7x	17.3x	24.3x

Marginal cost of each additional question is ~5 ms on this hardware, against ~500 ms to encode the image; that ratio is the whole mechanism.

There is no benefit at N=1 (0.99x, i.e. parity), which is expected: packing one question is just asking one question. The gain comes entirely from amortising the image encode, so it grows with the number of questions you ask per image.

Absolute latencies are image-dependent (a larger image means more vision tokens) and hardware-dependent; the speedup ratio is the portable number.

Order invariance

Permuting question order yields bitwise-identical logits within float tolerance (atol 1e-5). On choice and score tasks, 90/90 images produced byte-for-byte identical predictions packed vs. unpacked. Packing does not silently change answers.

---

Training

Base	Qwen3-VL-2B-Instruct (2,127,532,032 params, Apache 2.0)
Method	Full fine-tune, all parameters trainable (no LoRA)
Objective	Cross-entropy over candidate-answer logits only, label smoothing 0.05
Data	85,500 questions over 28,500 images, 3 questions per image
Epochs	1
LR	2e-5, OneCycle, 10% warm-up
Batch	8 images via gradient accumulation
Optimiser	8-bit AdamW (bitsandbytes), grad clip 1.0
Precision	bfloat16, gradient checkpointing enabled
Hardware	2x RTX 3090, ~2.5 hours

Training targeted ground-truth labels from the source datasets, not the base model's predictions, so the model learns to be correct rather than to imitate its teacher.

Training data

Assembled from 31 publicly available image datasets spanning natural photos, sketches, paintings, clipart, satellite imagery, textures, fine-grained species and product categories, plus synthetic image corruptions. The corpus itself is not redistributed with this model.

Licensing

Weights: CC BY-NC 4.0 (non-commercial). Not because the base model requires it, but because of training-data provenance. Of the twelve source dataset families, only EuroSAT and the content-safety set are permissively licensed (MIT); the other ten (CUB-200-2011, Stanford Cars, Food-101, SUN397, Oxford Flowers-102, Oxford-IIIT Pets, DTD, GTSRB, DomainNet, RESISC45) are distributed for academic or research use only. Whether a research-only dataset licence reaches the weights of a model trained on it is genuinely unsettled, so this release takes the cautious side rather than granting commercial rights it may not be in a position to grant.

Code: Apache 2.0. The qevi package carries no data-provenance encumbrance and is freely usable, including commercially.

Base model: Apache 2.0 (Qwen3-VL-2B-Instruct), unaffected by the above.

If you need commercially-licensable weights: the method, the code and the base model are all permissive. Retrain using the qevi package on data you own or that is licensed for commercial use, and the resulting weights carry no encumbrance from this corpus. The recipe is fully documented above (1 epoch, lr 2e-5, ~5 GPU-hours on two consumer cards).

---

Limitations

• Closed answers only. Every question needs a finite answer set declared up front. It cannot caption, describe, or answer something you did not anticipate.
• Free-form generation survives, but answers get terser. The fine-tune optimised only the typed-readout objective, so we checked whether ordinary .generate() still works. It does: across 5 held-out images x 3 open prompts, greedy decoding produced coherent, grammatical, on-topic text with no repetition collapse (degeneracy 0.002 vs 0.001 for base) and no empty outputs. The measurable change is length: mean 28.3 generated tokens vs 50.7 for the base, a ~44% drop. Having been trained on one-word answers, the model answers open questions more directly ("Spiders" where the base writes "The main object in this image is a spider."). This is a behavioural shift, not a capability loss, but if you want verbose descriptions the base model is the better choice.
• Answer tokens must be single-token-distinguishable. Candidates are matched on their first token, so options sharing a first token (e.g. "Poor" vs "Poorly") collide.
• English prompts only, matching the prompt templates used in training.
• Not a safety classifier. One training domain involved content-severity labels, but this model has not been validated for moderation use and should not be deployed for it without its own evaluation.

The qevi engine (bundled in this repo)

Packing, prompt construction and the readout live in qevi/ inside this repository — three files, ~350 lines, torch only, Apache-2.0. Nothing to pip install, and no trust_remote_code: it is ordinary Python you can read before you run it.

qevi/__init__.py   public API
qevi/core.py       prompt templates, build_readouts(), forward_logits()
qevi/pack.py       block-diagonal mask + M-RoPE position construction

Using it

snapshot_download fetches the whole repo (weights and engine together), then put it on the path:

import sys, torch
from PIL import Image
from huggingface_hub import snapshot_download
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

REPO = "MeerDevelopment/Qevi-2B"
local = snapshot_download(REPO)      # weights + qevi/ in one download
sys.path.insert(0, local)            # makes `import qevi` work

from qevi import (build_readouts, forward_logits, softmax,
                  noul_question, choice_question_generic)

model     = Qwen3VLForConditionalGeneration.from_pretrained(local, dtype=torch.bfloat16).cuda().eval()
processor = AutoProcessor.from_pretrained(local)
tok       = processor.tokenizer
image     = Image.open("photo.jpg").convert("RGB")

specs = [
    (noul_question("There is a ladder in this image."),  ["Yes", "No"]),
    (noul_question("There is a bicycle in this image."), ["Yes", "No"]),
    (choice_question_generic("Which is shown?",
         ["ladder", "bicycle", "car", "person"]),       ["A", "B", "C", "D"]),
]

readouts   = build_readouts(tok, specs)
logit_sets = forward_logits(model, processor, image, readouts)   # ONE forward pass

for (question, answers), lg in zip(specs, logit_sets):
    probs = softmax(lg)                                          # T=1.0, see above
    print(dict(zip(answers, [round(p, 4) for p in probs])))

All three questions share a single encoding of the image and are isolated from one another by the block-diagonal mask, so the answers are identical to asking them separately — verified bitwise, see Order invariance. Asking 30 questions instead of 3 costs about 5 ms more per question rather than another full forward pass each.

If you would rather vendor it

The three files have no dependency on this repo's layout. Copy qevi/ into your own project and import qevi directly; only torch, transformers and pillow are required.

Inspiration and prior art

Qevi is an independent, unaffiliated implementation, for images, of an idea published by TypeSafe. Their "System One" model Jev answers typed questions with type-safe structured values and calibrated probabilities rather than generating text: as they put it, "possible outputs and structure are defined in advance, the model never makes type errors, all answers are accompanied with calibrated probabilities and confidence scores."

Jev is a text model. Qevi asks the same question of images: if the answer is known in advance to be one of a small fixed set, why make a vision-language model write a sentence to say it?

The name reflects the debt. This project began as JEVI, short for Jev for images, and was later renamed Qevi (Qwen + Jevi) once it settled on a Qwen3-VL trunk.

What is and is not shared. The idea of typed, calibrated, non-generative outputs comes from TypeSafe's public writing. Everything here is otherwise independent: no code, weights, data or training method from TypeSafe is used, and this model is not endorsed by or affiliated with them. In particular Qevi does not implement their RLCD training method: it is trained with ordinary cross-entropy over candidate-answer logits with label smoothing, which is a proper scoring rule and is what produces the calibration improvements reported above.

TypeSafe's published notes on Jev's failure modes (literal reading of questions, unreliable counting) informed how this corpus was built and what this model does not claim to do.

Citation

@misc{qevi2b,
  title  = {Qevi-2B: typed question answering over images without text generation},
  year   = {2026},
  note   = {Full fine-tune of Qwen3-VL-2B-Instruct with packed native-logit readout}
}