Winnow-E4B

Winnow-E4B is a compact EldanRing fine-tune of Gemma 4 E4B IT for typed decisions, chat, and image input. Give it a shared state and questions with known answer options; it scores the candidates without generating an explanation. Typical uses include routing requests, choosing actions, checking conditions, and rating urgency.

The GGUF downloads contain the merged fine-tune. No separate adapter or base-model download is needed. Image input uses the matching vision projector.

Quickstart · Inference code · API reference · Benchmarks

Decision results

These published Q8 results compare the models on the same questions. Jev and Kev report top-choice accuracy; Typed reports agreement with synthetic teacher labels.

Evaluation	Winnow-E4B Q8	Winnow-12B Q8
JevBench public subset, 231 questions	80.52% (186/231)	85.71% (198/231)
Kev v9 clean, 1,046 questions	72.66%	81.45%
Kev v9 additional, 390 questions	55.90%	68.97%
Local typed decisions, 2,000 decisions	72.30%	70.10%

Winnow-12B leads on the public Jev and Kev suites. E4B leads by 2.2 points on the local Typed teacher-agreement panel. These frozen panels were used during development. JevBench reports public-subset accuracy rather than the composite leaderboard score.

The 12B Kev-clean comparator is 852/1,046, distinct from its separate 853/1,046 release campaign. Evaluation details cover uncertainty, calibration, and scope.

Downloads

File	Role	Size
Winnow-E4B-Q8_0.gguf	Q8 language model; measured decision, chat, and vision setup	8.01 GB / 7.46 GiB
Winnow-E4B-BF16.gguf	Higher-precision language model	15.05 GB / 14.02 GiB
mmproj-Winnow-E4B.gguf	F16 vision projector	990 MB / 0.922 GiB

Choose Q8 for a smaller memory footprint, or BF16 for higher-precision weights. Add the matching projector for images.

Use an exact target filename or a Winnow preset. Checksums identify every file.

Running the model

The quickstart covers installation, downloads, and launch commands. The server provides /v1/systemone for decisions and /v1/chat/completions for ordinary chat, streaming, and images.

F16 is the default and recommended target K/V cache. Cache precision is separate from the GGUF weight format; use --cache q8_0 for an explicit Q8 override. The pinned MTP assistant uses the target's shared K/V cache.

Question type	Input	Result
noul	A yes/no question	Probability of true
choice	Named options with optional descriptions	Selected option, probabilities, confidence
score	Ordered levels	Level probabilities, expected score, confidence

Questions share a state prefill and can reuse a cached prefix. Candidate probabilities are conditional on the supplied options; confidence measures concentration, not guaranteed correctness.

Measured performance

On an RTX 5070 Ti 16 GB, the matched 8K text workload delivered 147.8 decisions/s for E4B Q8 versus 67.2 decisions/s for 12B Q8 at 64 questions per request. These are medians of ten warm requests with Q8 weights and KV cache; results depend on workload and batching.

Q8 configuration	Observed peak device memory	Suggested GPU capacity
8K text-only decision workload	8.56 GiB	10 GB+ VRAM
64K vision and chat operational probes	10.72 GiB across the probes	12 GB+ VRAM

Capacity suggestions are inferred from measurements. Six cached questions over a 65,023-position image state took 129–131 ms after an 11.59 s first request. A separate 62,431-position multimodal prompt generated 512 tokens at 71.2 tokens/s. See timing and memory details.

Calibration and adaptive reasoning

For the direct text profile, Q8's fitted winnow.temperature is 1.2574172017327816, based on 778 calibration questions. BF16's separately fitted value is 1.3331553765162731. Temperature changes probabilities without changing the winning option; the values belong to their respective runtime profiles.

Direct decisions are the default. Adaptive reasoning adds generated context before rescoring a question. A calibrated 50:50 evaluation improved Brier 0.524807 → 0.481402 and NLL 1.591870 → 1.385153, while source-equal agreement changed +0.083 pp. This profile combines calibration with generated context.

A separate historical uncalibrated recipe gained on Jev and Kev but lost 57 Typed teacher agreements (1,447 → 1,390/2,000), for a net loss of five across all 3,277 decisions. Mean direct HTTP time inside that workflow was 37 ms; the full router averaged 234 ms, including bypasses. Reasoning evaluations give the policies and timing definitions.

MTP separately drafts ordinary chat tokens with the matching E4B assistant. The 8K Q8 vision+MTP4 preset sampled 10,197 MiB peak memory. Direct serving needs no assistant. See runtime profiles and assistant attribution.

Training and limitations

Winnow-E4B is a rank-32, alpha-64 LoRA fine-tune. Language tensors were trained; vision and audio modules remained frozen. The adapter was merged in FP32 before Q8_0 and BF16 export. The private training set combines contrastive synthetic decisions, verified labels, teacher distributions, semantic tasks, and targeted hard cases. Training, development, calibration, and reserved tests were separated; the training data and pipeline are not released.

Credits and license

Winnow-E4B is an independent fine-tune by EldanRing of Google DeepMind's Gemma 4 E4B IT, released under Apache 2.0. See LICENSE and NOTICE.

The separate inference code builds on llama.cpp and preserves its MIT license. Jev-style describes the typed-decision interface; Winnow is not affiliated with or endorsed by TypeSafe, Google, or llama.cpp.