Jev vs Winnow

The analysis

Where Jev and Winnow really differ, and why the bars above land where they do.

Winnow is the strongest system on the board that also chats and reads images, and in v1.5.6 it finished ahead of Jev. The full re-measure in v1.6 reversed that: it is #39 of 235 open-weights systems at 72.2, against 71.4 for Jev, which the board shows as an unranked reference, and its intelligence lead has gone — 59.5 against Jev’s 63.6, and 59.2 against 61.6 on the sealed half. Its small edge on the composite comes from the cost axis since Benchmark Heaven’s 29 October GPU-time cost basis, 67.0 against 54.7; on Capability, which leaves cost out, Jev’s reference score is well above it.

Jev keeps a wide edge on calibration, 90.6 against 83.0; Winnow’s own model card says its confidence is entropy-based and not a guaranteed probability of correctness. By request type Jev leads on choice, 77.4 against 70.1 above chance, and on score questions, 63.2 against 57.0, while Winnow edges it on yes/no, 51.5 against 50.3. Jev is also the faster of the two on the benchmark’s measure, 91.2 against 86.2. On Benchmark Heaven’s headline Capability ranking, which averages intelligence and calibration, Jev scores 77.1 and Winnow is #53 at 71.2.

Winnow’s case is breadth on your own hardware. The same loaded model serves /v1/systemone for decisions and /v1/chat/completions for chat, takes images through an optional vision projector, and was tested with a 64K context on a single consumer GPU. If you want one local model to decide, explain and look at a screenshot, nothing else on this site does all three.

What Winnow actually is

Winnow-12B, the Gemma 4 decision fine-tune from EldanRing, by EldanRing.

Winnow-12B is a LoRA fine-tune of google/gemma-4-12B-it, merged into the weights and released under Apache-2.0 as two GGUF files: an 8-bit Q8_0 build of about 12.7 GB, recommended for 16 GB GPUs, and a 16-bit BF16 build of about 23.8 GB. A separate 175 MB projector adds image input. It runs in the author’s llama.cpp-based inference server; no conversion step or separate adapter is needed.

For decisions, the server fills in the shared state once, forks one branch per question, and reads the logits of the answer tokens without generating any text, the same idea as Jev’s typed questions. It supports noul, choice and score. It states that it is not affiliated with or endorsed by TypeSafe.

The training data is private. The model card describes curated typed-decision examples covering routing, policy and rule application, evidence selection, workflow decisions, ordinal judgments, entailment, paraphrase and answerability, followed by a refinement pass. It makes no claim that public benchmarks were excluded from the base model’s pretraining data.

On the author’s RTX 5070 Ti test, a four-question request at near-full 64K context took 25 seconds cold and a median of 143ms once the context was cached, with peak GPU memory of 15.01 GiB including the desktop. JevBench v1.6.1 measured it on an RTX 6000 at a 0.12s median per decision before its self-hosting adjustment.

A second benchmark from the author

Winnow’s model card also reports a second suite, Kev-v9 clean, with 1,046 items, run by its author with Jev called through a hosted API aggregator on the same inputs and scoring rules. There Jev scores 87.00% against 81.55% for Winnow Q8. On the 231-item public JevBench subset the author reports a tie at 85.71%. These are the author’s measurements, not JevBench’s official composite.

Jev vs Winnow FAQ

Other Jev comparisons

Every number on these pages is quoted from a published source and was read on October 31, 2026.

• Jev vs Imajev-4BA 4B model that reads photos, against the model it copies on text.
• Jev vs decider-4bLevel on speed, far behind on the answers.
• Jev vs JevK5A fast open Qwen3.5-4B system, and far behind Jev on the answers.
• Jev vs CygnetNo training at all, forty-sixth on the open-weights board, and behind Jev on every accuracy axis.
• Jev vs decisioTenth on Capability on its biggest base, and it trains nothing of its own.
• Jev vs metask-jev-rainMore accurate answers than Jev on the benchmark, and two of three rows above it on the composite.
• Jev vs djevLevel with Jev when it was last measured, and more expensive to run.
• Jev vs SemIfA fast open rebuild on your own GPU, and where it still falls short.
• Jev vs OpenJevOne name, ten projects. Here is every one, and how the drop-in compares.
• Jev vs LayaA model you fine-tune on your data, against one that works out of the box.
• Jev vs Mercury DecideTwo hosted decision models that take the same request, one place apart.
• Run Jev locally?The open alternatives in one table, with licences and hardware.
• Jev vs LLMHow a decision model differs from a chat LLM and a BERT classifier.

Sources

• Benchmark Heaven — JevBench board v1.7.48 (open weights)
• Benchmark Heaven — JevBench API board
• EldanRing/Winnow-12B model card (Apache-2.0)
• EldanRing/winnow-inference — llama.cpp-based server
• TypeSafe documentation (Jev model card)