Jev vs Cygnet

The analysis

Where Jev and Cygnet really differ, and why the bars above land where they do.

Cygnet is still the clearest evidence that a general model, read the right way, can do much of what a trained decision model does. It led v1.5.6; on the re-measured v1.6 pool it is #13 of 135 at 68.6 against Jev’s 71.5 at #5. What it keeps is the cheap side of the board: speed 91.8 against Jev’s 91.5, from a 37ms raw median on the benchmark’s GPU, and a slightly better cost estimate, 56.4 against 54.7.

On the answers Jev is now ahead across the board: 63.6 against 54.8 on intelligence, and 61.6 against 51.3 on the sealed half. The split by request type that used to favour Cygnet on yes/no questions has closed — Jev leads there too, though only by 50.3 against 48.7 above chance, and more clearly on choice questions, 77.4 against 69.1, and score questions, 63.2 against 46.8. Calibration is 90.6 against 87.0, and on Benchmark Heaven’s headline Capability ranking, which averages intelligence and calibration, Jev is fifth at 77.1 and Cygnet eighteenth at 70.9.

The practical limits are the bigger difference. Cygnet rejects inputs over 16,384 tokens and choices with more than 26 options, and sends one chat request per decision. It has been measured on three GPU models, and the repository appeared on 24 September: it is a recipe you run, not a service.

What Cygnet actually is

Cygnet, typed decisions read from frozen Gemma 4 12B, by blockbrain.

Cygnet is a recipe from blockbrain. It runs google/gemma-4-12B-it at a pinned revision on unmodified vLLM 0.30.0, with a small shim in front, written with the Python standard library, that accepts JevBench’s /v1/systemone requests. Nothing about the model is trained or merged.

For each decision the shim sends one chat request with the state, the instructions and the options relabelled A, B, C and so on, and asks for one letter. vLLM restricts the answer position to those letters and returns their log-probabilities; the shim sums every token that decodes to each letter, renormalises over the options and applies a calibration temperature of 3.4. Output is one token per decision. The author credits the one-token readout to NInfer.

The temperature was fitted on 241 items the author generated; the README states that JevBench’s public items were used only to measure, never to fit. On the public set the author measured 203 of 231 correct on both an RTX A6000 and an L40S, with a 50–66ms median on the standard tier.

The shim is MIT. The weights are Google’s Gemma 4 12B under Apache-2.0, subject to Google’s Gemma Prohibited Use Policy.

Jev vs Cygnet FAQ

Other Jev comparisons

Every number on these pages is quoted from a published source and was read on October 9, 2026.

• Jev vs Imajev-4BA 4B model that reads photos, against the model it copies on text.
• Jev vs decider-4bLevel on speed, far behind on the answers.
• Jev vs JevK5A fast open Qwen3.5-4B system, and far behind Jev on the answers.
• Jev vs decisioFourth on the board on its biggest base, and it trains nothing of its own.
• Jev vs WinnowOne local model for decisions, chat and images, against a hosted decision model.
• Jev vs djevLevel with Jev when it was last measured, and more expensive to run.
• Jev vs SemIfA fast open rebuild on your own GPU, and where it still falls short.
• Jev vs OpenJevOne name, ten projects. Here is every one, and how the drop-in compares.
• Jev vs LayaA model you fine-tune on your data, against one that works out of the box.
• Jev vs Mercury DecideTwo hosted decision models that take the same request, two places apart.
• Run Jev locally?The open alternatives in one table, with licences and hardware.
• Jev vs LLMHow a decision model differs from a chat LLM and a BERT classifier.

Sources

• Benchmark Heaven — JevBench v1.6.1
• blockbrain-ai/cygnet-recipe — typed decisions from frozen Gemma 4 12B
• google/gemma-4-12B-it on Hugging Face
• TypeSafe documentation (Jev model card)