Cygnet
71.9#62 of 261 on the open-weights board · Capability #72
Model guide · Untrained recipe · frozen Gemma 4 12B on vLLM · MIT shim · updated October 26, 2026
Twenty-seventh on the board, and nothing about the model is trained.
Cygnet does not train anything. It serves Google’s Gemma 4 12B unchanged on stock vLLM, and a short MIT-licensed shim turns each decision into one masked answer letter whose probability it reads and calibrates with a single temperature. That was enough for first place in JevBench v1.5.6 and, on the re-measured v1.6 pool, #62 of 261 open-weights systems on board v1.7.48. It is the clearest evidence that a general model, read the right way, can do much of what a trained decision model does.
Cygnet is a third-party project by blockbrain. Jev AI does not host it; this page is a guide to what it is and how to run it yourself.
Try it on this page
Edit the example or paste your own text, then run it. Sign in to run: new accounts get 20 welcome credits, and a failed run costs nothing.
Jev AI does not host Cygnet, so this editor runs Jev 1.14 on the same kind of decision. Compare the answers with your own Cygnet setup.
Support ticket triage · Jev 1.14
1 Input
Hi, this is the third time I am writing. Our payouts have failed every day since Monday and my team cannot pay suppliers. I already re-entered the bank details twice. If this is not fixed today we will have to move to another provider.
2 Questions
One benchmark measured every system under one method. It ranks 261 open-weights systems on one board and 54 hosted APIs on another; Jev, a hosted API, is ranked on the second and shown on the first as an unranked reference. The full board is on the JevBench results page.
Cygnet
71.9#62 of 261 on the open-weights board · Capability #72
Jev 1.13.0
71.4Open-weights reference row · #18 of 54 on the API board · Capability #15
IntelligenceHow often it picks the right answer, half from sealed decisions
CalibrationWhether 0.8 really means about 80%
SpeedMeasured response time
Open half 58.3 · sealed half 51.3; chance-corrected by request type: choice 69.1, yes/no 48.7, score 46.8. The composite also weighs a fourth axis, cost, which these bars leave out. Self-hosted latency is adjusted by the benchmark to approximate production load.
For each decision the shim sends one chat request with the state, the instructions and the options relabelled A, B, C and so on, and asks for one letter. vLLM restricts the answer position to those letters and returns their log-probabilities; the shim sums every token that decodes to each letter, renormalises over the options and applies a calibration temperature of 3.4. Output is one token per decision. The author credits the one-token readout to NInfer.
The temperature was fitted on 241 items the author generated; the README states that JevBench’s public items were used only to measure, never to fit. On the public set the author measured 203 of 231 correct on both an RTX A6000 and an L40S, with a 50–66ms median on the standard tier.
The repository appeared on 24 September 2026. It is a recipe you run, not a service: there is no hosted Cygnet to call.
JevBench v1.7.48 ranks Cygnet #62 of 261 open-weights systems at 71.9, measured on an RTX PRO 6000 at a 37ms raw median. Speed is 91.4 and calibration 87.0; intelligence is 54.8, with 58.3 on the open half and 51.3 on the sealed half. Jev, the board’s unranked reference, scores 71.4 with intelligence 63.6 and leads on all three request types. On the Capability ranking Cygnet is #72 at 70.9, without training its base model; decisio read out of Gemma 4 31B at #32 and deck-31B at #45 are the frozen-base systems above it on the composite.
Serve gemma-4-12B-it on vLLM at the pinned revision, start the shim in front of it, and send /v1/systemone requests to the shim. The author’s runs used 48 GB GPUs.
git clone https://github.com/blockbrain-ai/cygnet-recipe
cd cygnet-recipe
# Follow README.md: vLLM 0.30.0 serving google/gemma-4-12B-it at the pinned revision,
# then the shim, which accepts JevBench's /v1/systemone requests.
curl http://localhost:8000/v1/systemone \
-H "Content-Type: application/json" \
-d '{"state": "Chat: Can you cancel my subscription and refund this month?",
"questions": {"intent": {"type": "choice", "instructions": "What does the customer want?",
"criteria": {"cancel": null, "refund": null, "both": null, "other": null}}}}'From the project’s own documentation and JevBench v1.7.48.
Cygnet gets remarkably far with no training, which makes it worth measuring on your own cases rather than taking either ranking on trust. Run them on Jev here with no setup and keep the answers; the gap on your decisions is the only one that matters.
A recipe from blockbrain: Google’s gemma-4-12B-it served unchanged on stock vLLM, with a short MIT shim that turns each typed decision into one masked answer letter and reads its probability. Nothing is trained or merged.
No. The only fitted value is one calibration temperature, 3.4, which the author fitted on 241 self-generated items rather than on JevBench’s public set.
#62 of 261 open-weights systems at 71.9 in JevBench v1.7.48, with speed 91.4 and calibration 87.0. It led the previous release, v1.5.6, on the old pool. Jev, shown on the current board as an unranked reference, scores 71.4, half a point lower on the composite but ahead on intelligence, 63.6 against 54.8.
The author measured it on an RTX A6000 and an L40S, both 48 GB, and JevBench measured it on an RTX PRO 6000. Other cards have not been measured.
No. Both use Gemma 4 12B, but Winnow fine-tunes it and Cygnet uses the stock model. They are separate projects by different authors.
Every figure on this page is quoted from a published source and was read on October 26, 2026.
Cygnet and every other product named on this page belongs to its respective owner and is not affiliated with Jev AI. Figures are quoted from the sources above as they read on October 26, 2026 and can change.