Jev vs OpenJev

The analysis

Where Jev and OpenJev really differ, and why the bars above land where they do.

Of everything called openJev, razorback16’s is the one built to replace Jev directly. It is a genuine drop-in: it answers on the /v1/systemone wire format and accepts the model name jev-latest, so TypeSafe’s own SDKs work against it without a code change. JevBench v1.6.1 still does not measure its default configuration; the last release that did put it well below Jev on intelligence and calibration.

Its thinking mode was measured, and on the answers it beats Jev: intelligence 70.3 against 63.6, though calibration is 81.4 against Jev’s 90.6. It ranks only #75 of 135 at 17.3 because thinking needs far more compute per decision: its cost score is 28.5 and its speed 72.6, and the harmonic mean punishes weak axes hard. If accuracy on genuinely ambiguous inputs is what you are buying, that configuration is more accurate than Jev and you should know it.

What razorback16’s OpenJev actually is

OpenJev, the Jev-compatible decision server on DiffusionGemma, by razorback16 / Codiv.

It runs DiffusionGemma 26B-A4B-it, quantised to NVFP4, about 18GB of weights, under an Apache-2.0 licence from NVIDIA and Google. A diffusion model denoises many tokens at once rather than writing them one after another, and OpenJev turns that into a decision engine: it builds a masked canvas in which only the answer slots are uncertain, then takes one read-only forward pass and reads the probability distribution over the label tokens.

The model never writes into those slots, which is a real structural guarantee rather than a prompt instruction — an answer cannot go off-schema. When the distribution is too uncertain, above 0.1 entropy, it re-reads up to four times with fresh noise and averages the results.

It supports the same three primitives as Jev: noul, choice with up to 128 options, and score over 2 to 10 ranked levels with an expected value. It adds things Jev does not have: up to eight images per request, a tunable 1–8 denoising steps, 1–32 samples per read, and a thinking budget of up to 4,096 tokens.

On hardware, vLLM on a 24GB NVIDIA GPU handles up to 64 in-flight reads at about 94ms p50 at concurrency 1; the MLX backend runs on Apple silicon with 16GB free at 0.2–0.4s on an M3 Ultra, one read at a time. Codiv also hosts it at api.codiv.ai/v1/systemone.

Where its confidence number comes from

OpenJev computes confidence as 1 − H(p)/ln K: one when the model is certain, zero when the distribution is flat. That is a measure of how concentrated the answer is, not a measure of how often it is right. The documentation publishes no independent calibration check, and the last JevBench release to measure it, v1.4.2.2, scored its calibration well below Jev’s.

The practical consequence is the same as with every uncalibrated system on this site: a high number tells you the model was not torn between options, which is not the same as telling you it was correct. Validate the threshold on your own labelled data before you let it auto-approve anything.

The author is straightforward about the rest, too: answer quality is the quality of DiffusionGemma 26B-A4B used this way, choices cap at 128 against Jev’s 255, images cannot be combined with thinking mode, and the Apple silicon backend does not return reasoning content separately.

openJev Verdict: the cautionary one

Verdict is the openJev most likely to mislead you, because its published numbers are excellent and completely honest at the same time. On its own held-out set of 1,000 examples it reports 95.00% top-1 accuracy, a Brier score of 0.0756 and an expected calibration error of 1.13%. In-domain on Banking77 it reaches 95.2%.

On data it was not fitted to, it collapses. Its own author measures 48.07% on TypeSafe’s 337-case benchmark against 90.80% for Jev, and in JevBench v1.6.1 both Verdict builds answer the sealed decisions barely above chance, so their intelligence axis is near zero and they rank #115 and #120 of 135. Version 1.3.0 of the benchmark, which began measuring intelligence above chance and penalising anything below 50, had already dropped it from #4 to #36 without the model changing at all.

That is not dishonesty and it is barely even a flaw — it is what a 151M encoder is. Verdict was trained on contexts under 71 tokens and runs a 512-token budget with at most 25 candidates. Its README is unusually candid about the failure modes: abstention recall falls from 75.5% to 18.0% against hard-negative siblings, accuracy from 96.0% to 84.75% when distractors are confusable rather than random, and reordering the options flips 4.5% of choices. Fit it to your distribution and it is excellent. Point it at open-ended production text and you get the numbers above.

Jev vs OpenJev FAQ

Other Jev comparisons

Every number on these pages is quoted from a published source and was read on October 9, 2026.

• Jev vs Imajev-4BA 4B model that reads photos, against the model it copies on text.
• Jev vs decider-4bLevel on speed, far behind on the answers.
• Jev vs JevK5A fast open Qwen3.5-4B system, and far behind Jev on the answers.
• Jev vs CygnetNo training at all, thirteenth on the board, and behind Jev on every accuracy axis.
• Jev vs decisioFourth on the board on its biggest base, and it trains nothing of its own.
• Jev vs WinnowOne local model for decisions, chat and images, against a hosted decision model.
• Jev vs djevLevel with Jev when it was last measured, and more expensive to run.
• Jev vs SemIfA fast open rebuild on your own GPU, and where it still falls short.
• Jev vs LayaA model you fine-tune on your data, against one that works out of the box.
• Jev vs Mercury DecideTwo hosted decision models that take the same request, two places apart.
• Run Jev locally?The open alternatives in one table, with licences and hardware.
• Jev vs LLMHow a decision model differs from a chat LLM and a BERT classifier.

Sources

• Benchmark Heaven — JevBench v1.6.1
• razorback16/openjev — Jev-compatible decision server on DiffusionGemma
• openJev Verdict — RLCD ModernBERT decision engine (Apache-2.0)
• Zefan-Cai/Open-Jev — LoRA decision adapters
• SemIf — the project formerly named openjev
• TypeSafe documentation (Jev model card)