On this site
Jev 1.13.0
TypeSafe · System One model
Works on text it has never seen, with nothing to train first.
75.4JevBench v1.2 rank #1
Jev comparison · JevBench v1.2 · updated September 21, 2026
Jev vs openJev
Near-perfect on the data it was trained for, and shaky on anything else.
vs
Alternative
openJev
Heman10x-NGU · Calibrated 151M ModernBERT engine
Near-perfect on the data it was trained for, and only there.
72.5JevBench v1.2 rank #4
- JevHosted APIWhere it runsopenJevSelf-hosted on CPU, or in a browser tab
- JevNot what it is built forOn the data it was trained foropenJev95.00% correct, and well calibrated
- Jev90.80% correctOn data it has never seenopenJev48.07% correct
- Jev64k tokens — a long documentContext windowopenJev512 tokens — a paragraph or two
- Jev74.1% correctHardest test casesopenJev37.7% correct
- JevNo native abstain optionAbstainingopenJevCan report that the right option is not in the list
JevBench v1.2, measured the same way
One benchmark ran every system below under one method, so these bars are comparable with each other in a way that vendor-published figures are not.
Composite score
The top four finish within three points of each other. The composite hides where they actually differ, so the two charts below break it apart.
Capability, higher is better
IntelligenceHow often it picks the right answer
CalibrationWhether 0.8 really means about 80%
SpeedMeasured response time
Accuracy by how hard the decision is
Easy72 straightforward cases
Standard96 everyday cases
Judge146 evaluation-style calls
Hard220 genuinely ambiguous cases
Easy and standard decisions separate almost nothing. The hard tier, highlighted, is where these systems stop agreeing.
One thing these charts leave out: the composite score above also weighs a fourth axis, cost, scored from each system's public list price, and Jev places last of the five on it. We do not reproduce those figures, because Jev AI is an independent reseller metering its own credits and a list price is not what you would pay here. The full table is at the source below, and our rates are on the pricing page.
Every system on the board
The capability scores are charted above; this is the ranking and the deployment detail behind them. Jev and openJev are highlighted.
| Model | Rank | Score | Measured latency | Runs on |
|---|---|---|---|---|
| Jev 1.13.0 (TypeSafe) | #1 | 75.4 | 0.65s median / 0.72s p95 | Hosted production API |
| SemIf (Qwen3.5-4B) | #2 | 74.7 | 0.20s raw, 0.55s adjusted | Self-hosted, RunPod GPU |
| djev (Maisa, DiffusionGemma) | #3 | 74.3 | 0.24s median / 0.31s p95 | Hosted API, free preview |
| openJev Verdict 1.4 | #4 | 72.5 | 0.31s raw, 0.78s adjusted | Self-hosted, CPU |
| Laya (ModernBERT-large) | #7 | 70.1 | 0.79s raw, 1.72s adjusted | Self-hosted, CPU |
JevBench v1.2 scored 42 systems on 534 decisions (72 easy, 96 standard, 146 judge, 220 hard) on 21 September 2026. The composite is a geometric mean of four equally weighted axes: intelligence, calibration, speed and cost. Self-hosted latency is adjusted by x2 plus 0.15s to approximate production conditions, so self-hosted numbers are not directly comparable with the raw latency their authors publish.
Jev and openJev feature by feature
| Attribute | Jev 1.13.0 | openJev |
|---|---|---|
| What it is | Hosted System One model from TypeSafe, version jev-1.13.0 | Self-hosted 151M ModernBERT + GLiClass head, Apache-2.0 |
| Question types | Noul, choice, score | Noul, choice, score, plus an explicit abstention slot |
| Context | 64k tokens per request | 512 tokens; trained on contexts under 71 tokens |
| Options per question | No fixed small ceiling | 25 maximum (24 substantive plus abstention) |
| In-domain accuracy | Not the design target — Jev is zero-shot | 95.00% top-1, ECE 1.13% on its own held-out set; 95.2% on Banking77 |
| Out-of-domain accuracy | 90.80% on TypeSafe’s 337-case benchmark | 48.07% on the same 337 cases, as published by its author |
| Latency | 0.65s median on the production API | Under 35ms per forward pass on CPU; 0.31s raw in JevBench |
| Cost to run | Metered in Jev AI credits — see the pricing page | ~$0.0039 per 1,000 decisions in your own compute |
What each one is better at
Where Jev wins
- 90.80% against 48.07% on TypeSafe’s 337-case benchmark, by the Verdict author’s own measurement.
- Hard-tier 74.1% against 37.7%, and judge-tier 94.5% against 56.2%.
- 64k of context against 512 tokens. Most real support tickets, contracts and agent traces do not fit in 512 tokens.
- Stable under paraphrase, near-miss options and option reordering — the three places Verdict’s own README reports sharp drops.
- No training data required. Verdict’s best numbers come from a distribution it was fitted to; Jev is asked to work on text it has never seen.
Where openJev wins
- One of the cheapest systems on the board to run: JevBench scores its cost axis at 82.4, the best in the top four.
- Runs anywhere: CPU via ONNX, or entirely in a browser tab through WebGPU. No GPU, no API key, no network call.
- Genuinely calibrated in-domain — 1.13% ECE is better than most things of any size.
- Native abstention. It can tell you the right answer is not among the options you gave it.
- Apache-2.0 weights and code, so you can fine-tune it on your own labels.
The analysis
Where Jev and openJev really differ, and why the bars above land where they do.
On its own held-out set of 1,000 examples, openJev Verdict reports 95.00% top-1 accuracy, a multiclass Brier score of 0.0756 and an expected calibration error of 1.13%. Those are excellent numbers and they are honestly reported. In-domain on Banking77 it reaches 95.2%.
On someone else’s data it falls over. The repository publishes its own result on TypeSafe’s 337-case benchmark: 48.07% against 90.80% for Jev. JevBench v1.2 measures the same shape — 86.1% on easy decisions, 67.7% standard, 56.2% judge, 37.7% hard, against Jev’s 100 / 99 / 94.5 / 74.1.
That is not a flaw so much as a description of what a 151M encoder is. Verdict was trained on contexts under 71 tokens and runs a 512-token budget with a maximum of 25 candidates. Feed it the distribution it was fitted to and it is superb and nearly free at around $0.0039 per 1,000 decisions. Feed it open-ended production text and the numbers above are what you get.
Four things called openJev
openJev Verdict, the 151M ModernBERT decision engine, by Heman10x-NGU.
openJev Verdict (author Heman10x-NGU, weights at heman10x/rlcd-modernbert-151m) is the entry in JevBench v1.2 and the subject of the rest of this page. It is ModernBERT-base with a GLiClass bi-encoder classification head, 151,378,177 parameters, Apache-2.0.
openjev-sglang (ekzhang) is a different thing entirely: Qwen3.6-35B-A3B served on B200-class GPUs, scaling to zero on Modal, implementing TypeSafe’s /v1/systemone wire format so the official SDKs point at it unchanged. Its documentation states plainly that its probabilities are not calibrated estimates of correctness.
razorback16/openjev is an open, Jev-compatible System One decision server built on DiffusionGemma — closer in spirit to djev than to Verdict.
SemIf was called openjev before it was renamed. If you read a 2026 benchmark table that puts openJev second overall on a Qwen3.5-4B, that is SemIf under its old name, not Verdict.
How openJev Verdict works
Verdict is non-autoregressive: one forward pass through an encoder, no tokens generated. Candidate labels are framed as natural-language inference sentences (“It is {description}”) to match how the base model was pretrained, and the scores are then temperature-scaled by an L-BFGS-fitted calibrator. Training combines cross-entropy with the Brier score, which is what the author means by RLCD — reinforcement learning for calibrated decisions, the same idea TypeSafe names for Jev, applied at a very different scale.
It supports the same three primitives as Jev: choice, score against an ordered rubric, and noul returning P(true). It adds an abstention slot, so it can say the right option is not in the list — something Jev does not do natively. A forward pass runs in under 35 milliseconds, on CPU via ONNX Runtime or in a browser through WebGPU and WebAssembly, with FP16 quality parity.
Version 1.4 is a serious engineering pass on the inference engine rather than a new model: the calibrator now auto-loads (it previously silently ran at temperature 1.0), labels are templated as NLI sentences, and the context budget dropped from 1,024 to 512 tokens to avoid positional drift. The author measures easy-tier accuracy up 2.1 points, standard up 6.9, and hard-tier calibration error down 60.4%.
A v2.0 exists as well, reporting 77.10% accuracy, 0.0636 Brier and 0.0144 ECE on a LocalLLaMA typed-decisions set. It is not the configuration JevBench v1.2 scored, so the numbers on this page refer to 1.4.
The limitations the author publishes
Verdict’s README is unusually candid, and the failure modes are worth reading before you adopt it. Abstention recall drops from 75.5% on standard missing-option cases to 18.0% against hard-negative siblings and 23.5% against paraphrased prompts. Accuracy falls from 96.0% with random distractors at five candidates to 84.75% with confusable siblings at nine. Randomly permuting the option order flips 4.5% of choices, concentrated in low-confidence predictions.
Those are exactly the conditions production data arrives in: near-miss categories, rephrased inputs, and option lists whose order nobody thought about.
So which should you use?
Pick Jev if
- The label set changes, or you are defining questions at request time rather than training for them.
- Inputs are long or messy: tickets with history, documents, transcripts, agent traces.
- You cannot assemble a labelled training set before shipping, which is the usual situation.
Pick openJev if
- You have a fixed, known label set and enough labelled examples to fine-tune and calibrate on it.
- Inputs are short — comfortably under 512 tokens, ideally much less.
- Volume is high enough that your own compute bill dominates, or the decision has to run offline or in the browser.
Try Jev on the cases you are actually arguing about
These two are not really competitors; they are stages. Use Jev while you are still learning what the decision is and what the labels should be — the playground here gives you that for five free credits. Once a task has settled into a fixed, short, high-volume shape, its answers become the training set you distil into something like Verdict.
Jev vs openJev FAQ
Which project is the real openJev?
There is no single one. openJev Verdict (151M ModernBERT) is the JevBench v1.2 entry; openjev-sglang serves Qwen3.6-35B-A3B in TypeSafe’s wire format; razorback16/openjev is a DiffusionGemma decision server; and SemIf used to be called openjev before it was renamed.
Is openJev Verdict as accurate as Jev?
In-domain, it can be better — 95.00% on its own held-out set with 1.13% calibration error. Out of domain it is not close: its author measures 48.07% on TypeSafe’s 337-case benchmark against 90.80% for Jev.
Can openJev run without a GPU?
Yes. At 151M parameters it runs under 35 milliseconds per forward pass on CPU through ONNX Runtime, and it runs in a browser via WebGPU or WebAssembly.
What is the openJev context limit?
512 tokens as of version 1.4, reduced from 1,024 to avoid positional drift, with training contexts under 71 tokens. Jev accepts 64k tokens per request.
Is openJev free?
The Verdict weights and code are Apache-2.0, so there is no licence fee. JevBench estimates running cost at about $0.0039 per 1,000 decisions on CPU.
Other Jev comparisons
Every number on these pages is quoted from a published source and was read on September 21, 2026.
- Jev vs djevThe fastest decision API, against the best calibrated one.
- Jev vs SemIfThe strongest open rebuild on the board, and where it still falls short.
- Jev vs LayaA model you fine-tune on your data, against one that works out of the box.
Sources
Jev AI is an independent service and is not affiliated with or endorsed by TypeSafe. Jev answers are probabilistic; review important decisions before acting on them. openJev and every other product named on this page belongs to its respective owner and is not affiliated with Jev AI. Figures are quoted from the sources above as they read on September 21, 2026 and can change.