On this site
Jev 1.13.0
TypeSafe · System One model
Fifth overall, fifth on Capability, and 25.1 points ahead on intelligence.
71.5JevBench v1.6.1 rank #5 · Capability #5
Jev comparison · JevBench v1.6.1 · updated October 9, 2026
Jev vs JevK5
A fast open Qwen3.5-4B system, and far behind Jev on the answers.
vs
Alternative
JevK5
allebee · Open Qwen3.5-4B with a distilled LoRA
Forty-first of 135 for version 0.3, the fastest of the systems charted here, and the weights are yours to run.
37.4JevBench v1.6.1 rank #41 · Capability #29
- JevHosted API, generally availableWhere it runsJevK5Self-hosted on your GPU, about 9GB in bf16
- JevProprietary, hostedLicenceJevK5Apache-2.0 weights and code
- Jev63.6 on JevBenchIntelligenceJevK538.5 (v0.3) · 36.3 (v0.2)
- Jev61.6Sealed decisionsJevK538.7 (v0.3) · 32.7 (v0.2)
- Jev50.3 above chanceYes/no questionsJevK521.7 (v0.3) · 18.3 (v0.2)
- Jev91.5SpeedJevK593.9 (v0.3) · 91.6 (v0.2)
- JevUp to 255 options, 64k tokensOptions and input sizeJevK5Up to 16 options, 16,384 tokens
JevBench v1.6.1, measured the same way
One benchmark measured all 173 systems under one method and ranked 135, so these bars are comparable with each other in a way that vendor-published figures are not. The full board and how to read it are on the JevBench results page.
Composite score
The top four ranked systems finish within 2.3 points of each other. The composite hides where systems actually differ, so the charts below break it apart. Benchmark Heaven’s headline Capability ranking, shown next to each score above, averages intelligence and calibration among Jev-class systems.
Capability, higher is better
IntelligenceHow often it picks the right answer, half from sealed decisions
CalibrationWhether 0.8 really means about 80%
SpeedMeasured response time
Intelligence on open and sealed decisions
Open300 decisions from public sources
Sealed1,200 private decisions nobody could tune for
By request type, chance-corrected
ChoicePick one of the options
NoulProbability that a statement is true
ScorePlace the case on a graded scale
The sealed half, highlighted, is where results cannot have been tuned: nobody outside the benchmark has seen those decisions. Request types are scored above chance, so 0 means guessing and a value below 0 means worse than guessing.
One thing these charts leave out: the composite score above also weighs a fourth axis, cost. We do not reproduce it here; the full table is at the source below.
Every system on the board
The capability scores are charted above; this is the ranking and the deployment detail behind them. Jev and JevK5 are highlighted.
| Model | Rank | Score | Measured latency | Runs on |
|---|---|---|---|---|
| Sage 1.3.0 (Levanto Labs, hosted API) | #1 | 74.0 | 0.15s median / 0.29s p95 | Hosted API |
| H2O-Lightning-4B v1.1 (H2O.ai, Qwen3.5-4B) | #2 | 72.5 | 0.03s raw, 0.21s adjusted | Self-hosted GPU |
| Mercury Decide (Inception, diffusion decision API) | #3 | 72.4 | 0.33s median / 0.85s p95 | Hosted API |
| decisio v0.8.0 on Gemma 4 31B (Amin Roudaki, frozen base) | #4 | 71.7 | 0.05s raw, 0.24s adjusted | Self-hosted, H100 |
| Jev 1.13.0 (TypeSafe) | #5 | 71.5 | 0.24s median / 0.30s p95 | Hosted API |
| Quyet-1.0-Large (Chinh Nguyen, Gemma 4 31B decoder) | #6 | 71.4 | 0.12s raw, 0.38s adjusted | Self-hosted, H100 |
| decider-12b v2 (Mapika, Gemma 4 12B) | #7 | 70.9 | 0.05s raw, 0.26s adjusted | Self-hosted, RTX PRO 6000 |
| wity-1 (Wity, closed API, reasoning auto) | #8 | 70.8 | 1.57s median / 4.21s p95 | Hosted API |
| decider-12b v1 (Mapika, stock Gemma 4 12B) | #9 | 70.4 | 0.05s raw, 0.26s adjusted | Self-hosted, RTX PRO 6000 |
| torchcast-decision-12b (Torchcast AI, Gemma 4 12B) | #10 | 69.9 | 0.04s raw, 0.22s adjusted | Self-hosted, H100 |
| Winnow-12B Q8 (EldanRing, Gemma 4 12B) | #11 | 68.9 | 0.12s raw, 0.38s adjusted | Self-hosted, RTX 6000 |
| deck-31B (krishna765, frozen Gemma 4 31B) | #12 | 68.7 | 0.13s raw, 0.40s adjusted | Self-hosted GPU |
| Cygnet (blockbrain, frozen Gemma 4 12B) | #13 | 68.6 | 0.04s raw, 0.22s adjusted | Self-hosted, RTX PRO 6000 |
| decisio v0.8.0 on Gemma 4 12B (Amin Roudaki, frozen base) | #14 | 68.2 | 0.10s raw, 0.35s adjusted | Self-hosted GPU |
| Jev-Omni (akhilaaa3, Gemma 4 12B) | #16 | 67.7 | 0.15s raw, 0.46s adjusted | Self-hosted, RTX 6000 |
| Xor 26B-A4B (Juspay, Gemma 4 26B-A4B) | #17 | 67.4 | 0.06s raw, 0.27s adjusted | Self-hosted GPU |
| Plumb-4B (crh225, JevK5 v0.2 + LoRA) | #30 | 48.3 | 0.02s raw, 0.19s adjusted | Self-hosted, H100 |
| decider-4b v2 (Mapika, Qwen3.5-4B) | #39 | 41.2 | 0.02s raw, 0.20s adjusted | Self-hosted, RTX 5090 |
| JevK5 v0.3 (allebee, Qwen3.5-4B) | #41 | 37.4 | 0.02s raw, 0.19s adjusted | Self-hosted, H100 |
| Hopper (HopitAI, Qwen3.5-4B LoRA) | #52 | 28.9 | 0.06s raw, 0.26s adjusted | Self-hosted, RTX A6000 |
| Imajev-4B (Mohit Garg, Qwen3.5-4B LoRA) | #53 | 28.7 | 0.07s raw, 0.29s adjusted | Self-hosted, RTX 5090 |
| SemIf (Qwen3.5-4B) | #80 | 15.5 | 0.05s raw, 0.25s adjusted | Self-hosted, RTX 5090 |
| Laya (ModernBERT-large) | #127 | 0.0 | 0.83s raw, 1.80s adjusted | Self-hosted, CPU |
JevBench v1.6 is a full re-measure on a new decision pool, not an update of v1.5. Every system answers 1,500 decisions: 300 open decisions from public sources and 1,200 sealed decisions whose text stays private, one request at a time. v1.6.1 put the hosted APIs on that same full set instead of the 600-item subset v1.6.0 gave them, so their accuracy axes are no longer equated onto the self-hosted scale. The official score is a harmonic mean of four equally weighted axes (intelligence, calibration, speed and cost), so one weak axis pulls it down hard; an axis below chance counts as zero, which zeroes the score. The sealed decisions make up half of intelligence, and a system whose open-minus-sealed gap exceeds the field median loses intelligence in proportion. Choice, noul and score requests carry equal weight. v1.6.1 also prices every token-priced system on one common item set — all 1,500 decisions except the 23 that fall outside Jev’s accepted input range — so a system that answered those long items no longer pays for their tokens while one that refused them does not. Refusals still count against intelligence. Benchmark Heaven also leads its page with a Capability ranking: the mean of intelligence and calibration among Jev-class systems, those inside a budget frozen at twice Jev’s v1.5 cost and median latency. v1.6.1 lists 173 systems and ranks 135; the rest were not re-measured, and 13 of them keep a separately dated v1.5.x score. Self-hosted latency is adjusted by x2 plus 0.15s to approximate production load, so it is not comparable with the raw latency authors publish. Scores are not comparable with any v1.5 or v1.4 release.
Jev and JevK5 feature by feature
| Attribute | Jev 1.13.0 | JevK5 |
|---|---|---|
| What it is | Hosted System One model from TypeSafe, version jev-1.13.0 | Open-weight Qwen3.5-4B with a merged, distilled LoRA |
| Licence | Proprietary, hosted | Apache-2.0 for weights and code |
| Request format | POST /v1/systemone | Its own server accepts the TypeSafe-style /v1/systemone request shape |
| Question types | Noul, choice, score — many per call, answered in parallel | Noul, choice, score — each question evaluated separately |
| Options per choice | Up to 255 | Up to 16 |
| Context | 64k tokens per request | Refuses inputs over 16,384 tokens |
| Languages | Best in English; other languages work with lower accuracy | English only |
| Calibration | Post-trained with RLCD; JevBench 90.6 | One temperature fitted on held-out questions; JevBench 89.4 (v0.2), 89.9 (v0.3) |
| Hardware | None — it is an API call | Your own GPU, about 9GB in bf16; the author’s figures are on an H100 and an L40S |
What each one is better at
Where Jev wins
- Intelligence 63.6 against 38.5 for version 0.3 and 36.3 for version 0.2.
- Sealed half 61.6 against 38.7 and 32.7.
- Yes/no questions 50.3 above chance, against 21.7 and 18.3.
- 64k tokens per request against a 16,384-token limit, and up to 255 options per choice against 16.
- Nothing to operate, and questions in one call answered in parallel rather than serialized on one GPU.
Where JevK5 wins
- Twenty-ninth on the Capability ranking in JevBench v1.6.1, which leaves speed and cost out, against Jev’s fifth.
- Faster: speed 93.9 against Jev’s 91.5, the highest of any system charted here; the author reports about 13ms per short decision on an H100.
- Apache-2.0 weights you can run offline or inside your own network, with the training recipe published.
- Plumb-4B, fine-tuned from JevK5 v0.2, ranks #30 at 48.3, eleven places ahead of JevK5 itself: the base takes further fine-tuning well.
The analysis
Where Jev and JevK5 really differ, and why the bars above land where they do.
JevBench v1.6.1 ranks JevK5 version 0.3 #41 of 135 at 37.4, with Jev #5 at 71.5, and the older version 0.2 #48 at 32.1. Plumb-4B, a LoRA fine-tune of version 0.2 by another author, is well ahead of both at #30 with 48.3. On Benchmark Heaven’s headline Capability ranking, which averages intelligence and calibration, Jev is fifth and JevK5 v0.3 twenty-ninth.
Jev keeps a large lead on the answers. Intelligence is 63.6 against 36.3 for version 0.2 and 38.5 for version 0.3, and on the sealed half 61.6 against 32.7 and 38.7. The weakest spot is yes/no questions: version 0.2 scores 18.3 above chance and version 0.3 scores 21.7, against Jev’s 50.3. JevK5 is the faster of the two, 93.9 on speed for version 0.3 against Jev’s 91.5, and calibration is close, 89.9 against 90.6.
The other differences are limits rather than scores. JevK5 is English-only, handles up to 16 options per choice, refuses inputs over 16,384 tokens and serializes requests on one GPU. It is also new: the repository appeared on 22 September, version 0.2.0 followed, and its author says its quality on Jev’s published real-world workflows has not been measured yet.
What JevK5 actually is
JevK5, the open-weight Jev alternative from allebee, by allebee.
JevK5 is Qwen3.5-4B with a distilled LoRA merged into the weights, published on Hugging Face as alibiserikbay/JevK5 under Apache-2.0, with the code on GitHub. It reads answers the way SemIf does: a softmax over the next-token logits of the answer letters with one temperature, so every option gets a probability in a single forward pass and no text is generated. It states that it is not affiliated with TypeSafe, and it does not reproduce Jev’s unpublished architecture.
The training is what separates it from SemIf. Qwen3.6-27B, with thinking, wrote documents carrying hard typed questions across 17 business domains and 11 decision families, answered each question twice, and kept it only when both answers matched. JevK5 was trained on 3,272 of those questions plus 3,272 human-labelled items from public datasets such as MMLU-Pro, BoolQ and banking77. The author states that no JevBench item and no Jev output was used for training.
The author also documents a correction made on 23 September: a small hand-written set used to choose the calibration setting echoed the wording of a few public benchmark items. It was rewritten, and the author reports that no accuracy figure changed.
It installs from GitHub as a Python package and ships a server that accepts the TypeSafe-style /v1/systemone request shape. In bf16 it needs about 9GB of GPU memory. On one H100 the author measures a 13.5ms median on easy and standard items and a 30ms median on hard items with 1–4k-token documents.
So which should you use?
Pick Jev if
- Inputs are long documents, traces or tickets over 16k tokens, or choices with more than 16 options.
- The same question gets worded differently by different people or systems, where Jev’s answers hold more consistently.
- You want a hosted, versioned model with published rate limits instead of a new project to operate.
Pick JevK5 if
- You need the weights: offline use, data residency, or a model you can inspect and fine-tune.
- Your inputs are English, fit in 16k tokens and have at most 16 options.
- You already run GPUs and want millisecond-level decisions on your own hardware.
Try Jev on the cases you are actually arguing about
JevK5 is close enough that the honest comparison is on your own cases. Run them on Jev here — it takes a minute with no setup — and keep the answers: if JevK5 agrees on the cases you care about, you have a strong reason to self-host, and if it does not, you know exactly where the gap is.
Jev vs JevK5 FAQ
Is JevK5 better than Jev?
Not on the answers. JevBench v1.6.1 ranks JevK5 v0.3 #41 at 37.4 against Jev at #5 with 71.5, and it trails Jev on intelligence, 38.5 against 63.6; version 0.2 is #48. It is faster, and its weights are open.
Is JevK5 made by TypeSafe?
No. It is an independent Apache-2.0 project by allebee and states that it is not affiliated with TypeSafe AI. It uses open Qwen3.5-4B weights and SemIf’s option-logit readout rather than Jev’s architecture.
Can I send Jev requests to JevK5?
Its server accepts the TypeSafe-style /v1/systemone request shape, so requests written for Jev need little or no change. The project does not claim full SDK compatibility, and it caps choices at 16 options and inputs at 16,384 tokens, so test your own requests before switching.
What hardware does JevK5 need?
About 9GB of GPU memory in bf16. The author’s figures are on an H100, 13.5ms median on easy and standard items, and an L40S, about 21ms for a demo decision. Without the optional flash-linear-attention package, long inputs run slower.
How is JevK5 different from SemIf?
It uses SemIf’s readout on the same Qwen3.5-4B base, then adds a LoRA distilled from Qwen3.6-27B on double-checked hard questions and a fitted temperature. In JevBench v1.6.1, JevK5 v0.3 ranks #41, JevK5 v0.2 #48 and SemIf #80.
Other Jev comparisons
Every number on these pages is quoted from a published source and was read on October 9, 2026.
- Jev vs Imajev-4BA 4B model that reads photos, against the model it copies on text.
- Jev vs decider-4bLevel on speed, far behind on the answers.
- Jev vs CygnetNo training at all, thirteenth on the board, and behind Jev on every accuracy axis.
- Jev vs decisioFourth on the board on its biggest base, and it trains nothing of its own.
- Jev vs WinnowOne local model for decisions, chat and images, against a hosted decision model.
- Jev vs djevLevel with Jev when it was last measured, and more expensive to run.
- Jev vs SemIfA fast open rebuild on your own GPU, and where it still falls short.
- Jev vs OpenJevOne name, ten projects. Here is every one, and how the drop-in compares.
- Jev vs LayaA model you fine-tune on your data, against one that works out of the box.
- Jev vs Mercury DecideTwo hosted decision models that take the same request, two places apart.
- Run Jev locally?The open alternatives in one table, with licences and hardware.
- Jev vs LLMHow a decision model differs from a chat LLM and a BERT classifier.
Sources
JevK5 and every other product named on this page belongs to its respective owner and is not affiliated with Jev AI. Figures are quoted from the sources above as they read on October 9, 2026 and can change.