On this site
Jev 1.13.0
TypeSafe · System One model
A 71.4 reference score, above 186 of 235 open-weights systems, and 25.1 points ahead on intelligence.
71.4JevBench v1.7.48 · Open-weights reference row · #15 of 48 on the API board · Capability #12
Jev comparison · JevBench v1.7.48 · updated October 31, 2026
Jev vs JevK5
A fast open Qwen3.5-4B system, and far behind Jev on the answers.
vs
Alternative
JevK5
allebee · Open Qwen3.5-4B with a distilled LoRA
#119 of 235 open-weights systems for version 0.3, the fastest of the systems charted here, and the weights are yours to run.
39.2JevBench v1.7.48 · #119 of 235 on the open-weights board · Capability #91
- JevHosted API, generally availableWhere it runsJevK5Self-hosted on your GPU, about 9GB in bf16
- JevProprietary, hostedLicenceJevK5Apache-2.0 weights and code
- Jev63.6 on JevBenchIntelligenceJevK538.5 (v0.3) · 36.3 (v0.2)
- Jev61.6Sealed decisionsJevK538.7 (v0.3) · 32.7 (v0.2)
- Jev50.3 above chanceYes/no questionsJevK521.7 (v0.3) · 18.3 (v0.2)
- Jev91.2SpeedJevK593.8 (v0.3) · 91.6 (v0.2)
- JevUp to 255 options, 64k tokensOptions and input sizeJevK5Up to 16 options, 16,384 tokens
Try it on this page
Run Jev 1.14 right here
Edit the example or paste your own text, then run it. Sign in to run: new accounts get 20 welcome credits, and a failed run costs nothing.
Support ticket triage · Jev 1.14
1 Input
Hi, this is the third time I am writing. Our payouts have failed every day since Monday and my team cannot pay suppliers. I already re-entered the bank details twice. If this is not fixed today we will have to move to another provider.
2 Questions
- Choice Which team should handle this ticket?
- Yes / No Does the customer need a response today?
- Score How frustrated is the customer?
- Yes / No Does the customer threaten to leave or switch providers?
JevBench v1.7.48, measured the same way
One benchmark measured every system under one method — it ranks 235 open-weights systems on one board and 48 hosted API offerings on another, with the same scores in both — so these bars are comparable with each other in a way that vendor-published figures are not. The full board and how to read it are on the JevBench results page.
Composite score
Plain numbers are ranks on the open-weights board; “API” marks a rank on the separate board for hosted API offerings. Jev, a hosted API, is ranked there, and appears on the open-weights board only as an unranked reference row, where its score falls between #49 and #50. The composite hides where systems actually differ, so the charts below break it apart. Benchmark Heaven’s headline Capability ranking, shown next to each score above, averages intelligence and calibration among Jev-class systems.
Capability, higher is better
IntelligenceHow often it picks the right answer, half from sealed decisions
CalibrationWhether 0.8 really means about 80%
SpeedMeasured response time
Intelligence on open and sealed decisions
Open300 decisions from public sources
Sealed1,200 private decisions nobody could tune for
By request type, chance-corrected
ChoicePick one of the options
NoulProbability that a statement is true
ScorePlace the case on a graded scale
The sealed half, highlighted, is where results cannot have been tuned: nobody outside the benchmark has seen those decisions. Request types are scored above chance, so 0 means guessing and a value below 0 means worse than guessing.
One thing these charts leave out: the composite score above also weighs a fourth axis, cost. We do not reproduce it here; the full table is at the source below.
Every system on the board
The capability scores are charted above; this is the ranking and the deployment detail behind them. Jev and JevK5 are highlighted.
| Model | Rank | Score | Measured latency | Runs on |
|---|---|---|---|---|
| H2O-Lightning-4B v1.1 (H2O.ai, Qwen3.5-4B) | #2 | 77.9 | 0.03s raw, 0.21s adjusted | Self-hosted GPU |
| Xor 26B-A4B (Juspay, Gemma 4 26B-A4B) | #21 | 74.0 | 0.06s raw, 0.27s adjusted | Self-hosted GPU |
| Arcbright Elf 1 31B | #23 | 74.0 | 0.06s raw, 0.27s adjusted | Self-hosted GPU |
| Quyet-1.0-Large (Chinh Nguyen, Gemma 4 31B decoder) | #25 | 73.8 | 0.12s raw, 0.39s adjusted | Self-hosted, H100 |
| Sage 1.3.0 (Levanto Labs, hosted API) | API #11 | 73.7 | 0.17s median / 0.41s p95 | Hosted API |
| decider-12b v2 (Mapika, Gemma 4 12B) | #27 | 73.7 | 0.05s raw, 0.26s adjusted | Self-hosted, RTX PRO 6000 |
| decisio v0.8.0 on Gemma 4 31B (Amin Roudaki, frozen base) | #32 | 73.2 | 0.05s raw, 0.24s adjusted | Self-hosted, H100 |
| decider-12b v1 (Mapika, stock Gemma 4 12B) | #33 | 73.2 | 0.05s raw, 0.26s adjusted | Self-hosted, RTX PRO 6000 |
| Mercury Decide (Inception, diffusion decision API) | API #14 | 72.2 | 0.34s median / 0.96s p95 | Hosted API |
| Winnow-12B Q8 (EldanRing, Gemma 4 12B) | #39 | 72.2 | 0.12s raw, 0.40s adjusted | Self-hosted, RTX 6000 |
| deck-31B (krishna765, frozen Gemma 4 31B) | #45 | 71.9 | 0.13s raw, 0.41s adjusted | Self-hosted GPU |
| Cygnet (blockbrain, frozen Gemma 4 12B) | #46 | 71.9 | 0.04s raw, 0.24s adjusted | Self-hosted, RTX PRO 6000 |
| Jev 1.13.0 (TypeSafe) | API #15 | 71.4 | 0.24s median / 0.31s p95 | Hosted API |
| Jev-Omni (akhilaaa3, Gemma 4 12B) | #52 | 71.0 | 0.16s raw, 0.47s adjusted | Self-hosted, RTX 6000 |
| decisio v0.8.0 on Gemma 4 12B (Amin Roudaki, frozen base) | #54 | 70.6 | 0.12s raw, 0.38s adjusted | Self-hosted GPU |
| torchcast-decision-12b (Torchcast AI, Gemma 4 12B) | #62 | 69.8 | 0.04s raw, 0.24s adjusted | Self-hosted, H100 |
| wity-1 (Wity, closed API, reasoning auto) | API #20 | 69.0 | 1.85s median / 7.97s p95 | Hosted API |
| Plumb-4B (crh225, JevK5 v0.2 + LoRA) | #96 | 50.4 | 0.02s raw, 0.19s adjusted | Self-hosted, H100 |
| decider-4b v2 (Mapika, Qwen3.5-4B) | #111 | 43.1 | 0.03s raw, 0.21s adjusted | Self-hosted, RTX 5090 |
| JevK5 v0.3 (allebee, Qwen3.5-4B) | #119 | 39.2 | 0.02s raw, 0.19s adjusted | Self-hosted, H100 |
| Imajev-4B (Mohit Garg, Qwen3.5-4B LoRA) | #133 | 30.0 | 0.07s raw, 0.29s adjusted | Self-hosted, RTX 5090 |
| Hopper (HopitAI, Qwen3.5-4B LoRA) | #136 | 27.7 | 0.06s raw, 0.26s adjusted | Self-hosted, RTX A6000 |
| SemIf (Qwen3.5-4B) | #160 | 16.1 | 0.06s raw, 0.27s adjusted | Self-hosted, RTX 5090 |
| Laya (ModernBERT-large) | #227 | 0.0 | 0.93s raw, 2.01s adjusted | Self-hosted, CPU |
JevBench method v1.7 applies a 10-second limit per decision: a response taking longer than 10 seconds counts as wrong and contributes zero calibration for that item. Speed uses all decisions, including failures. Benchmark Heaven re-scored recorded per-decision times; it did not run every model again. Tokens, draws and raw outputs are unchanged. Since method v1.8.0 (29 October 2026), self-hosted open-weights rows are costed by GPU time on a monthly-rented RTX PRO 6000 with an efficient inference engine instead of base-model list prices; encoders, architectures outside the six calibrated Gemma/Qwen classes and rows without a verified token basis keep their old costs, and API costs and both caps are unchanged. Intelligence, calibration and latency did not move, but most open-weights cost scores rose, so their composites rose and the Jev-class pool grew. Four rows recorded on a loaded CPU host await re-measurement and remain unranked with their dated v1.6 values; five CPU rows were measured again on GPU and replaced in full. The official composite combines intelligence, calibration, speed and cost using the frozen scorer. Each full-set system answers 1,500 decisions, 300 open and 1,200 sealed; most hosted APIs use an equated 600-item set. Half of intelligence comes from sealed decisions. The open-weights and API boards compare the same measurements within separate hosting groups. Board v1.7.48 ranks 235 open-weights systems and 48 API offerings; an additional API result is preliminary and unranked. There are 110 newer ranked open-weights rows tagged with their draw. Jev 1.13.0 remains an unranked reference on the open-weights board and is ranked on the API board. Capability averages intelligence and calibration among systems inside caps frozen at twice Jev’s v1.5 reference: 217 open-weights systems and 33 APIs qualify. The caps do not track current Jev latency. Self-hosted latency uses an estimated 2× measured latency plus 0.15s; API times include the network. Scores and splits on this site use method v1.8 (v1.7 scoring with the v1.8 cost basis); historical v1.6, v1.5 and v1.4 results are not current standings.
Jev and JevK5 feature by feature
| Attribute | Jev 1.13.0 | JevK5 |
|---|---|---|
| What it is | Hosted System One model from TypeSafe, version jev-1.13.0 | Open-weight Qwen3.5-4B with a merged, distilled LoRA |
| Licence | Proprietary, hosted | Apache-2.0 for weights and code |
| Request format | POST /v1/systemone | Its own server accepts the TypeSafe-style /v1/systemone request shape |
| Question types | Noul, choice, score — many per call, answered in parallel | Noul, choice, score — each question evaluated separately |
| Options per choice | Up to 255 | Up to 16 |
| Context | 64k tokens per request | Refuses inputs over 16,384 tokens |
| Languages | Best in English; other languages work with lower accuracy | English only |
| Calibration | Post-trained with RLCD; JevBench 90.6 | One temperature fitted on held-out questions; JevBench 89.4 (v0.2), 89.8 (v0.3) |
| Hardware | None — it is an API call | Your own GPU, about 9GB in bf16; the author’s figures are on an H100 and an L40S |
What each one is better at
Where Jev wins
- Intelligence 63.6 against 38.5 for version 0.3 and 36.3 for version 0.2.
- Sealed half 61.6 against 38.7 and 32.7.
- Yes/no questions 50.3 above chance, against 21.7 and 18.3.
- 64k tokens per request against a 16,384-token limit, and up to 255 options per choice against 16.
- Nothing to operate, and questions in one call answered in parallel rather than serialized on one GPU.
Where JevK5 wins
- Ninety-first on the Capability ranking in JevBench v1.7.48, which leaves speed and cost out, at 64.2 against Jev’s 77.1.
- Faster: speed 93.8 against Jev’s 91.2, the highest of any system charted here; the author reports about 13ms per short decision on an H100.
- Apache-2.0 weights you can run offline or inside your own network, with the training recipe published.
- Plumb-4B, fine-tuned from JevK5 v0.2, ranks #96 at 50.4, twenty-three places ahead of JevK5 itself: the base takes further fine-tuning well.
The analysis
Where Jev and JevK5 really differ, and why the bars above land where they do.
JevBench v1.7.48 ranks JevK5 version 0.3 #119 of 235 open-weights systems at 39.2 and the older version 0.2 #126 at 33.6; Jev, an unranked reference row on that board, scores 71.4. Plumb-4B, a LoRA fine-tune of version 0.2 by another author, is well ahead of both at #96 with 50.4. On Benchmark Heaven’s headline Capability ranking, which averages intelligence and calibration, Jev scores 77.1 and JevK5 v0.3 is #91 at 64.2.
Jev keeps a large lead on the answers. Intelligence is 63.6 against 36.3 for version 0.2 and 38.5 for version 0.3, and on the sealed half 61.6 against 32.7 and 38.7. The weakest spot is yes/no questions: version 0.2 scores 18.3 above chance and version 0.3 scores 21.7, against Jev’s 50.3. JevK5 is the faster of the two, 93.8 on speed for version 0.3 against Jev’s 91.2, and calibration is close, 89.8 against 90.6.
The other differences are limits rather than scores. JevK5 is English-only, handles up to 16 options per choice, refuses inputs over 16,384 tokens and serializes requests on one GPU. It is also new: the repository appeared on 22 September, version 0.2.0 followed, and its author says its quality on Jev’s published real-world workflows has not been measured yet.
What JevK5 actually is
JevK5, the open-weight Jev alternative from allebee, by allebee.
JevK5 is Qwen3.5-4B with a distilled LoRA merged into the weights, published on Hugging Face as alibiserikbay/JevK5 under Apache-2.0, with the code on GitHub. It reads answers the way SemIf does: a softmax over the next-token logits of the answer letters with one temperature, so every option gets a probability in a single forward pass and no text is generated. It states that it is not affiliated with TypeSafe, and it does not reproduce Jev’s unpublished architecture.
The training is what separates it from SemIf. Qwen3.6-27B, with thinking, wrote documents carrying hard typed questions across 17 business domains and 11 decision families, answered each question twice, and kept it only when both answers matched. JevK5 was trained on 3,272 of those questions plus 3,272 human-labelled items from public datasets such as MMLU-Pro, BoolQ and banking77. The author states that no JevBench item and no Jev output was used for training.
The author also documents a correction made on 23 September: a small hand-written set used to choose the calibration setting echoed the wording of a few public benchmark items. It was rewritten, and the author reports that no accuracy figure changed.
It installs from GitHub as a Python package and ships a server that accepts the TypeSafe-style /v1/systemone request shape. In bf16 it needs about 9GB of GPU memory. On one H100 the author measures a 13.5ms median on easy and standard items and a 30ms median on hard items with 1–4k-token documents.
So which should you use?
Pick Jev if
- Inputs are long documents, traces or tickets over 16k tokens, or choices with more than 16 options.
- The same question gets worded differently by different people or systems, where Jev’s answers hold more consistently.
- You want a hosted, versioned model with published rate limits instead of a new project to operate.
Pick JevK5 if
- You need the weights: offline use, data residency, or a model you can inspect and fine-tune.
- Your inputs are English, fit in 16k tokens and have at most 16 options.
- You already run GPUs and want millisecond-level decisions on your own hardware.
Try Jev on the cases you are actually arguing about
JevK5 is close enough that the honest comparison is on your own cases. Run them on Jev here — it takes a minute with no setup — and keep the answers: if JevK5 agrees on the cases you care about, you have a strong reason to self-host, and if it does not, you know exactly where the gap is.
Jev vs JevK5 FAQ
Is JevK5 better than Jev?
Not on the answers. JevBench v1.7.48 ranks JevK5 v0.3 #119 of 235 open-weights systems at 39.2, against 71.4 for Jev, shown there as an unranked reference, and it trails Jev on intelligence, 38.5 against 63.6; version 0.2 is #126. It is faster, and its weights are open.
Is JevK5 made by TypeSafe?
No. It is an independent Apache-2.0 project by allebee and states that it is not affiliated with TypeSafe AI. It uses open Qwen3.5-4B weights and SemIf’s option-logit readout rather than Jev’s architecture.
Can I send Jev requests to JevK5?
Its server accepts the TypeSafe-style /v1/systemone request shape, so requests written for Jev need little or no change. The project does not claim full SDK compatibility, and it caps choices at 16 options and inputs at 16,384 tokens, so test your own requests before switching.
What hardware does JevK5 need?
About 9GB of GPU memory in bf16. The author’s figures are on an H100, 13.5ms median on easy and standard items, and an L40S, about 21ms for a demo decision. Without the optional flash-linear-attention package, long inputs run slower.
How is JevK5 different from SemIf?
It uses SemIf’s readout on the same Qwen3.5-4B base, then adds a LoRA distilled from Qwen3.6-27B on double-checked hard questions and a fitted temperature. In JevBench v1.7.48, JevK5 v0.3 ranks #119, JevK5 v0.2 #126 and SemIf #160 on the open-weights board.
Other Jev comparisons
Every number on these pages is quoted from a published source and was read on October 31, 2026.
- Jev vs Imajev-4BA 4B model that reads photos, against the model it copies on text.
- Jev vs decider-4bLevel on speed, far behind on the answers.
- Jev vs CygnetNo training at all, forty-sixth on the open-weights board, and behind Jev on every accuracy axis.
- Jev vs decisioTenth on Capability on its biggest base, and it trains nothing of its own.
- Jev vs metask-jev-rainMore accurate answers than Jev on the benchmark, and two of three rows above it on the composite.
- Jev vs WinnowOne local model for decisions, chat and images, against a hosted decision model.
- Jev vs djevLevel with Jev when it was last measured, and more expensive to run.
- Jev vs SemIfA fast open rebuild on your own GPU, and where it still falls short.
- Jev vs OpenJevOne name, ten projects. Here is every one, and how the drop-in compares.
- Jev vs LayaA model you fine-tune on your data, against one that works out of the box.
- Jev vs Mercury DecideTwo hosted decision models that take the same request, one place apart.
- Run Jev locally?The open alternatives in one table, with licences and hardware.
- Jev vs LLMHow a decision model differs from a chat LLM and a BERT classifier.
Sources
JevK5 and every other product named on this page belongs to its respective owner and is not affiliated with Jev AI. Figures are quoted from the sources above as they read on October 31, 2026 and can change.