On this site
Jev 1.13.0
TypeSafe · System One model
Nothing for you to run, and still ahead where the cases get hard.
75.4JevBench v1.2 rank #1
Jev comparison · JevBench v1.2 · updated September 21, 2026
Jev vs SemIf
The strongest open rebuild on the board, and where it still falls short.
vs
Alternative
SemIf
TheoLeeCJ · Open logit reader on Qwen3.5-4B
Gets within 0.7 points of Jev on a graphics card you already own.
74.7JevBench v1.2 rank #2
- JevHosted API — nothing for you to operateWhere it runsSemIfSelf-hosted on a GPU you own or rent
- JevProprietary, hostedLicenceSemIfMIT code over open weights
- Jev94.5% correctEvaluation-style casesSemIf95.2% correct
- Jev74.1% correctHardest test casesSemIf59.5% correct
- JevCalibrated as delivered — 82.7 / 100ProbabilitiesSemIfNeeds fitting on your own workload — 72.6 / 100
- JevSent to TypeSafe’s APIWhere your data goesSemIfNever leaves your network
JevBench v1.2, measured the same way
One benchmark ran every system below under one method, so these bars are comparable with each other in a way that vendor-published figures are not.
Composite score
The top four finish within three points of each other. The composite hides where they actually differ, so the two charts below break it apart.
Capability, higher is better
IntelligenceHow often it picks the right answer
CalibrationWhether 0.8 really means about 80%
SpeedMeasured response time
Accuracy by how hard the decision is
Easy72 straightforward cases
Standard96 everyday cases
Judge146 evaluation-style calls
Hard220 genuinely ambiguous cases
Easy and standard decisions separate almost nothing. The hard tier, highlighted, is where these systems stop agreeing.
One thing these charts leave out: the composite score above also weighs a fourth axis, cost, scored from each system's public list price, and Jev places last of the five on it. We do not reproduce those figures, because Jev AI is an independent reseller metering its own credits and a list price is not what you would pay here. The full table is at the source below, and our rates are on the pricing page.
Every system on the board
The capability scores are charted above; this is the ranking and the deployment detail behind them. Jev and SemIf are highlighted.
| Model | Rank | Score | Measured latency | Runs on |
|---|---|---|---|---|
| Jev 1.13.0 (TypeSafe) | #1 | 75.4 | 0.65s median / 0.72s p95 | Hosted production API |
| SemIf (Qwen3.5-4B) | #2 | 74.7 | 0.20s raw, 0.55s adjusted | Self-hosted, RunPod GPU |
| djev (Maisa, DiffusionGemma) | #3 | 74.3 | 0.24s median / 0.31s p95 | Hosted API, free preview |
| openJev Verdict 1.4 | #4 | 72.5 | 0.31s raw, 0.78s adjusted | Self-hosted, CPU |
| Laya (ModernBERT-large) | #7 | 70.1 | 0.79s raw, 1.72s adjusted | Self-hosted, CPU |
JevBench v1.2 scored 42 systems on 534 decisions (72 easy, 96 standard, 146 judge, 220 hard) on 21 September 2026. The composite is a geometric mean of four equally weighted axes: intelligence, calibration, speed and cost. Self-hosted latency is adjusted by x2 plus 0.15s to approximate production conditions, so self-hosted numbers are not directly comparable with the raw latency their authors publish.
Jev and SemIf feature by feature
| Attribute | Jev 1.13.0 | SemIf |
|---|---|---|
| What it is | Hosted System One model from TypeSafe, version jev-1.13.0 | Self-hosted logit reader over frozen open models |
| Licence | Proprietary, hosted | MIT for the project code; model weights keep upstream licences |
| Base model | Undisclosed, post-trained with RLCD | Qwen3.5-4B primary; also Qwen3-0.6B, MiniCPM5-2B, Qwen3-Reranker-4B |
| Hardware | None — it is an API call | Your own GPU; the published numbers are on an RTX 3090 |
| Calibration | Post-trained for calibrated probabilities; JevBench 82.7 | Conditional on the options you supply; per-workload calibration required. JevBench 72.6 |
| Judge tier accuracy | 94.5% | 95.2% — SemIf wins this one |
| Hard tier accuracy | 74.1% | 59.5% |
| Cost to run | Metered in Jev AI credits, with no idle cost — see the pricing page | ~$0.022 per 1,000 decisions on a busy rented GPU; idle time still bills |
What each one is better at
Where Jev wins
- Hard-tier accuracy 74.1% against 59.5%. The ambiguous cases are where a production queue actually hurts.
- Calibration 82.7 against 72.6, and the probabilities are meant to be honest out of the box rather than after you fit them.
- Nothing to operate: no GPU, no vLLM, no capacity planning, no 3am page when the serving process dies.
- 64k context per request against a 4B model’s practical window, and rate limits published at 250,000 tokens per second and 1,200 requests per minute.
Where SemIf wins
- Beats Jev on the judge tier: 95.2% against 94.5%.
- MIT code you can read, fork and audit, running on weights you control. Nothing leaves your network.
- No per-call price. Once the GPU is paid for, throughput is free — 20.03 decisions per second with parallel suffix reuse on one 3090.
- Runs offline, including a WebGPU demo in the browser.
The analysis
Where Jev and SemIf really differ, and why the bars above land where they do.
Give SemIf its due: 74.7 against Jev’s 75.4 in JevBench v1.2, second place overall, and on the 146-decision judge tier it is actually ahead — 95.2% against Jev’s 94.5%. If somebody tells you open models cannot do typed decisions, SemIf is the counterexample.
The gap is concentrated in two places. On the 220 hard decisions Jev scores 74.1% and SemIf 59.5%, a 14.6-point spread. And on calibration Jev scores 82.7 against 72.6, because SemIf reads conditional probabilities off a frozen model rather than being post-trained to make those probabilities honest — its own README says so and asks you to calibrate per workload.
The third gap is not in the model at all. SemIf’s ~$0.022 per 1,000 decisions assumes a rented GPU that stays busy. Idle GPUs bill the same as busy ones, so bursty traffic moves that number in a hurry — and somebody on your team now owns a serving stack.
What SemIf actually is
SemIf, formerly OpenJev, from TheoLeeCJ, by TheoLeeCJ.
SemIf takes three things: an unstructured state, criteria you write at request time, and a set of typed options. It runs one forward pass through an open model and reads the native logits for the option tokens. Because nothing is generated, there is no answer sentence to parse and no malformed JSON to retry. The project is explicit that it reproduces Jev’s interface, not Jev’s undisclosed model or training.
It runs on Qwen3-0.6B, MiniCPM5-2B, Qwen3.5-4B (the primary baseline and the configuration JevBench measured) and Qwen3-Reranker-4B. The code is MIT; the model weights keep their own upstream licences. There is a WebGPU browser demo, and deployment is self-hosted only — there is no SemIf service to call.
The speed argument is the strongest part of the project. On an RTX 3090 with a frozen state and 21 binary criteria, reading typed logits directly takes 1.023 seconds against 5.332 seconds to generate the same JSON array. With parallel suffix reuse the author measures 20.03 decisions per second against 2.33 for the autoregressive path.
On accuracy the author is careful rather than promotional: on a 102-row subset of TypeSafe’s cases, direct logits reach 0.845 modal agreement against Jev’s published 0.883, and balanced accuracy on authored decisions is 0.813. The README notes that this covers 102 aligned rows, not TypeSafe’s full 711-row aggregate.
A note on the name
SemIf was previously called OpenJev, and you will still find the old name in benchmark tables and blog posts. It is not the same project as openJev Verdict, openjev-sglang or the other repositories carrying that name — those are separate efforts by different authors. We untangle all of them on the Jev vs openJev page.
So which should you use?
Pick Jev if
- Traffic is bursty, or you do not want to run a GPU serving stack at all.
- Your inputs include long, noisy or contested cases — the hard tier is where the 14.6-point gap sits.
- You want thresholds that work on day one instead of after a calibration exercise.
Pick SemIf if
- You already own idle GPU capacity, or your volume is high and steady enough to keep a rented one busy.
- The data legally cannot leave your network.
- Your decisions are mostly easy, standard or judge-tier rather than genuinely hard, and you are willing to calibrate on your own labels.
Try Jev on the cases you are actually arguing about
The honest comparison is on your data, not on either scoreboard. Running Jev here takes a minute and five free credits; if the answers on your awkward cases are close enough, SemIf is a legitimate way to take the cost per decision to nearly nothing on hardware you already own.
Jev vs SemIf FAQ
Is SemIf the same as OpenJev?
SemIf is the current name of the project formerly called OpenJev, by TheoLeeCJ. Other projects also use the openJev name — openJev Verdict and openjev-sglang, among others — and they are unrelated codebases by different authors.
Is SemIf as accurate as Jev?
Close on most tiers and ahead on one. In JevBench v1.2 it matches Jev on easy decisions, trails slightly on standard, wins the judge tier 95.2% to 94.5%, and falls well behind on hard decisions, 59.5% to 74.1%.
What hardware do I need to run SemIf?
The published numbers are on an RTX 3090 with a 4B model. Smaller supported models such as Qwen3-0.6B and MiniCPM5-2B run on less, and there is a WebGPU browser demo, but production throughput assumes a GPU you keep running.
Does SemIf give calibrated probabilities?
It returns probabilities conditional on the options you supply and the README asks you to calibrate and validate per workload. JevBench scores its calibration at 72.6 against Jev’s 82.7.
What does it cost to run SemIf?
JevBench estimates roughly $0.022 per 1,000 decisions on a GPU that stays busy. That figure excludes idle GPU time and the engineering cost of running the stack, which is where self-hosting usually gets expensive. Jev AI’s own pricing is on the pricing page.
Other Jev comparisons
Every number on these pages is quoted from a published source and was read on September 21, 2026.
- Jev vs djevThe fastest decision API, against the best calibrated one.
- Jev vs openJevNear-perfect on the data it was trained for, and shaky on anything else.
- Jev vs LayaA model you fine-tune on your data, against one that works out of the box.
Sources
Jev AI is an independent service and is not affiliated with or endorsed by TypeSafe. Jev answers are probabilistic; review important decisions before acting on them. SemIf and every other product named on this page belongs to its respective owner and is not affiliated with Jev AI. Figures are quoted from the sources above as they read on September 21, 2026 and can change.