On this site
Jev 1.13.0
TypeSafe · System One model
Answers on day one, with no labelled data and no training run.
75.4JevBench v1.2 rank #1
Jev comparison · JevBench v1.2 · updated September 21, 2026
Jev vs Laya
A model you fine-tune on your data, against one that works out of the box.
vs
Alternative
Laya
Convai Innovations · Apache-2.0 421M decision head
Open weights, tens of milliseconds, the cheapest decision on the board.
70.1JevBench v1.2 rank #7
- JevHosted APIWhere it runsLayaSelf-hosted on Apache-2.0 weights
- JevWorks as deliveredWithout fine-tuningLayaScores below a majority-class baseline
- JevEnglish; others less reliablyLanguagesLayaEnglish, plus 100+ on the smaller checkpoint
- Jev64k tokens — a long documentContext windowLaya512 tokens per question
- Jev74.1% correctHardest test casesLaya34.1% correct
- Jev0.65s median on the hosted APIResponse timeLaya33–40ms on a Tesla T4
JevBench v1.2, measured the same way
One benchmark ran every system below under one method, so these bars are comparable with each other in a way that vendor-published figures are not.
Composite score
The top four finish within three points of each other. The composite hides where they actually differ, so the two charts below break it apart.
Capability, higher is better
IntelligenceHow often it picks the right answer
CalibrationWhether 0.8 really means about 80%
SpeedMeasured response time
Accuracy by how hard the decision is
Easy72 straightforward cases
Standard96 everyday cases
Judge146 evaluation-style calls
Hard220 genuinely ambiguous cases
Easy and standard decisions separate almost nothing. The hard tier, highlighted, is where these systems stop agreeing.
One thing these charts leave out: the composite score above also weighs a fourth axis, cost, scored from each system's public list price, and Jev places last of the five on it. We do not reproduce those figures, because Jev AI is an independent reseller metering its own credits and a list price is not what you would pay here. The full table is at the source below, and our rates are on the pricing page.
Every system on the board
The capability scores are charted above; this is the ranking and the deployment detail behind them. Jev and Laya are highlighted.
| Model | Rank | Score | Measured latency | Runs on |
|---|---|---|---|---|
| Jev 1.13.0 (TypeSafe) | #1 | 75.4 | 0.65s median / 0.72s p95 | Hosted production API |
| SemIf (Qwen3.5-4B) | #2 | 74.7 | 0.20s raw, 0.55s adjusted | Self-hosted, RunPod GPU |
| djev (Maisa, DiffusionGemma) | #3 | 74.3 | 0.24s median / 0.31s p95 | Hosted API, free preview |
| openJev Verdict 1.4 | #4 | 72.5 | 0.31s raw, 0.78s adjusted | Self-hosted, CPU |
| Laya (ModernBERT-large) | #7 | 70.1 | 0.79s raw, 1.72s adjusted | Self-hosted, CPU |
JevBench v1.2 scored 42 systems on 534 decisions (72 easy, 96 standard, 146 judge, 220 hard) on 21 September 2026. The composite is a geometric mean of four equally weighted axes: intelligence, calibration, speed and cost. Self-hosted latency is adjusted by x2 plus 0.15s to approximate production conditions, so self-hosted numbers are not directly comparable with the raw latency their authors publish.
Jev and Laya feature by feature
| Attribute | Jev 1.13.0 | Laya |
|---|---|---|
| What it is | Hosted System One model from TypeSafe, version jev-1.13.0 | Open-weights decision head on a frozen ModernBERT/mmBERT encoder |
| Licence | Proprietary, hosted | Apache-2.0, weights on Hugging Face, installable from PyPI |
| Size | Undisclosed | 421M (English) or 322M (multilingual, 100+ languages) |
| Languages | Best in English; other languages work with lower accuracy | English checkpoint, plus a multilingual checkpoint covering 100+ languages |
| Zero-shot quality | 90.4 intelligence in JevBench; designed to be called without training | 0.362 on typed decisions against a 0.461 majority-class baseline — needs fine-tuning |
| Calibration | ECE 0.246 by Convai’s measurement; JevBench calibration 82.7 | ECE 0.081 after temperature fitting by Convai’s measurement; JevBench calibration 62.5 |
| Context | 64k tokens per request | 512 tokens per question; JevBench notes long states are truncated by the package |
| Options per choice | No fixed small ceiling | Degrades past ~20 options; 0.425 accuracy on a 77-label task |
| Latency | 0.65s median on the production API | 32.8–39.5ms on a Tesla T4; 193–464ms on CPU |
| Cost to run | Metered in Jev AI credits — see the pricing page | Free to self-host; ~$0.0029 per 1,000 decisions in JevBench’s estimate |
What each one is better at
Where Jev wins
- Works zero-shot. Laya without fine-tuning scores below a majority-class baseline on unseen typed decisions, by Convai’s own measurement.
- Hard-tier accuracy 74.1% against 34.1%, and judge-tier 94.5% against 69.2%, in JevBench v1.2.
- 64k of context against 512 tokens per question, so long tickets, documents and traces do not get truncated.
- No ceiling around 20 options. Laya’s accuracy falls to 0.425 on a 77-label task.
- Nothing to train, fit or serve before you can answer the first question.
Where Laya wins
- Cheapest per decision of anything in the JevBench top ten — cost score 86.2, roughly $0.0029 per 1,000.
- Tens of milliseconds on a GPU, and fast enough on CPU to sit inline in a request path with no network hop.
- Genuine multilingual coverage from the 322M mmBERT checkpoint. Jev is explicitly best in English.
- Apache-2.0 weights: fine-tune them, ship them in your product, run them offline.
- Excellent calibration once fitted — Convai measures 0.081 ECE after temperature scaling.
The analysis
Where Jev and Laya really differ, and why the bars above land where they do.
Convai publishes sharp numbers and they are not wrong: 32.8ms for the 322M multilingual checkpoint and 39.5ms for the 421M English one on a Tesla T4, an expected calibration error of 0.081 after temperature fitting, Apache-2.0 weights, free to self-host. Against Jev they claim 7.8x faster single calls, 20x faster batched, and 3x better calibration.
Convai is equally straightforward about the catch. Zero-shot, Laya scores 0.362 on typed decisions and the multilingual variant 0.352 — above a 0.318 random baseline but below the 0.461 majority-class baseline. In plain terms: out of the box, on an unseen task, always guessing the most common label beats it. Laya is a model you fine-tune, not a model you call.
JevBench v1.2 measured the untuned package on CPU and put it seventh at 70.1, with 34.1% on hard decisions against Jev’s 74.1%, and noted that long hard-tier states were truncated by Laya’s own 512-token per-question budget. Those two pictures are not contradictory — they are the tuned and untuned versions of the same model.
What Laya actually is
Laya, the 421M open-weights decision model from Convai Innovations, by Convai Innovations.
Laya is a decision head bolted onto a frozen encoder. ModernBERT-large supplies 395M of the 421M parameters; on top sit two transformer layers, a scorer that rates each answer option, and a head that decides whether to act or escalate. Convai ships three checkpoints: 421M ModernBERT-large for English, 322M mmBERT-base covering 100+ languages at roughly 2.2x the speed, and a 421M typed-decisions checkpoint aimed at agent pipelines. Everything is Apache-2.0, on PyPI and on Hugging Face.
The interesting design choice is that the answer space is defined at request time rather than baked in. Each option gets its own marker, is scored independently, and the scores are normalised across the set — so you can add a new label without retraining the whole classifier. That is what makes it Jev-shaped rather than a conventional fixed-class model.
Convai aims it at invoice processing, ticket routing, moderation, guardrails and agent-trace observability: the classification layer around an LLM stack rather than the LLM itself. On hardware, expect 32.8–39.5ms on a T4 and 193–464ms on CPU.
The limitations are published alongside the claims, which is to their credit. Choice questions degrade past about 20 options — 0.425 accuracy on a 77-label task. Base checkpoints sit near random at about 0.35 without fine-tuning. Temperature calibration is required to reach the quoted 0.081 ECE.
Why the two sets of latency numbers disagree
Convai measures 32.8ms on a Tesla T4 and cites 236–276ms for Jev. JevBench v1.2 measures Laya at 0.79s raw on CPU and Jev at 0.65s median on its production API. Neither is dishonest; they are different machines and different task mixes.
Three things drive the spread. Laya on a GPU is an order of magnitude faster than Laya on a CPU, and JevBench ran it on CPU. JevBench then applies a x2 plus 0.15s adjustment to every self-hosted latency to approximate production conditions, which penalises self-hosted entries relative to their raw figures. And network round-trip is in Jev’s number but not in a local forward pass. If you put Laya on a GPU in your own datacentre, Convai’s figure is the one you will see.
So which should you use?
Pick Jev if
- You do not have a labelled training set yet, which is where almost every project starts.
- Questions are defined at request time and change as you learn what the task is.
- Inputs are long, or a choice has more than a couple of dozen options.
Pick Laya if
- You have labelled data for your task, or can produce it, and the label set is stable.
- You need non-English coverage or strict data residency.
- Volume is high and the decision is narrow — that is where free self-hosted inference actually pays.
Try Jev on the cases you are actually arguing about
The useful move is to treat these as sequential rather than exclusive. Run the task on Jev first — five free credits here, no setup — and keep the answers. If the task settles into something narrow and high-volume, those stored runs are the labelled set you fine-tune Laya on, and you get the cheap fast model with the calibration already validated against a stronger one.
Jev vs Laya FAQ
Is Laya a drop-in replacement for Jev?
No. It answers the same three question types, but Convai reports that base checkpoints score near random zero-shot, below a majority-class baseline. Laya expects to be fine-tuned on your task; Jev expects to be called on text it has never seen.
Is Laya really 7.8x faster than Jev?
On Convai’s hardware, yes — 32.8ms on a Tesla T4 against 236–276ms cited for Jev. JevBench v1.2, running Laya on CPU and applying its self-hosted latency adjustment, measured 0.79s raw against Jev’s 0.65s median. The GPU figure is the one you would see self-hosting on a GPU.
Does Laya support other languages?
Yes. The 322M mmBERT-base checkpoint covers over 100 languages and runs about 2.2x faster than the English checkpoint. This is a real advantage: TypeSafe states Jev is most accurate in English.
How much does Laya cost?
The weights are Apache-2.0 and free. JevBench estimates about $0.0029 per 1,000 decisions in compute — the cheapest in its top ten — plus whatever it costs you to run and maintain the service.
Can Laya handle a long document?
Not in one piece. It uses a 512-token budget per question, and JevBench noted that the package itself truncated long hard-tier states. Jev accepts 64k tokens per request.
Other Jev comparisons
Every number on these pages is quoted from a published source and was read on September 21, 2026.
- Jev vs djevThe fastest decision API, against the best calibrated one.
- Jev vs SemIfThe strongest open rebuild on the board, and where it still falls short.
- Jev vs openJevNear-perfect on the data it was trained for, and shaky on anything else.
Sources
Jev AI is an independent service and is not affiliated with or endorsed by TypeSafe. Jev answers are probabilistic; review important decisions before acting on them. Laya and every other product named on this page belongs to its respective owner and is not affiliated with Jev AI. Figures are quoted from the sources above as they read on September 21, 2026 and can change.