Jev vs Laya

The analysis

Where Jev and Laya really differ, and why the bars above land where they do.

Convai publishes sharp numbers and they are not wrong: 32.8ms for the 322M multilingual checkpoint and 39.5ms for the 421M English one on a Tesla T4, an expected calibration error of 0.081 after temperature fitting, Apache-2.0 weights you can self-host. Against Jev they claim 7.8x faster single calls, 20x faster batched, and 3x better calibration.

Convai is equally straightforward about the catch. Zero-shot, Laya scores 0.362 on typed decisions and the multilingual variant 0.352 — above a 0.318 random baseline but below the 0.461 majority-class baseline. In plain terms: out of the box, on an unseen task, always guessing the most common label beats it. You can call the base checkpoints here to evaluate your task; fine-tuning remains an option when their quality is insufficient.

JevBench measured the untuned package on CPU. In v1.6.1 its answers are barely above chance on both halves, 1.8 on each, so its intelligence axis is 1.8, and on board v1.7.48 it places #227 of 235 open-weights systems with a score of 0.0. Convai’s multilingual and typed-decisions checkpoints, added to the board since v1.5.0, score 0.0 and 0.2. It had already fallen from #7 in v1.2 to #33 in v1.3.0 — not because the model changed, but because the benchmark started measuring intelligence above chance. That is the same fact Convai reports about zero-shot use, now reflected in the ranking. Both pictures are true; they are the tuned and untuned versions of the same model.

What Laya actually is

Laya, the 421M open-weights decision model from Convai Innovations, by Convai Innovations.

Laya is a decision head bolted onto a frozen encoder. ModernBERT-large supplies 395M of the 421M parameters; on top sit two transformer layers, a scorer that rates each answer option, and a head that decides whether to act or escalate. Convai ships three checkpoints: 421M ModernBERT-large for English, 322M mmBERT-base covering 100+ languages at roughly 2.2x the speed, and a 421M typed-decisions checkpoint aimed at agent pipelines. Everything is Apache-2.0, on PyPI and on Hugging Face.

The interesting design choice is that the answer space is defined at request time rather than baked in. Each option gets its own marker, is scored independently, and the scores are normalised across the set — so you can add a new label without retraining the whole classifier. That is what makes it Jev-shaped rather than a conventional fixed-class model.

Convai aims it at invoice processing, ticket routing, moderation, guardrails and agent-trace observability: the classification layer around an LLM stack rather than the LLM itself. On hardware, expect 32.8–39.5ms on a T4 and 193–464ms on CPU.

The limitations are published alongside the claims, which is to their credit. Choice questions degrade past about 20 options — 0.425 accuracy on a 77-label task. Base checkpoints sit near random at about 0.35 without fine-tuning. Temperature calibration is required to reach the quoted 0.081 ECE.

Why the two sets of latency numbers disagree

Convai measures 32.8ms on a Tesla T4 and cites 236–276ms for Jev. JevBench v1.6.1 measures Laya at 0.83s raw on CPU and Jev at 0.24s median on its production API. Neither is dishonest; they are different machines and different task mixes.

Three things drive the spread. Laya on a GPU is an order of magnitude faster than Laya on a CPU, and JevBench ran it on CPU. JevBench then applies a x2 plus 0.15s adjustment to every self-hosted latency to approximate production conditions, which penalises self-hosted entries relative to their raw figures. And network round-trip is in Jev’s number but not in a local forward pass. If you put Laya on a GPU in your own datacentre, Convai’s figure is the one you will see.

Jev vs Laya FAQ

Other Jev comparisons

Every number on these pages is quoted from a published source and was read on October 31, 2026.

• Jev vs Imajev-4BA 4B model that reads photos, against the model it copies on text.
• Jev vs decider-4bLevel on speed, far behind on the answers.
• Jev vs JevK5A fast open Qwen3.5-4B system, and far behind Jev on the answers.
• Jev vs CygnetNo training at all, forty-sixth on the open-weights board, and behind Jev on every accuracy axis.
• Jev vs decisioTenth on Capability on its biggest base, and it trains nothing of its own.
• Jev vs metask-jev-rainMore accurate answers than Jev on the benchmark, and two of three rows above it on the composite.
• Jev vs WinnowOne local model for decisions, chat and images, against a hosted decision model.
• Jev vs djevLevel with Jev when it was last measured, and more expensive to run.
• Jev vs SemIfA fast open rebuild on your own GPU, and where it still falls short.
• Jev vs OpenJevOne name, ten projects. Here is every one, and how the drop-in compares.
• Jev vs Mercury DecideTwo hosted decision models that take the same request, one place apart.
• Run Jev locally?The open alternatives in one table, with licences and hardware.
• Jev vs LLMHow a decision model differs from a chat LLM and a BERT classifier.

Sources

• Benchmark Heaven — JevBench board v1.7.48 (open weights)
• Benchmark Heaven — JevBench API board
• Laya by Convai Innovations
• convaiinnovations/laya-typed-decisions on Hugging Face
• TypeSafe documentation (Jev model card)