Jev AI

Jev comparison · JevBench v1.7.48 · updated November 14, 2026

Jev vs Mercury Decide

Two hosted decision models that take the same request, one place apart.

On this site

Jev 1.13.0

TypeSafe · System One model

Ranked #18 on the API board, fifteenth on its Capability ranking, better calibrated, and 64k of context.

71.4JevBench v1.7.48 · Open-weights reference row · #18 of 54 on the API board · Capability #15

vs

Alternative

Mercury Decide

Inception · Hosted diffusion decision model

Seventeenth of 54 on the API board, and the cheaper of the two to run.

72.2JevBench v1.7.48 · #17 of 54 on the API board · Capability #20

  1. JevSystem One: state plus typed questionsRequest schemaMercury DecideThe same System One schema
  2. JevReference answers (jev-1.13.0)Agreement with JevMercury Decide91.7% of 133 answers in our replay
  3. Jev64k tokensContext windowMercury Decide32k tokens
  4. Jev#18 of 54 on the JevBench v1.7.48 API board at 71.4External benchmarkMercury Decide#17 of 54 at 72.2
  5. Jev0.65s median on the hosted APIThroughput claimMercury DecideUp to 14 decisions per second, per Inception
  6. JevGenerally availableAvailabilityMercury DecideEarly access through a hosted API aggregator

The comparison below still stands. Mercury Decide is temporarily unavailable on Jev AI: the upstream provider has no endpoints right now. Mercury Decide model guide →

Try it on this page

Run Jev 1.14 right here

Edit the example or paste your own text, then run it. Sign in to run: new accounts get 20 welcome credits, and a failed run costs nothing.

Open the full playground

Support ticket triage · Jev 1.14

1 Input

Hi, this is the third time I am writing. Our payouts have failed every day since Monday and my team cannot pay suppliers. I already re-entered the bank details twice. If this is not fixed today we will have to move to another provider.

2 Questions

  • Choice Which team should handle this ticket?
  • Yes / No Does the customer need a response today?
  • Score How frustrated is the customer?
  • Yes / No Does the customer threaten to leave or switch providers?

JevBench v1.7.48, measured the same way

One benchmark measured every system under one method — it ranks 261 open-weights systems on one board and 54 hosted API offerings on another, with the same scores in both — so these bars are comparable with each other in a way that vendor-published figures are not. The full board and how to read it are on the JevBench results page.

Composite score

#3 H2O-Lightning-4B77.9
#29 Xor 26B-A4B74.0
#31 Arcbright Elf 1 31B74.0
#34 Quyet-1.0-Large73.8
API #14 Sage 1.3.073.7
#36 decider-12b v273.7
#42 decisio 31B73.2
#43 decider-12b v173.2
API #17 Mercury Decide72.2
#54 Winnow72.2
#61 deck-31B71.9
#62 Cygnet71.9
API #18 Jev71.4
#68 Jev-Omni71.0
#70 decisio v0.8.070.6
#78 torchcast-12b69.8
API #24 wity-169.0
#112 Plumb-4B50.4
#129 decider-4b43.1
#139 JevK5 v0.339.2
#153 Imajev-4B30.0
#156 Hopper27.7
#183 SemIf16.1
#253 Laya0.0

Plain numbers are ranks on the open-weights board; “API” marks a rank on the separate board for hosted API offerings. Jev, a hosted API, is ranked there, and appears on the open-weights board only as an unranked reference row, where its score falls between #65 and #66. The composite hides where systems actually differ, so the charts below break it apart. Benchmark Heaven’s headline Capability ranking, shown next to each score above, averages intelligence and calibration among Jev-class systems.

Capability, higher is better

IntelligenceHow often it picks the right answer, half from sealed decisions

Jev63.6
Mercury Decide65.9

CalibrationWhether 0.8 really means about 80%

Jev90.6
Mercury Decide81.3

SpeedMeasured response time

Jev91.2
Mercury Decide84.8

Intelligence on open and sealed decisions

Open300 decisions from public sources

Jev65.6
Mercury Decide67.1

Sealed1,200 private decisions nobody could tune for

Jev61.6
Mercury Decide64.7

By request type, chance-corrected

ChoicePick one of the options

Jev77.4
Mercury Decide76.9

NoulProbability that a statement is true

Jev50.3
Mercury Decide58.0

ScorePlace the case on a graded scale

Jev63.2
Mercury Decide62.8

The sealed half, highlighted, is where results cannot have been tuned: nobody outside the benchmark has seen those decisions. Request types are scored above chance, so 0 means guessing and a value below 0 means worse than guessing.

One thing these charts leave out: the composite score above also weighs a fourth axis, cost. We do not reproduce it here; the full table is at the source below.

Every system on the board

The capability scores are charted above; this is the ranking and the deployment detail behind them. Jev and Mercury Decide are highlighted.

ModelRankScoreMeasured latencyRuns on
H2O-Lightning-4B v1.1 (H2O.ai, Qwen3.5-4B)#377.90.03s raw, 0.21s adjustedSelf-hosted GPU
Xor 26B-A4B (Juspay, Gemma 4 26B-A4B)#2974.00.06s raw, 0.27s adjustedSelf-hosted GPU
Arcbright Elf 1 31B#3174.00.06s raw, 0.27s adjustedSelf-hosted GPU
Quyet-1.0-Large (Chinh Nguyen, Gemma 4 31B decoder)#3473.80.12s raw, 0.39s adjustedSelf-hosted, H100
Sage 1.3.0 (Levanto Labs, hosted API)API #1473.70.17s median / 0.41s p95Hosted API
decider-12b v2 (Mapika, Gemma 4 12B)#3673.70.05s raw, 0.26s adjustedSelf-hosted, RTX PRO 6000
decisio v0.8.0 on Gemma 4 31B (Amin Roudaki, frozen base)#4273.20.05s raw, 0.24s adjustedSelf-hosted, H100
decider-12b v1 (Mapika, stock Gemma 4 12B)#4373.20.05s raw, 0.26s adjustedSelf-hosted, RTX PRO 6000
Mercury Decide (Inception, diffusion decision API)API #1772.20.34s median / 0.96s p95Hosted API
Winnow-12B Q8 (EldanRing, Gemma 4 12B)#5472.20.12s raw, 0.40s adjustedSelf-hosted, RTX 6000
deck-31B (krishna765, frozen Gemma 4 31B)#6171.90.13s raw, 0.41s adjustedSelf-hosted GPU
Cygnet (blockbrain, frozen Gemma 4 12B)#6271.90.04s raw, 0.24s adjustedSelf-hosted, RTX PRO 6000
Jev 1.13.0 (TypeSafe)API #1871.40.24s median / 0.31s p95Hosted API
Jev-Omni (akhilaaa3, Gemma 4 12B)#6871.00.16s raw, 0.47s adjustedSelf-hosted, RTX 6000
decisio v0.8.0 on Gemma 4 12B (Amin Roudaki, frozen base)#7070.60.12s raw, 0.38s adjustedSelf-hosted GPU
torchcast-decision-12b (Torchcast AI, Gemma 4 12B)#7869.80.04s raw, 0.24s adjustedSelf-hosted, H100
wity-1 (Wity, closed API, reasoning auto)API #2469.01.85s median / 7.97s p95Hosted API
Plumb-4B (crh225, JevK5 v0.2 + LoRA)#11250.40.02s raw, 0.19s adjustedSelf-hosted, H100
decider-4b v2 (Mapika, Qwen3.5-4B)#12943.10.03s raw, 0.21s adjustedSelf-hosted, RTX 5090
JevK5 v0.3 (allebee, Qwen3.5-4B)#13939.20.02s raw, 0.19s adjustedSelf-hosted, H100
Imajev-4B (Mohit Garg, Qwen3.5-4B LoRA)#15330.00.07s raw, 0.29s adjustedSelf-hosted, RTX 5090
Hopper (HopitAI, Qwen3.5-4B LoRA)#15627.70.06s raw, 0.26s adjustedSelf-hosted, RTX A6000
SemIf (Qwen3.5-4B)#18316.10.06s raw, 0.27s adjustedSelf-hosted, RTX 5090
Laya (ModernBERT-large)#2530.00.93s raw, 2.01s adjustedSelf-hosted, CPU

JevBench method v1.7 applies a 10-second limit per decision: a response taking longer than 10 seconds counts as wrong and contributes zero calibration for that item. Speed uses all decisions, including failures. Benchmark Heaven re-scored recorded per-decision times; it did not run every model again. Tokens, draws and raw outputs are unchanged. Since method v1.8.0 (29 October 2026), self-hosted open-weights rows are costed by GPU time on a monthly-rented RTX PRO 6000 with an efficient inference engine instead of base-model list prices; encoders, architectures outside the six calibrated Gemma/Qwen classes and rows without a verified token basis keep their old costs, and API costs and both caps are unchanged. Intelligence, calibration and latency did not move, but most open-weights cost scores rose, so their composites rose and the Jev-class pool grew. Four rows recorded on a loaded CPU host await re-measurement and remain unranked with their dated v1.6 values; five CPU rows were measured again on GPU and replaced in full. The official composite combines intelligence, calibration, speed and cost using the frozen scorer. Each full-set system answers 1,500 decisions, 300 open and 1,200 sealed; most hosted APIs use an equated 600-item set. Half of intelligence comes from sealed decisions. The open-weights and API boards compare the same measurements within separate hosting groups. Board v1.7.48 ranks 261 open-weights systems and 54 API offerings. There are 136 newer ranked open-weights rows tagged with their draw; since release v1.8.10 (12 November 2026) the 27 October regular draw is equated to the reference scale with four anchor systems (intelligence −2.9, calibration −1.9 points) and 31 of its rows are ranked, while four stay unranked because their top-five places lie within the uncertainty of that shift. Jev 1.13.0 remains an unranked reference on the open-weights board and is ranked on the API board. Capability averages intelligence and calibration among systems inside caps frozen at twice Jev’s v1.5 reference: 243 open-weights systems and 39 APIs qualify. The caps do not track current Jev latency. Self-hosted latency uses an estimated 2× measured latency plus 0.15s; API times include the network. Scores and splits on this site use method v1.8 (v1.7 scoring with the v1.8 cost basis); historical v1.6, v1.5 and v1.4 results are not current standings.

Jev and Mercury Decide feature by feature

AttributeJev 1.13.0Mercury Decide
What it isHosted System One model from TypeSafe, version jev-1.13.0Hosted decision model from Inception, built on diffusion language modelling
Request formatstate plus noul, choice and score questionsThe same schema; works with the same Jev AI request
ProbabilitiesPer option, with a confidence valuePer option, with a confidence value, read from the model’s distribution
Context64k tokens per request32,768 tokens per request
External score71.4 composite, #18 of 54 on the JevBench v1.7.48 API board72.2 composite, #17 of 54
Intelligence and calibrationIntelligence 63.6, calibration 90.6Intelligence 65.9, calibration 81.3
Agreement on our examplesReference122 of 133 answers (91.7%) match Jev
Speed0.24s median on the production API in JevBench v1.6.10.33s median and 0.85s p95 in JevBench v1.6.1; 0.67s median wall-clock in our own run
Model name on Jev AIjev-latest, jev-preview, jev-1.14.0 or jev-1.13.0mercury-decide (Decisions API alias inception/mercury-decide)

What each one is better at

Where Jev wins

  • Twice the context: 64k tokens against 32k, so long tickets, transcripts and documents fit in one request. Mercury Decide refused all 23 items in the benchmark’s pool that exceed its window, and the refusals counted against it.
  • Far better calibrated: 90.6 against 81.3, so a 0.8 is closer to meaning 80%.
  • Fifteenth on the API board’s Capability ranking at 77.1 against #20 at 73.6, the ranking that drops speed and cost and keeps only intelligence and calibration.
  • The faster of the two on the benchmark’s own measure, 91.2 against 84.8, from a 0.24s median against 0.33s.
  • Generally available rather than an early-access endpoint, with pinnable versions such as jev-1.13.0.
  • Most of our disagreements were on borderline yes/no and rubric scores, where Jev is the model these templates were written and checked against.

Where Mercury Decide wins

  • #17 of 54 on the JevBench v1.7.48 API board at 72.2, one place above Jev on the composite.
  • More accurate on the answers: intelligence 65.9 against 63.6, and 64.7 against 61.6 on the sealed half.
  • Clearly ahead on yes/no questions, 58.0 above chance against 50.3, the axis Jev is weakest on.
  • The better cost score of the two, 62.1 against 54.7, and served free during early access.
  • Diffusion decoding built for throughput: Inception quotes up to 14 decisions per second.
  • A drop-in alternative on the same schema, so you can switch with one field and keep every judge and template.
  • Close agreement with Jev on everyday tasks: 53 of 54 choices matched in our replay.
  • Slightly fewer counted input tokens on the same requests in our sample.

The analysis

Where Jev and Mercury Decide really differ, and why the bars above land where they do.

Inception released Mercury Decide on 30 September 2026 as a decision model rather than a chat model. It accepts the same request Jev does and returns probabilities read from the model’s own distribution instead of generated text, which is why output tokens cost nothing. Benchmark Heaven measured it on 7 October and added it on 8 October — its first external score. Hosted offerings are ranked on their own board, separately from open weights, and there Mercury Decide is #17 of 54 at 72.2, one place above Jev at #18 with 71.4.

The composite hides a clean split. Mercury Decide is the more accurate of the two on the answers — intelligence 65.9 against 63.6, and 64.7 against 61.6 on the sealed half nobody could tune for — and the cheaper to run, 62.1 against 54.7 on the cost axis. Jev is the better calibrated by a wide margin, 90.6 against 81.3, and the faster, 91.2 against 84.8 from a 0.24s median against 0.33s. On the Capability ranking, which averages intelligence and calibration and so drops speed and cost, the order reverses: on the API board Jev is fifteenth at 77.1 and Mercury Decide #20 at 73.6.

We also replayed every recorded Jev request on this site — 8 homepage demos and 33 use-case templates, 41 requests and 133 questions — on Mercury Decide and compared the answers. 91.7% agree: 53 of 54 choices, 37 of 42 yes/no answers on the same side of 0.5, and 32 of 37 scores at the same level.

Agreement is not accuracy. These examples have no ground-truth labels, so where the two differ we cannot say which is right; read the numbers as how interchangeable the models are on everyday triage, routing, moderation and matching tasks. On that measure they are close, with most disagreement on borderline yes/no and graded questions rather than on clear-cut choices.

What Mercury Decide actually is

Mercury Decide, Inception’s diffusion-based decision model, by Inception.

Mercury Decide comes from Inception, the company behind the Mercury diffusion language models. Like Mercury 2.5, it refines many positions in parallel over a few passes instead of producing one token at a time. For a decision model that matters less for the answer itself than for speed: the answer is a probability over a short set of labels, not a paragraph.

It supports the three System One question types: a choice among named options, a score on an ordered rubric, and a yes/no probability. Inception aims it at moderation, routing, fraud triage, agent tool selection, evaluation loops, grading, policy checks and escalation gates — the same territory as Jev.

It is served through a hosted API aggregator as an early-access endpoint with a 32,768-token context window. Inception notes that real latency depends on request size, network conditions, provider load and concurrency, and that a public benchmark cannot establish accuracy or calibration on your own data. Benchmark Heaven measured it through that endpoint on 7 October.

How we ran the head-to-head

Our own replay used the site’s published, fictional examples: each homepage demo and each use-case template, exactly as the playground sends them. Jev’s side is the recorded jev-1.13.0 answer already shown on those pages; Mercury’s side was run on 1 October 2026 against snapshot inception/mercury-decide-20260930, one request at a time. No request failed.

A choice agrees when both pick the same option. A yes/no answer agrees when both probabilities fall on the same side of 0.5; the mean gap between the two probabilities was 0.147. A score agrees when both round to the same rubric level; the mean gap was 0.148 of a level. The script is in the repository as scripts/bench-mercury-vs-jev.ts, so the run can be repeated.

Mercury’s median wall-clock time in this run was 0.67s, measured from one client and including the network hop to the hosting aggregator, so it is not comparable with lab latency. On the eight demos where Jev’s usage was recorded, Mercury counted 3,364 input tokens against Jev’s 3,749 for the same requests; the models tokenise differently.

So which should you use?

Pick Jev if

  • Inputs are long — a full transcript, contract or multi-message thread.
  • You need a version you can pin, and calibration you can threshold on rather than the higher composite.
  • Calibrated yes/no and rubric scores on borderline cases matter more than raw throughput.

Pick Mercury Decide if

  • You run large volumes of short decisions and want a second fast model on the same schema.
  • You want a second opinion: send the same request to both and escalate when they disagree.
  • Inputs stay well under 32k tokens.

Try Jev on the cases you are actually arguing about

Because the request is identical, this is the rare comparison you can settle on your own data in minutes. Run your judge on Jev and on Mercury Decide side by side, look at the cases where they disagree, and keep whichever model is right more often on those. Where both agree, you can trust the answer more than either alone.

Jev vs Mercury Decide FAQ

Is Mercury Decide compatible with Jev requests?

Yes. It uses the same System One schema: a state plus typed noul, choice and score questions, including saved judges. It is temporarily unavailable on Jev AI because the upstream provider has no endpoints right now.

Is Mercury Decide better than Jev?

On the official JevBench v1.7.48 composite, yes, narrowly: #17 of 54 on the API board at 72.2 against Jev’s #18 at 71.4, and it is the more accurate of the two on the answers, intelligence 65.9 against 63.6. Jev is much better calibrated, 90.6 against 81.3, and the faster of the two, so on the API board’s Capability ranking, which keeps only intelligence and calibration, Jev is fifteenth and Mercury Decide #20. In our replay of 41 recorded requests the two agreed on 91.7% of 133 answers.

How long can the input be?

Mercury Decide accepts 32,768 tokens per request; Jev accepts 64k. Longer inputs to Mercury Decide are rejected without a charge — that happened to all 23 oversized items in JevBench’s pool, and the benchmark counted those refusals against its intelligence.

Can I call Mercury Decide through the Jev AI API?

Not right now. Mercury Decide is temporarily unavailable: the upstream provider has no endpoints right now. Requests for mercury-decide fail before any charge and do not call upstream. The model guide stays up as a reference.

Other Jev comparisons

Every number on these pages is quoted from a published source and was read on November 14, 2026.

Sources

Mercury Decide and every other product named on this page belongs to its respective owner and is not affiliated with Jev AI. Figures are quoted from the sources above as they read on November 14, 2026 and can change.