Jev vs metask-jev-rain

The analysis

Where Jev and metask-jev-rain really differ, and why the bars above land where they do.

Three metask-jev-rain rows are ranked, and all three answer the benchmark’s questions better than Jev does. Intelligence runs 67.8 for the 12B adapter, 70.7 for the 31B and 72.6 for the 31B service v3, against Jev’s 63.6. That puts them at #6, #5 and #3 on Benchmark Heaven’s headline Capability ranking — the mean of intelligence and calibration among the 117 Jev-class systems — where Jev’s reference row scores 77.1 and falls between deck-31B and René-1 31B FP8.

The official composite tells the other half. It weighs intelligence, calibration, speed and cost equally as a harmonic mean, and the metask rows are costlier to serve than the frozen Jev-class budget: cost scores of 51.5, 49.2 and 47.9 against Jev’s 54.7, which the board puts at roughly 1.3×, 1.5× and 1.7× Jev’s v1.5 reference per decision. Their speed scores are 73.4, 83.2 and 84.7 against 91.5. So the composite order inverts the Capability order — the most accurate row, service v3, is the lowest-placed of the three at #23, while the 12B adapter is the highest at #15.

Jev keeps calibration, 90.6 against 86.6 to 87.9, and keeps the budget. The practical read: if you are buying answer quality per decision and can run a 31B adapter on your own card, metask-jev-rain is one of the few systems on the board that genuinely out-answers Jev. If you are buying a hosted call with a published price and a probability you can trust at face value, that is not what these rows are.

What metask-jev-rain actually is

the metask-jev-rain family of quantised decision adapters, by metask.

metask-jev-rain is a set of decision adapters that Benchmark Heaven runs on hardware it rents, which is what puts them on the open-weights board rather than the API one. The board classifies the three ranked rows as adapters at 12B and 31B, quantised to W4A16 INT4, and records their base model as undisclosed. It publishes no repository, model card or author page for them, so unlike every other rival this site compares, there is no second source to check the board against.

The family is larger than the three ranked rows. metask-jev-4b, on Qwen3.5-4B, is the oldest and the only one with a published base; it is #46 of 162 at 37.3 and is one of the twelve anchor systems the benchmark uses to put new sealed draws on the v1.6.1 scale. Metask rain 4B, added on 9 October, is #25 at 62.7. An untrained metask-jev-rain-12B was measured on the same date but is not ranked.

The three rows this page is about arrived later: metask-jev-rain-12B-trained and metask-jev-rain-31B-trained on 12 October, and metask-jev-rain-31B-trained service v3 on 13 October. All three carry the board’s v1.6.18 draw tag, meaning they answered the sealed draw of 12 October rather than the v1.6.1 reference pool.

Why the most accurate row ranks lowest of the three

The composite is a harmonic mean of four equally weighted axes, so one weak axis pulls the whole score down much harder than a strong axis lifts it. Across the three rows intelligence climbs from 67.8 to 70.7 to 72.6, but the cost score falls from 51.5 to 49.2 to 47.9 and never recovers it. The result is a composite that runs the other way: 67.6, 66.9, 63.4.

The cost axis is not a price Jev AI charges or quotes; it is Benchmark Heaven’s estimate of what each system costs to serve on the hardware it rents, scored against a budget frozen at twice Jev’s v1.5 reference. All three rows sit inside that budget — the board marks them amber, meaning inside the 2× cap but costlier than the reference — so all three stay in the Jev-class pool that the Capability ranking draws from.

The Capability ranking exists precisely to show the other view: it drops speed and cost and averages intelligence and calibration among the systems already inside the budget. Read that way the order is service v3, then the 31B, then the 12B, and all three are above Jev.

It never hedges on yes/no questions, and that cuts both ways

Benchmark Heaven records a decisive rate for yes/no questions: the share of valid answers where the probability lands at 0.80 or above, or 0.20 or below. All three metask-jev-rain rows are decisive on 100% of them, and are right on 86%, 88% and 89% of the answers they commit to. Jev commits on 79% and is right on 98% of those.

Neither number is better in the abstract; they describe different products. A system that always commits is easy to wire into code that only reads the top label. A system that hedges on a fifth of its cases is telling you which cases to route elsewhere, and Jev’s 98% accuracy on the ones it does commit to is what makes that signal worth having. If your pipeline branches on confidence, that difference matters more than the two points of intelligence between them.

The decisive rate is a diagnostic on the board and does not feed any score.

What the board has not measured

These rows come from the sealed draw of 12 October, not the v1.6.1 reference pool. Benchmark Heaven equates a new draw to that scale with twelve anchor systems measured on both; until those anchor runs finish, the offset for this draw is zero and the rows are shown as published. The board notes that systems on the newer draws score 6 to 13 sealed points below the reference field’s line on the shared public items, and that their rows are, if anything, understated. It cannot yet say whether the draw is harder or those systems were tuned on the public items.

For the same reason there is no equated open-versus-sealed split and no per-request-type breakdown for them on the v1.6.1 scale, which is why this page carries no bar chart: the figures that would go in it do not exist yet in a comparable form. The three rows are also marked “not measured” in the board’s language and category tables, so there is nothing published about how they handle non-English decisions or any particular subject area.

Finally, the base model is undisclosed and no weights link is published alongside the rows. Everything above is Benchmark Heaven’s measurement of a system whose internals its author has not described.

Jev vs metask-jev-rain FAQ

Other Jev comparisons

Every number on these pages is quoted from a published source and was read on October 15, 2026.

• Jev vs Imajev-4BA 4B model that reads photos, against the model it copies on text.
• Jev vs decider-4bLevel on speed, far behind on the answers.
• Jev vs JevK5A fast open Qwen3.5-4B system, and far behind Jev on the answers.
• Jev vs CygnetNo training at all, tenth on the open-weights board, and behind Jev on every accuracy axis.
• Jev vs decisioSecond on the open-weights board on its biggest base, and it trains nothing of its own.
• Jev vs WinnowOne local model for decisions, chat and images, against a hosted decision model.
• Jev vs djevLevel with Jev when it was last measured, and more expensive to run.
• Jev vs SemIfA fast open rebuild on your own GPU, and where it still falls short.
• Jev vs OpenJevOne name, ten projects. Here is every one, and how the drop-in compares.
• Jev vs LayaA model you fine-tune on your data, against one that works out of the box.
• Jev vs Mercury DecideTwo hosted decision models that take the same request, one place apart.
• Run Jev locally?The open alternatives in one table, with licences and hardware.
• Jev vs LLMHow a decision model differs from a chat LLM and a BERT classifier.

Sources

• Benchmark Heaven — JevBench board v1.7.46 (open weights)
• Benchmark Heaven — JevBench API board
• TypeSafe documentation (Jev model card)