Jev vs Imajev-4B

The analysis

Where Jev and Imajev-4B really differ, and why the bars above land where they do.

The ranking reversed between releases. In JevBench v1.4.2.2 Imajev-4B was first, 67.4 against Jev’s 63.3. v1.6 re-measured every system on a new pool of 1,500 decisions, 1,200 of them sealed. There Imajev-4B, measured on an RTX 5090, ranks #133 of 235 open-weights systems at 30.0. Jev scores 71.4; since board v1.7.0 Benchmark Heaven shows it on that board as an unranked reference and ranks it with the hosted APIs instead, where it is ranked #15 of 48. On the headline Capability ranking, which averages intelligence and calibration, Jev scores 77.1 and Imajev-4B is #103 at 61.9.

The gap is intelligence: 63.6 against 34.6. Jev scores 61.6 on the sealed half against Imajev-4B’s 34.2, and the widest difference is on yes/no questions, where Jev scores 50.3 above chance against 0.3 — on the new pool Imajev-4B is barely above guessing on them, which is what drags its score down to 30.0. On choice questions they are closer, 77.4 against 64.4. Imajev-4B keeps an edge on the benchmark’s cost estimate, 77.8 against 54.7, but no longer on speed, 90.4 against 91.2, or calibration, 89.2 against 90.6.

The larger difference is still what it can look at. Jev reads text only; Imajev takes up to two photos in the same request and checks them against the record you send, and every answer carries a trained probability that the evidence cannot settle the question. JevBench measures neither. On Benchmark Heaven’s separate Image JevBench v0.3.0 it is fourth on the composite and seventh on the Capability ranking, well ahead of Jev-Omni; the Imajev-4B vs Jev-Omni page covers it.

What Imajev-4B actually is

Imajev-4B, the open image-capable decision model, by Mohit Garg.

Imajev is a family of open decision models by Mohit Garg in three sizes, 2B, 4B and 9B, each a LoRA adapter and a 255-code decision readout on Qwen3.5. The 4B, the recommended default, keeps Qwen3.5-4B’s vision tower frozen. It reads the state, up to two images and each question, then takes the probability of each option from one position without generating text. The adapters and code are Apache-2.0, as is the Qwen base.

Its server accepts the same /v1/systemone request and response as TypeSafe’s Jev and adds images, an unknown probability and an abstained flag; the README says text-only Jev requests work unchanged. Per request it takes up to two images, a state up to 32 KB, one to eight questions and 2 to 254 options. It runs on Apple silicon through MLX or on a CUDA GPU, and the 4B needs a 9.3 GB base-model download. The author measures about 0.1s per question raw on one H100, and about 0.35s as shipped, averaging four option orders.

It was trained on about a million decisions in four stages, labelled by people or by open-weight teacher models. The author states that no Jev outputs, no paid-API outputs and no JevBench items were used, and that JevBench items were screened out with an 8-gram check. JevBench measured the released 4B adapter on an RTX 5090, with one option order and the shipped calibration file.

What its author says about its limits

The README is candid. On the public hard split alone, two other open models, JevK5 and Eikos-4B, are ahead of every Imajev size. Without its calibration file the model is over-confident on hard items. An empty field can be read as “no” instead of unknown, and English is the only supported language.

Its own image benchmark, ImajevBench, uses AI-generated images that have not yet had a human audit, and it was one input to choosing the released checkpoints. Benchmark Heaven’s separate Image JevBench is an external check on the photos; the Imajev-4B vs Jev-Omni page covers it.

Jev vs Imajev-4B FAQ

Other Jev comparisons

Every number on these pages is quoted from a published source and was read on October 31, 2026.

• Jev vs decider-4bLevel on speed, far behind on the answers.
• Jev vs JevK5A fast open Qwen3.5-4B system, and far behind Jev on the answers.
• Jev vs CygnetNo training at all, forty-sixth on the open-weights board, and behind Jev on every accuracy axis.
• Jev vs decisioTenth on Capability on its biggest base, and it trains nothing of its own.
• Jev vs metask-jev-rainMore accurate answers than Jev on the benchmark, and two of three rows above it on the composite.
• Jev vs WinnowOne local model for decisions, chat and images, against a hosted decision model.
• Jev vs djevLevel with Jev when it was last measured, and more expensive to run.
• Jev vs SemIfA fast open rebuild on your own GPU, and where it still falls short.
• Jev vs OpenJevOne name, ten projects. Here is every one, and how the drop-in compares.
• Jev vs LayaA model you fine-tune on your data, against one that works out of the box.
• Jev vs Mercury DecideTwo hosted decision models that take the same request, one place apart.
• Run Jev locally?The open alternatives in one table, with licences and hardware.
• Jev vs LLMHow a decision model differs from a chat LLM and a BERT classifier.

Sources

• Benchmark Heaven — JevBench board v1.7.48 (open weights)
• Benchmark Heaven — JevBench API board
• mohit67890/imajev — open typed-decision models that also take images (Apache-2.0)
• mohit67890/imajev-4b on Hugging Face
• TypeSafe documentation (Jev model card)