TypeSafe releases JevBench benchmark for AI models

Jev: Jev is the typed-decision model released by TypeSafe on September 15 that returns structured answers with probabilities instead of prose. Version 1.13.0 of Jev leads the JevBench composite score because the benchmark factors in latency, calibration quality, and cost alongside accuracy. Its design targets practical, bounded decision workloads in software applications. JevBench: JevBench is a benchmark created specifically for AI models that produce bounded software decisions rather than open-ended text. It evaluates systems on a composite that integrates intelligence, calibration, speed, and cost to better reflect deployment realities. The benchmark was released in connection with TypeSafe’s September 15 launch of its Jev model. TypeSafe: TypeSafe is the company that developed and released the Jev model on September 15. Jev processes application state together with fixed options to output typed answers accompanied by probability estimates. TypeSafe’s release directly prompted the creation of the JevBench evaluation framework. GPT-5.6 Luna: GPT-5.6 Luna is a model noted for achieving higher hard-case accuracy than Jev 1.13.0 on certain tasks. Despite this strength, it ranks lower overall on JevBench due to comparatively weaker results in calibration, speed, and cost. The comparison illustrates how JevBench distinguishes performance on narrow, typed decision scenarios. Scoring Method: A geometric mean is used so that strength on any single dimension cannot fully offset weakness on another. Benchmark Scope: JevBench focuses on models whose output is a bounded software decision with probabilities rather than open-ended prose. Composite Evaluation: The benchmark deliberately combines intelligence, calibration, speed, and cost because high accuracy alone can still lead to deployment failures.