Skip to content
All builds /Research and data/OpenRouter’s Ori Eval
xpost·Research and data

OpenRouter’s Ori Eval

1/ Jev, a decision model by @typesafeai, sparked a burst of projects and discussion. We tested it using Ori Eval against popular LLMs on OpenRouter at judging. Jev was >5x faster than the next fastest model, and even its slowest requests beat every other model's median.

OpenRouter’s Ori Eval
OpenRouter

OpenRouter

@OpenRouter

1/ Jev, a decision model by @typesafeai, sparked a burst of projects and discussion. We tested it using Ori Eval against popular LLMs on OpenRouter at judging. Jev was >5x faster than the next fastest model, and even its slowest requests beat every other model's median.

01 / The Decision

Evaluate input state and return typed decision for OpenRouter’s Ori Eval.

Jev sits between the incoming context and the next system action. Rather than generating lengthy, slow natural language that requires brittle regex parsing, Jev returns non-autoregressive, calibrated probabilities that downstream code can immediately execute.

02 / Typed Outputs
01actionPrimary action
02confidenceConfidence score
03fallbackFallback route

Related Research and data Builds

xpost

Search and tagging on keep.md

Cloudflare Workers has Jev now so I'm putting it to the test on keep.md - 7x faster search rerank compared to the current hybrid - 50x faster tagging of content vs GLM 4.7 Flash with no failures

Inspect
xpost

A visual reference finder

I built a visual reference finder with Jev One single prompt → 100 images from Cosmos, NASA, and The Met my new rabbit hole for creative work 🌻

Inspect
xpost

Classify rows in DuckDB

I made a DuckDB extension where you can use @typesafeai 's Jev to do quick classification of rows in any csv/parquet file or duckdb table about 10sec for 1k rows ~ better than using an LLM, way more ergonomic than a classifier game-changing for data analysis!

Inspect