O

Ori Eval

AI agent

by OpenRouter

Ori Eval explores a codebase, runs model evaluations, and grades answers with an LLM judge.

Find alternatives to Ori Eval

What it does

• Evaluation and observability

Features 6

• Codebase scan for model surfacesScans a codebase for every place a model runs, showing the surface, code location and current model.
• Model comparisonRuns evaluations across several models in parallel and recommends one with quantitative proof.
• Pinned model and effortPins the model and effort so evaluation runs are repeatable and consistent.
• Bug-to-test evalsTurns each bug into an eval that fails now, passes once fixed and prevents regressions.
• CI regression checksFails the build in CI when a change regresses an eval so bugs do not reach users.
• Scheduled re-evaluationEvals are code that can be re-run when a new model drops or scheduled monthly.

Sources

Pricing

Official documentation · 14 Oct 2026

"An eval usually costs $10 to $30."

Description

Official documentation · 14 Oct 2026

"explores your codebase, runs evaluations on models"

What it does

Official documentation · 14 Oct 2026

"We use LLM-as-a-judge to grade open-ended answers."

Sold within

Official documentation · 14 Oct 2026

"Ori Eval is part of"

Name

Official documentation · 14 Oct 2026

"What is Ori Eval?"

URL

Official documentation · 14 Oct 2026

"What is Ori Eval?"

Deployment

Official documentation · 14 Oct 2026

"Ori sends its requests through OpenRouter."

Interfaces

Official documentation · 18 Oct 2026

Page: 'Using OpenRouter MCP? Run /spawn-ori-eval in your agent to get started'.

Also from OpenRouter

• OpenRouter Model API

  Single API that routes requests to models from many providers and bills them through one account.

• Ori Developer tool

  OpenRouter's suite of agentic developer tools for using any hosted model "right where you are": Ori Harness (CLI) and Ori Code (terminal coding agent) are GA; Ori Eval finds the best model for a task; Ori Intern (Slack) is in private testing and Ori Desktop (Mac) is waitlisted.

Open OpenRouter in the directory

Something wrong here? Send a correction — quote this product id: openrouter-ori-eval.