EvalRouter
Hosted service that runs versioned benchmarks and environments across model providers from a CLI, SDKs or an HTTP API, in parallel isolated sandboxes, grades each sample, compares scores and costs and exports JSON, CSV or HTML reports; pay as you go with a platform fee, plus an Enterprise tier.
Find alternatives to EvalRouter
What it does
- Evaluation and observability
Sources
- Pricing
-
Site: "Pay only for what you run. No subscription, no minimums." with "Platform fee 12.5% on top of model and sandbox cost" and an Enterprise column. The $5 sign-up credit is one-off, not a free tier.
- Description
-
Official documentation · 21 Oct 2026
Site: "Established benchmarks and the models you use, all through one API." and "Run many evaluations in parallel, each in its own isolated sandbox, without managing any infrastructure."
- What it does
-
Official documentation · 21 Oct 2026
Site: "Compare model scores and costs, inspect answers and export reports." under Per-sample grading.
- Sold within
-
Site: self-serve sign-up with "$5 in free credit at sign-up" and a published pay-as-you-go column.
- Name
-
Official documentation · 21 Oct 2026
News page: "Introducing EvalRouter Announcements October 3, 2026"; the product's own site is named "EvalRouter" (https://evalrouter.ai/).
- URL
-
Official documentation · 21 Oct 2026
The product's own site, on its own domain, linked from the Kimpton menu as "EvalRouter".
- Deployment
-
Official documentation · 21 Oct 2026
Evaluations run in sandboxes the vendor operates; saas is recorded as the vendor-operated reading.
- Interfaces
-
Official documentation · 21 Oct 2026
Site: "Terminal Agent Python TypeScript" install tabs and "CLI, SDKs, HTTP API, ACP and agents" in the pricing table.
- Modalities
-
Official documentation · 21 Oct 2026
Site: each result comes back as a JSON object with a score and a cost, and the pricing table lists "JSON, CSV and HTML exports"; models are called with text prompts.
Also from Kimpton
-
Kimpton Koliseum Platform
Private evaluation environments and benchmarks built to order for a customer's models: the customer defines the task and the scoring, Kimpton designs the environment around the decisions to be tested, and checkpoints are compared on outcomes and recorded trajectories; public examples include a simulated S&P 500 Trading Floor and a six-max poker room. Sold by commission and pilot; no prices are published.
Something wrong here? Send a correction — quote this product id: kimpton-evalrouter.