Evaluate
Braintrust's offline-evaluation pillar: versioned datasets built from production logs or curated by hand, scorers written as code, as LLM judges or as human review, side-by-side experiments across prompts and models, and CI gates that block a regression before it ships.
What it does
- Evaluation and observability
Sources
- Pricing
-
Plan names, rates, allowances and overage rates read from braintrust.dev/pricing on 2026-09-19.
- Description
-
Official documentation · 19 Sep 2026
Read 2026-09-19: "Build datasets, define scorers, run experiments, and gate deployments. Measure quality and improve iteratively." Its own four-step lifecycle is "Build datasets", "Define scorers", "Run experiments" and "Gate deployments", with "Combine deterministic checks, LLM judges, and human review in one platform."
- What it does
-
Official documentation · 19 Sep 2026
Scoring model outputs against datasets and gating releases on the result is evaluation-observability.
- Sold within
-
Carried by a Braintrust plan: the pricing page meters it ("Scores ... 10K / month included then $2.50 per 1,000") but sells no Evaluate SKU; the purchased unit is the platform fee.
- Name
-
Official documentation · 19 Sep 2026
Braintrust's own Product menu lists three products -- Observe, Evaluate and Discover -- and links each to its own page; this page's heading is "Evaluate".
- URL
-
Official documentation · 19 Sep 2026
The product's own page under braintrust.dev/product/, read in full on 2026-09-19. It is absent from braintrust.dev/sitemap.xml, which lists only the homepage, pricing, legal, careers and content pages -- the pages were found from the vendor's own navigation instead.
- Deployment
-
The Enterprise tier's own line: "premium support with on-prem or hosted deployment for high volume or privacy-sensitive data"; the two self-serve plans are hosted.
Also from Braintrust
-
Braintrust Platform
Stores production agent traces in its own datastore, turns the patterns it finds there into evaluation datasets, and scores every release against them.
-
Loop AI agent
Loop is an AI agent that investigates Braintrust logs, experiments, and datasets.
-
Patterns Application
Patterns uses Loop to investigate production traces and record recurring problems and trends.
-
Observe Platform
Braintrust's production-tracing pillar: every prompt, response, tool call and retrieved context logged from the SDK or its gateway, searchable across millions of logs, with latency, cost and quality scored live and alerted on when they drift.
-
Discover Platform
Braintrust's investigation pillar: it applies a model to a project's own production traces on a schedule, surfaces recurring failure modes as evidence-backed patterns, classifies traffic into topics, and turns a confirmed finding into a regression dataset and scorer.
Open Braintrust in the directory
Something wrong here? Send a correction — quote this product id: braintrust-evaluate.