Opik
Open-source and hosted platform for tracing, evaluating and debugging LLM applications and agents: agent tracing and analysis, Test Suites with plain-text assertions, an Agent Playground, 30+ LLM-as-a-judge metrics, production monitoring with guardrails, and the Ollie AI debugging assistant.
What it does
- Evaluation and observability
Features 11
- Tracing and DebuggingCaptures and visualizes every action an agent takes so teams can annotate and fix traces.
- LLM-as-a-Judge EvaluationScores traces with 30+ metrics such as answer relevance, hallucination and task completion.
- Production Monitoring and GuardrailsEvaluates production traces in real time with alerts and guardrails against policy violations.
- Test Suites & AssertionsPlain-text assertions for what an agent should and shouldn't do, with clear pass/fail results.
- OllieCoding assistant that analyzes traces, suggests fixes and implements them in the development code.
- Agent PlaygroundRuns an entire agent in Opik to test how model, prompt and parameter changes affect the system.
- Prompt OptimizerOffers prompt optimization algorithms to improve agent performance.
- DiagnosticsScans a project's traces on demand or daily to surface problems in logged LLM application runs.
- Annotation QueuesA list of traces or threads set aside for human review, shown the way reviewers judge each item.
- Prompt LibraryA central place to manage the prompts an agent depends on.
- Online EvaluationAutomatically scores production traces with evaluation rules so they need not be reviewed by hand.
How it compares
-
Langfuse
Opik vs. Langfuse
Company disclosure · 13 Oct 2026 -
Arize Phoenix
Opik vs. Arize Phoenix
Company disclosure · 13 Oct 2026 -
Galileo
Opik vs. Galileo
Company disclosure · 13 Oct 2026 -
Braintrust
Opik vs. Braintrust
Company disclosure · 13 Oct 2026 -
LangSmith
Opik vs. LangSmith
Company disclosure · 13 Oct 2026 -
W&B Weave
Opik vs. Weights & Biases Weave
Company disclosure · 13 Oct 2026
Sources
- Pricing
-
Opik ladder: Open Source $0, Free Cloud $0 (25k spans per month), Pro Cloud $19 per month (100k spans per month), Enterprise "Custom".
- Description
-
Official documentation · 18 Oct 2026
"Define plain-text assertions for your desired outcomes in Test Suites, auto-implement fixes with the Ollie coding harness, and test run your entire agent in Agent Playground."; "Score performance with 30+ metrics"; "Apply guardrails to proactively block content and policy violations".
- Constraints
-
Derived from the security facts on this page, from the same source.
- Security
-
Company-wide Comet trust center (SafeBase, script-rendered; read in the in-app browser). Lists SOC 2 Type 2 and ISO/IEC 27001:2022; the other sections say details are available on request.
- Sold within
-
Comet's pricing page lists Opik "Open Source" at $0 ("Download, install, & run Opik your way"), a "Free Cloud" plan, and "Pro Cloud" at $19 "Per month" with "Start Free Trial"; Enterprise is priced on request.
- Deployment
-
Official documentation · 13 Oct 2026
Page: 'You can download the code from GitHub and run it locally'.
- Interfaces
-
Official documentation · 18 Oct 2026
Docs navigation: "Opik SDK & API reference", "MCP Server"; "Install the Opik MCP server and drive your entire workspace from chat"; Try Opik Cloud sign-up.
- Intended for
-
"Built for developers" with a generous free tier; also "trusted by the world's largest enterprise teams" with audit logs and governance/compliance features.
Also from Comet ML
-
Comet Experiment Tracking Developer tool
Comet's experiment-tracking product for logging, comparing, visualizing and reproducing machine-learning training runs.
-
Comet Artifacts Developer tool
Comet's dataset- and model-versioning product for tracking artifacts and lineage from training through production.
-
Comet Model Production Monitoring Platform
Comet's production-monitoring product for tracking deployed-model performance and data drift across the ML lifecycle.
-
Agent Optimizer Developer tool
Opik feature (also available as an open-source SDK) that automatically iterates and tunes system prompts across seven optimization algorithms before freezing them for production.
-
Cost Intelligence Application
Opik feature that tracks and attributes AI/coding-agent token spend (Claude Code, Codex) across engineering teams, surfacing recoverable-spend opportunities.
-
Ollie AI agent
AI debugging assistant built into Opik: investigates any logged trace, finds the root cause, reads your source files through opik connect, proposes code fixes for your approval, reruns your agent and adds the failing trace to a test suite to confirm the fix.
Open Comet ML in the directory
Something wrong here? Send a correction — quote this product id: comet-opik.