Arthur Engine
Open-source, real-time evaluation engine for generative AI and traditional ML models: instant analysis of outputs, active guardrails, and built-in plus customizable metrics, run inside a user's own infrastructure independently of the hosted Arthur platform.
Find alternatives to Arthur Engine
What it does
- Evaluation and observability
Features 7
- GuardrailsConfigurable real-time detection of PII leakage, hallucination, prompt injection and toxic language.
- OpenTelemetry trace collectionCollects and analyzes OpenInference OTEL traces from agentic workflows and LLM apps.
- Continuous evaluationsRuns evaluations on live traces to catch regressions and quality issues.
- Prompt managementManages, versions and iterates on prompts across applications.
- Prompt experimentsCompares prompt variants and measures their impact on quality metrics.
- RAG evaluationEvaluates and monitors Retrieval-Augmented Generation pipelines end-to-end.
- Custom metricsNative support for custom metrics and an extensible API.
Sources
- Pricing
-
Official documentation · 13 Sep 2026
GitHub API repository metadata: license.spdx_id = MIT.
- Description
-
Arthur's own announcement: "open-source, real-time AI evaluation engine for both Generative AI and Traditional ML models"; "developers can use the free Engine independently" of the paid Arthur Platform, which 'layers on' for enterprise features.
- Sold within
-
Official documentation · 15 Sep 2026
GitHub repo (already cited official-docs elsewhere on this record); downloaded directly, independent of the hosted Arthur platform.
- URL
-
Official documentation · 13 Sep 2026
Repository home; GitHub API license field returns MIT.
- Deployment
-
Announcement: runs "inside your own stack" with no third-party data transmission.
- Interfaces
-
Platform listing · 14 Oct 2026
PyPI listing published by Arthur AI: 'Arthur Observability SDK — telemetry, prompt management, and framework instrumentation'.
- Intended for
-
Official documentation · 13 Oct 2026
Page: 'For builders and hackers that want to move fast'.
Also from Arthur AI
-
Arthur Platform
AI observability platform monitoring model performance, bias, drift and behavior for ML and generative AI/agentic applications in production, built on an open-source evals engine.
-
Arthur Agent Security & Governance Application
Module within the Arthur AI Platform providing security scanning, policy enforcement and governance controls specifically for autonomous agent deployments (tool-use permissions, action auditing, risk scoring).
-
Arthur Observability Platform
Monitors model performance, model metrics, and data drift.
-
Arthur Agent Development Toolkit Developer tool
Toolkit for building, debugging and shipping AI agents: version prompts, A/B test prompts, models and RAG configs on production data, trace every agent step, run continuous evals and guardrail checks for hallucination, PII, prompt injection and toxicity on live traffic, and alert on quality drift.
Open Arthur AI in the directory
Something wrong here? Send a correction — quote this product id: arthur-engine.