Deepchecks LLM Evaluation
Platform for testing, validating and monitoring LLM and agent applications, with automated evaluators, version comparison and continuous production monitoring.
Find alternatives to Deepchecks LLM Evaluation
What it does
- Evaluation and observability
Features 5
- Version ComparisonCompares versions of prompts, models, agents and AI systems.
- Auto-Scoring PipelineSets up an auto-scoring pipeline that addresses nuanced quality constraints.
- Datasets and LLM JudgesGenerates datasets and creates LLM judges within minutes.
- Annotation and Data SlicingUses auto-scoring for annotations and for slicing and dicing data.
- CI/CD Testing and MonitoringTests LLM apps in CI/CD and monitors them in production.
Sources
- Pricing
-
Four options read live: Basic (free trial, 3 seats, 1 AI application, 5K DPUs/month, 3-month retention), Scale (demo required, 5 seats, 3 applications, 20K DPUs/month), Enterprise (custom), plus Deepchecks on SageMaker AI and Deepchecks Dedicated for on-prem/bare-metal. No dollar figure named on any tier.
- Description
-
Official documentation · 9 Sep 2026
Fetched 2026-09-09: feature list names Agents, Evaluators, Version Comparison, Auto-Scoring.
- Constraints
-
Derived from the security facts on this page, from the same source.
- Security
-
Homepage security section, read as rendered page text on 2026-10-08: "SOC2 Type 2 | GDPR | HIPAA Compliance | Single Sign On | AWS GovCloud Supported". GDPR and HIPAA are stated without a DPA or BAA so are omitted; "Single Sign On" is recorded as sso-saml, the vocabulary's SSO term. Training, retention and residency not stated.
- Sold within
-
Bought and metered on its own pricing page -- no parent offering it ships inside.
- URL
-
Official documentation · 9 Sep 2026
Product page, linked from the homepage's own "Try LLM Evaluation" call to action.
- Deployment
-
Cloud-hosted tiers (Basic/Scale/Enterprise) are SaaS; "Deepchecks on Sagemaker AI" names AWS-managed with on-premises data locality; "Deepchecks Dedicated" names single-tenant, on-prem and bare-metal options explicitly.
- Interfaces
-
Platform listing · 14 Oct 2026
PyPI listing published by deepchecks: 'The SDK client for communicating with Deepchecks LLM service'.
- Intended for
-
Official documentation · 13 Oct 2026
Page: 'Deepchecks supports secure access controls, data isolation, and auditability to meet the requirements of regulated and security-conscious organizations'.
Also from Deepchecks
-
Deepchecks ML Testing Developer tool
Open-source Python package that runs pre-built suites of checks over machine-learning data and models -- data integrity, train/test distribution mismatch and model performance -- in a research notebook or in CI/CD.
Open Deepchecks in the directory
Something wrong here? Send a correction — quote this product id: deepchecks-llm-evaluation.