Modal Inference
Serve, scale and optimise model inference on Modal's runtime, with sub-second cold starts and autoscaling across regions.
Find alternatives to Modal Inference
What it does
- Model inference
- Model hosting
Features 6
- Shared EndpointsOptimized open-source model serving billed per token with no infrastructure to manage.
- Dedicated endpointsDeploys a dedicated endpoint tuned to a team's traffic, workload and performance needs.
- Deploy custom modelsOne-click deploy of leading open models or bring your own weights and serving configurations.
- Fine-grained observabilityProvides engine-level and container-level metrics for debugging.
- Custom optimizationsMirrors production traffic to a trial serving recipe to compare quality, latency, throughput and cost.
- Custom inference stackDeploys SGLang, vLLM or a custom server on the chosen GPUs while Modal handles infrastructure.
How it compares
-
Baseten Model APIs
Modal Labs is Baseten's closest competitor
Research report · 8 Oct 2026
Sources
- Pricing
-
Metered per second of compute on Modal's one rate card — Nvidia B200 $0.001736/sec, H200 SXM $0.001261/sec, H100 SXM5 $0.001097/sec, A100 80GB $0.000694/sec — with $30 a month of free compute. The pro
- Description
-
Official documentation · 5 Sep 2026
Modal's own product page for it, fetched 2026-09-05, HTTP 200.
- Constraints
-
Official documentation · 2 Oct 2026
Derived from the security facts on this page, from the same source.
- Security
-
Official documentation · 2 Oct 2026
Platform-wide: security, data-residency, region-selection, audit-logs, SCIM and RBAC docs. HIPAA BAA is Enterprise plan only and excludes Volumes v1, Images (most), Memory Snapshots and user code. SSO, SCIM and audit logs are Enterprise-plan features; SCIM is beta. ZDR unknown at platform level: only Inference endpoints are described as zero data retention. Training is from the Modal terms: without prior written consent Modal will not 'train any AI model using Customer Data' (this page adds that Modal will not access inputs, outputs or stored data). Customer Supplied Encryption Keys exist but are Alpha, so customer-managed-keys not recorded. Trust portal trust.modal.com is script-rendered.
- Sold within
-
Official documentation · 5 Sep 2026
Modal's pricing page prices compute, not products: every one of the six entries in its Products nav runs on the same account and the same meter.
- Interfaces
-
Official documentation · 8 Oct 2026
"Run pip install modal to install the modal Python package" -- a Python SDK and a CLI workflow for every product on the same account/meter; modal-notebooks additionally has a browser notebook UI.
- Intended for
-
"AI infrastructure that developers love"; "Code-first inference" staying in application code, customer quote praising it for "power and flexibility."
Also from Modal
-
Modal Infrastructure service
Serverless GPU compute: a Python decorator puts a function on an accelerator, scales it from zero to thousands of containers and stops billing when it stops running — aimed at inference, fine-tuning and batch jobs rather than reserved clusters.
-
Modal Training Infrastructure service
Managed training runs on Modal's fleet, configured in Python alongside the rest of a team's code; also gives instant access to GPU compute, including RDMA-enabled GPUs.
-
Modal Sandboxes Infrastructure service
Isolated, instantly-started containers for running untrusted or agent-generated code at scale — the primitive behind AI app-generation products.
-
Modal Notebooks Developer tool
Hosted notebooks backed by Modal's GPUs, for profiling and experimenting without provisioning a machine.
-
Modal Batch Infrastructure service
Batch execution of large jobs across Modal's fleet, described as one line of code on the product page.
Something wrong here? Send a correction — quote this product id: modal-inference.