Together Batch Inference
Asynchronous bulk inference for workloads that do not need a real-time response, priced below Together's serverless rate.
Find alternatives to Together Batch Inference
What it does
- Model inference
Features 5
- Up to 50% cost savingsBatch jobs run at up to half the cost of the real-time API for most serverless models.
- 30B enqueued tokens per modelBatch jobs scale to 30 billion enqueued tokens per model per user, with custom limits on request.
- Under-24-hour processing SLAJobs are stated to finish well under 24 hours, often within hours.
- Any serverless or dedicated modelBatch jobs run across any serverless model or private deployment.
- JSONL file uploadLaunch batch jobs by uploading a JSONL file, with no orchestration or monitoring setup.
Sources
- Pricing
-
Together AI publishes per-unit and per-GPU-hour rates; each service is metered separately and bought on its own.
- Description
-
Official documentation · 6 Sep 2026
Summarised from the page's own title, <h1> and meta description, fetched 2026-09-06.
- Constraints
-
Official documentation · 2 Oct 2026
Derived from the security facts on this page, from the same source.
- Security
-
Official documentation · 2 Oct 2026
Privacy policy (platform services). GPU clusters and custom training left out (policy routes their data to email requests). Certifications not read: trust.together.ai is script-only (see needs_browser). Training stated as opt-in only.
- Sold within
-
Together AI publishes per-unit and per-GPU-hour rates; each service is metered separately and bought on its own. Corrected at apply time: parent set to null.
standalonemeans no parent OFFERING stands between the buyer and the product; the vendor's own company name is not such an offering. - URL
-
Official documentation · 6 Sep 2026
Fetched 2026-09-06: HTTP 200, <title> 'Batch Inference | Together AI', <h1> 'Process massive workloads asynchronously'.
- Interfaces
-
Official documentation · 12 Oct 2026
Page: 'We rely on the Batch Inference API to process very large amounts of requests'.
- Modalities
-
Official documentation · 12 Oct 2026
Page: 'from a single uploaded JSONL file / batch_output.jsonl'.
- Intended for
-
page describes capabilities as "purpose built for AI natives" processing offline workloads, matching serverless-inference's developer framing
Also from Together AI
-
Together GPU Clusters Infrastructure service
Reserved NVIDIA GPU clusters for training and large-scale inference; also takes a trained model to a production endpoint without switching providers.
-
Together Inference Model API
Hosted API serving open-weight text, image and audio models, billed per token.
-
Together Fine-Tuning API service
A managed service for fine-tuning open-source models on a customer's own data and serving the result on Together's infrastructure.
-
Together Custom Training Developer tool
Custom model training service covering supervised fine-tuning and direct preference optimization, billed per token.
-
Together Dedicated Container Inference Infrastructure service
Dedicated, reserved GPU containers for model inference with guaranteed performance, billed per GPU-hour.
-
Together Provisioned Throughput API service
Reserved inference capacity for open models, bought in Provisioned Throughput Units that each hold a fixed tokens-per-minute rate, with a 99% uptime SLA and overflow routed to Together's serverless fleet at list price.
-
Together Dedicated Model Inference Infrastructure service
Dedicated single-tenant GPU endpoints for open-weight or custom models pulled from Hugging Face or S3, with autoscaling, canary and rolling rollouts and multi-region failover, billed per GPU-hour.
-
Together Frontier AI Factory Infrastructure service
Private GPU cloud built to order for frontier-scale training, fine-tuning and inference, from 1,000 to 100,000+ NVIDIA GPUs with managed Kubernetes or Slurm and direct access to Together's research team.
Open Together AI in the directory
Something wrong here? Send a correction — quote this product id: together-batch-inference.