Is the Batch API's 50% Discount Worth It for AI Agents?
Updated 2026-10-25. Prices from models.json, checked against official pricing pages on 2026-10-25.
OpenAI, Anthropic, and Google all sell the same tokens at half price if you send them through their batch APIs and wait for the results. For most teams that's the biggest discount on the table. But for the live agent loop the honest answer is usually no. The discount pays off on the work around your agent: evals, replays, backfills, summaries, and any fleet of jobs that doesn't need an answer right now.
What you get, and what you give up
- OpenAI Batch API: "50% cost discount compared to synchronous APIs," with "substantially more headroom" on rate limits. Each batch completes within 24 hours, and often sooner.
- Anthropic Message Batches: "All usage is charged at 50 percent of the standard API prices," covering input, output, and special tokens. Anthropic says most batches finish in less than 1 hour (results are available within 24 hours).
- Google Gemini API: the paid tier lists "Batch API (50% cost reduction)," and the pricing page shows batch rates at half of standard.
What you give up is latency. You submit a file of independent requests, poll for status, and collect the results later. You don't get a streamed answer in a second or two.
Batch prices on popular agent models
Per 1M tokens, standard tier vs batch, short context. Prices are from official pricing pages, retrieved 2026-10-25. Standard rates match TokenTab's models.json.
| Model | Standard in / out | Batch in / out |
|---|---|---|
| GPT-6 Astra | $10.00 / $50.00 | $5.00 / $25.00 |
| GPT-6.1 Sol | $2.00 / $10.00 | $1.00 / $5.00 |
| GPT-6 Luna | $0.10 / $0.50 | $0.05 / $0.25 |
| Claude Opus 5.5 | $4.00 / $20.00 | $2.00 / $10.00 |
| Claude Sonnet 5.5 | $2.00 / $10.00 | $1.00 / $5.00 |
| Claude Haiku 5.5 (≤100K prompt) | $0.10 / $0.50 | $0.05 / $0.25 |
| Gemini 4 Argon* | $2.00 / $10.00 | $1.00 / $5.00 |
| Gemini 3.8 Flash* | $0.75 / $3.75 | $0.375 / $1.875 |
*Google promotional pricing. Gemini 4 Argon's standard rate rises to $4/$20 on Dec 1, 2026 (batch $2/$10). Gemini 3.8 Flash rises to $1.50/$7.50 on Jan 1, 2027 (batch $0.75/$3.75).
Why batch doesn't fit the agent loop
An agent is a chain. Step N's prompt is built from step N-1's output: the tool result, the updated plan, the new state. Batch pricing rewards independent requests that the provider can schedule whenever it has spare capacity. Put a 20-step agent task into a batch queue and, in the worst case, every step waits for its own batch to finish. Even at Anthropic's typical under-an-hour turnaround, that's hours for something a user expects in minutes.
The UK AI Security Institute's Inspect framework, which supports batch mode for OpenAI, Anthropic, Google, xAI and Together AI, warns about this. Evals with a large or variable number of sequential generations, such as agentic tasks, "can often take many hours or days" in batch mode because of the path dependency.
So the live loop stays on real-time (standard) pricing. Prompt caching (see our companion guide) and routing easy steps to cheaper models are the levers that actually reduce in-loop cost.
Where batch is worth it
Most agent systems produce a lot of token volume that nobody is waiting on:
- Offline evals and regression runs. Re-running test tasks against a new prompt or model.
- Trajectory scoring and labeling. Grading yesterday's agent runs with an LLM judge.
- Memory summarization. Nightly compaction of long histories into short notes.
- Document processing and classification. Bulk extraction, tagging, or embedding backfills for RAG.
- Fleets of independent jobs. If 20 subagents each need one self-contained, non-urgent completion, they can go out as one real batch.
Worked example: nightly eval
Say you replay 500 recorded agent tasks every night through an LLM judge. Each call has 40,000 input tokens and 4,000 output tokens, on Claude Sonnet 5.5:
- Standard: 500 × (40,000 × $2/M + 4,000 × $10/M) = 500 × ($0.08 + $0.04) = $60 per night
- Batch: 500 × (40,000 × $1/M + 4,000 × $5/M) = $30 per night
Over 30 nights that's $1,800 vs $900, so $900 a month saved on a job nobody watches live. The same workload on GPT-6.1 Sol has identical list prices and the same savings. On GPT-6 Luna or Haiku 5.5 it's $3 vs $1.50 per night.
Stacking batch with prompt caching
The discounts combine. Anthropic says prompt caching multipliers "stack with other pricing modifiers, including the Batch API discount." OpenAI's pricing page also lists batch cached-input rates. GPT-6.1 Sol's batch cached input is $0.05/M, compared with $0.10/M at standard.
Watch out for one thing: batches can take longer than a short cache lifetime. Anthropic suggests the 1-hour cache for batches that share context (a 2x write instead of 1.25x), so later requests in the batch can still hit the cache. Model the cache side in the prompt caching calculator. Put your shared rubric or system prompt first and keep it identical across requests.
Where batch doesn't apply
- Anthropic fast mode is "not available with the Batch API."
- Claude Managed Agents sessions have no batch mode, since they're "stateful and interactive."
- User-facing chat or voice agents can't tolerate the delay.
- Time-sensitive tool calls (bookings, live data) shouldn't sit in a queue.
A quick decision rule
Ask one question for each LLM call your system makes: does anything wait on this result within the next hour?
- Yes: keep it real-time, and use caching and model routing to cut the cost.
- No: batch it and pay half.
Measure your own split. To see what that split means for your monthly bill, plug your steps, tokens, and model into the TokenTab AI agent cost calculator, then halve the token cost of the share you can batch.
Sources
Prices and docs retrieved 2026-10-25.
- OpenAI Batch API guide (50% discount, 24-hour completion, higher rate limits): https://developers.openai.com/api/docs/guides/batch
- OpenAI API pricing (Batch tab: model rates, batch cached input): https://developers.openai.com/api/docs/pricing
- Anthropic Message Batches API (50% on all usage, most batches finish in less than 1 hour, 1-hour cache suggestion): https://platform.claude.com/docs/en/build-with-claude/batch-processing
- Anthropic pricing (batch table, caching stacks with batch, fast mode and Managed Agents exclusions): https://platform.claude.com/docs/en/about-claude/pricing
- Google Gemini API pricing (Batch 50% reduction, batch rates, promo dates): https://ai.google.dev/gemini-api/docs/pricing
- UK AISI Inspect, batch mode (agentic evals and path dependency): https://inspect.aisi.org.uk/models-batch.html
- TokenTab models.json: /models.json