Is Self-Hosting an Open-Weights LLM Cheaper Than the API? The Break-Even Math
Updated 2026-10-25. Checked against official provider docs and pricing pages on 2026-10-25.
Renting a GPU and running an open-weights model yourself sounds cheaper than paying per token. Whether it actually is depends on one number most people never measure: how many tokens per second your setup sustains on your workload, around the clock. This guide gives you the formula, real list prices for both sides, and the throughput you'd need to break even. You plug in your own benchmark result. We don't invent one.
The formula
A rented GPU costs the same per hour whether it's busy or idle. An API costs per token. So:
Self-hosted $ per 1M tokens = (GPU $/hour × 1,000,000) / (tokens per second × 3,600 × utilization)Break-even is where that equals the API's blended price. Solving for throughput:
Break-even tokens/sec = (GPU $/hour × 1,000,000) / (3,600 × API blended $ per 1M)- tokens per second is aggregate throughput across all concurrent requests (input and output), measured with your real prompt and output lengths on your serving stack.
- utilization is the share of paid hours the GPU is actually busy. A box you pay for 24/7 but use 8 hours a day is at roughly one-third utilization.
- API blended $/1M mixes input and output prices in your workload's ratio.
Before buying GPUs, compare against the cheapest hosted LLMs for AI agents and the discount from the batch API for non-urgent jobs.
Inputs: GPU rental prices (official pages)
Retrieved 2026-10-25. List prices, before tax.
| Provider | GPU | Price |
|---|---|---|
| Lambda, 8x instance | NVIDIA H100 SXM 80 GB | $3.99 / GPU / hr |
| Lambda, 8x instance | NVIDIA B200 SXM6 180 GB | $6.69 / GPU / hr |
| Lambda, 1x instance | NVIDIA H100 SXM 80 GB | $4.29 / hr |
| Lambda, 1x instance | NVIDIA B200 SXM6 180 GB | $6.99 / hr |
| RunPod Pods, Secure Cloud (Community Cloud) | H100 SXM 80 GB | $3.99 / hr ($2.69) |
| RunPod Pods, Secure Cloud (Community Cloud) | H200 141 GB | $5.29 / hr ($3.59) |
| RunPod Pods, Secure Cloud (Community Cloud) | B200 180 GB | $7.99 / hr ($5.98) |
| RunPod Pods, Secure Cloud (Community Cloud) | L40S 48 GB | $1.09 / hr ($0.79) |
RunPod's on-demand prices above are Secure Cloud rates, with the cheaper Community Cloud rate in brackets. Lambda prices are per GPU-hour and rise for smaller instances (H100 SXM: $3.99 on 8x, $4.09 on 4x, $4.19 on 2x, $4.29 on 1x).
Running 24/7 (730 hours a month), one H100 SXM at $3.99 (RunPod Secure Cloud) costs $2,912.70 a month. A full 8x B200 Lambda instance ($53.52/hr) costs $39,069.60 a month.
Inputs: API prices for open-weights models (models.json)
The fair comparison is to the same open-weights model served by its maker's API, not a closed frontier model. From TokenTab's models.json (lastUpdated 2026-10-25), per 1M tokens:
| Model (open weights) | Input | Output | Cached input |
|---|---|---|---|
| DeepSeek V4 Pro (peak) | $1.32 | $3.96 | $0.044 |
| DeepSeek Flash (peak) | $0.30 | $1.20 | $0.006 |
| Mistral Small 4 | $0.15 | $0.60 | $0.015 |
DeepSeek's off-peak prices are half of peak ($0.66/$1.98 and $0.15/$0.60).
Worked break-even numbers
Assumption (labeled, not measured): an input-heavy agent mix of 3 input tokens to 1 output token, no cache hits. Blended price = (3 × input + output) ÷ 4.
| Compare against | Blended $/1M | Break-even on 1x H100 SXM ($3.99/hr, RunPod Secure) | Break-even on Lambda 8x B200 ($53.52/hr) |
|---|---|---|---|
| DeepSeek V4 Pro (peak) | $1.98 | ~560 tok/s sustained | ~7,508 tok/s |
| DeepSeek Flash (peak) | $0.525 | ~2,111 tok/s | ~28,317 tok/s |
| Mistral Small 4 | $0.2625 | ~4,222 tok/s | ~56,635 tok/s |
The same thresholds as monthly volume at 100% utilization: an H100 has to serve about 1.47B tokens a month to match DeepSeek V4 Pro's API, 5.5B to match DeepSeek Flash, and 11.1B to match Mistral Small 4. At 33% utilization, the throughput you need while the GPU is busy roughly triples.
This table doesn't tell you whether a model fits on that hardware or how fast it actually runs. Those depend on parameter count, quantization, context length, batch size, and the serving engine. Measure them with a benchmark such as vLLM's benchmark suites on your real prompts. Then compare your measured tok/s to the break-even column.
Why the API usually wins for agents
- Prompt caching tilts the math further toward the API. Agent prompts repeat. Cached input on DeepSeek Flash is $0.006/M, which is 2% of its normal input price. A self-hosted engine can reuse prefixes too, but that only saves GPU time you're already paying for. Recompute the blended price with your real cache-hit rate and the break-even throughput goes up.
- Agents are bursty. Traffic arrives in spikes, and you size the GPU for the peak but pay for it at the trough. Utilization is the term that usually ruins self-hosting spreadsheets.
- Off-peak prices. DeepSeek's off-peak discount halves the API price, which doubles the break-even throughput.
- Costs that aren't GPU hours. Engineering time, monitoring, storage, egress, redundancy, and on-call aren't in the formula. Add them as a monthly dollar figure on the GPU side.
When self-hosting can win
- Steady, high, predictable volume that keeps GPUs busy most hours.
- Data residency or privacy rules that rule out third-party APIs. Then cost isn't the deciding factor anyway.
- Fine-tuned or custom models that no API serves.
- Small models on cheap GPUs. An L40S at $1.09/hr (RunPod Secure Cloud) needs about 577 tok/s sustained to beat DeepSeek Flash's $0.525 blend. Whether a model that fits in 48 GB reaches that speed on your workload is the thing to benchmark.
How to decide in 15 minutes
- Estimate your monthly tokens and input/output mix. The TokenTab AI agent cost calculator gives you the API-side monthly cost per model.
- Compute the break-even tok/s for your GPU price with the formula above.
- Benchmark the model on that GPU with your real prompts.
- Pick self-hosting only if measured throughput × realistic utilization clearly beats break-even after adding non-GPU costs.
Sources
Retrieved 2026-10-25.
- Lambda pricing (instance tables, per GPU-hr, 8x/4x/2x/1x): https://lambda.ai/pricing
- RunPod pricing (Pods, Secure Cloud and Community Cloud on-demand rates): https://www.runpod.io/pricing
- TokenTab models.json, lastUpdated 2026-10-25 (DeepSeek V4 Pro, DeepSeek Flash, Mistral Small 4 prices and open-weights notes): /models.json
- vLLM documentation, Benchmark Suites: https://docs.vllm.ai/en/stable/benchmarking/