LLM Self-Hosted vs API Calculator
Calculate when self-hosting an LLM pays off vs using cloud APIs: the token volume where rented GPUs beat API prices, and what each costs at yours.
Qwen 2.5 32B · 1 GPU per copy on A100 80G
GPU and cost detailsA100 80G · 70% util · 40% cached
GPU
Rental defaults: RunPod on-demand $1.59/h × 730 h. Maintenance is a % of GPU rental + storage; storage adds ~$25/TB-month. GPUs are added automatically when the model or your volume needs more.
Self-hosting breaks even at
~435Mtokens / month
You're at 2B: self-hosting saves $3,511 a month.
- API · GPT-6.1 Sol
- $6,120/mo
- Self-host · 2× A100 80G
- $2,609/mo
- Setup paid back in
- 1.1 months
API list prices blended 50/50 input/output · GPU rental from RunPod on-demand · throughput is a rough heuristic
Prices as of Oct 4, 2026; check the provider for current rates. Terms
How it’s worked outMonthly costs at each volume; break-even where the lines cross
- API monthly = tokens × (1 − cached %) ÷ 1M × $/1M × (1 − batch discount).
- Self-host monthly = GPU rental + storage + network + maintenance. Rented GPUs include power. GPUs are whole: enough per copy for the model’s VRAM, and enough copies for your volume, assuming ~1,000 tokens/s per copy for an 8B-class model on an A100, scaled by model size, GPU and utilization.
- The break-even volume is the lowest monthly volume where self-hosting costs less per month. Setup (1.5× the first month) is paid back in setup ÷ monthly saving.
- Thresholds for the 1–3 year advice: payback under 12 months → self-host; over 36 months → API; in between → hybrid.
How this works: Method, 6 sources,
How this works
Method
API cost = monthly tokens × (1 − cached share) ÷ 1M × the model’s list price (input and output blended 50/50) × (1 − batch discount). Self-hosting is whole rented GPUs (enough for the model’s memory, with copies added until roughly 1,000 tokens a second per copy for an 8B-class model on an A100, scaled by model size, GPU and utilization, covers your volume) plus storage, network and maintenance; setup is taken as 1.5× a month and paid back in setup ÷ monthly saving.
Sources
- RunPod pricing (Secure Cloud, on-demand GPUs) (RunPod, prices checked 2026-10-04)
- OpenAI API pricing (OpenAI, prices checked 2026-10-04)
- Claude API pricing (Anthropic, prices checked 2026-10-04)
- Gemini API pricing (Google, prices checked 2026-10-04)
- Together AI pricing (Together AI, prices checked 2026-10-04)
- DeepInfra pricing (DeepInfra, prices checked 2026-10-04)
Limits
Prices as of Oct 4, 2026; check the provider for current rates.
Changes
- Added a printable report with the cost-by-volume chart, the one- and three-year costs and the self-hosting breakdown.
- Redesigned around a cost-by-volume chart with the break-even point; API prices now come from provider price lists, GPU rental from RunPod, and whole GPUs are sized to fit each model.
- A price you type in is now kept when you switch API models.
- Results now update as you type, starting from a filled-in example.
- First published.
Frequently Asked Questions
What is the break-even point?
The break-even point is when your total self-hosted costs (including the one-time setup) equal your total API costs. After this point, self-hosting becomes more cost-effective. The calculator shows exactly when this happens based on your input.
How do I calculate my monthly token volume?
Check your API provider's dashboard for monthly token usage. For self-hosted estimates, multiply average requests per month by average tokens per request. Most chat applications use 500-2000 tokens per user interaction.
Which GPU should I choose for self-hosting?
A100 80GB is best for production workloads with large models. H100 offers better performance but at higher cost. 4090 24GB is great for development/testing or smaller models under 7B parameters.
What are the hidden costs of self-hosting?
Beyond hardware, consider network bandwidth, backup storage, monitoring tools, and DevOps time (and power and cooling if you own the hardware; rented cloud GPUs already include them). The calculator includes maintenance, network, and storage costs but operational overhead varies by team.
When should I stick with cloud APIs?
Stick with APIs if: you need the latest model versions, require high availability without operational overhead, have unpredictable traffic patterns, or plan to stop usage within 12 months.
Can I use hybrid approach?
Yes! Many teams use a hybrid approach: handle most traffic with self-hosted models for cost savings, but use APIs for edge cases, model updates, or overflow during peak times.
How does batching affect costs?
Batching can reduce API costs by 10-30% for supported models. For self-hosted, batching improves GPU utilization, effectively lowering per-token costs. The calculator includes batch discount and utilization inputs.