AI provider list prices are not directly comparable
Updated August 27, 2026 · first published August 27, 2026
Comparing AI providers by the headline price per million tokens produces false savings. Providers bill different token types, discounts, context rules, regions, and tool calls. A useful comparison measures effective cost per successful task for the same workload and quality target.
Normalize the billable unit
Build a price sheet keyed by provider, model, endpoint, region, token type, cache state, batch status, and effective date. Keep input, output, cache-read, cache-write, reasoning, image, audio, and tool charges separate. Record minimum commitments and credits outside the unit rate so the comparison does not hide a contract assumption.
Normalize the workload
Replay a fixed sample with the same prompts, context, output constraints, tool permissions, and success test. Measure p50 and p95 latency, completion rate, retries, human repair, and output quality. A model that is 30% cheaper per token but needs twice as many retries may be more expensive per successful task.
Include operating cost
Provider arbitrage has an engineering price. Add gateway changes, observability, routing logic, evaluation, support, data-transfer cost, and failure handling. Run the calculation over a realistic traffic mix, not a single ideal prompt. The cheapest model for summarization may be the wrong model for extraction or high-value reasoning.
Route by capability-per-dollar
Use a capability matrix with a minimum quality score and a maximum effective cost per workload. Route eligible requests to the cheapest provider that passes both. Keep a fallback route, but count its expected cost and test it during failover drills. Record why each request was routed so savings can be reconciled to invoices.
Provider price comparison is not a leaderboard. It is a controlled experiment with a common workload, a quality bar, and a fully loaded denominator.
Related
- LLM provider arbitrage
- Model routing without quality regressions
- Cost per successful task
Related
Want this applied to your own LLM spend? FinOps LLM runs a free audit of your AI costs and shows where the savings are. Book free audit →