Token prices fell. Your LLM bill still went up.
Updated August 27, 2026 · first published August 27, 2026
Lower token prices do not guarantee a lower LLM bill. In production, usage can grow faster than rates fall: longer contexts, more agent steps, larger outputs, retries, and new features can erase a 40% price reduction. The useful diagnosis is not which provider got cheaper It is four-way variance: rate, volume, mix, and efficiency.
Rate is only one part of the bill
Suppose a provider cuts your blended price from $10 to $6 per million tokens. If monthly usage rises from 100 million to 220 million tokens, the bill moves from $1,000 to $1,320. The price dropped 40%; the invoice rose 32%. A dashboard that reports only price per token will call this a win while finance sees the opposite.
Track the provider rate actually billed by model and token type. Input, output, cache-read, cache-write, and reasoning tokens should not be collapsed into one average. The average is a reporting convenience, not a causal explanation.
Volume, mix, and efficiency
Volume is how many tokens you consumed. It rises when traffic grows, context becomes longer, or agents take more steps. Mix is where the tokens went: a premium reasoning model, a new geography, or a high-context feature can change the bill without changing total requests. Efficiency is the cost of serving the same useful outcome: retries, failed tool calls, oversized prompts, and output that users never consume.
Instrument every model call with provider, model, feature, team, environment, request type, input tokens, output tokens, cache tokens, reasoning tokens, retries, and outcome. Join those events to the provider invoice. Without the invoice join, you can explain usage but not prove spend.
The review that catches the problem
Run a weekly bridge from the prior period to the current one. Start with the baseline invoice, apply rate changes, then volume changes, model-mix changes, and efficiency changes. The residual should be small enough to investigate. A practical alert is not ‘tokens rose 20%’; it is ‘the feature’s traffic-adjusted cost per successful task rose 18%.’
| Variance | Question | Typical action |
|---|---|---|
| Rate | Did the provider price change? | Refresh the price sheet and contract assumptions. |
| Volume | Did traffic or context grow? | Forecast usage and add capacity controls. |
| Mix | Did work move to a dearer model? | Review routing and quality thresholds. |
| Efficiency | Did each outcome require more work? | Fix retries, prompts, tools, or stopping rules. |
What to optimize first
Do not start by forcing every request onto the cheapest model. First find the variance category with the largest dollars. If volume dominates, fix context growth or agent loops. If mix dominates, create a routing policy. If efficiency dominates, remove retries and unused output. If rate dominates, renegotiate or move eligible work to batch pricing. The right lever is the one attached to the observed variance.
Falling prices are good news, but they are not a cost-control strategy. A production team needs a reconciled view of price, usage, workload mix, and outcome quality. That is how a cheaper market becomes a cheaper product.
Related
Related
Want this applied to your own LLM spend? FinOps LLM runs a free audit of your AI costs and shows where the savings are. Book free audit →