Adaptive thinking and cost forecasting
Updated September 1, 2026 · first published September 1, 2026
The reasoning budget used to be a number you set. You passed thinking: {type: "enabled", budget_tokens: N}, and N was both a control and a forecast: worst case, every request cost you N reasoning tokens, and finance could multiply.
That parameter is deprecated on Opus 4.6 and Sonnet 4.6, and on the newest models — Fable 5 and 5.1, Sonnet 5, Opus 5, 4.8 and 4.7 — sending it returns a 400. The replacement is thinking: {type: "adaptive"}, where the model decides how much to think based on the difficulty of the request.
This is better output per dollar on average. It is also the removal of a ceiling, and if your forecast was built on that ceiling, the forecast is now wrong in a way that will not show up until the month closes.
What changes in the numbers
Mean cost per request usually goes down, because easy requests stop paying for reasoning they never needed. Variance goes up, because hard requests are no longer truncated at your cap. The two move in opposite directions, and a budget expressed as a per-request maximum has nothing left to attach to.
The practical failure is not the average. It is the tail: a prompt change, a new document type, or a user who starts asking harder questions moves a slice of your traffic into deeper reasoning, and nothing in your configuration limits it.
Forecast the distribution, not the cap
Three changes, in order of how much they buy you.
Track reasoning tokens as their own series. They are already in the usage object on every response. Separate them from input and output in your telemetry, and you can see the shift the day it happens instead of at month end.
Budget on percentiles. Replace "N tokens per request" with a p50 and a p99 per route. The p50 tells you what the workload costs; the gap between p50 and p99 tells you how exposed you are to a bad week.
Alert on the ratio, not the total. Reasoning tokens as a share of total tokens per route is the single number that moves first when a prompt change makes the model work harder. A total-spend alert fires days later, once the volume has accumulated.
Where a cap still belongs
Removing the per-request knob does not mean removing all limits. Route-level and tenant-level spend limits still work, still catch runaway loops, and are where the guardrail belongs now — at the boundary you own, not inside a request parameter that no longer exists. For latency-sensitive paths where deep reasoning is never worth it, the lever is model choice, not a token budget.
Related
Related
Want this applied to your own LLM spend? FinOps LLM runs a free audit of your AI costs and shows where the savings are. Book free audit →