Why do server-side tools cost more than the tokens suggest?

They bill twice. Several carry a per-use charge on the provider's infrastructure, and their results are appended to the conversation, so every following turn re-sends them as input tokens. The second cost is usually larger and is invisible in a token-only model.

How should we attribute agent tool costs?

Log which tool ran, how many times, and the token size each result added to the context — then compute cost per agent run rather than per API call. A run is what the user triggered; the call is one of many things that run did.

What is the most effective control?

A hard cap on tool calls per user request, because it bounds both the per-use fee and the context growth at once. Truncating results before they enter the context and caching the stable prompt prefix are the next two.

Quick answer: Most LLM cost models have one meter: tokens in, tokens out, price per million. That model was correct when the only thing a request did was generate text. It stopped being correct the moment...

Server tools are a separate invoice

Updated September 1, 2026 · first published September 1, 2026

Most LLM cost models have one meter: tokens in, tokens out, price per million. That model was correct when the only thing a request did was generate text. It stopped being correct the moment server-side tools arrived.

Web search, web fetch and code execution run on the provider's infrastructure, and several of them carry their own per-use charge on top of the tokens they produce. A single agent turn can therefore bill you twice: once for the searches it ran, and again for the tokens those results added to the context.

The compounding part

The second charge is the one people miss. Every tool result gets appended to the conversation, and every subsequent turn re-sends the whole thing as input. Ten searches early in a long agent run are not ten charges — they are ten charges plus their result payloads re-read on every turn that follows.

This is why agent costs feel non-linear when the token math says they should be linear. The per-use fee is the visible part; the context growth it causes is the larger part, and it lands in your input-token line where nothing labels it as tool-driven.

Attribute it or you cannot manage it

Log tool invocations per request alongside the usage object: which tool, how many calls, and the token size of what each result added. That last field is the one that makes the compounding visible, and it is the one nobody records by default.

Then compute cost per agent run, not cost per API call. A run is what a user actually triggered; an API call is one of the twenty things that run happened to do. Any budget or unit-economics number attached to the call rather than the run will be wrong in the direction that flatters you.

Three controls that actually work

Cap the tool calls per run. A hard limit on searches or fetches per user request bounds both charges at once. Most workloads that hit a limit of ten were looping, not researching.

Truncate what enters the context. You rarely need a whole fetched page. Summarising or trimming a result before it is appended cuts every downstream turn, not just the current one.

Cache the stable prefix. If the system prompt and tool definitions are constant across a run — and they usually are — prompt caching removes the largest repeated cost the tools created.

Check your provider's current rate card before modelling any of this: which tools charge per use, and at what rate, differs by tool and changes over time. The structural point holds regardless — the token meter is no longer the whole bill.

Want this applied to your own LLM spend? FinOps LLM runs a free audit of your AI costs and shows where the savings are. Book free audit →

Back to research