Agent total cost of ownership: the 70% of agent cost that isn't tokens

Updated 19 July 2026

Teams running production AI agents often track only one cost: model tokens. The bill from OpenAI or Anthropic feels like "the" LLM cost. But that's typically only 10–30% of the real expense. The rest hides in evaluation burn, observability tooling, platform staffing, model drift correction, compliance overhead, and retry waste — costs that grow faster than token spend and often go unbudgeted until they surprise you.

This article gives you a practical total cost of ownership (TCO) formula, breaks down each component with realistic numbers, and walks through a worked example. By the end, you'll know where to measure and what levers to pull to reduce agent cost.

The TCO anatomy: where agent cost really lives

Production token spend (10–30%) — tokens consumed by live agent execution. This is the most visible cost and the one most teams track first.

Evaluation token burn (20–60%) — tokens consumed by test suites, validation runs, and regression testing. A comprehensive evaluation suite runs on every release and often consumes more tokens than production traffic. Industry reports on code-review agents show evaluation token spend as high as 59% of total token consumption.

Observability and monitoring (5–15%) — infrastructure for logging, tracing, metrics, and alerting. As agents get more complex, observability cost grows faster than production workload.

Platform staffing (15–35%) — engineering time to maintain the agent framework, handle model updates, debug failures, and tune performance. A team running 8–15 agents might need 2–3 full-time platform engineers; attributing this cost fairly shows why most solo agents are expensive.

Model drift correction (5–10%) — evaluation and retraining runs triggered by new model releases or performance degradation. When a provider ships a new model version, you run your eval suite against it; when performance drifts, you run corrective fine-tuning or re-prompting.

Compliance and audit overhead (3–8%) — token audits, data retention, anonymization, and regulatory logging. In regulated industries (finance, healthcare), this cost is often overlooked in early pilots but becomes material at scale.

Retry and loop waste (2–15%) — tokens burned on failed attempts, retries after errors, and loops that produce no user value. Poorly tuned retry logic or uncontrolled agentic loops can waste 10–20% of your token budget.

The TCO formula

Total Cost of Ownership (annual) =

Then divide by annual successful agent executions to get real cost per useful task:

This differs from "cost per token" or "cost per execution" because it includes all the hidden expenses and only counts tasks that produce value.

Worked example: a code-review agent

Let's build out a realistic scenario for a team running an AI code-review agent in production. This agent reviews pull requests, checks for common issues, and suggests refactors. The team is 30 engineers, and ~50 PRs are opened per day (or ~1,500 per month).

Component 1: Production token spend

Component 2: Evaluation token burn

The team maintains a regression test suite of 800 test cases (real PRs with known good answers). They run it before every release (2×/week), plus on-demand when debugging failures. Each test case consumes ~4,000 tokens end-to-end.

Key observation: Evaluation tokens (2.765K/year) exceed production tokens (308/year) by ~9×. This is typical for code-review agents where correctness matters and model updates need validation.

Component 3: Observability infrastructure

The agent logs every review request + response to a tracing backend (OpenTelemetry), metrics to Prometheus, and stores structured logs for audit. Infrastructure cost breakdown:

Component 4: Platform staffing

The team has 1.5 FTE platform engineers (a lead + a junior, shared across 3 agents) dedicated to: agent framework maintenance, model updates, debugging, performance tuning, and integration with the CI/CD pipeline. Allocated staffing cost per agent:

Component 5: Model drift correction

When Claude ships a new version (2–3× per year), the team runs the full eval suite against it and benchmarks performance. If performance dips, they run targeted re-evals and sometimes fine-tuning experiments. Estimated annual cost:

Component 6: Compliance and audit overhead

The team's company is in fintech and subject to compliance audits. For each agent, they log: who triggered it, inputs, outputs, reasoning chain, timestamp. This requires:

Component 7: Retry and loop waste

The agent is configured to retry failed reviews up to 2 times before escalating to a human. In practice:

Annual TCO and cost per useful task

Putting it all together:

The team runs 1,500 reviews/month; 92% are successful on first pass, yielding ~16,560 useful reviews per year. Cost per useful task:

The leverage points: where to cut

Reduce evaluation frequency and coverage. The example spends $2,765/year on evals — 9× production tokens. If the team runs evals 2×/month instead of 2×/week, they save ~$2,070/year. But cut too much and you miss drift or regressions. The minimum is probably 1–2 eval runs per release, not on-demand.

Consolidate observability tooling. Running a full OpenTelemetry stack, Prometheus, and custom audit logging costs $5,760/year. Many teams use a SaaS platform (Datadog, New Relic) instead, which costs $500–1,000/month but includes all three. For a single agent, SaaS is cheaper; at 3+ agents, self-hosted infrastructure wins because you share the cost across agents.

Share platform staffing. The biggest lever is staffing. One agent with 1.5 dedicated FTE is expensive; 10 agents with a shared 3-FTE team is much cheaper per agent. But this only works if the team operates a common framework that agents can plug into. The crossover is around 8–15 agents where dedicated platform investment starts paying for itself.

Automate drift detection. Instead of running full evals on every new model release, run a fast smoke test (10% of test cases) first. If that passes, assume the model is safe. If it fails, escalate to a full eval. This cuts model drift cost from $4,700 to ~$1,000/year.

Set retry budgets and escalate early. Poorly configured retry loops can waste 10–20% of tokens. Set a hard retry limit (2 attempts max) and escalate to humans. The token waste is small in this example ($22/year) because retries are already capped.

Batch compliance audits. Instead of preparing audit data on demand, run a batch job quarterly. This shifts from $600/year of manual work to $200/year of automated ETL. Requires upfront investment in audit log infrastructure but pays for itself quickly.

When to build vs. buy agent infrastructure

The staffing cost in the example ($80K/agent/year) suggests a buy-vs-build decision:

For 1–5 agents: Buy. Use an existing platform (Langchain, Anthropic's Agents API, or a SaaS wrapper). Your agent can't justify a dedicated platform team, so outsourced infrastructure is cheaper.

For 5–15 agents: Evaluate. A 2-FTE platform team (~$240K/year) can support 8–15 agents, pushing per-agent staffing cost down to $16–30K/year. At 10 agents, that's a $50–70K savings vs. buying a SaaS for each. But only if you build a reusable framework; ad-hoc infrastructure per agent is expensive and doesn't scale.

For 15+ agents: Build. A shared platform pays for itself and becomes a competitive advantage. Teams at scale (50+ agents) invest in proprietary frameworks and dedicated platform teams.

Measuring your own agent TCO

Start here:

• Production tokens: Query your LLM provider's dashboard or billing API. Break down by agent and model.
• Evaluation tokens: Track eval runs in a spreadsheet. Multiply (test cases × tokens per case × runs per month × 12). Don't estimate; measure for one month and extrapolate.
• Observability cost: Audit your cloud and SaaS bills. Tag expenses by agent. If you don't tag, use allocation: (observability cost) × (agent traffic / total traffic).
• Platform staffing: Ask your engineering leadership. Get a load factor (% of time each engineer spends on agent infrastructure). Allocate fairly across agents.
• Drift correction & compliance: Use the examples above as a starting point. Your numbers will differ based on industry and release cadence.
• Retry waste: Log every LLM call with (attempt_number, success). Calculate (tokens where attempt > 1) / (total_tokens).

Once you have a baseline TCO, prioritize cuts by impact. Staffing is usually 50%+ of cost; observability and evals are the next levers. Focus there first.

Related

• Agent economics: how much does a coding agent actually cost? - subscription vs. API pricing for user-facing agents.
• Agent spend attribution - how to track costs across multiple agents and projects.
• Eval cost allocation - breaking down evaluation spend by model, task, and test suite.
• Hidden LLM costs - infrastructure, retry chains, and observability overhead beyond tokens.

Ready to audit your agent TCO? FinOps LLM runs a free analysis of your LLM spend—production, evaluation, infrastructure, staffing—and shows where to cut. Book free audit →

Back to research