LLM cost tracking
LLM cost tracking is the practice of collecting, tagging, and aggregating the data you need to explain every dollar of AI spend. Without it, the monthly invoice is just a number. With it, you can answer who spent what, on which model, for which feature, and whether it was worth it.
What to tag on every request
The minimum viable attribution set is:
- Team or cost center — who owns the workload.
- Project or feature — what the workload supports.
- Model and provider — which model was called and through which provider.
- Environment — production, staging, or development.
- Token type — input, output, cache-read, cache-write, reasoning.
- Request context — user ID, session ID, or task ID for tracing.
Where the data comes from
There are three sources, and each has gaps:
- Provider invoices — the ground truth for dollars, but coarse-grained and delayed.
- Gateway or proxy logs — rich per-request detail, but may miss retries, fallbacks, and reasoning tokens.
- Application telemetry — links cost to business outcomes, but needs instrumentation.
Reconcile all three. Use the invoice as the final number, gateway logs for attribution, and application telemetry for cost-per-task.
Metrics that matter
- Cost per request — the average cost of a single API call.
- Cost per task — the cost to complete a business outcome.
- Spend concentration — which endpoints and teams drive most of the cost.
- Variance — actual spend vs. budget, by week or by feature.
Getting started
Start with gateway logs if you use a proxy like LiteLLM, Helicone, or Langfuse. Add custom tags to your requests and export the data to a dashboard. If you do not use a proxy, instrument your application to log model, tokens, and task ID on every call. See also our guides to LLM cost management and LLM cost monitoring.
Want this applied to your own LLM spend? FinOps LLM runs a free audit of your AI costs and shows where the savings are. Book free audit →
FAQ
How do you track LLM costs?
Track LLM costs by tagging every request with team, project, feature, model, provider, and environment, then aggregating provider invoices and gateway logs into a dashboard that shows spend, tokens, and cost per task.
What metrics should you track for LLM costs?
Track total spend, tokens by type, cost per request, cost per task, latency, error rate, and spend concentration across teams and features.
What is the best tool for LLM cost tracking?
The best tool is the one your team will actually use. Options range from provider dashboards and gateway logs for small teams to observability platforms and custom pipelines for larger deployments.