Chargeback when your provider won't bill per team
19 July 2026
The cleanest path to LLM chargeback is billing at the credential boundary. Give each team their own API key, each gets its own invoice line, finance is happy, done. But the world is messier. Everyone shares one org key. The provider invoice is a single undifferentiated number. The credential boundary does not match the team boundary. This is the single most common blocker to AI chargeback in practice. Here are the four ways to work around it, with honest tradeoffs for each.
1. Key-per-team (simplest but sprawling)
Create an API key for each team and route their requests through it. At request time, select which key to use based on headers, metadata, or request context. Each team gets their own provider invoice line.
Pros: Billing is automatic. Finance sees one line per team with no allocation math. No shared infrastructure to maintain. The provider view is clean and matches your org structure perfectly.
Cons: Key sprawl. Hundreds of keys to rotate, audit, and revoke. Quota limits fragment—a team with a low quota cannot borrow unused quota from teammates. Team-level rate limits mean one team cannot spike into another's allocation. Key rotation at scale becomes a rotation incident. If teams are microservices or ephemeral (short-lived jobs, containers), managing per-team keys becomes infeasible. Cold-start jobs fight over quota because they all spin up at once.
When to use it: You have fewer than ~20 teams and keys live long (not ephemeral). Teams are stable and well-known. Quota isolation between teams is a business requirement, not just a nice-to-have.
2. Gateway or proxy (best fidelity, hot-path risk)
Sit a stateless reverse proxy between your application and the provider. Every request passes through it. The proxy tags each request with team, feature, and environment metadata—either extracted from headers or injected by the caller—then logs the tagged request with its usage before forwarding. The proxy selects which API key to use (or uses a shared key) and passes the request through.
Pros: Request-level fidelity. You see team, feature, environment, and workload metadata for every call. Log format is consistent because it's enforced at one point. Requests are tagged before any retry or caching layer, so you capture failure modes. No application code needs to know about chargeback; the proxy is transparent. You can filter, rate-limit, or audit at the gateway layer.
Cons: Sits in the hot path. Every request latency is affected; a poorly tuned gateway adds tail latencies or becomes a bottleneck. Logging overhead (writes to the log pipeline) directly impacts throughput. If the gateway fails, all LLM traffic fails. Key rotation, credential management, and quota handling happen here too—bugs are production incidents. Requres deployment and SRE overhead. Testing is harder because you must test the full path including the proxy.
When to use it: You have traffic volume and latency sensitivity that can absorb a few milliseconds of proxy overhead. You have SRE capacity to maintain an additional service in the hot path. Request-level attribution is essential for your business model (per-customer or per-feature billing, or precise chargeback). Your log pipeline is reliable enough that every request tag reaches your data warehouse without loss.
3. Application-level telemetry with OpenTelemetry GenAI (safe, incomplete)
Do not route traffic through a proxy. Instead, instrument the application layer that calls the LLM provider. Use OpenTelemetry GenAI conventions (or equivalent structured logging) to tag every request with team, feature, endpoint, and workload metadata. Send those tags to your observability platform alongside the provider usage data. Match requests to usage by correlation ID and timestamp.
Pros: No proxy to maintain. Application layer already has the metadata (team from auth context, feature from code path, endpoint from URL). Does not sit in the hot path; tags are written asynchronously to telemetry backends. Failures in the telemetry pipeline do not break LLM requests. Teams can instrument at different rates—gradual rollout is safe. Standard OpenTelemetry format means tools and training exist; not a custom solution.
Cons: Only covers instrumented code. Legacy codebases or third-party SDKs that call the LLM provider without instrumentation are invisible. Async telemetry means you may miss requests on application crashes. Matching requests to provider usage requires correlation IDs and clock sync; off-by-one errors in timestamp matching cause orphaned costs. Incomplete coverage means a portion of spend is always unattributed and needs statistical fallback. Instrumentation drift—if teams update code without updating tags—breaks showback downstream.
When to use it: Your codebase is new, well-structured, and under your control. You can require instrumentation as part of the LLM caller contract. Workload is not deadline-critical, so async telemetry overhead is fine. You have a mature observability platform (Datadog, New Relic, Honeycomb) that speaks OpenTelemetry natively. Incomplete coverage is acceptable as long as it covers your top spend paths.
4. Statistical allocation (last resort, auditable only with care)
Log which teams made requests, but not their usage. Allocate total provider spend across teams based on request ratios, model popularity, or proxy metrics (token counts from your cache, batch job metadata). Work backwards from the provider invoice.
Pros: Minimal integration. Works with existing shared API keys. Requires no gateway or application changes if you already have team-level request logs. Statistical models are fast and cheap to compute.
Cons: Allocations are rough guesses. One team's expensive reasoning request can vanish into the noise if you allocate by request count. Model pricing changes between invoice periods and attribution lags; you may allocate against stale prices. Retries and fallbacks are invisible—you lose attribution for failed requests. Finance will demand reconciliation and you will be unable to explain why Team A was charged $8,000 for 2,000 requests when the same team elsewhere used 2,000 requests for $3,000. Teams will argue the allocation is unfair. Audit trails are weak; you cannot point to any single request and explain why it was charged to that team.
When to use it: This is genuinely a last resort. Use it only if none of the other three approaches are feasible—perhaps you are in a hard vendor lock-in, or you have a short time window before migrating systems. Even then, use it as a temporary bootstrap and a signal to move away from undifferentiated billing. If you must use statistical allocation, layer it with application-level telemetry for your top 80% of spend, and allocate the remaining 20% statistically. This keeps finance happy without needing perfect coverage.
Reconciliation to the provider invoice
Whatever method you choose, reconcile it back to the raw provider invoice monthly. The total allocated spend across all teams should match (or nearly match) the provider invoice line for that month. Small gaps are normal—rounding, delayed invoice lines, or provider-specific bulk discounts. Large gaps mean your log pipeline is losing traffic, your price tables are stale, or your proxy is dropping requests.
The reconciliation check is not optional. It is your guarantee that chargeback numbers survive finance review. Without it, teams will find discrepancies, audit will find gaps, and you will lose the trust needed to actually charge them.
Set up a monthly report that shows: total provider invoice, total allocated spend, difference in dollars and percentage, and the top 10 unallocated requests (if any). Flag any month where the gap exceeds 5%. This is your early warning system for pipeline rot.
What "good enough" attribution coverage means
Do not chase 100% attribution. It is a trap. Perfect coverage requires perfect instrumentation, perfect telemetry, perfect logging, perfect reconciliation—and the cost of reaching the last 5% often exceeds the value of the accuracy gain.
Aim for 80–90% coverage: the requests that drive 95% of your spend should have clear team and feature attribution. The remaining 10–20% (lower-cost or edge-case requests) can be statistically allocated or left visible as unallocated. This is good enough for showback, good enough for optimization ranking, and good enough for chargeback budgets.
Coverage is measured by dollars, not by request count. A system that attributes 80% of requests by count might only cover 40% of spend if the unattributed requests are expensive. Track coverage by cost, not by volume.
Staged rollout: showback first
Never go straight to chargeback. Start with showback: build your attribution method (whichever of the four you choose), send reports to teams showing their spend, and let them validate the numbers for a month or two. They will find bugs. You will discover missing teams or features. Trust will build.
Only move to chargeback once:
- Attribution coverage is 80%+ of spend and stable month-over-month.
- Teams have seen showback reports and challenged the numbers. You have fixed the obvious bugs.
- Reconciliation to the provider invoice is solid. Finance has audited it.
- Teams understand how to optimize. They cannot see a button labeled "chargeback" and assume they are now blocked from using the LLM.
Even after you move to chargeback, keep the showback reports running in parallel for 3–6 months. Let teams see both the allocation and the charge. If discrepancies appear, you can revert to showback-only mode without losing the entire system.
Related
- LLM chargeback and showback - designing allocation models.
- LLM cost attribution - attribution foundations and dimensions.
- OpenTelemetry GenAI conventions - structured telemetry for LLM workloads.
- What is LLM FinOps? - the complete operating discipline.
Want this applied to your own LLM spend? FinOps LLM runs a free audit of your AI costs and shows where the savings are. Book free audit →