LLM cost dashboard: what to put on it
A good LLM cost dashboard answers two questions at a glance: where is the money going, and is anything about to break the budget. Most dashboards fail because they show one big number - total spend - and stop there. Total spend tells you that costs went up; it never tells you why.
What belongs on the dashboard
- Spend by dimension - broken out by team, product surface, environment, and model, not just one grand total.
- Cost per request and per successful task - the unit economics that reveal whether growth is efficient or just expensive.
- Cache-read ratio - the share of input tokens served from cache, the clearest signal of input-cost efficiency.
- Budget burn - spend against a monthly budget, with projected month-end based on run rate.
- Anomalies - sudden spikes by team or model, surfaced rather than buried in a chart.
Separate the estimate from the truth
An LLM cost dashboard has two layers, and conflating them is the most common mistake. Gateway and proxy logs give a near-real-time estimate - fast, useful for catching spikes the same day, but approximate. The provider invoice is the truth - slow, but authoritative. Show both, label which is which, and reconcile the estimate against the invoice each month so the dashboard keeps finance's trust.
Two audiences, two views
Engineers need request-level granularity to act - which endpoint, which model, which call pattern. Finance needs roll-ups they can put in a forecast - spend by team, trend, budget variance. A single dashboard can serve both, but design the views separately rather than forcing one audience to read the other's numbers.
Common mistakes that make a dashboard misleading
- Averaging cache-read and input tokens together, which hides the biggest cost lever.
- Tracking money as floating-point, which lets cent-level drift accumulate into reports.
- Showing spend with no owner - a dashboard with low attribution coverage looks precise but cannot drive a single decision.
- Reporting only lagging invoice data, so spikes are discovered weeks after they happen.
Related
- LLM cost monitoring - the monitoring practice behind the dashboard.
- LLM cost attribution - how the per-dimension breakdowns are built.
- Anomaly detection for AI spend - surfacing spikes automatically.
- LLM budget governance - turning the burn chart into budgets and alerts.
Want this applied to your own LLM spend? FinOps LLM runs a free audit of your AI costs and shows where the savings are. Book free audit →
FAQ
What should an LLM cost dashboard show?
Spend broken out by team, product surface, environment, and model; cost per request and per successful task; cache-read ratio; budget burn with a projected month-end; and surfaced anomalies. A single total-spend number tells you costs went up but never why.
Why separate the estimate from the invoice on a cost dashboard?
Gateway and proxy logs give a fast, near-real-time estimate that is useful for catching spikes the same day but approximate. The provider invoice is the slow but authoritative truth. Show both, label which is which, and reconcile monthly so the dashboard keeps finance's trust.
What makes an LLM cost dashboard misleading?
Averaging cache-read and input tokens together, tracking money as floating-point, showing spend with no owner (low attribution coverage), and reporting only lagging invoice data so spikes are found weeks after they happen.