Prompt caching ROI and attribution

Published 1 August 2026

Prompt caching is the most misreported optimization in LLM cost management. Teams enable it, see cache-read tokens on their invoice, and claim 50% savings. The real figure is usually half that. The mistake is counting cache-read tokens as full savings instead of calculating the actual discount.

This article shows how to calculate real savings, attribute them to the right workloads, and decide whether caching is worth the complexity for your use case.

How prompt caching actually works

Providers cache the prefix of your prompt: system instructions, few-shot examples, retrieved context. If the next request shares that prefix, the provider charges a discounted rate for the repeated portion. The discount varies by provider:

The economics are not intuitive. Anthropic charges a premium to write to cache, so the first request is more expensive. Savings only materialize on subsequent requests within the TTL window. OpenAI and Gemini charge full price for cache writes, so the first request costs the same as without caching.

Calculating real savings

The formula is:

Savings = (input_tokens_without_caching × input_price)
        - (cache_write_tokens × cache_write_price)
        - (cache_read_tokens × cache_read_price)
        - (output_tokens × output_price)

A concrete example. A support summarization endpoint sends 4,000 input tokens per request: 3,000 tokens of system prompt + 1,000 tokens of user ticket. Without caching, at $2.50 per 1M input tokens:

• Cost per request: 4,000 × $2.50/1M = $0.010
• 1,000 requests/day: $10.00

With caching, 70% of requests share the 3,000-token system prompt. OpenAI charges $1.25 per 1M cache-read tokens:

• 300 requests (miss): 4,000 × $2.50/1M = $0.010 each
• 700 requests (hit): 3,000 × $1.25/1M + 1,000 × $2.50/1M = $0.00625 each
• Daily cost: (300 × $0.010) + (700 × $0.00625) = $3.00 + $4.375 = $7.375
• Real savings: 26%, not 50%

The 50% figure would come from counting all cache-read tokens as free. That is wrong. Cache reads are discounted, not free, and cache misses still cost full price.

Attributing savings correctly

Cache savings belong to the workload that generates the cache hits. If you share a system prompt across support, sales, and engineering endpoints, the savings belong to each endpoint proportionally to its hit rate.

Attribution requires per-request records with:

• cache_write_tokens (full price)
• cache_read_tokens (discounted price)
• cache_hit (boolean or hit rate)
• feature / endpoint

Without this split, you cannot answer: "Did the support endpoint save money, or did the eval suite?" The answer matters because you optimize the endpoint with the worst hit rate, not the one with the most tokens.

When caching is not worth it

Three cases where caching adds cost instead of saving it:

Low hit rate. Below 30%, the cache write premium (Anthropic) or TTL expiry (all providers) eats the discount. Short, unique prompts do not benefit.

Short prompts. A 500-token prompt saves fractions of a cent per request. The engineering time to implement and monitor caching exceeds the savings.

Anthropic with low volume. Anthropic charges a 25% premium on cache writes. If your volume is low and your hit rate is below 50%, the first-request premium exceeds subsequent discounts.

Measuring hit rate

Hit rate is the percentage of requests that share a cached prefix with a previous request within the TTL. Measure it per endpoint, not globally:

SELECT
  feature,
  COUNT(*) AS total_requests,
  SUM(CASE WHEN cache_read_tokens > 0 THEN 1 ELSE 0 END) AS cache_hits,
  SUM(cache_read_tokens) AS total_cache_read,
  AVG(cache_read_tokens * 1.0 / input_tokens) AS avg_prefix_ratio
FROM llm_usage
WHERE date >= CURRENT_DATE - 7
GROUP BY feature
ORDER BY cache_hits DESC

A healthy cache shows hit rate >50% and prefix ratio >0.6 (most of the input is cached). A hit rate below 30% means the TTL is too short or the prompts are too unique.

Provider-specific gotchas

OpenAI. Cache is automatic for prompts >1,024 tokens. No explicit cache control. TTL is short (5-10 minutes), so batch workloads with gaps between requests lose the cache.

Anthropic. Explicit cache control via `cache_control` markers. You choose what to cache. This is powerful but requires code changes. The 25% write premium means you need hit rate >55% to break even on a two-request sequence.

Gemini. Context caching is explicit and has a 1-hour TTL. Good for batch jobs that run within the hour. The 75% discount on cache reads is the most aggressive, but the setup is the most manual.

Decision framework

Use this checklist before enabling caching:

1. Is the average input prompt >1,000 tokens? If no, skip.
2. Is the hit rate likely >30%? If no, skip.
3. Is the workload latency-sensitive? If yes, cache reads also reduce latency — a secondary benefit.
4. Can you measure cache_write_tokens and cache_read_tokens separately? If no, implement metering first.
5. Is the provider Anthropic? If yes, require hit rate >55% to justify the write premium.

If you answer yes to all, caching is likely your highest-ROI optimization. If you answer no to any, model routing or prompt compression is probably a better first lever.

Related

• Prompt caching explained - the mechanics of how providers cache prefixes.
• LLM usage metering and billing - how to meter cache tokens correctly.
• Semantic cache economics - caching semantically similar prompts, not just exact prefixes.
• Model routing - when routing beats caching as the first lever.

Want this applied to your own LLM spend? FinOps LLM runs a free audit of your AI costs and shows where the savings are. Book free audit →

Back to research