Quick answer: A prompt-cache dashboard can tell you that reuse fell. It cannot, by itself, tell you which request change caused the fall or how much it cost. OpenAI's September 8, 2026 release made Prompt Cache...

Find the cache miss behind your OpenAI bill

Updated September 26, 2026 · first published September 26, 2026

A prompt-cache dashboard can tell you that reuse fell. It cannot, by itself, tell you which request change caused the fall or how much it cost. OpenAI's September 8, 2026 release made Prompt Cache Diagnostics generally available in the Responses API for GPT-5.6 and later supported models. The useful FinOps move is to turn a drop in cached tokens into a short, repeatable investigation.

Compare the request that missed

Save the ID of a recent, completed response whose prefix you expected to reuse. On the next comparable Responses API request from the same organization, set prompt_cache_options.comparison_response_id to that ID. Then read prompt_cache_diagnostics on the new response, alongside usage.input_tokens_details.cached_tokens. The comparison option requests a diagnosis; it does not load the earlier conversation or change cache behavior. OpenAI's diagnostics guide includes working request examples.

const next = await client.responses.create({
  model: model,
  instructions: stableInstructions,
  input: nextInput,
  tools: stableTools,
  prompt_cache_options: { comparison_response_id: baseline.id }
});
console.log(next.prompt_cache_diagnostics);
console.log(next.usage.input_tokens_details.cached_tokens);

The snippet assumes baseline is a recent completed response and the other variables are your own request data. Keep a stable prefix long enough to cache: OpenAI documents a 1,024-token minimum for GPT-5.6 and later. A small or radically different prompt is not a meaningful cache-miss test.

Fix the cause, not the metric

A cache_miss result may point to a changed model, service tier, tool definitions or ordering, cache key, response format, reasoning effort, verbosity, or compacted context. For example, a harmless-looking tool-schema rename can invalidate the reusable prefix. Compare request configuration and the earliest prompt bytes, restore stability where the change was accidental, and rerun the comparison against the same baseline. OpenAI reports the first cause it finds, so another cause may appear after the first fix.

Do not force a hit if the change was deliberate. A different model may lower total task cost even while losing cache reuse; compaction may reduce future context. Judge the entire request and task, not cache-hit percentage in isolation. An expired baseline or an unavailable result is inconclusive, not proof that caching failed.

Translate the finding into money

cache_missed_tokens estimates the reusable tokens lost relative to the comparison response. It is not a billed-token count. A cache_hit result likewise does not establish dollar savings. For realized input cost, collect each response's total input_tokens, cached_tokens, and cache_write_tokens, then apply the current rate for that model and processing tier. For GPT-5.6 and later, OpenAI's prompt-caching guide states that cache reads cost 0.1 times the uncached input rate and cache writes cost 1.25 times that rate.

ordinary = input_tokens - cached_tokens - cache_write_tokens
weighted_input = ordinary + 0.1 * cached_tokens + 1.25 * cache_write_tokens
input_cost = weighted_input * input_price_per_million / 1_000_000

Those are exclusive token categories: do not add a cache-write surcharge to tokens already counted at the write rate. This formula covers input only; include output, tool, retry, and other charges when calculating cost per successful task. Use the provider's current price table rather than a hard-coded article price.

A useful weekly check

  1. Group comparable traffic by workload, model, and service tier; chart cached-token share and input cost per completed task.
  2. Sample a sudden regression, compare it with a recent baseline, and record the diagnostic reason and owning code change.
  3. Fix accidental prefix or configuration drift; keep intentional quality or routing changes if total task economics improve.
  4. Validate the result across representative production traffic and reconcile the observed savings with billing.

Diagnostics themselves have no extra feature fee, but extra test requests are billed normally. The payoff is not a prettier hit-rate graph. It is a defensible explanation for why a particular workload's input cost changed—and whether the proposed fix really lowered its bill.

Questions teams ask

Does a cache-hit diagnosis prove a request was cheaper?

No. It says no miss was detected against the selected baseline. Check actual cached_tokens and the request's priced token categories before claiming savings.

Can diagnostics compare any two OpenAI requests?

No. Use a recent completed baseline from the same organization, and only expect this workflow on supported GPT-5.6-and-later Responses API models. A comparison record can expire.

Should every cache miss be removed?

No. A model switch, different service tier, or compacted context can be intentional. Compare quality, latency, and cost per successful task before reversing the change.

Related

Want this applied to your own LLM spend? FinOps LLM runs a free audit of your AI costs and shows where the savings are. Book free audit →

Back to research