Does JSON mode cost extra?

Not as a surcharge, but as tokens. The schema is input tokens on every call, the braces, quotes and field names are billed output tokens, and every validation failure costs a full retry of both sides.

How do you reduce structured output overhead?

Cache the schema by keeping it in a stable prompt prefix, request only fields you actually consume, use short field names and flat structures, and track your validation failure rate.

Do long field names really matter?

On a low-volume feature, no. On a million calls a day, every extra token in a field name is generated and billed on every response, so verbose naming becomes a permanent recurring charge.

Quick answer: Asking a model for structured output looks like a formatting choice. On the bill it is a volume choice, and it lands in three places at once.The schema is an input cost on every requestA response...

The token cost of JSON mode and schemas

Updated September 1, 2026 · first published September 1, 2026

Asking a model for structured output looks like a formatting choice. On the bill it is a volume choice, and it lands in three places at once.

The schema is an input cost on every request

A response format or tool definition is serialised into the request. A modest schema — a dozen fields, descriptions on each, a couple of nested objects — is a few hundred tokens. That is small until you multiply it by every call, on every turn, for the life of the feature. A 400-token schema on a million calls is 400 million input tokens you are paying for in order to describe a shape that never changes.

Worse, schemas usually sit after the volatile part of the prompt in naive implementations, which breaks the cached prefix. The schema then costs full price every time and destroys the discount on everything before it.

Syntax is an output cost

Every brace, quote, comma and field name in the response is a generated token you are billed for at the output rate. Verbose field names are a recurring charge: customer_shipping_address_line_one costs more, forever, than addr1. Deeply nested objects pay for their own scaffolding. For short answers wrapped in a large envelope, the structure can genuinely exceed the content.

Retries are the expensive part

Constrained decoding on the major providers makes schema violations rare, but validation failures still happen — a field the model cannot fill honestly, an enum with no right answer, a schema that is ambiguous to a reader. Each failure costs a full retry: all input tokens again, all output tokens again, plus latency. A 2% invalid rate on a high-volume endpoint is a 2% cost increase with nothing to show for it.

What to actually do

Put the schema in the stable prefix so it caches, and keep it identical across calls. Ask only for fields you consume — extra fields are paid for twice, once in the schema and once in the response. Prefer short field names and flat structures. Where the output is a single value, do not wrap it in an object at all. And measure your validation failure rate: it is the one part of this that is pure waste.

Want this applied to your own LLM spend? FinOps LLM runs a free audit of your AI costs and shows where the savings are. Book free audit →

Back to research