Prompt Caching

When your requests start with the same instructions, tools, or conversation history, prompt caching lets the model provider reuse that work. This can lower input costs and make responses start sooner. The model still generates an answer for each request.

The repeated beginning of a prompt is its prefix. Keep it unchanged and put new questions or other changing content after it.

Use the section for the API your application already calls. Supported OpenAI, Azure OpenAI, and Gemini models cache automatically. Claude needs a cache_control field, as shown below.

INFO

The examples show request bodies. Replace the context placeholders with real content: Claude Sonnet 4.6 needs at least 1,024 tokens before the cache boundary; Opus 4.6 needs 4,096. Other models have different minimums. A short example can return an answer without caching anything.

Chat Completions — `/v1/chat/completions`

For Claude, add cache_control next to model. This asks to cache through the last eligible part of the prompt. As you append conversation turns, the cache boundary moves forward with them.

{
  "model": "claude-sonnet-4-6",
  "cache_control": { "type": "ephemeral" },
  "messages": [
    { "role": "system", "content": "<long, stable instructions>" },
    { "role": "user", "content": "Today's question" }
  ]
}

The default Claude cache lifetime is 5 minutes. For supported OpenAI, Azure OpenAI, and Gemini models, automatic caching needs no cache_control field.

Cache only a stable part of the prompt

A breakpoint marks where the cached prefix ends. Put cache_control on a content block when you want to reuse fixed instructions while changing the question after them. The marked message's content must be an array.

{
  "model": "claude-sonnet-4-6",
  "messages": [
    {
      "role": "system",
      "content": [
        {
          "type": "text",
          "text": "<long, stable instructions>",
          "cache_control": { "type": "ephemeral" }
        }
      ]
    },
    { "role": "user", "content": "Today's question" }
  ],
  "tools": [
    {
      "type": "function",
      "cache_control": { "type": "ephemeral" },
      "function": { "name": "lookup", "parameters": { "type": "object" } }
    }
  ]
}

This example also marks a tool definition, which Claude supports. Claude's prefix order is tools, system instructions, then messages; a marker includes all earlier content in that order.

Claude accepts up to 4 cache points. You can combine block markers with the top-level field, which uses one slot if it adds a new point. LLM API removes the earliest markers when there are too many. If the final block is already marked, that marker takes precedence over the top-level field, including its lifetime.

Provider support and limits

A successful request does not guarantee that its cache controls were used. LLM API translates or omits them according to the selected model and provider.

Provider	Top-level `cache_control`	Block markers
Anthropic and Claude on Azure	Forwarded	Supported blocks and tools
Cache-capable Claude on Bedrock	Forwarded, or converted to a marker on the last eligible block	Supported blocks and tools
Supported Nova models on Bedrock	Converted to one checkpoint	One checkpoint, with placement limits below
OpenAI with explicit cache support	Automatic caching stays enabled	Last 3 eligible content markers; tool markers and their lifetimes are omitted
Other automatic-caching models	No added control	Omitted
Models without prompt-cache support	Omitted	Omitted

OpenAI: GPT-5.6 and later support explicit breakpoints. Translation must also be enabled for the selected model and provider in LLM API. Use cache_control on this endpoint: native OpenAI fields such as prompt_cache_key, prompt_cache_retention, prompt_cache_options, and client-supplied prompt_cache_breakpoint are not passed through. Use /v1/responses for those controls.

Nova: current default support covers Micro, Lite, Pro, and Nova 2 Lite. Only the last eligible checkpoint survives. Without tools, it can follow system or message content. With tools, only system checkpoints are supported: message and tool markers are omitted. A top-level marker with tools requires system content. A "1h" lifetime causes the entire marker to be omitted, rather than changed to "5m".

System checkpoints on Nova can produce repeated writes without reads; check the read count before relying on them for savings. These LLM API restrictions can be narrower than the native Bedrock API.

Omitted cache controls do not produce a client-facing warning. Controls that are forwarded still undergo provider validation. On Claude, put longer-lived markers first: a "1h" marker after a "5m" marker can fail the request.

Anthropic Messages — `/v1/messages`

Use Anthropic's cache_control fields with your existing Messages request. This example marks the reusable system instructions and leaves the question outside the cache boundary.

{
  "model": "claude-sonnet-4-6",
  "max_tokens": 1024,
  "system": [
    {
      "type": "text",
      "text": "<long, stable instructions>",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [{ "role": "user", "content": "Today's question" }]
}

LLM API preserves your cache controls on this endpoint. It does not convert or trim them as it does on Chat Completions.

WARNING

Using Claude on Bedrock? Use block-level markers, as in the example. Some Bedrock paths reject or ignore top-level cache_control; this endpoint does not convert it for you.

Other marker locations and validation rules

Anthropic accepts markers at the top level, on tool definitions, on system blocks, and on message content blocks, subject to the upstream model's support. A top-level marker asks for caching through the last eligible block.

At most 4 cache points are allowed; excess points can return 400.
Put "1h" markers before "5m" markers.
Top-level caching needs a free slot unless the final block is already marked with the same lifetime. A conflicting lifetime can return 400.

For the full contract, see Anthropic's prompt-caching guide.

Responses — `/v1/responses`

Supported OpenAI models cache automatically. You do not need to add a field to start benefiting from repeated prefixes.

For GPT-5.6 and later, use the following example when you want to cache only the stable context. mode: "explicit" turns off the automatic breakpoint; the marked content is eligible for caching, and the changing question after it is not written to the cache.

{
  "model": "gpt-5.6-sol",
  "prompt_cache_options": { "mode": "explicit", "ttl": "30m" },
  "input": [
    {
      "role": "user",
      "content": [
        {
          "type": "input_text",
          "text": "<long, stable context>",
          "prompt_cache_breakpoint": { "mode": "explicit" }
        },
        { "type": "input_text", "text": "Today's question" }
      ]
    }
  ]
}

This endpoint uses OpenAI's cache fields. Do not send Anthropic's cache_control: LLM API does not translate it here, and OpenAI can reject it.

OpenAI cache fields and model differences

LLM API forwards these fields without translating or trimming them. The upstream model validates them.

prompt_cache_key: a stable key helps earlier models route related requests to a cache. On GPT-5.6 and later it separates cache accounting. It does not guarantee a hit.
prompt_cache_retention: earlier models support "in_memory", "24h", or a subset of those values. Defaults depend on the model and the upstream organization's retention policy.
prompt_cache_options: on GPT-5.6 and later, mode is "implicit" (default) or "explicit". The only ttl is "30m", also the default.
prompt_cache_breakpoint: on GPT-5.6 and later, add { "mode": "explicit" } to a supported message content part. Up to 4 cache writes are allowed; implicit mode reserves one, leaving 3 explicit write slots.

Explicit-only mode with no breakpoints produces neither cache reads nor writes. Top-level instructions and tool definitions cannot carry breakpoints. To mark reusable instructions, put them in an input_text block in a developer message.

Earlier models do not support the newer breakpoint and options fields. GPT-5.6 and later use prompt_cache_options instead of prompt_cache_retention. Unsupported combinations can return a provider validation error. See OpenAI's prompt-caching guide.

Check that caching worked

Send a request with enough unchanged context to meet your model's minimum.
Wait for the response, then repeat the request promptly. For a conversation, you can instead append a turn while keeping the earlier messages unchanged.
Inspect the second response's cache read count. A number above zero means the provider reused part of the prompt.

If LLM API response caching is enabled, turn it off for this check so both requests reach the provider.

Look inside the response's usage object:

Endpoint	Read count	Write count
Chat Completions	`cached_tokens`	`cache_creation_tokens`
Messages	`cache_read_input_tokens`	`cache_creation_input_tokens`
Responses	`cached_tokens` inside `input_tokens_details`	`cache_write_tokens` inside `input_tokens_details`

Writes mean content was saved; reads mean it was reused. The first request may have writes but no reads. Zero-valued optional fields may be absent. See the request's cost breakdown for the amount charged.

Cache writes split by lifetime

For Claude on Chat Completions, usage may also include cache_creation_5m_tokens and cache_creation_1h_tokens. OpenAI's 30-minute writes use the aggregate cache_creation_tokens field.

On Messages, Claude reports the split inside usage.cache_creation as ephemeral_5m_input_tokens and ephemeral_1h_input_tokens.

Cost and cache lifetime

Caching pays off when a saved prefix is reused. A write can cost more than ordinary input, so repeatedly saving content that nobody reuses can increase your bill. Output tokens are still charged normally.

Claude: the default lifetime is 5 minutes. Add "ttl": "1h" inside cache_control for one hour where supported. Provider write rates are 1.25× ordinary input for 5 minutes and 2× for one hour. Reads commonly cost 0.1×, with lower rates on some models.
OpenAI GPT-5.6 and later: the lifetime is 30 minutes. Writes cost 1.25× ordinary input, including automatic writes. Read rates depend on the model.
Earlier OpenAI models: there is no extra write premium; retention and read rates depend on the model.
Nova through LLM API: use the default or "5m"; "1h" markers are omitted.

Successful reuse refreshes the lifetime on Claude and OpenAI GPT-5.6 and later. For your bill, use the selected model's LLM API price and request cost breakdown. Provider references: Claude pricing and OpenAI pricing.

When does a longer lifetime save money?

At a 0.1× read rate, one 5-minute Claude write plus one full read costs 1.35× the prefix's ordinary input price, versus 2× for processing it twice without caching. One read is enough to save money.

A 1-hour write plus one read costs 2.1×. It needs more reuse to save money; choosing a longer lifetime does not automatically make caching cheaper.

No cache hits?

Check the length. The prefix before the breakpoint must meet your model's minimum. The total request length alone is not enough.
Keep the prefix stable. Timestamps, request IDs, reordered tools, and changed instructions can prevent reuse. Put changing content after an explicit breakpoint.
Check the timing. Wait for the first response before testing reuse, then send the next request before the entry expires.
Check support. A model may not support your marker's location or lifetime. Expand the provider limits above; a successful request can still be uncached.
Check routing. Different models or providers do not share a cache. Selecting provider/model narrows routing but does not guarantee a hit.

Prompt caching is separate from response caching, which replays a saved answer without calling the model. You do not need to enable response caching to use prompt caching, and replaying an answer does not refresh the provider's prompt cache.

For current model minimums and provider-specific behavior, see Claude, OpenAI, Bedrock, and Gemini.