What is prompt caching? How it works on Claude, GPT and Gemini

As of November 2026, prompt caching means the model provider stores the processed prefix of a prompt and reuses it on any later request that starts with the same tokens. Anthropic bills cache reads at 0.1x the input price by default (0.05x on Opus 5.5 and Sonnet 5.5, 0.025x on Fable 5.1 and Mythos 5.1). OpenAI's GPT-5.6 and later charge 1.25x to write a cache that lives 30 minutes by default; older GPT models cache for free. Gemini caches implicitly, with explicit caches on most models.

How does prompt caching work?

Every request to a large language model starts with a prefill step, where the model reads all the input tokens and builds an internal state for each one (the key-value cache, or KV cache). Token, in this context, means a chunk of text of roughly three quarters of a word. Prefill is the slow, expensive part of a long prompt, and in an agent or chat app most of that prompt is the same on every call: the system prompt, the tool definitions, the documents, the conversation so far.

Prompt caching keeps that KV cache around for a while. When the next request arrives, the provider compares it to what it has stored, token by token from the start. The tokens that match are read from the cache at a discount; the provider then runs prefill only on the tokens after the first difference.

That exact-match rule decides almost everything about how you structure prompts. Put the stable content first (tools, system prompt, reference documents) and the content that changes last (the new user message, retrieved snippets, timestamps). A date string at the top of the system prompt turns every request into a miss, and nobody notices until they look at the bill.

Prompt caching is a different thing from semantic caching, which returns a stored answer when a new question looks similar to an old one. We compare the two in prompt caching vs semantic caching.

How does Anthropic prompt caching work?

Anthropic makes you ask for caching. You mark the end of a cacheable block with cache_control, and the API caches everything from the start of the request up to that point. Per Anthropic's prompt caching docs, a request can carry up to 4 breakpoints, and the cache prefix is built in a fixed order: tools, then system, then messages.

That order is the cache key in practice. Change a tool definition and the system prompt and messages caches are invalidated too, because they sit after the tools in the prefix. Change only the latest message and the tools and system prompt still hit.

import os
import anthropic
 
client = anthropic.Anthropic()
MODEL = os.environ["CLAUDE_MODEL"]
 
response = client.messages.create(
    model=MODEL,
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": open("system_prompt.md").read(),
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[{"role": "user", "content": "Summarise the open tickets."}],
)
 
print(response.usage.cache_creation_input_tokens)
print(response.usage.cache_read_input_tokens)

The two usage fields at the bottom are the ones to log. cache_creation_input_tokens is what you paid the write premium on; cache_read_input_tokens is what you got at the read price. If the read number stays at zero across a session, the prefix is changing somewhere.

Now, the pricing. Anthropic's pricing page expresses caching as multipliers on each model's base input price. The write multipliers are the same on every model; the read multiplier is lower on the newest ones:

The 5 minute TTL refreshes every time the cache is read, so a busy agent can keep one prefix warm for hours at the read price. The 1 hour option is for gaps longer than 5 minutes: a user who thinks before replying, or a batch job that runs every 20 minutes.

There is also a minimum prefix length per model, listed in the caching docs. It is 512 tokens on Opus 5, Opus 5.5, Sonnet 5.5, Fable and Mythos, and up to 4,096 tokens on Opus 4.5, Opus 4.6 and Haiku 4.5. Below the minimum, the request is processed normally and nothing is cached, with no error.

How does OpenAI prompt caching work?

OpenAI changed the billing model with GPT-5.6, so the answer depends on which generation you call. Both generations cache automatically, both report hits in usage.prompt_tokens_details.cached_tokens, and per OpenAI's prompt caching guide the prefix has to be at least 1,024 tokens.

One setting helps on every model. prompt_cache_key is a string you send with the request, and OpenAI uses it together with the start of the prompt to route requests to the same machine. The Prompt Caching 201 cookbook recommends one key per user or per conversation and warns that a single key receiving very high request rates spills onto extra machines, which lowers hits.

GPT-5.6 and later: a write fee and a longer TTL

Per OpenAI's latest model guide and the GPT-6 caching announcement, GPT-5.6 and later bill cache writes at 1.25x the uncached input rate and cache reads at a discount set per model (check the pricing page for yours). Caching still happens implicitly, or you can mark explicit breakpoints with prompt_cache_breakpoint, up to 4 per request, the same pattern as Anthropic's cache_control.

Retention moved to prompt_cache_options.ttl, which replaces prompt_cache_retention on these models. The default is 30 minutes. A 12 hour TTL, added on October 20, 2026, bills writes at 2.5x. The usage object now reports cache_write_tokens next to cached_tokens, so you can see both sides of the trade.

The break-even works out like Anthropic's: a 30 minute write at 1.25x plus one read beats sending the prefix twice at full price for any read price below 0.75x, so one reuse pays off. A 12 hour write at 2.5x needs two reads at 0.1x to win (2.7x against 3x uncached).

GPT-5.5 and earlier: automatic and free to write

On older models there is no write fee and no code change. The discount started at 50% when OpenAI launched caching in 2024 and reached 90% on later models, matching in 128-token increments. Because writes cost the normal input price, a miss costs exactly what an uncached request costs.

Retention here is in memory by default and cleared after a few minutes of inactivity. Extended retention, set with prompt_cache_retention on supported models, keeps prefixes longer; the guide lists which models support it.

The structure advice is the same on every generation: static instructions, examples and tool schemas first, user-specific content last. Tool definitions and structured output schemas are part of the prefix, so reordering them between calls is a miss.

How does Gemini prompt caching work?

Google calls it context caching and offers two modes, documented in the Gemini API caching guide.

Implicit caching is on by default for current Gemini models. When a request repeats a recent prefix, the discount applies automatically and shows up in usage_metadata.cached_content_token_count. There is no setup and no storage fee, but also no guarantee: a hit depends on the request landing where the prefix is still held.

Explicit caching is the guaranteed version. You create a cache object holding the content (a long PDF, a codebase, a video), give it a TTL (1 hour if you set nothing), and reference it by name in later requests. Every request that uses it gets cached-token pricing, and the trade is a storage fee, billed per million cached tokens per hour and listed per model on the Gemini pricing page. Gemini 4 Argon supports implicit caching only, so explicit caches are not available there.

Minimum cache sizes are larger than on Anthropic or OpenAI: 2,048 tokens on the Gemini 2.5 models and 4,096 tokens on Gemini 3.x and 4 models. Explicit caching pays off when many requests hit the same large document within the TTL; for a chatbot with a shared system prompt, implicit caching usually does the job for free.

How do the three providers compare?

For the per-model dollar figures (Opus vs Sonnet vs GPT vs Gemini Pro and Flash), see our cache read price comparison across providers.

What does caching save on an agent loop?

Take a coding agent, the kind of workload where caching does the most work. Every turn resends the tools, the system prompt and the whole conversation so far, plus whatever the last tool call returned.

Our assumptions (all Requesty assumptions, not measurements):

• The fixed prefix (tool definitions plus system prompt) is 20,000 tokens.
• Each turn adds 1,500 new tokens (the previous reply plus the new tool output and instruction).
• A session runs 30 turns, each less than 5 minutes after the last, so the 5 minute cache never expires.
• The model is an illustrative Claude model with a base input price of $3 per million tokens and the default 0.1x read multiplier. Swap in your own model's price; the percentages hold for any base price as long as reads are 0.1x.
• Output tokens are left out, since caching does not change them.

Without caching, turn k sends 20,000 + 1,500 × (k - 1) tokens. Over 30 turns that is 30 × 20,000 + 1,500 × 435 = 1,252,500 input tokens, which at $3 per million is $3.76 per session.

With a breakpoint at the end of each request, turn 1 writes the 20,000-token prefix. Every later turn reads everything the previous turn sent and writes only the new 1,500 tokens. That gives 1,189,000 tokens read at $0.30 per million ($0.36) and 63,500 tokens written at $3.75 per million ($0.24), for $0.59 per session. Input cost drops by ~84%, and almost all of what is left is the write premium on new tokens.

On a model with 0.05x reads (Opus 5.5 or Sonnet 5.5 at the same illustrative price), the reads cost $0.18 instead of $0.36, the session comes to $0.42 and the saving rises to ~89%.

Now, the break-even. A 5 minute write costs 1.25x and each read saves 0.9x, so one read already wins (1.25 + 0.1 = 1.35x, against 2x uncached). A 1 hour write at 2x needs two reads to pay off (2 + 0.2 = 2.2x, against 3x uncached). The only losing case is a cache that is never read: if every turn in our session arrived more than 5 minutes apart, all 1,252,500 tokens would be writes at $3.75 per million, or $4.70, which is 25% worse than not caching.

Even if we double the new tokens per turn to 3,000 (bigger tool outputs, longer replies), the uncached session costs $5.72 and the cached one $0.94, a saving of ~84% again. The longer the conversation grows relative to the prefix, the more of each request is old history, and old history is exactly what gets read from cache. At 1,000 sessions a day, the base case is the difference between ~$3,760 and ~$590 a day in input tokens.

How does Claude Code use prompt caching?

Claude Code leans on prompt caching harder than almost any other client: its system prompt and tool definitions are large and identical on every turn. Anthropic's Claude Code caching docs describe it placing breakpoints automatically, and the DISABLE_PROMPT_CACHING environment variable exists to turn that off, which you almost never want.

Which TTL does Claude Code use?

The same docs set the cache TTL by how you pay, with subagents on their own bucket:

So a developer on an API key who stops to read a diff for six minutes comes back to a cold cache and pays the write again. From v2.1.242, you can change this: the promptCacheTtl setting and the CLAUDE_CODE_PROMPT_CACHE_TTL environment variable set the main conversation TTL, with subagent equivalents listed on the same page. ENABLE_PROMPT_CACHING_1H asks for the 1 hour TTL where you would otherwise get 5 minutes, and FORCE_PROMPT_CACHING_5M does the reverse.

Pick with the break-even above in mind: 1 hour writes cost 2x, so they only pay off if your gaps between turns regularly exceed 5 minutes. To see how you are doing, /usage or the status line shows the cache hit ratio for the session.

Why does Claude Code miss the cache through a gateway?

The problem shows up when Claude Code talks to anything other than Anthropic's own API: a gateway, a proxy, a cloud provider or a local model. Recent versions put an x-anthropic-billing-header text block at the very start of the system prompt, and part of its value changes between requests. Any backend that caches by exact prefix sees new first tokens on every call, and per the prefix rule above, a change at the start is a miss on everything.

GitHub issue reports date the block to versions from around v2.1.29 to v2.1.36 onward, and the claude-code-cache-fix project documents users seeing near-zero cache reads because of it. Requests to Anthropic's own endpoint are not affected, per Claude Code's LLM gateway protocol docs; the misses hit gateways, proxies and local models.

The fix is one variable, documented in Claude Code's environment variable reference: CLAUDE_CODE_ATTRIBUTION_HEADER=0. Despite the name, the block is part of the request body (inside the system prompt), not a transport header, so the setting takes effect on the client and travels through any gateway in front of the model. Put it in the env block of settings.json, since reports differ on whether a plain shell export is picked up.

1. 1
   Point Claude Code at your gateway

   In the env block of your Claude Code settings.json, set ANTHROPIC_BASE_URL to the gateway and ANTHROPIC_AUTH_TOKEN to your key. For Requesty that is https://router.requesty.ai. Claude Code appends /v1/messages itself; the /v1 suffix (https://router.requesty.ai/v1) is for OpenAI-compatible SDKs.

2. 2
   Turn off the attribution block

   Add "CLAUDE_CODE_ATTRIBUTION_HEADER": "0" to the same env block so every session picks it up.

3. 3
   Check the cache reads

   Run a session of 5 or more turns, then check /usage or the status line for the cache hit ratio, or cache read tokens in your gateway's analytics. Reads should climb with every turn once the first turn has written the prefix.

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://router.requesty.ai",
    "ANTHROPIC_AUTH_TOKEN": "your-requesty-api-key",
    "CLAUDE_CODE_ATTRIBUTION_HEADER": "0"
  }
}

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://router.requesty.ai",
    "CLAUDE_CODE_ATTRIBUTION_HEADER": "0",
    "ENABLE_PROMPT_CACHING_1H": "1"
  }
}

The project version leaves the key in each developer's user settings and asks for the 1 hour TTL, which suits a team on API keys with long pauses between turns. What we do not know is which upstream providers strip the attribution block on their side, so setting the variable to 0 is the safe default everywhere.

If you run Claude Code against non-Anthropic models, the same setting applies; we cover that setup in how to use Claude Code with DeepSeek, GLM and Kimi.

So what?

The teams that save the most are the ones resending the most: coding agents, support bots with long policy documents, RAG apps that stuff the same reference material into every call, and anything with long multi-turn conversations. A short one-shot prompt saves nothing, because below the minimum prefix (512 tokens on the newest Claude models, 1,024 on OpenAI, 2,048 to 4,096 on Gemini) no provider caches it.

What to do this week:

1. Log the cache fields (cache_read_input_tokens, cached_tokens, cache_write_tokens, cached_content_token_count) next to total input tokens and compute a hit rate per endpoint.
2. Move every timestamp, user ID and retrieved snippet out of the system prompt and to the end of the request.
3. If you call GPT-5.6 or later, budget for the 1.25x write and check that each cached prefix is read at least once in its 30 minute window.
4. On Anthropic, check whether your gaps between requests exceed 5 minutes; if they do, test the 1 hour TTL against the break-even above.
5. If you run Claude Code through any gateway or proxy, set CLAUDE_CODE_ATTRIBUTION_HEADER to 0 in settings.json.

If you route traffic through Requesty, caching sits in the gateway. Auto caching applies prompt caching for you, and manual caching lets you control the cache_control blocks yourself. The dashboards show cache savings per model and per team, which is the hit-rate number from step 1 without writing the logging yourself. For per-team and per-customer views of that spend, see how to track LLM costs per user, team or customer, and if you are still deciding whether a gateway belongs in your stack, start with what an LLM gateway is.