Engineering 8 min read· 1 Aug 2026· By Engineering
Prompt caching — save 50% on repeated context
How cache hits work, when they trigger, and how to structure your prompts to maximize savings.
If your app resends the same system prompt or RAG context on every turn, prompt caching cuts input cost by 50-90%.
How it works#
The provider hashes the leading tokens of your prompt. If the same prefix arrives within the cache window (5 min for GPT, 1 hour for Gemini), input tokens for that prefix are billed at a fraction of the normal rate.
Structure your prompt for cache hits#
- Static system prompt at the top
- RAG context next (largest static block)
- Dynamic user message last
- Never interleave static and dynamic — put dynamic at the tail
Cached input tokens are marked in
X-Cache-Hit-Tokens on the response for observability.