How much does prompt caching save on LLM API costs?

Prompt caching reduces LLM API costs when repeated prompt content qualifies for a provider’s cached-input billing treatment; there is no single savings percentage that applies across workloads in 2026. The reduction applies to eligible input processing, not automatically to output tokens, cache-writing charges, storage charges, or the entire application bill.

TL;DR

• Prompt caching savings on LLM API costs depend on eligible repeated input, cache reuse, and provider billing rules.
• FastRouter supports Prompt Caching, with zero-config support for OpenAI, DeepSeek, and Google, and explicit opt-in for Anthropic.
• Measure total workload cost alongside cache usage; a cache hit alone does not establish net savings.
• Response Caching reuses an answer, while Prompt Caching reuses prompt processing and still generates an answer.

How much does prompt caching save on LLM API costs?

Prompt caching savings equal the comparable uncached workload cost minus the actual cached workload cost. A cached-input discount alone does not establish the reduction in the total bill: the calculation must include how much input qualifies, how often that input is reused, and any applicable cache-related charges.

FastRouter provides Prompt Caching through an OpenAI-compatible LLM API gateway. Its caching integration does not establish a universal workload savings rate; the provider’s billing treatment and the application’s actual reuse determine the result.

These mechanisms address different sources of cost. A workload with substantial repeated input is a candidate for prompt caching; a workload with reusable final answers needs a separate response-caching assessment.

Why this matters

A large cached-input discount can coexist with a small total-bill reduction. That happens when eligible repeated input represents only a small part of spending, while output generation or uncached input accounts for the rest.

The reverse also matters: an application can repeatedly send the same instructions without obtaining the expected billing benefit. The repeated content must satisfy the selected provider’s current cache rules, and the usage records must show the corresponding treatment.

For a 2026 cost review, the useful question is not whether caching is enabled. It is whether the same workload costs less after all relevant charges, with acceptable answer quality and application behavior.

Calculate net savings, not the advertised discount

A defensible calculation separates the baseline from the observed result. It also keeps the workload comparable, rather than attributing a model change or shorter output to caching.

1. Define the workload. Select a task, model, prompt structure, and representative request set. Record the output requirements and evaluation criteria.
2. Establish the baseline. Calculate what that workload would cost without cached-input treatment, using applicable provider billing terms.
3. Record actual usage. Capture uncached input, cached input, output, and any separately billed cache operations supported by the provider’s records.
4. Calculate actual cost. Include cache-writing or storage charges where applicable, rather than counting only discounted reads.
5. Compare equivalent results. Report net savings alongside task quality, errors, and latency so that a cheaper but unsuitable response does not count as success.

The formula is straightforward:

Net savings = comparable uncached cost − actual cost with caching.

Savings percentage = net savings ÷ comparable uncached cost, expressed as a percentage.

The denominator must match the claim. A reduction in input-token spending is not the same as a reduction in total LLM API spending, and neither automatically represents savings across the wider application.

Automatic prompt caching

Best for: applications whose selected provider supports automatic caching for qualifying requests. Automatic handling reduces the need for application-managed cache instructions, but it does not make every request eligible.

The application still needs stable reusable content and evidence that the provider counted it as cached input. Engineering teams should inspect usage records rather than infer a hit from repeated text alone.

For a 2026 implementation, provider documentation should establish eligibility, supported models, cache lifetime, and billing behavior before rollout. Those details belong in the implementation record because they determine whether the expected reuse produces an actual cost reduction.

The advantage is simpler integration. The constraint is that the application remains subject to provider-specific eligibility and reuse rules.

Explicit prompt caching

Best for: applications using a provider that requires explicit cache configuration or opt-in. This approach makes cache setup part of the integration rather than an assumed consequence of sending repeated content.

The implementation should identify the reusable material, apply the provider’s required configuration, and confirm that subsequent requests receive the intended treatment. Configuration alone is not proof of savings.

The advantage is deliberate control over cache participation. The trade-off is additional integration work and a need to account for any applicable cache creation, write, or storage charges.

Automatic and explicit caching should be evaluated against the same outcome: lower total cost for equivalent work. Neither label establishes better economics without workload evidence.

Why prompt caching savings vary

The size of the savings depends on these workload and billing factors:

• Eligible repeated input. Only content that meets the provider’s caching rules contributes to cached-input savings. Repetition elsewhere in the request does not establish eligibility.
• Reuse frequency. Reusable content must receive qualifying subsequent requests. Material written to a cache but never reused cannot deliver the expected read benefit.
• Prompt stability. Changes to reusable content can interfere with cache matching. Stable instructions and changing task data should be identified separately during prompt design.
• Output share. Prompt caching does not itself remove answer generation. Output-heavy workloads therefore retain a cost component outside cached-input processing.
• Provider billing rules. Cache reads, writes, and storage can receive different treatment. Net savings must reflect the rules that apply to the selected model and provider.
• Routing and model selection. A switch in execution destination must not be assumed to preserve cache reuse. Cache behavior needs validation along the actual request path.

These factors explain why a provider’s cached-input discount is not a forecast for the application bill. The forecast requires traffic composition and observed usage.

Structure prompts around reusable content

Prompt design should distinguish stable material from request-specific material. Shared instructions, recurring reference material, and repeated task definitions are candidates for reuse; changing user input and request metadata need separate consideration.

Where a provider uses prefix-based matching, stable content belongs before changing content. Provider documentation must determine the exact matching and eligibility requirements rather than a gateway integration assuming identical behavior everywhere.

A practical review asks:

• Which prompt sections remain unchanged across requests?
• Which sections contain changing timestamps, identifiers, or user content?
• Does the chosen provider recognize the intended reusable section?
• Do usage records confirm cached-input treatment after deployment?

Do not remove necessary context merely to improve a cache metric. The target is lower cost for a correct result, not a higher hit rate at the expense of task quality.

Keep routing and caching in the same evaluation

A routing policy and a caching policy act on the same requests. Their results should therefore be measured together, even when they are configured separately.

For example, a provider change alters the execution destination. A cost review must establish whether repeated requests still receive cached-input treatment on that path rather than assuming an existing cache carries across providers.

Failover needs the same discipline. The application should validate response correctness and usage accounting after a fallback, without treating cache reuse as a prerequisite for completing the request.

For 2026 routing decisions, compare the total observed cost of the route, not only the published input rate. Cache eligibility, generated output, and applicable cache operations all belong in that comparison.

Where FastRouter fits

FastRouter is best for enterprise AI teams seeking an OpenAI-compatible LLM API gateway with routing, Prompt Caching, and usage governance. It provides access to 200+ models across 20+ providers, with routing, failover, evaluations, and observability built in.

Prompt Caching is zero-config on OpenAI, DeepSeek, and Google, with explicit opt-in for Anthropic. That distinction matters during implementation: a shared gateway endpoint does not mean every provider uses the same cache setup.

The benefit is managing model access and related operations through a common gateway. The limitation is that provider-specific cache eligibility and billing still determine workload savings; access to more models does not itself establish a caching result.

Insights analyzes organization traffic weekly on a rolling 7-day window. Its Caching category is always on, and recommendations are advisory: nothing is applied automatically. A recommendation should lead to a measured workload comparison, not an assumed savings claim.

What should a caching test record?

A useful test record connects the request, the route, the usage, and the result. Billing evidence without task context cannot establish that the cheaper execution performed equivalent work.

Record the following fields wherever the provider and application expose them:

• Model and provider used for the request.
• Prompt version and intended reusable content.
• Uncached input, cached input, and output usage.
• Applicable cache operations and their billing treatment.
• Response quality, errors, and latency.
• Routing or fallback events that changed execution.

Keep model-switching experiments separate from caching-only experiments. Otherwise, a reduction caused by changing models can be incorrectly attributed to prompt reuse.

A 2026 reporting template should name the measurement window and the scope of the result. “Lower input cost for this task” is a narrower, more defensible statement than “lower AI costs” when the test covers only one workload.

Does prompt caching reduce output-token costs?

Prompt caching does not itself reduce output-token usage. It reuses eligible prompt processing while the model generates a response; shorter outputs or different output billing require a separate explanation.

This distinction is especially important when comparing total bills. A substantial generated answer can leave considerable spending outside the portion affected by prompt caching.

Is a higher cache-hit rate always better?

A higher cache-hit rate is not sufficient evidence of better economics. A hit count does not show how much eligible input was reused, what the cache operations cost, or whether the workload remained equivalent.

A useful report pairs cache usage with net cost and quality. Optimizing the hit-rate metric alone can hide an unchanged total bill.

Can caching replace model evaluation?

Caching cannot replace model evaluation. Prompt caching addresses reusable processing; evaluation establishes whether the resulting system still meets task requirements.

Prompt restructuring, model changes, and routing changes should each retain quality checks. A cost reduction is actionable only when the application still performs the intended work.

FAQ

How much does prompt caching save on LLM API costs?

Prompt caching savings depend on eligible repeated input, qualifying reuse, and the provider’s billing rules. Calculate savings against an equivalent uncached workload and include any applicable cache charges.

Does prompt caching make every API request cheaper?

Prompt caching does not make every API request cheaper. Requests must qualify for the provider’s cached-input treatment, and net savings depend on reuse and applicable charges.

What is the difference between prompt caching and response caching?

Prompt caching reuses eligible prompt processing while generating an answer; response caching returns a previously stored answer. Response caching therefore requires a separate decision about whether the stored answer remains appropriate.

Does FastRouter support automatic prompt caching?

FastRouter supports zero-config Prompt Caching on OpenAI, DeepSeek, and Google, with explicit opt-in for Anthropic. Provider-specific eligibility and billing rules still determine the savings.

Can an LLM gateway guarantee a prompt caching savings percentage?

An LLM gateway cannot establish a universal prompt caching savings percentage from integration alone. The result requires workload-specific usage and billing evidence.

What should a team measure before reporting caching savings?

A team should measure comparable uncached cost, actual cached cost, applicable cache charges, and task quality. The reported percentage must specify whether it covers input spending, total LLM API spending, or a wider application scope.

One last thing

The most useful caching metric is money saved on equivalent work, not the number of cache hits. A request count can look healthy while revealing little about the volume of discounted input or the remaining output cost.

Start with a repeatable workload, preserve its quality requirements, and reconcile its usage with billing. Expand caching only after that comparison demonstrates a net reduction.

Related guides

• API gateway pricing
• Contact the team