Prompt Caching: The Free Insights Lever With Nothing to Lose

FastRouter's traffic already knows where your savings are. The question has always been who has time to go looking. Insights, found in the dashboard, runs a weekly, read only analysis of your organization's actual requests and turns them into evidence backed recommendations, each with a projected saving, the evidence behind it, and exactly where to make the change. Prompt Caching is one of two recommendation types included in free insights, on by default, at no cost to generate.

What the Caching Recommendation Detects

Insights looks for a system prompt prefix that repeats across many requests on the same key, where prompt caching is not currently in use. A long system prompt, a static tool schema, or a repeated RAG payload sent unchanged on most calls is exactly the pattern it is built to catch, and exactly the pattern that is easy to miss inside normal request volume.

Analysis runs per Key + Model scope, and a scope needs at least 1,000 text generation requests in the past 7 days to qualify. Within a qualifying scope, your requests are bucketed into hourly windows across the measurement period and grouped by system prompt hash.

How the Saving Is Modelled

For every hour containing a repeated prefix, the first request is priced as a cache write and every subsequent request as a cache read, using the relevant provider's published cache multipliers. Cache reads run up to 90% off. That modelled cost is compared against what you actually paid, and the saving shown is the sum across every hourly bucket in the window:

// per hour, per repeated prefix

actual = requests × prefix_tokens × input_rate

cached = prefix_tokens × write_rate

+ (requests − 1) × prefix_tokens × read_rate

saving = actual − cached

What a Recommendation Shows You

A caching card names the model and key it applies to in plain language, the projected weekly saving, and the evidence behind it: how many repeated prefixes were found, how often they repeat, and the cost gap between what you are paying now and what caching would bring it down to. From there it tells you exactly what to do: mark the stable prefix with cache_control, and enable prompt caching on that key.

Prompt Caching: The Free Insights Lever With Nothing to Lose PS 1

Prompt Caching: The Free Insights Lever With Nothing to Lose PS2

Applying the Recommendation

Every figure on a caching card comes from that week's actual traffic, so what you are looking at is your usage, not an estimate. Because caching leaves the model, the prompt, and the output completely unchanged, it is the lowest risk of Insights' recommendation types to act on. The main variable is timing, not risk. Cache entries expire, so a prefix repeated many times but spread thinly across the week saves less than the same volume concentrated into bursts.

For zero config providers, OpenAI, Google AI Studio, Google Vertex AI, DeepSeek, Grok, Moonshot AI, and Baseten, caching activates automatically once a request crosses that provider's minimum token threshold. Anthropic Claude requires an explicit cache_control marker, since it does not cache automatically, which is exactly the change a caching recommendation points you toward.

Where This Fits Into Routing Intelligence

Caching recommendations are the clearest, lowest friction expression of Routing Intelligence at FastRouter: a system that keeps re-reading your traffic every week, at no cost to run, and hands you back a specific, evidenced action instead of a dashboard you still have to interpret.

Learn more: