Prompt caching and batch routing: two discounts most teams never turn on

Two structural cost reductions exist inside the AI providers themselves, available to any account that qualifies, and most teams never enable either one — not because the discounts aren't worth having, but because nobody has flagged which specific features actually qualify. Prompt caching discounts a repeated static prompt prefix. Batch routing discounts workloads that don't need a real-time response. Neither requires a model change or a quality tradeoff.

The reason they go unused isn't technical difficulty — both are well-documented provider features. It's that finding out which of your features actually match the pattern each discount requires means looking at real call data, not something you can determine by reading a feature's spec.

Prompt caching: paying once for a prefix you send every time

Prompt caching lets you pay a fraction of the normal input price when the beginning of your prompt is identical to one sent recently. Anthropic's cache discount runs around 90% off the normal rate on cached reads for models like Claude Sonnet — a $3.00-per-million-token input rate drops to roughly $0.30 per million on the cached portion. OpenAI's prompt caching is automatic for any prompt prefix of 1,024 tokens or more that's identical across calls, at roughly a 50% discount on the cached tokens. The exact mechanics differ by provider — Anthropic requires an explicit cache-control marker on the content block you want cached; OpenAI applies it automatically with no code change.

The feature this benefits most is exactly the one described in the context tax guide — a large, mostly-static system prompt sent on every call. Cognocient flags features where the same large system prompt appears in more than half of calls, with the projected monthly saving from enabling caching attached directly to the recommendation, rather than leaving it to a manual audit to notice the pattern.

Batch routing: 50% off for workloads nobody is waiting on

Batch APIs accept a job, process it within a turnaround window — typically up to 24 hours — and charge roughly half the synchronous price in exchange. That tradeoff is irrelevant for a chat interface where a user is watching a response stream in. It's essentially free money for anything that runs in the background: nightly report generation, bulk contract or document analysis, data enrichment and classification pipelines, evaluation and QA scoring runs, and embedding generation for search indexes.

The pattern that flags a feature as batch-eligible is timing, not task type: calls triggered by a cron job or webhook rather than a live user request, with no one waiting on the response in real time. Cognocient's recommendations surface this directly — a feature where the large majority of calls are non-interactive is flagged with the projected monthly saving from routing to the batch API instead of the synchronous endpoint.

Why this requires detection, not just documentation

Both discounts are publicly documented by every major provider — this isn't hidden information. The obstacle is that “which of our fifteen features has a stable-enough system prompt to benefit from caching” and “which of our features are actually triggered by a scheduler rather than a user” are questions about real call patterns, not questions a team can answer by re-reading the provider's pricing page. Cognocient's Recommendations engine analyzes actual call history to answer both questions automatically, per feature, with the dollar saving attached to each flag.

That evidence-first structure matters because it turns “we should probably enable prompt caching at some point” into a specific, prioritized action: “enable caching on support-chat — a 12,000-token system prompt repeated 4,800 times last month, projected saving $1,240/month” is something a team can act on the same day it's surfaced.

What this looks like in practice

Consider a hypothetical nightly report-generator feature that summarizes the day's activity for internal distribution — triggered entirely by a scheduled job, with no user ever waiting on its response. That timing pattern surfaces as a batch-routing recommendation with the specific projected saving attached, since nothing about the feature requires a synchronous response in the first place.

Separately, a support-chat feature on the same account carries a large, stable system prompt repeated on nearly every call — a caching recommendation with its own projected saving, entirely independent of the batch-routing flag on the other feature. Two different discounts, two different features, two different mechanisms — both invisible without looking at real call data. This is illustrative of how the recommendations behave, not a specific customer's reported outcome.

Both discounts are structural fixes, distinct from token maxing — a feature can have the right model tier and still be leaving both of these discounts unused. See the full guide to the five AI spend waste categories for how caching and batch routing fit alongside model mismatch and the other patterns.