How do I save up to 75% with prompt caching and batch routing?
Two structural cost reductions most teams leave on the table: prompt caching (up to 75% off cached reads) and async batch routing (50% off non-real-time jobs). Cognocient AI detects opportunities automatically and shows them on the Recommendations page. (New feature)
Prompt caching discounts repeated static content in your prompts by up to 75%. Batch routing saves 50% on non-real-time jobs. Cognocient AI detects opportunities for both automatically and surfaces them on the Recommendations page.
| Discount | Value | Detail |
|---|---|---|
| Anthropic prompt cache discount | 75% | on cache_read tokens |
| OpenAI prompt cache discount | 50% | on cached prefix reads |
| Batch API discount | 50% | gpt-4o, claude-sonnet, and more |
Prompt caching
Prompt caching is a feature on Anthropic and OpenAI that lets you pay a fraction of the normal price when the beginning of your prompt is identical across calls. If you have a 10,000-token system prompt that stays the same, only the first call charges you full price — subsequent calls with the same prefix pay the cached read rate.
How cached tokens are costed (since 2026-10-28). Cognocient AI reads the cache counts each provider reports: OpenAI-compatible cached_tokens, Anthropic cache_read_input_tokens and cache_creation_input_tokens, and Bedrock cacheReadInputTokens and cacheWriteInputTokens. Cache reads are costed at the cache-read rate, cache writes at the cache-write rate, and only the rest of the prompt at the full input rate. Before this date, cached OpenAI tokens were costed at the full input rate and Anthropic cache tokens weren't counted at all, so cached traffic now shows a more accurate (OpenAI lower, Anthropic higher) cost. Earlier calls keep the cost recorded at the time. For Anthropic calls, prompt_tokens in the response now includes cached tokens, matching OpenAI's convention.
1-hour cache writes (since 2026-10-30). Anthropic charges cache writes with the 1-hour lifetime ("ttl": "1h") at 2× input, not the 1.25× of the default 5-minute lifetime. Cognocient AI reads the split Anthropic reports (cache_creation.ephemeral_1h_input_tokens; cacheDetails on Bedrock) and charges each at its own rate. Before this date every cache write was charged at the 5-minute rate. The cache-write prices in the table below are the 5-minute ones.
| Provider | Cache model | Normal input | Cache write | Cache read | Saving |
|---|---|---|---|---|---|
| Anthropic | claude-sonnet-4-6 | $3.00 / MTok | $3.75 / MTok | $0.30 / MTok | 90% on cache reads |
| Anthropic | claude-haiku-4-5 | $1.00 / MTok | $1.25 / MTok | $0.10 / MTok | 90% on cache reads |
| OpenAI | gpt-4o | $2.50 / MTok | $2.50 / MTok | $1.25 / MTok | 50% on cache reads |
| OpenAI | gpt-4o-mini | $0.15 / MTok | $0.15 / MTok | $0.075 / MTok | 50% on cache reads |
Cognocient AI analyses your call patterns and flags features where the same large system prompt appears in more than 50% of calls. On the Recommendations page you'll see: "Enable Anthropic prompt caching on 'support-chat' — saves $1,240/month. Large system prompt (12,000 tokens) repeated 4,800 times last month."
Automatic prompt caching
Anthropic only caches what a request marks with cache_control, and most code never marks anything. Turn on Settings → Safety & Limits → Add Anthropic prompt caching automatically (or PATCH /api/account/prompt-cache with {"prompt_cache_mode": "auto"}) and Cognocient AI adds the marks for you on Claude calls to the Anthropic API, both through /v1/chat/completions and /v1/messages:
- the last tool definition, which caches the tool list
- the end of the system prompt, which caches tools and system prompt together
- on multi-turn requests only, the last block of the conversation, so the next turn reads the conversation so far from cache
Single-turn prompts are never marked: a cache write costs 1.25× input and a one-off prompt is never read back. A request that already has any cache_control is left exactly as sent, and a request under about 1,024 tokens is skipped. The response header x-cog-prompt-cache: auto; breakpoints=N shows when marks were added. To choose per request, send X-Cog-Prompt-Cache: auto or off.
It is off by default, on every plan. Check the result in cache reads and writes on each call. It does not yet cover Claude on Bedrock or Vertex AI.
Enabling prompt caching — Anthropic
Anthropic requires you to add cache_control: {type: 'ephemeral'} to the content blocks you want cached. Cognocient AI passes this through and records the cache_read_input_tokens at the discounted rate.
Enabling prompt caching — OpenAI
OpenAI prompt caching is automatic — no code change needed. OpenAI caches any prompt prefix of 1,024+ tokens that is identical across calls. Cognocient AI tracks and reports on cached_tokens from the usage object.
Batch routing (50% off async workloads)
Batch APIs let you submit jobs that complete within 24 hours at half the normal price. If your workload doesn't need a real-time response — report generation, bulk analysis, nightly summarisation — batch routing is the single largest cost lever available.
| Model | Sync price | Batch price | Max turnaround |
|---|---|---|---|
gpt-4o | $2.50 / MTok in | $1.25 / MTok in | 24h |
gpt-4o-mini | $0.15 / MTok in | $0.075 / MTok in | 24h |
claude-sonnet-4-6 | $3.00 / MTok in | $1.50 / MTok in | 24h |
claude-haiku-4-5 | $0.80 / MTok in | $0.40 / MTok in | 24h |
Batch-eligible workloads
| Workload | Detail |
|---|---|
| Report generation | Nightly AI summaries, weekly analytics reports |
| Bulk document analysis | Processing uploaded PDFs, contract review |
| Data enrichment | Tagging, classification, extraction at scale |
| Evaluation pipelines | Automated QA scoring, model evals |
| Email summarisation | End-of-day digest, CRM enrichment |
| Embedding generation | Building vector indexes, semantic search |
Do not use batch routing for features where users wait for a response (chat, search, generation in the UI). Batch responses take minutes to hours. It is designed for background processing pipelines only.
Batch API example — OpenAI
Finding batch-eligible features
On the Cognocient AI Recommendations page, features eligible for batch routing are flagged with: "report-generator: 89% of calls are non-interactive (triggered by cron/webhook, not user action). Switching to Batch API would save $640/month." Cognocient AI detects non-interactive calls by analysing call timing patterns and the presence of job-scheduler headers.
Related articles