Context tax: the recurring cost of resending the same system prompt every call
Context tax is the token cost paid repeatedly for a large, mostly unchanging system prompt sent with every API call. When a feature's system prompt makes up 80–90% of its total input tokens on every request, that static portion is a tax in the literal sense — the same cost, paid again and again, for information that almost never changes between calls.
It's easy to miss because nothing about it looks wrong in isolation. Each individual call is billed correctly for the tokens it sent. The waste only becomes visible when you look across hundreds or thousands of calls to the same feature and notice that the input token count barely moves — which means the model is reading nearly the same block of text, at full price, every single time.
Where it comes from
RAG-based features are especially prone to this pattern — sending large reference documents, tool schemas, or policy text with every query dramatically inflates token count per call, even when the user's actual question is a single short sentence. A support-bot feature with a 10,000-token system prompt covering product docs, tone guidelines, and escalation policy pays that 10,000 tokens on every single turn of every conversation, whether the user asked a one-word question or a paragraph-long one.
The design choice that causes it is usually reasonable on its own — a large, comprehensive system prompt genuinely improves answer quality and consistency. The problem isn't that the prompt is large. It's that the same large block gets billed at full input price on every call, when only a small fraction of it — the part the model actually needs to reference for a given question — differs from the last call.
How to actually detect it
The signal isn't the size of any single call's prompt — a large prompt on its own might be entirely justified. The signal is variance: measure the ratio of static tokens (the portion that's identical call to call) to variable tokens (the portion that actually changes, like the user's question), across many calls to the same feature. Low variance in total input token count — the number barely moves even though the user's question length obviously does — means most of what's being billed is the same fixed block, not new information.
Cognocient's Context Tax Analyser calculates this ratio automatically per feature from real call history, and quantifies the exact saving available from enabling prompt caching on the static portion — typically a 60–80% reduction on that portion of input cost. See the context tax glossary entry for the underlying definition.
What prompt caching actually changes
Prompt caching is a provider-side feature that discounts the price of a prompt prefix that's identical to one sent recently — the model provider caches the processed representation of that prefix instead of reprocessing it from scratch. The first call in a cache window pays full price to write the cache; every call after that, within the cache's lifetime, pays a substantially discounted rate to read it. It reduces context tax significantly. It does not remove the design pattern that caused it — the static block is still being sent on every call, it's just billed differently once cached.
That distinction matters for how you evaluate whether caching is worth enabling for a given feature: it helps most when the static prefix is genuinely large and genuinely stable call to call, and helps least when the “static” portion actually changes more often than it looks like it should — a system prompt with a timestamp or a per-user variable embedded near the top invalidates the cache far more often than the feature owner might expect.
What this looks like in practice
Consider a hypothetical document-QA feature built on a 3,200-token system prompt containing product documentation and answer-formatting instructions. Across a month of calls, the total input token count per call varies by only a few dozen tokens — almost entirely explained by the length of the user's question, not by any change in the reference material. That pattern is the signature the Context Tax Analyser is built to catch: a consistently large base with low variance around it.
Enabling caching on the static portion of that prompt turns most of the 3,200 tokens from a full-price cost on every call into a discounted cache-read cost on every call after the first in each cache window — without changing the prompt itself or the quality of the answers it produces. This is illustrative of how the mechanism behaves, not a specific customer's reported outcome.
Context tax and token maxing are separate waste categories with separate fixes, and a feature can carry both at once — an expensive model with a large static prompt pays twice, once for the model tier and once for the prompt size. See the full guide to the five AI spend waste categories for how they fit together, and the guide to prompt caching and batch routing for how to actually enable caching provider by provider.