How do I enforce LLM budget limits before the API call?
LLM budget enforcement means blocking, degrading to a cheaper model, or alerting on AI spend before it exceeds a limit — enforced at the proxy, before the provider bills you. Set per-feature, per-department, and per-run budgets.
LLM budget enforcement means stopping AI spend before it happens, not flagging it after. Cognocient AI checks every API call against your configured budgets before the call reaches the provider. Set a budget in the dashboard and choose your enforcement mode: Block (hard stop), Degrade (auto-switch to cheaper model), or Alert (allow through, flag it).
Most cost tools read billing exports that are 24–48 hours late. Cognocient AI checks budget before the call reaches the provider — so overspend is impossible, not just notifiable. The check is a single atomic Redis operation that runs in sub-millisecond time on every request.
What are the three budget enforcement modes?
| Mode | What happens when the limit is hit | Best for |
|---|---|---|
| Alert | Call goes through, notification sent | Baseline measurement, new features |
| Block | HTTP 429 returned — provider never sees the call | Experiments, agents, dev/test |
| Degrade | Request silently rerouted to a cheaper model | Production features with SLA requirements |
The Base plan supports up to 3 active budgets. Growth and Business plans have unlimited budgets.
How do I create a budget in Cognocient AI?
Go to Dashboard → Budgets → New Budget and fill in:
- Name — descriptive label, e.g.
chatbot feature,Engineering department,Org total - Scope — Global (all traffic), Feature (by
X-Cost-Feature), Department (byX-Cost-Department), User (byX-Cost-User), Environment (byX-Cost-Environment), Key (one proxy key, Growth+), Customer (byX-Cost-Customer, Growth+), Spend group (a named set of people, Growth+) or Model group (a list of models sharing one limit, Growth+) - Resets — every month (default) or every day. A daily budget on a key catches a leaked key or a runaway loop within hours; a monthly cap alone can be drained in an hour and still be "within limits"
- Limit — the spending cap in USD for that period
- Enforcement mode — Alert, Block, or Degrade
- Alert thresholds — default 50%, 80%, 100% — each one alerts once per month by email (every plan) and Slack (Base+); see Alerts
Alert mode
All API calls pass through normally. Cognocient AI sends an alert when each configured threshold is crossed: by email to the account owner by default, and to Slack if connected.
What the alert says (email subject "Cognocient AI: 🟡 Budget Alert — 80%"):
Block mode
When the budget is exhausted, Cognocient AI returns HTTP 429 immediately. The AI provider never sees the request. This applies to every call the budget covers, whatever its X-Cost-Workload tag.
How concurrent calls and streams are counted
Each call's estimated cost is reserved against the budget atomically before the call is let through, then corrected to the real cost when it finishes. If two calls arrive and only one fits, the second gets the 429. Streamed responses have max_tokens lowered when needed so that a single stream can't run past what's left.
If a stream stops before the provider's final usage report (for example, your client disconnects or times out), Cognocient AI charges an estimate rather than nothing: the prompt, plus roughly one token per four characters of output already streamed. The provider still bills for that work, so the budget counts it too.
How big is each reservation?
- The worst case, near the limit. The reservation is the prompt (counted locally, plus a 10% margin because tokenizers differ between providers) plus the full
max_tokensyou allow. If you don't setmax_tokens(ormax_completion_tokens), 4,096 output tokens are reserved, and with a Block budget the forwarded limit is also clamped to what the remaining budget can pay for. - The typical case, far from it. Reserving every call's full
max_tokenscan lock most of a budget in reservations that are never spent when many calls run in parallel, so real calls get refused while actual spend is far under the cap. Once a model and feature have at least 50 successful calls in the last 7 days, Cognocient AI reserves their p95 output length (plus 20%) instead, but only while the budget has more than 50 worst-case calls of room left. Inside that margin it goes back to the full worst case, so the ceiling stays strict at the end. The rare long reply is corrected when the call settles. - Reservations can't leak. Every reservation has a 15-minute lease. If the worker handling a call dies before settling it, the lease expires and the reservation is released, so the budget doesn't slowly lock itself. If that call does settle later, its real cost is still counted, once.
- The right period. A call reserved at 23:59:58 and finished at 00:00:03 is settled against the period it was reserved in, not the next one.
Shrink to fit instead of refusing (optional)
Near the limit, a call is refused when its worst case doesn't fit, even if a shorter answer would. Send X-Cog-Shrink-To-Fit: true and Cognocient AI lowers the call's max_tokens to what the remaining budget can pay for, as long as at least 256 output tokens still fit, and reserves that instead. The response carries x-cog-max-tokens-shrunk (the new limit) and x-cog-max-tokens-original. The answer may stop early with finish_reason: "length", so only opt in where a shorter answer beats none.
Timeouts: when the bill is unknown
If the provider received a call but no response came back (a read timeout, or the connection dropped mid-response), the provider may still have completed and billed it. Cognocient AI charges the call's worst-case estimate instead of releasing it, so a retry can't spend the same money twice. The 504 carries x-cog-billing-outcome: unknown. Calls that never left (the provider couldn't be reached at all) are released in full. Shadow Spend Reconciliation (Growth+) compares these estimates with the provider's own billing.
To stop a retry from opening a second reservation while the first attempt is still running, send an X-Idempotency-Key: a second request with the same key gets a 409 until the first finishes.
If the budget check itself is unreachable
Each budget chooses what happens if the store its counter lives in (Redis) can't be reached:
- Fail open (default): calls go through. Your product stays up, but a loop that starts during the outage isn't capped. Add an outage allowance to cap it: while the store is unreachable, at most that many dollars are let through (counted per Cognocient AI server process), then calls get a 429. Set it with
outage_allowance_usd. - Fail closed: calls are blocked until the check is back.
Optional: cheaper-model fallback allowance
Block is a hard stop by default. If you'd rather some calls keep running on a cheaper model for a while, set a fallback allowance on the budget: an extra dollar amount per month, chosen in advance. After the limit is used up, calls run on the cheaper model from the table in Degrade mode and are reserved against the allowance the same way. Once the allowance is spent, calls get the 429.
- Off unless you set it. A budget with no allowance never falls back.
- Calls tagged
X-Cost-Workload: agentic-writenever use the allowance. A weaker model shouldn't be choosing write actions. - Spend on the fallback still counts toward the budget, so the dashboard shows the overrun.
- Fallback responses carry
x-cog-degraded: trueandx-cog-degraded-reason: budget_exceeded_fallback_allowance.
Set it under Budgets → Create / Edit → Block → Cheaper-model fallback allowance, or with fallback_allowance_usd on POST /api/budgets. On PATCH, 0 removes it.
Handle it gracefully in your application:
Degrade mode (recommended for production)
When the budget threshold is reached, Cognocient AI transparently rewrites requests to a cheaper model. Your application keeps working — cost drops immediately.
| Original model | Degraded to | Cost saving |
|---|---|---|
gpt-4o | gpt-4o-mini | ~94% |
claude-opus | claude-sonnet-4-6 | ~80% |
claude-sonnet-4-6 | claude-haiku | ~91% |
gemini-1.5-pro | gemini-1.5-flash | ~87% |
The response includes headers so you can optionally surface a UI indicator:
Degrade mode switches models; it never blocks. If you also need a hard ceiling, use Block mode with a fallback allowance.
Budget scopes
| Scope | What it applies to | Example |
|---|---|---|
| Global | All traffic through your account | Monthly org cap: $5,000 |
| Feature | Calls with matching X-Cost-Feature | chatbot ≤ $800/mo |
| Department | Calls with matching X-Cost-Department | customer-success ≤ $2,000/mo |
| User | Calls from a specific X-Cost-User | Per-seat billing caps |
| Environment | Calls with matching X-Cost-Environment (lower-cased) | ci ≤ $300/mo, block |
| Key | Every call made with one proxy key (Growth+) | A contractor's key ≤ $200/mo |
| Customer | Calls with matching X-Cost-Customer: one of your customers (Growth+) | acme-corp ≤ $150/mo, so one heavy account can't erase its own margin |
| Spend group | Calls whose X-Cost-User is a member of a spend group (Growth+) | platform-team ≤ $3,000/mo across everyone in it |
| Model group | Calls to any of a list of models: exact ids, or a prefix ending in * (Growth+) | claude-opus-*, gpt-5 ≤ $1,000/mo shared |
Stack budgets. A feature budget of $800/mo and a department budget of $2,000/mo coexist — both are checked, the most restrictive wins.
A model group matches on the model the call is made with, including a provider/ prefix (openai/gpt-5 matches gpt-5). It covers chat completions, embeddings, image generation and the Anthropic /v1/messages endpoint.
Rollover
Turn on Roll unused budget into the next period (Growth+) and whatever a budget didn't spend last month (or yesterday, for a daily budget) is added to this period's limit, up to one period's limit. A $1,000/month budget that spent $700 in October enforces $1,300 in November; if November spends $1,300, December starts at $1,000 again. Nothing carries for a period before the budget existed. The carried amount is shown on the budget card and is worked out from the recorded spend of the period before, when the new period starts. See Hierarchical Budgets for how to set up nested limits.
Per-run limits for agents
For agentic workloads, add a per-run limit so a single runaway agent execution can't consume your entire monthly budget.
Configure in Budgets → New Budget → Per-run limit. When a run exceeds its limit, only that run is blocked — other concurrent runs continue normally.
Pass X-Cost-Run-ID on all agent calls (see Track Cost Per Agent Run).
Every call in a run comes back with its ceiling, so the loop can stop itself before the next step instead of finding out from a 429:
Check remaining budget before your next agent step
For long-running agents, query remaining budget before each step to exit gracefully instead of being hard-stopped mid-execution:
Frequently asked questions
Related articles
Spend Groups
Named groups of people, managed in the app or pushed by your identity provider, that one budget caps together.
Alerts: Email & Slack
Budget-threshold and anomaly alerts by email (every plan) and Slack (Base+): defaults, recipients, quiet hours, hourly cap.
CI Cost Gate
Fail a CI build when a change makes your AI calls more expensive than the baseline. Base plan.