Budgets & Control

How do I enforce LLM budget limits before the API call?

LLM budget enforcement means blocking, degrading to a cheaper model, or alerting on AI spend before it exceeds a limit — enforced at the proxy, before the provider bills you. Set per-feature, per-department, and per-run budgets.

In the app: Budgets · every planOpen in app →No account? Start free

LLM budget enforcement means stopping AI spend before it happens, not flagging it after. Cognocient AI checks every API call against your configured budgets before the call reaches the provider. Set a budget in the dashboard and choose your enforcement mode: Block (hard stop), Degrade (auto-switch to cheaper model), or Alert (allow through, flag it).

Most cost tools read billing exports that are 24–48 hours late. Cognocient AI checks budget before the call reaches the provider — so overspend is impossible, not just notifiable. The check is a single atomic Redis operation that runs in sub-millisecond time on every request.

What are the three budget enforcement modes?

ModeWhat happens when the limit is hitBest for
AlertCall goes through, notification sentBaseline measurement, new features
BlockHTTP 429 returned — provider never sees the callExperiments, agents, dev/test
DegradeRequest silently rerouted to a cheaper modelProduction features with SLA requirements

The Base plan supports up to 3 active budgets. Growth and Business plans have unlimited budgets.

How do I create a budget in Cognocient AI?

Go to Dashboard → Budgets → New Budget and fill in:

  • Name — descriptive label, e.g. chatbot feature, Engineering department, Org total
  • Scope — Global (all traffic), Feature (by X-Cost-Feature), Department (by X-Cost-Department), User (by X-Cost-User), Environment (by X-Cost-Environment), Key (one proxy key, Growth+), Customer (by X-Cost-Customer, Growth+), Spend group (a named set of people, Growth+) or Model group (a list of models sharing one limit, Growth+)
  • Resets — every month (default) or every day. A daily budget on a key catches a leaked key or a runaway loop within hours; a monthly cap alone can be drained in an hour and still be "within limits"
  • Limit — the spending cap in USD for that period
  • Enforcement mode — Alert, Block, or Degrade
  • Alert thresholds — default 50%, 80%, 100% — each one alerts once per month by email (every plan) and Slack (Base+); see Alerts

Alert mode

All API calls pass through normally. Cognocient AI sends an alert when each configured threshold is crossed: by email to the account owner by default, and to Slack if connected.

What the alert says (email subject "Cognocient AI: 🟡 Budget Alert — 80%"):

🟡 Budget Alert — 80%
Budget: Ticket resolver
Applies to: feature: ticket-resolver
Spent this month: $402.10 of $500.00
[View in Cognocient AI →]

Block mode

When the budget is exhausted, Cognocient AI returns HTTP 429 immediately. The AI provider never sees the request. This applies to every call the budget covers, whatever its X-Cost-Workload tag.

{
  "error": {
    "message": "Budget limit exceeded: chatbot",
    "type": "budget_error",
    "enforcement": "hard_stop"
  }
}

How concurrent calls and streams are counted

Each call's estimated cost is reserved against the budget atomically before the call is let through, then corrected to the real cost when it finishes. If two calls arrive and only one fits, the second gets the 429. Streamed responses have max_tokens lowered when needed so that a single stream can't run past what's left.

If a stream stops before the provider's final usage report (for example, your client disconnects or times out), Cognocient AI charges an estimate rather than nothing: the prompt, plus roughly one token per four characters of output already streamed. The provider still bills for that work, so the budget counts it too.

How big is each reservation?

  • The worst case, near the limit. The reservation is the prompt (counted locally, plus a 10% margin because tokenizers differ between providers) plus the full max_tokens you allow. If you don't set max_tokens (or max_completion_tokens), 4,096 output tokens are reserved, and with a Block budget the forwarded limit is also clamped to what the remaining budget can pay for.
  • The typical case, far from it. Reserving every call's full max_tokens can lock most of a budget in reservations that are never spent when many calls run in parallel, so real calls get refused while actual spend is far under the cap. Once a model and feature have at least 50 successful calls in the last 7 days, Cognocient AI reserves their p95 output length (plus 20%) instead, but only while the budget has more than 50 worst-case calls of room left. Inside that margin it goes back to the full worst case, so the ceiling stays strict at the end. The rare long reply is corrected when the call settles.
  • Reservations can't leak. Every reservation has a 15-minute lease. If the worker handling a call dies before settling it, the lease expires and the reservation is released, so the budget doesn't slowly lock itself. If that call does settle later, its real cost is still counted, once.
  • The right period. A call reserved at 23:59:58 and finished at 00:00:03 is settled against the period it was reserved in, not the next one.

Shrink to fit instead of refusing (optional)

Near the limit, a call is refused when its worst case doesn't fit, even if a shorter answer would. Send X-Cog-Shrink-To-Fit: true and Cognocient AI lowers the call's max_tokens to what the remaining budget can pay for, as long as at least 256 output tokens still fit, and reserves that instead. The response carries x-cog-max-tokens-shrunk (the new limit) and x-cog-max-tokens-original. The answer may stop early with finish_reason: "length", so only opt in where a shorter answer beats none.

Timeouts: when the bill is unknown

If the provider received a call but no response came back (a read timeout, or the connection dropped mid-response), the provider may still have completed and billed it. Cognocient AI charges the call's worst-case estimate instead of releasing it, so a retry can't spend the same money twice. The 504 carries x-cog-billing-outcome: unknown. Calls that never left (the provider couldn't be reached at all) are released in full. Shadow Spend Reconciliation (Growth+) compares these estimates with the provider's own billing.

To stop a retry from opening a second reservation while the first attempt is still running, send an X-Idempotency-Key: a second request with the same key gets a 409 until the first finishes.

If the budget check itself is unreachable

Each budget chooses what happens if the store its counter lives in (Redis) can't be reached:

  • Fail open (default): calls go through. Your product stays up, but a loop that starts during the outage isn't capped. Add an outage allowance to cap it: while the store is unreachable, at most that many dollars are let through (counted per Cognocient AI server process), then calls get a 429. Set it with outage_allowance_usd.
  • Fail closed: calls are blocked until the check is back.

Optional: cheaper-model fallback allowance

Block is a hard stop by default. If you'd rather some calls keep running on a cheaper model for a while, set a fallback allowance on the budget: an extra dollar amount per month, chosen in advance. After the limit is used up, calls run on the cheaper model from the table in Degrade mode and are reserved against the allowance the same way. Once the allowance is spent, calls get the 429.

  • Off unless you set it. A budget with no allowance never falls back.
  • Calls tagged X-Cost-Workload: agentic-write never use the allowance. A weaker model shouldn't be choosing write actions.
  • Spend on the fallback still counts toward the budget, so the dashboard shows the overrun.
  • Fallback responses carry x-cog-degraded: true and x-cog-degraded-reason: budget_exceeded_fallback_allowance.

Set it under Budgets → Create / Edit → Block → Cheaper-model fallback allowance, or with fallback_allowance_usd on POST /api/budgets. On PATCH, 0 removes it.

Handle it gracefully in your application:

When the budget threshold is reached, Cognocient AI transparently rewrites requests to a cheaper model. Your application keeps working — cost drops immediately.

Original modelDegraded toCost saving
gpt-4ogpt-4o-mini~94%
claude-opusclaude-sonnet-4-6~80%
claude-sonnet-4-6claude-haiku~91%
gemini-1.5-progemini-1.5-flash~87%

The response includes headers so you can optionally surface a UI indicator:

x-cog-degraded: true
x-cog-original-model: gpt-4o
x-cog-degraded-model: gpt-4o-mini

Degrade mode switches models; it never blocks. If you also need a hard ceiling, use Block mode with a fallback allowance.

Budget scopes

ScopeWhat it applies toExample
GlobalAll traffic through your accountMonthly org cap: $5,000
FeatureCalls with matching X-Cost-Featurechatbot ≤ $800/mo
DepartmentCalls with matching X-Cost-Departmentcustomer-success ≤ $2,000/mo
UserCalls from a specific X-Cost-UserPer-seat billing caps
EnvironmentCalls with matching X-Cost-Environment (lower-cased)ci ≤ $300/mo, block
KeyEvery call made with one proxy key (Growth+)A contractor's key ≤ $200/mo
CustomerCalls with matching X-Cost-Customer: one of your customers (Growth+)acme-corp ≤ $150/mo, so one heavy account can't erase its own margin
Spend groupCalls whose X-Cost-User is a member of a spend group (Growth+)platform-team ≤ $3,000/mo across everyone in it
Model groupCalls to any of a list of models: exact ids, or a prefix ending in * (Growth+)claude-opus-*, gpt-5 ≤ $1,000/mo shared

Stack budgets. A feature budget of $800/mo and a department budget of $2,000/mo coexist — both are checked, the most restrictive wins.

A model group matches on the model the call is made with, including a provider/ prefix (openai/gpt-5 matches gpt-5). It covers chat completions, embeddings, image generation and the Anthropic /v1/messages endpoint.

Rollover

Turn on Roll unused budget into the next period (Growth+) and whatever a budget didn't spend last month (or yesterday, for a daily budget) is added to this period's limit, up to one period's limit. A $1,000/month budget that spent $700 in October enforces $1,300 in November; if November spends $1,300, December starts at $1,000 again. Nothing carries for a period before the budget existed. The carried amount is shown on the budget card and is worked out from the recorded spend of the period before, when the new period starts. See Hierarchical Budgets for how to set up nested limits.

Per-run limits for agents

For agentic workloads, add a per-run limit so a single runaway agent execution can't consume your entire monthly budget.

Configure in Budgets → New Budget → Per-run limit. When a run exceeds its limit, only that run is blocked — other concurrent runs continue normally.

Pass X-Cost-Run-ID on all agent calls (see Track Cost Per Agent Run).

Every call in a run comes back with its ceiling, so the loop can stop itself before the next step instead of finding out from a 429:

x-cog-run-limit-usd: 5.000000
x-cog-run-committed-usd: 1.500000      # spent + this call's reservation
x-cog-run-remaining-usd: 3.500000

Check remaining budget before your next agent step

For long-running agents, query remaining budget before each step to exit gracefully instead of being hard-stopped mid-execution:

import httpx
 
async def check_budget(feature: str, run_id: str) -> bool:
    status = (await httpx.AsyncClient().get(
        "https://api.cognocient.com/api/budgets/status",
        headers={"Authorization": f"Bearer {COGNOCIENT_KEY}"},
        params={"feature": feature, "run_id": run_id},
    )).json()
 
    if not status["can_proceed"]:
        return False
    # Stop if any budget is below $0.05 remaining
    return all(b["remaining_usd"] > 0.05 for b in status.get("budgets", []))
 
# In your agent loop:
for step in planned_steps:
    if not await check_budget("research-agent", run_id):
        return {"status": "budget_limit_reached", "completed": completed}
    result = await execute_step(step)

Frequently asked questions