How do I fail a CI build when a change makes AI calls more expensive?
Run your test suite through the Cognocient AI proxy with a run id, then call one endpoint that compares the run's cost with a baseline and fails the build on a regression. Base plan.
A prompt that doubles its context, a switch to a pricier model or a new retry loop usually shows up on the invoice weeks later. The cost gate catches it in the pull request.
How it works
- Your tests run with LLM calls going through Cognocient AI, each sending
X-Cost-Run-Id: ci-<commit>. - After the tests, CI calls
POST /api/cost-gate/checkwith that run id. - Cognocient AI totals the run's calls and compares them with a named baseline (default
main). - The response says whether the check passed, and why. The CI step fails if it did not.
- On your main branch,
update_baseline: truemakes a passing run the new baseline.
1. Send the run id from your app
Adding X-Cost-Environment: ci also lets you see and budget CI spend separately.
2. Add the gate to your workflow
Any CI system works the same way: it is one HTTP call.
Options
| Field | Default | Meaning |
|---|---|---|
run_id | required | The run to check. Letters, digits and . _ : / -, up to 128 characters. |
baseline | main | Name of the baseline to compare with. |
max_increase_pct | 10 | Fail if cost rises by more than this percentage. |
max_cost_usd | none | Fail if the run costs more than this, baseline or not. |
compare | total | total run cost, or per_call average (steadier when the number of tests changes). |
max_feature_increase_pct | none | Also fail if any single X-Cost-Feature rises by more than this. |
min_calls | 1 | Fewer calls than this is reported as no_data and fails. |
settle_seconds | 10 | Calls are logged just after each response; the check waits (0-30 s) until the run's call count stops changing. |
update_baseline | false | If the check passes (or no baseline exists yet), make this run the baseline. |
commit_sha, branch | none | Shown in the dashboard history. |
Results
status | passed | Meaning |
|---|---|---|
pass | true | Within every limit. |
fail | false | Over a limit. reasons says which. |
no_baseline | true | Nothing to compare with yet (unless max_cost_usd was exceeded). |
no_data | false | The run recorded fewer than min_calls calls. Usually a setup problem. |
The response also includes the run and baseline totals, the change in percent and a per-feature breakdown. Control → CI Cost Gate shows the history, lets you inspect each check and accept any run as a baseline.
. _ -.Other endpoints
GET /api/cost-gate/checks?limit=50: recent checksGET /api/cost-gate/baselines: baselinesPOST /api/cost-gate/baselineswith{"name": "main", "run_id": "..."}: set a baseline from any runDELETE /api/cost-gate/baselines/{name}
Related articles
Budget Enforcement
Hard limits enforced at the proxy before charges reach your card.
Spend Groups
Named groups of people, managed in the app or pushed by your identity provider, that one budget caps together.
Alerts: Email & Slack
Budget-threshold and anomaly alerts by email (every plan) and Slack (Base+): defaults, recipients, quiet hours, hourly cap.