The token-velocity circuit breaker: catching a cost spike in under 60 seconds
A $50-a-month agent that starts an infinite loop at $500 an hour will not trip any monthly budget for many hours — the math simply hasn't accumulated to a meaningful fraction of the monthly limit yet, even though the rate of spend is already ten times normal. By the time a cumulative budget check would catch it, the loop has already run for most of a day.
That's not a flaw in budget checks — cumulative monthly guards are built for a slower question, “are we on pace to overspend this month,” and they answer it well. It's a different question entirely from “is something happening right now, in the last minute, that looks nothing like normal usage,” which requires a fast, rate-based check running on a completely different timescale.
Budget checks and velocity checks solve different problems
A budget is a cumulative spend guard: it tracks total dollars against a ceiling over a period — a day, a month — and blocks once that ceiling is reached. It is, by design, slow to react to a sudden change, because it's answering a slow question. A velocity check is the opposite: it tracks the rate of token consumption per minute for a given API key, independent of any monthly total, and reacts to a sudden change in that rate within the same minute it happens.
Neither replaces the other. A budget catches sustained overspend that accumulates gradually and never looks abnormal minute to minute. A velocity check catches a sudden abnormal burst that might never accumulate into a monthly budget violation at all if caught early — which is exactly the point, since catching it early is what keeps it from accumulating into one.
How the velocity check actually decides to trip
Every call increments a per-minute token counter for the API key making it, using a sliding 60-second window. The decision to trip runs through three checks, in order:
1. Hard manual limit
If a tokens-per-minute ceiling has been explicitly set for the key, exceeding it trips the breaker immediately — a hard, deterministic limit chosen by the team, no baseline needed.
2. Auto-detection: 10x baseline spike
Once a key has enough history to establish a normal baseline tokens-per-minute rate — and the baseline itself is high enough to be meaningful, filtering out keys with too little traffic to produce a reliable signal — a current rate ten times that baseline trips the breaker as an unusual velocity spike, distinct from any hard limit.
3. Cold-start safety net
A brand-new key with no baseline history yet is still protected: a fixed tokens-per-minute ceiling applies until a personal baseline can be established after a few days of usage, so a new integration isn't exposed during the exact period it has the least history behind it.
The check runs on the same Redis connection already used elsewhere in the request path, adding a single pipelined command rather than a new round-trip — the design goal is sub-millisecond overhead on the happy path, so protection against the rare runaway case doesn't cost latency on every normal call. If the check itself fails for infrastructure reasons, it fails open rather than blocking legitimate traffic — velocity protection is a safety net, not a point of fragility in the request path.
What this looks like in practice
Consider a hypothetical agent integration with a normal baseline around 300 tokens per minute — modest, steady usage from a background automation. A code change introduces an unintended loop: a tool call fails, the agent retries immediately, the retry fails the same way, and the pattern repeats without backoff. Within the first minute of the loop starting, the token rate for that key crosses ten times its established baseline.
The velocity check trips before a second minute of the loop completes, returning a clear error naming the current rate, the baseline it's being compared against, and the spike multiple — “possible agent loop or retry storm” — rather than a generic rate-limit message. Calls from that key stay blocked until the rate normalizes on its own or the underlying loop is fixed. This is illustrative of how the circuit breaker behaves, not a specific customer's reported outcome.
Velocity catches a fast, abnormal rate spike on a single key. It's a different mechanism from run-level budget ceilings that cap total spend across every subagent sharing one workflow run, and from the Failure Loop Breaker that catches a stuck agent repeating the same failing call even when each individual call is cheap enough to never trip a rate or budget threshold at all. The three are designed to run together, each catching a failure mode the others structurally cannot.