How do I spread traffic across multiple API keys for one provider?
Add extra API keys next to a provider's primary key and Cognocient balances requests across them by weight, skipping any key that is rate-limited. Growth plan and above.
Provider rate limits are per API key. When one busy key is the bottleneck, add more keys (from different projects or organizations) and let Cognocient spread the load.
Set it up
- Connect the provider normally under Settings → AI Providers. That key is the primary key.
- Open Load Balancer → API key pool, choose the provider, give the new key a label, an optional weight and paste it.
Requests to that provider are then split across the primary key (weight 1) and each pooled key by weight. A key with weight 3 gets three times the traffic of one with weight 1. You can pause or remove a key at any time, and changes apply within about 15 seconds.
Health
Cognocient watches the real result of real calls. After two failures (429, 408, 500, 502, 503 or 504) within 60 seconds, a key is skipped for 45 seconds and the page shows rate-limited. There are no synthetic probes, and state is shared across proxy workers. If that state is unreachable, balancing keeps working and treats every key as healthy.
Notes
- Keys are encrypted at rest and cannot be read back; the list shows only the last four characters.
- Up to 10 keys per provider including the primary. Labels must be unique per provider.
- The pool applies to chat, audio, rerank and batch requests. Embeddings and image generation still use the primary key.
- Streaming requests use the key chosen when the request starts.
- Secret-manager references (Business) work for a provider's primary key only; pooled keys must be plain API keys.
Related articles