Three questions that narrow the list
1. Do you want to run it yourself?
Self-hosted: LiteLLM, Bifrost, and Portkey’s open-source gateway. Managed: Cloudflare AI Gateway, OpenRouter, Portkey’s cloud, Cognocient AI. Self-hosting keeps traffic in your network and costs nothing in licences, but you operate the database, cache, scaling and upgrades.
2. Whose provider accounts pay?
OpenRouter is a marketplace: by default you buy its credits and it pays providers. The others proxy to your own provider keys, so your negotiated rates and cloud commitments still apply.
3. How exact must a budget be?
Every option here can cap spend in some way. They differ on what a budget attaches to (key, team, model, feature, agent run), whether parallel calls can overshoot it, and what happens at the limit: an error, a cheaper model, or an alert.
The gateways, briefly
LiteLLM
The most widely used open-source gateway: 100+ providers behind an OpenAI-compatible proxy, with virtual keys and hard budget limits per key, user, team or org. You run it with PostgreSQL (spend tracking) and Redis. An experimental FOCUS 1.2 export arrived in 2026. Best for: teams that want maximum provider coverage and will self-host.
Portkey
A gateway with routing, guardrails and observability across 1,600+ models. Its gateway core is open source (Apache 2.0, since March 2026); Palo Alto Networks acquired the company in 2026 and is positioning it for AI-agent security. Budget limits on keys and workspaces sit in the Enterprise plan. Best for: security-led buyers.
Cloudflare AI Gateway
A free managed gateway on Cloudflare’s edge with caching, rate limiting, fallbacks, DLP and, since June 2026, dollar spend limits per provider, model or metadata (up to 20 rules per gateway, eventually consistent). Best for: teams already on Cloudflare that want basic limits at no cost.
Bifrost (Maxim AI)
A fast open-source Go gateway (Apache 2.0) with virtual keys, budgets nested across keys, teams and customers, semantic caching and an MCP gateway; its README reports microsecond overhead at 5,000 RPS. Best for: self-hosters who need raw performance.
OpenRouter
A model marketplace: hundreds of models, one credit balance, automatic provider fallback, per-key credit limits. No markup on token prices; fees on credit purchases and on BYOK usage above an allowance. Best for: breadth of models without separate provider accounts.
Cognocient AI
A managed OpenAI-compatible gateway built around spend: budgets per feature, team, user, key, run or org, reserved atomically before each call, with block, degrade-to-cheaper-model or alert at the limit; agent loop breakers and an account-wide Emergency Freeze; and a finance layer (chargeback, FOCUS export, board reports). Not open source, 10 provider types plus any OpenAI-compatible endpoint. Best for: teams whose AI bill needs to be controlled and explained.
What about Helicone?
Helicone, a popular open-source proxy for LLM logging, was acquired by Mintlify in March 2026 and is in maintenance mode: security fixes and new models, no new features. Existing users are not being cut off, but for a new project it is a deliberate bet. See our Helicone comparison for migration notes.
Comparison table
| LiteLLM | Portkey | Cloudflare | Bifrost | OpenRouter | Cognocient AI | |
|---|---|---|---|---|---|---|
| Hosting | Self-host | Cloud or self-host | Managed (edge) | Self-host | Managed | Managed |
| Open source | ✅ | ✅ core (Apache 2.0) | ❌ | ✅ (Apache 2.0) | ❌ | ❌ |
| Your own provider keys | ✅ | ✅ | ✅ or Unified Billing | ✅ | BYOK optional | ✅ always |
| Dollar budgets | Key, user, team, org | Key, workspace (Enterprise) | Provider, model, metadata (20 rules) | Virtual key, team, customer | Credit limit per key | Feature, team, user, key, run, org |
| At the limit | Error | Key expires | 429 or fallback model | Blocked | Requests fail | Block, degrade, alert, shrink-to-fit |
| Concurrency-safe budget | Not documented | Not documented | No (eventually consistent) | Not documented | Not documented | ✅ atomic reservation |
| Per-run agent budget | Not documented | Not documented | Not documented | Not documented | Not documented | ✅ |
| FOCUS export | Experimental | Not documented | Not documented | Not documented | Not documented | ✅ Growth+ |
| Board reports & GL chargeback | Not documented | Not documented | Not documented | Not documented | Not documented | ✅ Base+ |
| Pricing | Free OSS; enterprise licence | Free dev, $49/mo, Enterprise | Core free | Free OSS; enterprise | Fees on credits/BYOK | Free (1 provider), $99–$1,299/mo |
"Not documented" means we could not find it in the vendor’s public docs on November 2, 2026, not that it is impossible. Corrections are welcome and will be applied.
When to choose each
Pick a self-hosted gateway (LiteLLM, Bifrost) when
- Traffic must stay inside your network
- You have a platform team to run Postgres, Redis and upgrades
- Per-key and per-team budgets are enough
- Licence cost matters more than operating time
Pick Cognocient AI when
- You want a managed gateway with nothing to run
- Budgets must hold exactly for parallel agents and follow features and runs
- You need to degrade to a cheaper model instead of failing at the limit
- Finance needs chargeback, FOCUS data or board reports from the same data
Frequently asked questions
What is an LLM gateway?
A service between your application and model providers that exposes one API (usually OpenAI-compatible) for many models and adds routing, fallbacks, caching, logging and spend controls. It is also called an AI gateway or LLM proxy.
What is the best open-source LLM gateway?
LiteLLM has the widest provider coverage and community; Bifrost focuses on performance with nested budgets; Portkey’s gateway core is also open source. All three are self-hosted.
LiteLLM vs Portkey: which should I use?
LiteLLM if you want to self-host with the most providers and hard budgets per key, user or team. Portkey if guardrails and AI-agent security are your priority and you are comfortable with its Enterprise plan for budgets.
Which LLM gateway enforces budgets before the call?
LiteLLM, Portkey (Enterprise), Cloudflare AI Gateway, Bifrost and Cognocient AI all check a limit before forwarding. Cognocient AI reserves each call’s worst-case cost atomically so concurrent calls cannot overshoot, and can degrade to a cheaper model instead of failing.
There is no single best LLM gateway: the right one depends on whether you self-host, whose accounts pay the providers, and how much your budgets need to be exact and explainable. If cost control and attribution are the reason you are adding a gateway at all, try Cognocient AI on your own data first: import a usage CSV and see the dashboards before changing any code.
Not ready to route production traffic through a gateway yet? Import a CSV of usage you already have, with no proxy and no code change, and see the dashboards for free first. See the importer docs.
Start your 10-day free trial →