Self-Hosted and Open-Weight Models

I. The “it's free” framing that misleads

Teams that stand up Ollama or vLLM behind their own application usually do it to escape a metered bill, and the framing that follows is understandable: no per-token invoice means no cost problem, and no cost problem means no need for the attribution, budgets, or anomaly detection built for a metered API. That framing collapses two different things into one. A metered bill is a line item. Cost is what the underlying compute actually consumes, whether or not a vendor itemizes it per call. GPU time behind a self-hosted endpoint is real infrastructure spend — reserved capacity, a cloud GPU instance, or amortized hardware — it is simply not billed per token the way a provider API is. Removing the invoice does not remove the spend; it only removes the itemization that would otherwise show which feature or team is driving it.

II. Attribution does not stop at the metered boundary

Cognocient proxies ten provider types through one base_url — OpenAI, Anthropic, Google Gemini, Mistral, Groq, Together AI, Azure OpenAI, AWS Bedrock, Google Vertex AI, and a custom OpenAI-compatible endpoint for anything else that speaks the same chat-completions protocol. A self-hosted server registered under Settings → Providers → Custom is not a bolt-on special case in this list; it is provider ten, called the same way as the other nine. Send custom/<model> as the model name — the custom/ prefix selects your endpoint and is stripped before the request is forwarded — and every attribution header your application already sends (feature, department, user, session) applies to that call exactly as it would to a GPT-4o or Claude call. The dashboard, the per-feature breakdown, and the budget enforcement layer do not distinguish a self-hosted call by kind; they distinguish it by tag, the same as everything else.

Cost tracking on a self-hosted call defaults to $0, since there is no list price to draw from — Cognocient does not silently substitute a generic fallback price, which would invent spend that never happened. Enter your own effective per-1M-token rate (an amortized GPU cost per token, for example) and every downstream view — volume, latency, cost by feature, cost by department — uses it. Left blank, those same views still work on volume, latency, and call counts; only the dollar figure reads zero. Either way, a self-hosted deployment is visible in the same breakdown as everything metered, not absent from it because there was no bill to parse.

III. Budgets and velocity still apply

A budget expressed in dollars has little to enforce against a $0-tracked call — but the risk a budget hierarchy and a velocity circuit breaker exist to catch was never only about the dollar figure. A retry loop with no backoff, or an agent stuck re-issuing the same call, saturates a self-hosted deployment's finite GPU capacity exactly as it would rack up a metered bill against a provider — the failure mode is unbounded request volume, not unbounded dollars, and a self-hosted endpoint has no more infinite headroom than a rate-limited provider key does. Set a real per-1M-token rate on a custom endpoint and dollar-denominated budgets and hierarchy apply to it directly, the same as any provider. Leave it at $0 and volume-based signals — call frequency, token-velocity, the same 60-second sliding-window check used against a provider key spike — still catch a runaway loop hammering your own infrastructure, which a metered dollar ceiling was never going to see in that case anyway.

IV. Routing treats it as one more deployment

The Auto Router classifies each request as simple, medium, complex, or reasoning and forwards it to whichever model you have assigned to that tier — nothing in that mechanism requires the tier model to be a metered API. A custom/llama3.1 endpoint is a legitimate assignment for the simple tier, with a frontier model reserved for complex and reasoning work, which is precisely the split many self-hosting decisions are trying to achieve manually in application code already. Load Balancing goes further: a load-balanced group's deployments “can mix providers, including Azure deployments, Bedrock and Vertex model IDs, and custom/<name> endpoints, as long as each model has a provider key in Settings” — a self-hosted deployment sits in the same weighted, round-robin, or lowest-cost group as three different clouds, with the same three-failures-in-sixty-seconds health check skipping it for thirty seconds if it starts erroring, exactly as it would skip a struggling provider.

One honest caveat belongs here rather than getting glossed over: the lowest-cost strategy picks the cheapest deployment from list price times estimated tokens. A self-hosted deployment with no rate entered prices at $0 and will always win that comparison, whether or not it is actually the right model for the request — a hidden failure mode as significant as the honest economics themselves, but a knowable one, and the fix is exactly the same discipline this report has already argued for: enter a real effective rate on the custom endpoint before routing traffic to it on cost.

V. The one real constraint: public HTTPS

This governance model has a genuine architectural limit, worth stating rather than working around in the fine print: Cognocient calls a custom endpoint from its own cloud servers, not from inside your network, so the endpoint must be reachable over https:// from the public internet. Addresses that resolve to localhost, private ranges (10.x, 192.168.x, 172.16–31.x), link-local space, or cloud-metadata endpoints are rejected — both when the endpoint is saved and again on every request — as a server-side-request-forgery protection, not an oversight. A model running purely on a laptop or an air-gapped internal network does not fit this model as-is; it needs a tunnel (Cloudflare Tunnel, ngrok, Tailscale Funnel) or a TLS-terminated reverse proxy in front of it, protected with its own API key, before it can be governed the same way as everything else in this report. That is a real deployment step, not a footnote — and it is the honest trade-off for treating a self-hosted model as a first-class provider instead of a special case with its own disconnected tooling.

VI. Bringing it together

“Free” describes the absence of a metered invoice, not the absence of cost, risk, or a routing decision worth making deliberately. A self-hosted or open-weight model behind a custom OpenAI-compatible endpoint is provider ten in a ten-provider list, not an exception to the other nine — attributed by the same headers, visible in the same dashboard, capped by the same budget and velocity controls once a real rate is entered, and eligible for the same Auto Router and load-balancing decisions as OpenAI, Anthropic, Bedrock, or Vertex. The one place this model asks for something extra is architectural, not philosophical: a public HTTPS endpoint, reachable from the cloud, protected the same way any other credentialed API would be.