How do I track LangChain costs per user or per feature?

Point ChatOpenAI (or ChatAnthropic) at https://api.cognocient.com/v1 with a Cognocient AI proxy key and add X-Cost-Feature / X-Cost-User headers with default_headers, or per call with extra_headers. Cost is computed server-side from the provider's token usage, so no callbacks are needed.

Can I use Claude or Gemini with ChatOpenAI?

Yes. Through the gateway, ChatOpenAI reaches any supported provider: set model to claude-sonnet-4-6, gemini-2.5-flash, a Bedrock ID, and so on. Tool calling and with_structured_output work with Claude on this route.

Do I still need get_openai_callback?

No. get_openai_callback only sees one process and uses LangChain's own price table. The gateway records every call, from every service, at the provider's reported token counts.

How do I stop a LangChain agent from running up a large bill?

Give each run an X-Cost-Run-ID and X-Cost-Run-Budget header. The gateway refuses further calls in that run once its dollar budget is used, while other runs continue. The velocity circuit breaker catches loops on the proxy key as a backstop.

LangChain cost tracking: per-feature and per-user LLM spend for ChatOpenAI and ChatAnthropic

Track and cap LangChain LLM costs by pointing ChatOpenAI or ChatAnthropic at an OpenAI-compatible gateway. Tag calls by feature, user and chain run, and enforce budgets before the call. Python and JavaScript.

LangChain's chat models are thin wrappers around provider SDKs, so the gateway slots in at the client level: set the base URL and key, and every chain, agent and retriever built on that model is tracked and governed. Cost is computed by the gateway from the provider's own token counts, so nothing in your chains has to change.

ChatOpenAI (any provider)

ChatAnthropic (native Claude)

If you prefer langchain-anthropic, point it at the native passthrough route (no /v1):

# pip install langchain-anthropic
from langchain_anthropic import ChatAnthropic
 
llm = ChatAnthropic(
    model="claude-sonnet-4-6",
    api_key="sk-cog-YOUR-KEY",
    base_url="https://api.cognocient.com",
    default_headers={"X-Cost-Feature": "support-agent"},
)

The native route passes Anthropic features (prompt caching, extended thinking) through untouched. See Anthropic for what each route supports.

Tag per user and per run

Client-level headers tag every call. For values that change per request, pass extra_headers when you invoke, or bind them:

llm.invoke(messages, extra_headers={"X-Cost-User": f"user_{user.id}"})
 
# or, for every call a chain makes during one request:
tagged = llm.bind(extra_headers={"X-Cost-User": f"user_{user.id}", "X-Cost-Run-ID": run_id})
chain = prompt | tagged | parser

X-Cost-Run-ID groups every LLM call of one chain or agent run and is what a run budget (below) counts against. The per-run and per-user views in Sessions need the Growth plan.

Cap an agent run

Send a dollar ceiling with the run ID. The first call of the run sets it; once the run has spent it, further calls in that run get a 429 with type: run_budget_exceeded, and other runs carry on.

tagged = llm.bind(extra_headers={"X-Cost-Run-ID": run_id, "X-Cost-Run-Budget": "0.50"})

Concurrent sub-agents sharing a run ID share the budget. See Budget enforcement and Debug runaway agent loops.

Structured output and tools with Claude through ChatOpenAI

with_structured_output and bind_tools work with Claude models on the OpenAI route: the gateway translates tools, tool_choice (including a forced function) and parallel_tool_calls, on streamed and non-streamed calls. max_tokens is honoured whether langchain-openai sends it as max_tokens or max_completion_tokens.

Embeddings too

OpenAIEmbeddings(model="text-embedding-3-small", api_key="sk-cog-...", base_url="https://api.cognocient.com/v1") routes embedding calls through the gateway, so vector-store ingestion jobs show up as their own feature.