Groq API through an LLM gateway: GPT-OSS, Llama and Qwen on Groq with cost tracking
Route Groq calls (gpt-oss, Llama, Qwen, Whisper) through an OpenAI-compatible gateway. See which model names go to Groq, track spend per feature and cap it with budgets.
Groq serves open-weight models on its own LPU hardware with very high output speed. Its API is OpenAI-compatible, so calls are forwarded as sent with your Groq key.
Model names that route to Groq
| Pattern | Examples |
|---|---|
openai/gpt-oss-* | openai/gpt-oss-120b, openai/gpt-oss-20b |
qwen/qwen3* (lower-case) | Groq's Qwen 3 models |
llama* | llama-3.1-8b-instant |
mixtral-, gemma*, deepseek-r1-distill* | Groq-hosted variants |
whisper-* | Speech-to-text on /v1/audio/* |
Same name on Together AI
Together AI also hosts openai/gpt-oss-120b. Through Cognocient AI that exact name always goes to Groq. Other org/model names go to Together AI, including Together's capitalised Qwen/... names.
Fast models make fast loops
At several hundred tokens per second, an agent stuck in a loop on Groq burns tokens quickly even though each call is cheap. The velocity circuit breaker trips on the proxy key's own baseline within a 60-second window, and a per-run budget caps a single agent execution.
Related articles
Supported Providers
OpenAI, Anthropic, Gemini, Mistral, Groq, Together, Azure, AWS Bedrock, Vertex AI, and custom OpenAI-compatible endpoints.
OpenAI
OpenAI SDK through the gateway: chat, Responses, embeddings, images, audio and batches with per-feature cost and budgets.
Anthropic (Claude)
Claude through the OpenAI SDK (translated) or the native Anthropic SDK (passthrough), with prompt-cache-aware cost.