Vertex AI Gemini with the OpenAI SDK: route Google Cloud Gemini through an LLM gateway
Use Gemini on Google Cloud Vertex AI from the OpenAI SDK by prefixing the model with vertex/. Service-account auth handled for you, with per-feature cost tracking and budgets enforced before the call.
Vertex AI is how Gemini is bought through Google Cloud: billed to your GCP project, under your organisation's IAM and data-residency settings. Cognocient AI authenticates with your service account and calls Vertex AI's OpenAI-compatible endpoint, so no request or response translation is involved.
Setup
- Create a service account with the Vertex AI User role (
roles/aiplatform.user) and download a JSON key. - In Settings → AI Providers → Vertex AI paste the JSON key and enter the project ID and region (for example
us-central1). - Prefix model names with
vertex/:
Streaming, tool calling and JSON output work as they do on Vertex AI directly. Short-lived access tokens are minted from the service account and refreshed automatically.
Gemini API and Vertex AI side by side
gemini-2.5-flash uses your Gemini API key; vertex/gemini-2.5-flash uses Vertex AI. Teams often keep prototypes on the Gemini API and production on Vertex AI, and compare the two in one dashboard.
Why put Vertex AI behind a gateway?
GCP billing exports show Vertex AI spend by SKU and project, a day or more later. The gateway adds real-time budgets, a per-feature and per-user breakdown from attribution headers, and an Emergency Freeze that stops every proxied call at once. Use Shadow Spend Reconciliation (Growth) to compare gateway totals with your GCP bill.
Related articles
Supported Providers
OpenAI, Anthropic, Gemini, Mistral, Groq, Together, Azure, AWS Bedrock, Vertex AI, and custom OpenAI-compatible endpoints.
OpenAI
OpenAI SDK through the gateway: chat, Responses, embeddings, images, audio and batches with per-feature cost and budgets.
Anthropic (Claude)
Claude through the OpenAI SDK (translated) or the native Anthropic SDK (passthrough), with prompt-cache-aware cost.