How does the Batch API work through Cognocient?

Upload your JSONL with POST /v1/files (purpose=batch), create the job with POST /v1/batches, then poll GET /v1/batches/{id} and download results with GET /v1/files/{id}/content, exactly as with OpenAI directly. Cognocient forwards each call with your OpenAI key.

When is a batch's cost recorded?

When it finishes. A batch's cost is only known at completion, so Cognocient records it once, at the Batch API's 50% price, attributed to the X-Cost-* tags you sent when you created it. It is picked up when you retrieve the batch, or within about 30 minutes by a background check if you never do.

Does it count against budgets?

Creating a batch is blocked if a blocking budget is already exhausted. The batch's actual cost is added to spend when it completes, so it cannot stop a batch that is already running.

OpenAI. Anthropic message batches and other providers are not supported yet.

Can I use the OpenAI Batch API through Cognocient?

Create and manage OpenAI batches through Cognocient (/v1/files and /v1/batches) and have each batch's discounted cost attributed to your tags the moment it finishes. Growth plan and above.

The Batch API runs large jobs asynchronously at half the price, with results within 24 hours. Cognocient sits in front of it so batch spend shows up next to everything else instead of appearing as an unexplained line on your OpenAI invoice.

from openai import OpenAI
 
client = OpenAI(base_url="https://api.cognocient.com/v1", api_key="<your proxy key>",
                default_headers={"X-Cost-Feature": "nightly-enrichment"})
 
f = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
batch = client.batches.create(input_file_id=f.id, endpoint="/v1/chat/completions", completion_window="24h")
 
# later
batch = client.batches.retrieve(batch.id)
if batch.status == "completed":
    results = client.files.content(batch.output_file_id).text

Attribution

Send your X-Cost-Feature, X-Cost-Department, X-Cost-Project and other headers when creating the batch. They are stored with the job and used when its cost is recorded, so a batch created by one team is billed to that team even if someone else collects the results.

The model is read from the first request line of your input file. If a file mixes several models, the whole batch is priced at the first line's model and the cost is an approximation.

What is supported

Endpoint	Notes
`POST /v1/files`	Only `purpose=batch`
`GET /v1/files/{id}`, `/content`, `DELETE`	Passed through
`POST /v1/batches`, `GET /v1/batches`, `GET /v1/batches/{id}`, `POST /v1/batches/{id}/cancel`	Passed through; finished batches are costed

Failed, expired and cancelled batches are closed out at $0 if OpenAI reports no usage for them.

Limits

OpenAI only, with your own OpenAI key.
Batch creation is not budget-reserved: cost is added when the batch completes.
Fine-tuning and assistant files are not accepted through this endpoint.

Can I use the OpenAI Batch API through Cognocient?

Attribution

What is supported

Limits

On this page