How do I find wasted LLM spend from retries, oversized models and context bloat?
Cognocient AI automatically detects four kinds of waste in your AI API spend: retry waste, model mismatch, context bloat and context starvation, by feature, with one-click fixes on the Growth plan.
Cognocient AI automatically scans every API call and flags spend that bought nothing, in four categories. The Waste page (Base plan) shows your own recoverable total by feature; one-click fixes are on Recommendations (Growth plan).
| Waste category | What it catches |
|---|---|
| Retry waste | Failed calls that were re-billed with no value |
| Model mismatch | Frontier models on tasks a smaller model handles equally well |
| Context bloat | Unbounded context window growth across conversation turns |
| Context starvation | Two calls within 60 seconds because data wasn't ready for the first |
Two related analyses go further on the Growth plan: Token Maxing (frontier models used for short outputs) and Context Tax (large static prompts resent on every call).
No configuration required. Detection activates automatically as soon as calls flow through the proxy. Results appear on the Waste page (Monitor → Waste), grouped by feature and category.
Retry waste
What it detects: Failed API calls that were automatically retried — you're billed full input tokens every time a retry fires, even though no useful output was produced. A 10,000-token prompt that times out and retries three times wastes 30,000 tokens.
The dashboard shows: "You wasted $342 on retried calls last month. 34 failed calls triggered automatic retries in pdf-extractor."
Fix — use exponential backoff, not immediate retries:
Don't retry on 400 errors (bad request), 401 (auth), or content policy violations. These will never succeed and you pay for every attempt.
Model mismatch
What it detects: Expensive frontier models used for simple tasks — sentiment analysis, classification, entity extraction, short summarisation. GPT-4o on a "classify this as positive/negative" task costs ~60× more than GPT-4o-mini with identical output quality.
The dashboard shows: "Switching sentiment-analysis from gpt-4o to gpt-4o-mini would save $822/month. AI confidence: high."
| Task type | Recommended model | Savings vs GPT-4o |
|---|---|---|
| Sentiment / classification | gpt-4o-mini | ~94% |
| Entity extraction | gpt-4o-mini | ~94% |
| Short summarisation | gpt-4o-mini | ~94% |
| Complex reasoning | gpt-4o | — |
| Long document understanding | claude-sonnet-4-6 | Comparable cost, higher quality |
Fix: Apply the one-click recommendation in Intelligence → Recommendations (Growth plan). Cognocient AI creates a routing rule that silently redirects matching calls to the cheaper model — no code changes required.
Context bloat
What it detects: Sessions where each turn sends the full conversation history. By turn 10, you're paying for turns 1–9 on every single new call. A 20-turn session can pay for the same early tokens 19 times.
The dashboard shows: "$193 wasted on bloated context windows. 15 sessions exceeded 50% context growth in support-chat."
Fix — summarise old turns instead of sending them all:
Context starvation
What it detects: Two API calls within 60 seconds where the second call's prompt is 50%+ larger than the first. This pattern means your app called the model before it had all the context it needed, then had to call again. You paid for both calls but could have made one.
The dashboard shows: "$127/month wasted on iterative prompts. 34 sequences where your app called the model before the prompt was fully assembled."
Fix — gather all context before the first call:
What happens after waste is detected
Once Cognocient AI identifies waste in a feature, it generates a specific recommendation in Intelligence → Recommendations (Growth plan) with:
- Estimated monthly saving
- Confidence level (High / Medium / Low)
- One-click apply — creates a routing rule, no code changes needed
Recommendations are re-evaluated daily as new call data arrives.
Frequently asked questions
Related articles