How do I enable semantic caching to eliminate duplicate API calls?

Goal: Eliminate redundant API calls — both exact duplicates and semantically equivalent prompts — so you pay $0.00 and get a response in under 10ms whenever the same question has been asked before.

Time: 5 minutes to enable. Cache hits appear immediately.

Best for: FAQ chatbots, help centre assistants, support bots, product Q&A — any workload where users ask similar questions repeatedly.

Step 1 — Add the cache header to eligible requests

Add X-Cog-Similarity-Cache: true to any request you want cached. That is the only change required.

client = OpenAI(
    api_key="sk-cog-YOUR-PROXY-KEY",
    base_url="https://api.cognocient.com/v1",
    default_headers={
        "X-Cost-Feature":         "support-bot",
        "X-Cog-Similarity-Cache": "true",
    }
)
# Every call from this client is now cached automatically

Step 2 — Verify a cache hit

Make two similar requests (not necessarily identical). Then check Dashboard → Live Calls — cache hits show $0.00 cost with <10ms latency.

You can also inspect response headers directly:

Step 3 — Tune the similarity threshold (optional)

The default threshold of 0.92 works well for most workloads. Adjust with X-Cog-Similarity-Threshold:

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "How do returns work?"}],
    extra_headers={
        "X-Cog-Similarity-Cache":     "true",
        "X-Cog-Similarity-Threshold": "0.90",  # more aggressive for FAQ
    }
)

Step 4 — Set cache TTL for time-sensitive content

By default, cached responses expire after 24 hours. Override with x-cog-cache-ttl (in seconds):

# Cache policy answers for 7 days
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What are your pricing plans?"}],
    extra_headers={
        "X-Cog-Similarity-Cache": "true",
        "x-cog-cache-ttl":        "604800",  # 7 days
    }
)
 
# Cache inventory answers for only 5 minutes
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Is product #4821 in stock?"}],
    extra_headers={
        "X-Cog-Similarity-Cache": "true",
        "x-cog-cache-ttl":        "300",     # 5 minutes
    }
)

What to expect after 7 days

Check Dashboard → Recommendations for your cache hit rate and savings estimate. Typical results:

Related: Semantic Similarity Caching · Prompt Cache + Batch Routing · Cut Your AI Bill with One Click