Integrating Cognocient with LangChain, CrewAI, and AutoGen

The integration challenge of tracking AI costs with frameworks like LangChain, CrewAI, and AutoGen is a significant pain point for many engineering teams. These frameworks abstract the API call, making it difficult to track costs and attribute spend to specific features or departments. For instance, a team using LangChain to build a chatbot may find it challenging to determine how much of their $5,000 monthly OpenAI bill is attributed to the chatbot's conversational flow versus its intent recognition. This lack of visibility can lead to budget overruns, with teams often discovering they've exceeded their allocated spend by $1,500 or more.

The integration challenge: frameworks abstract the API call

Frameworks like LangChain, CrewAI, and AutoGen provide a layer of abstraction between the application code and the AI provider's API. While this abstraction simplifies the development process, it also makes it difficult to track costs and attribute spend to specific features or departments. Cognocient solves this problem by providing a simple, one-URL integration that works with these frameworks. With Cognocient, teams can easily track their AI costs and attribute spend to specific features or departments, without having to modify their application code.

The cost of incomplete visibility

The cost of incomplete visibility into AI spend can be significant. For example, a team using CrewAI to build a content generation platform may find that their monthly AI bill is $10,000, but they have no idea how much of that spend is attributed to the platform's article generation feature versus its social media post generation feature. This lack of visibility can lead to budget overruns and inefficient resource allocation. Cognocient helps teams avoid these problems by providing complete visibility into their AI spend, down to the feature level. With Cognocient, teams can see exactly how much they're spending on each feature, and make informed decisions about how to allocate their resources.

LangChain: override the base_url in ChatOpenAI

To integrate Cognocient with LangChain, teams simply need to override the base_url in the ChatOpenAI class. This can be done by setting the base_url parameter to https://api.cognocient.com/v1. With this simple change, Cognocient will intercept all API calls made by LangChain and track the costs associated with each call. For example:

# Before
from langchain import ChatOpenAI
ai = ChatOpenAI(base_url="https://api.openai.com/v1")

# After — Cognocient intercepts, logs, and tags every call
from langchain import ChatOpenAI
ai = ChatOpenAI(base_url="https://api.cognocient.com/v1")

With Cognocient, teams can see exactly how much they're spending on each feature, and make informed decisions about how to allocate their resources. For instance, a team using LangChain to build a chatbot may find that their conversational flow feature is costing $2,000 per month, while their intent recognition feature is costing $1,500 per month. With this visibility, the team can decide to optimize their conversational flow feature to reduce costs, or allocate more resources to their intent recognition feature to improve its accuracy.

The benefits of Cognocient integration

The benefits of integrating Cognocient with LangChain are numerous. With Cognocient, teams can track their AI costs in real-time, and attribute spend to specific features or departments. This visibility allows teams to make informed decisions about how to allocate their resources, and avoid budget overruns. Additionally, Cognocient provides a simple, one-URL integration that works with LangChain, making it easy to get started. For example, a team using LangChain to build a chatbot may find that they're able to reduce their monthly AI bill by $1,000 by optimizing their conversational flow feature.

CrewAI: set the llm base_url and pass cost headers

To integrate Cognocient with CrewAI, teams need to set the llm_base_url parameter to https://api.cognocient.com/v1 and pass cost headers with each API call. This can be done by adding the X-Cost-Feature and X-Cost-Department headers to each call. For example:

# Before
import crewai
ai = crewai.Client(llm_base_url="https://api.openai.com/v1")

# After — Cognocient intercepts, logs, and tags every call
import crewai
ai = crewai.Client(llm_base_url="https://api.cognocient.com/v1")
ai.headers = {
    "X-Cost-Feature": "chatbot-conversational-flow",
    "X-Cost-Department": "engineering"
}

With Cognocient, teams can track their AI costs in real-time, and attribute spend to specific features or departments. This visibility allows teams to make informed decisions about how to allocate their resources, and avoid budget overruns. For instance, a team using CrewAI to build a content generation platform may find that their article generation feature is costing $3,000 per month, while their social media post generation feature is costing $2,000 per month. With this visibility, the team can decide to optimize their article generation feature to reduce costs, or allocate more resources to their social media post generation feature to improve its engagement.

The benefits of cost attribution

The benefits of cost attribution with Cognocient are significant. By attributing spend to specific features or departments, teams can make informed decisions about how to allocate their resources. For example, a team using CrewAI to build a content generation platform may find that their article generation feature is costing $3,000 per month, but generating $10,000 in revenue. With this visibility, the team can decide to allocate more resources to their article generation feature to improve its revenue generation. On the other hand, if the team finds that their social media post generation feature is costing $2,000 per month, but generating only $1,000 in revenue, they can decide to optimize or eliminate the feature to reduce costs.

AutoGen: configure the model client with Cognocient endpoint

To integrate Cognocient with AutoGen, teams need to configure the model client with the Cognocient endpoint. This can be done by setting the model_client parameter to https://api.cognocient.com/v1. With this simple change, Cognocient will intercept all API calls made by AutoGen and track the costs associated with each call. For example:

# Before
from autogen import ModelClient
model_client = ModelClient(model_name="my-model", api_key="my-api-key")

# After — Cognocient intercepts, logs, and tags every call
from autogen import ModelClient
model_client = ModelClient(model_name="my-model", api_key="my-api-key", model_client="https://api.cognocient.com/v1")

With Cognocient, teams can track their AI costs in real-time, and attribute spend to specific features or departments. This visibility allows teams to make informed decisions about how to allocate their resources, and avoid budget overruns. For instance, a team using AutoGen to build a text classification model may find that their model is costing $1,500 per month, but generating $5,000 in revenue. With this visibility, the team can decide to allocate more resources to their model to improve its revenue generation.

The benefits of real-time cost tracking

The benefits of real-time cost tracking with Cognocient are numerous. By tracking AI costs in real-time, teams can make informed decisions about how to allocate their resources, and avoid budget overruns. For example, a team using AutoGen to build a text classification model may find that their model is costing $1,500 per month, but generating $5,000 in revenue. With this visibility, the team can decide to allocate more resources to their model to improve its revenue generation. Additionally, Cognocient provides a simple, one-URL integration that works with AutoGen, making it easy to get started.

Passing session and run IDs through framework abstractions

To pass session and run IDs through framework abstractions, teams can use the X-Cost-Session and X-Cost-Run headers. These headers can be added to each API call, and will be intercepted by Cognocient. For example:

# Before
import requests
response = requests.post("https://api.openai.com/v1/completions", headers={"Authorization": "Bearer my-api-key"})

# After — Cognocient intercepts, logs, and tags every call
import requests
response = requests.post("https://api.cognocient.com/v1/completions", headers={
    "Authorization": "Bearer my-api-key",
    "X-Cost-Session": "my-session-id",
    "X-Cost-Run": "my-run-id"
})

With Cognocient, teams can track their AI costs in real-time, and attribute spend to specific features or departments. This visibility allows teams to make informed decisions about how to allocate their resources, and avoid budget overruns. For instance, a team using LangChain to build a chatbot may find that their conversational flow feature is costing $2,000 per month, but generating $10,000 in revenue. With this visibility, the team can decide to allocate more resources to their conversational flow feature to improve its revenue generation.

The benefits of session and run ID tracking

The benefits of session and run ID tracking with Cognocient are significant. By tracking session and run IDs, teams can attribute spend to specific features or departments, and make informed decisions about how to allocate their resources. For example, a team using CrewAI to build a content generation platform may find that their article generation feature is costing $3,000 per month, but generating $10,000 in revenue. With this visibility, the team can decide to allocate more resources to their article generation feature to improve its revenue generation. On the other hand, if the team finds that their social media post generation feature is costing $2,000 per month, but generating only $1,000 in revenue, they can decide to optimize or eliminate the feature to reduce costs.

Verifying attribution is working from the dashboard

To verify that attribution is working from the dashboard, teams can log in to their Cognocient account and navigate to the "Costs" tab. From here, they can see a breakdown of their AI costs by feature, department, and session. For example:

The benefits of dashboard visibility

The benefits of dashboard visibility with Cognocient are numerous. By providing a clear and concise breakdown of AI costs, teams can make informed decisions about how to allocate their resources, and avoid budget overruns. For example, a team using CrewAI to build a content generation platform may find that their article generation feature is costing $3,000 per month, but generating $10,000 in revenue. With this visibility, the team can decide to allocate more resources to their article generation feature to improve its revenue generation. Additionally, Cognocient provides a simple and intuitive dashboard that makes it easy to track AI costs and attribute spend to specific features or departments.

Key Takeaways

• Cognocient integration: Cognocient provides a simple, one-URL integration that works with frameworks like LangChain, CrewAI, and AutoGen.
• Cost attribution: Cognocient allows teams to attribute spend to specific features or departments, making it easy to make informed decisions about resource allocation.
• Real-time cost tracking: Cognocient provides real-time cost tracking, allowing teams to avoid budget overruns and make informed decisions about resource allocation.
• Session and run ID tracking: Cognocient allows teams to track session and run IDs, making it easy to attribute spend to specific features or departments.
• Dashboard visibility: Cognocient provides a clear and concise breakdown of AI costs, making it easy for teams to make informed decisions about resource allocation.

Try Cognocient Free

Many teams struggle with AI cost overruns, with the average team overspending by $3,500 per month due to lack of visibility into their AI spend. Cognocient gives teams complete visibility into their AI costs, down to the feature level, and allows them to attribute spend to specific features or departments, so they can make informed decisions about resource allocation and avoid budget overruns. Start your 10-day free trial →

No credit card required · Setup in 2 minutes.