Skip to content

Framework Integrations

Paygent ships first-party integrations for popular agent frameworks. They plug into the framework's native callback / middleware system and meter every LLM call the framework makes.

Framework Python TypeScript
LangChain LangChainCallback PaygentCallbackHandler (also covers LangGraph.js)
CrewAI CrewAICallback — (no JS port; use instrument())
Vercel AI SDK — (Python N/A) pg.aiMiddleware()

But — and this is important — you don't always need them.

When to use framework callbacks vs auto-instrumentation

LangChain and CrewAI both use the OpenAI / Anthropic SDKs internally. So when Paygent instruments chat.completions.create — via Python's monkey-patch, or pg.instrument(client) / autoInstrument: true in the JS SDK — that interception fires whether the call comes from your code or from inside a LangChain ChatOpenAI. Auto-instrumentation works out of the box.

You only need the framework callback if:

  1. You want framework-specific metadata (which agent, which task, which chain step) attached to events.
  2. You're using a custom LLM client that LangChain understands but Paygent's patcher doesn't.
  3. You disabled auto-instrumentation (auto_instrument=False) and want metering only via the callback.

For most apps, auto-instrumentation alone is enough. Try that first. If you find an LLM call sneaking through unmetered, add the framework callback as a backstop.

Don't double-count

If you use auto-instrumentation and the framework callback simultaneously, you'd meter the same call twice.

  • Python: the framework callbacks detect this and skip themselves when the patcher is active and a paygent_context is set, so you can stack them safely — the callback only fires when the patcher won't.
  • TypeScript: the LangChain PaygentCallbackHandler does not auto-deduplicate. Pick one path per client: either pg.instrument(client) (or autoInstrument: true) or the callback handler — not both on the same client.

LangChain integration

Install

pip install paygent[langchain]

This pulls in langchain-core. If you're already using a heavier extra (langchain, langchain-openai), you don't need the [langchain] extra — the callback only depends on langchain-core.

npm i @paygentjs/sdk @langchain/core

@langchain/core is an optional peer dependency — install it alongside the SDK. The handler is imported from the @paygentjs/sdk/integrations/langchain subpath.

Basic usage with ChatOpenAI

from langchain_openai import ChatOpenAI

from paygent import Paygent
from paygent.integrations import LangChainCallback

pg = Paygent.init(api_key="pg_live_...")

llm = ChatOpenAI(model="gpt-4o-mini")
cb = LangChainCallback(pg, user_id="user_123")

response = llm.invoke(
    "Summarize the French Revolution in two sentences.",
    config={"callbacks": [cb]},
)
print(response.content)

import { ChatOpenAI } from "@langchain/openai";
import { Paygent } from "@paygentjs/sdk";
import { PaygentCallbackHandler } from "@paygentjs/sdk/integrations/langchain";

const pg = await Paygent.init({ apiKey: "pg_live_..." });

const llm = new ChatOpenAI({ model: "gpt-4o-mini" });
const cb = new PaygentCallbackHandler(pg, { userId: "user_123" });

const response = await llm.invoke(
  "Summarize the French Revolution in two sentences.",
  { callbacks: [cb] },
);
console.log(response.content);

The callback reads token usage from the LangChain LLMResult (llmOutput.tokenUsage or the per-generation usage_metadata in newer LangChain), builds a UsageEvent, and pushes it to Paygent's queue. It guards on handleLLMStart (throwing PaygentLimitExceeded on a hard gate) and meters on handleLLMEnd.

Usage with chains

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    ("user", "{input}"),
])
chain = prompt | llm | StrOutputParser()

cb = LangChainCallback(pg, user_id="user_123")
result = chain.invoke({"input": "Explain monads"}, config={"callbacks": [cb]})

import { ChatPromptTemplate } from "@langchain/core/prompts";
import { StringOutputParser } from "@langchain/core/output_parsers";

const prompt = ChatPromptTemplate.fromMessages([
  ["system", "You are a helpful assistant."],
  ["user", "{input}"],
]);
const chain = prompt.pipe(llm).pipe(new StringOutputParser());

const cb = new PaygentCallbackHandler(pg, { userId: "user_123" });
const result = await chain.invoke({ input: "Explain monads" }, { callbacks: [cb] });

Usage with agents

from langchain.agents import AgentExecutor, create_openai_tools_agent
from langchain_core.tools import tool

@tool
def search(q: str) -> str:
    """Search the web for information."""
    return f"Results for '{q}'"

agent = create_openai_tools_agent(llm, [search], prompt)
executor = AgentExecutor(agent=agent, tools=[search])

cb = LangChainCallback(pg, user_id="user_123")
result = executor.invoke(
    {"input": "What's new in AI today?"},
    config={"callbacks": [cb]},
)

import { AgentExecutor, createOpenAIToolsAgent } from "langchain/agents";
import { DynamicStructuredTool } from "@langchain/core/tools";
import { z } from "zod";

const search = new DynamicStructuredTool({
  name: "search",
  description: "Search the web for information.",
  schema: z.object({ q: z.string() }),
  func: async ({ q }) => `Results for '${q}'`,
});

const agent = await createOpenAIToolsAgent({ llm, tools: [search], prompt });
const executor = new AgentExecutor({ agent, tools: [search] });

const cb = new PaygentCallbackHandler(pg, { userId: "user_123" });
const result = await executor.invoke(
  { input: "What's new in AI today?" },
  { callbacks: [cb] },
);

The handler also covers LangGraph.js — graph nodes run their LLM calls through the same callback events, so pass the handler in the graph's config.callbacks exactly the same way.

Auto-instrument + LangChain callback together

In Python, you can keep both — auto-instrument catches the LLM call, the callback catches things the patcher might miss — and the callback's de-dup logic decides per call which one meters.

In TypeScript, there's no auto-dedup: choose one path per client. Use the callback handler when you want LangChain metadata or are on a client the Proxy can't wrap; otherwise just instrument() the underlying provider client and skip the handler.

pg = Paygent.init(api_key="...")  # auto_instrument=True (default)

cb = LangChainCallback(pg, user_id="user_123")

with paygent_context(user_id="user_123"):  # patcher will meter
    result = chain.invoke({"input": "..."}, config={"callbacks": [cb]})
    # Patcher meters the call; callback detects this and skips.

# No paygent_context → patcher won't meter; callback DOES meter.
result = chain.invoke({"input": "..."}, config={"callbacks": [cb]})

// Pick ONE path — the JS callback does not auto-deduplicate.

// Option A — instrument the provider client (no callback):
const openai = pg.instrument(new OpenAI());
const llm = new ChatOpenAI({ model: "gpt-4o-mini", client: openai.chat });

// Option B — callback handler (do NOT also instrument the same client):
const cb = new PaygentCallbackHandler(pg, { userId: "user_123" });
await chain.invoke({ input: "..." }, { callbacks: [cb] });

In Python the decision is per-call: if paygent_context is set AND the SDK is instrumented, the callback skips. Otherwise, the callback meters. In TypeScript there is no such skip, so do not stack both on one client.

Complete LangChain example

"""A LangChain chain metered with Paygent."""

import os
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

from paygent import Paygent, PaygentLimitExceeded
from paygent.integrations import LangChainCallback

pg = Paygent.init(api_key=os.environ["PAYGENT_API_KEY"])
pg.on_soft_gate(lambda r: print(f"⚠ {r.message}"))

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0.2)
prompt = ChatPromptTemplate.from_messages([
    ("system", "You write concise summaries in 2 sentences."),
    ("user", "{topic}"),
])
chain = prompt | llm | StrOutputParser()

user_id = "user_123"
cb = LangChainCallback(pg, user_id=user_id, metadata={"feature": "summarize"})

try:
    for topic in ["Quantum entanglement", "Mitochondria", "Async/await"]:
        result = chain.invoke({"topic": topic}, config={"callbacks": [cb]})
        print(f"\n{topic}:\n  {result}")
except PaygentLimitExceeded as e:
    print(f"\nBlocked: {e.guard_result.message}")

pg.flush()
usage = pg.get_usage(user_id)
print(f"\nTotal: ${usage.period_cost:.4f}, {usage.period_tokens_total} tokens")

pg.shutdown()

// A LangChain chain metered with Paygent.
import { ChatOpenAI } from "@langchain/openai";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { StringOutputParser } from "@langchain/core/output_parsers";
import { Paygent, PaygentLimitExceeded } from "@paygentjs/sdk";
import { PaygentCallbackHandler } from "@paygentjs/sdk/integrations/langchain";

const pg = await Paygent.init({ apiKey: process.env.PAYGENT_API_KEY });
pg.onSoftGate((r) => console.warn(`⚠ ${r.message}`));

const llm = new ChatOpenAI({ model: "gpt-4o-mini", temperature: 0.2 });
const prompt = ChatPromptTemplate.fromMessages([
  ["system", "You write concise summaries in 2 sentences."],
  ["user", "{topic}"],
]);
const chain = prompt.pipe(llm).pipe(new StringOutputParser());

const userId = "user_123";
const cb = new PaygentCallbackHandler(pg, { userId, metadata: { feature: "summarize" } });

try {
  for (const topic of ["Quantum entanglement", "Mitochondria", "Async/await"]) {
    const result = await chain.invoke({ topic }, { callbacks: [cb] });
    console.log(`\n${topic}:\n  ${result}`);
  }
} catch (err) {
  if (err instanceof PaygentLimitExceeded) {
    console.log(`\nBlocked: ${err.guardResult.message}`);
  } else {
    throw err;
  }
}

await pg.flush();
const usage = pg.getUsage(userId);
console.log(`\nTotal: $${usage.totalCost.toFixed(4)}, ${usage.totalTokens} tokens`);

await pg.close();

CrewAI integration

CrewAI is Python-only

CrewAI has no JavaScript/TypeScript port, so there's no CrewAICallback in the JS SDK. The examples below are Python. JS/TS users building multi-agent systems should use pg.instrument(client) on the underlying provider client (or the Vercel AI SDK middleware / LangGraph.js path), which meters every call the agents make.

Install

pip install paygent[crewai]

Basic usage

CrewAI exposes a step_callback on Crew. Paygent's CrewAICallback is a callable, so you can pass it directly.

from crewai import Agent, Crew, Task

from paygent import Paygent
from paygent.integrations import CrewAICallback

pg = Paygent.init(api_key="pg_live_...")
cb = CrewAICallback(pg, user_id="user_123")

researcher = Agent(
    role="Researcher",
    goal="Find recent news on a topic",
    backstory="You are a meticulous researcher.",
)
task = Task(
    description="Find 3 facts about quantum computing.",
    expected_output="A bullet list of 3 facts.",
    agent=researcher,
)

crew = Crew(agents=[researcher], tasks=[task], step_callback=cb)
result = crew.kickoff()
print(result)

// CrewAI is Python-only. In JS, instrument the provider client your
// agents call — every call they make is metered:
//   const openai = pg.instrument(new OpenAI());

The callback runs after every agent step. It tries to extract token counts from the step output's token_usage, usage_metrics, or tokens attribute, plus any nested result.usage dict. If a step isn't an LLM call (no tokens captured, no model), the callback skips it.

Multi-agent crew

researcher = Agent(role="Researcher", goal="Gather facts", backstory="...")
writer = Agent(role="Writer", goal="Produce a summary", backstory="...")

research_task = Task(
    description="Find 5 facts about Mars.",
    expected_output="bullet list",
    agent=researcher,
)
writing_task = Task(
    description="Write a 3-sentence summary using the research.",
    expected_output="paragraph",
    agent=writer,
)

cb = CrewAICallback(pg, user_id="user_123")

crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, writing_task],
    step_callback=cb,
)
result = crew.kickoff()

// CrewAI is Python-only — see the note at the top of this section.

Auto-instrument + CrewAI callback together

Same de-dup story as LangChain. CrewAI uses litellm or the OpenAI SDK under the hood, so the patcher catches the calls. The callback only fires when the patcher won't (i.e. no paygent_context set).

In CrewAI, the typical pattern is callback only, no paygent_context. The callback's user_id does the attribution.

pg = Paygent.init(api_key="...", auto_instrument=False)  # callback-only mode

cb = CrewAICallback(pg, user_id="user_123")
crew = Crew(..., step_callback=cb)
crew.kickoff()

// CrewAI is Python-only — see the note at the top of this section.

If you do want auto-instrumentation as a backstop, leave auto_instrument=True and just use the callback. The de-dup logic handles it.

Complete CrewAI example

"""A CrewAI crew metered with Paygent."""

import os
from crewai import Agent, Crew, Task

from paygent import Paygent
from paygent.integrations import CrewAICallback

pg = Paygent.init(api_key=os.environ["PAYGENT_API_KEY"])
pg.on_usage(lambda e: print(f"  metered {e.total_tokens} {e.model} tokens"))

user_id = "user_123"
cb = CrewAICallback(pg, user_id=user_id, metadata={"crew": "research"})

researcher = Agent(
    role="Researcher",
    goal="Find current information",
    backstory="You're a thorough researcher.",
    verbose=False,
)

task = Task(
    description="Name 3 recent breakthroughs in renewable energy.",
    expected_output="A bullet list of 3 items.",
    agent=researcher,
)

crew = Crew(agents=[researcher], tasks=[task], step_callback=cb)
result = crew.kickoff()

print("\nResult:", result)
pg.flush()
print(f"Spend: ${pg.get_usage(user_id).period_cost:.4f}")
pg.shutdown()

// CrewAI is Python-only — see the note at the top of this section.

Other frameworks

Anything that uses the OpenAI / Anthropic SDK underneath works with auto-instrumentation out of the box. No special integration needed.

That includes:

  • Python: LlamaIndex, AutoGen, DSPy, Pydantic AI — all use openai / anthropic directly
  • TypeScript: LlamaIndex.TS, Mastra, and anything built on the openai / @anthropic-ai/sdk packages
  • Bare openai / anthropic SDK calls
  • Anything you wrote yourself

For these, instrument the client (or set a context) and wrap your call site:

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

from paygent import Paygent, paygent_context

pg = Paygent.init(api_key="pg_live_...")

documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)

with paygent_context(user_id="user_123"):
    # LlamaIndex makes OpenAI calls underneath; the patcher catches them.
    response = index.as_query_engine().query("What's in the docs?")

import OpenAI from "openai";
import { Paygent, paygentContext } from "@paygentjs/sdk";

const pg = await Paygent.init({ apiKey: "pg_live_..." });
// Instrument the shared client the framework uses underneath…
const openai = pg.instrument(new OpenAI());

await paygentContext({ userId: "user_123" }, async () => {
  // …every call the framework makes through this client is metered.
  const res = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "What's in the docs?" }],
  });
});

If a framework wraps the LLM call in a way that hides it from instrumentation (e.g. a custom HTTP client that doesn't go through chat.completions.create), you'll see zero events. In that case, fall back to:

  • The framework's own callback system if it has one (write a custom Paygent adapter — the LangChainCallback / PaygentCallbackHandler is ~100 lines and a good template)
  • pg.wrap() (Python: pg.wrap() / pg.awrap()) at the call site
  • Manually constructing a UsageEvent (advanced)

Vercel AI SDK

The Vercel AI SDK is the idiomatic JS-ecosystem integration. Paygent provides a LanguageModelMiddleware you attach with wrapLanguageModel() — it guards and meters both generateText and streamText, reading usage from the AI SDK's normalized result. This is TypeScript-only (the AI SDK has no Python equivalent).

Install

npm i @paygentjs/sdk ai @ai-sdk/openai

ai is an optional peer dependency.

Usage

import { generateText, wrapLanguageModel } from "ai";
import { openai } from "@ai-sdk/openai";
import { Paygent, paygentContext } from "@paygentjs/sdk";

const pg = await Paygent.init({ apiKey: "pg_live_..." });

const model = wrapLanguageModel({
  model: openai("gpt-4o"),
  middleware: pg.aiMiddleware({ userId: "user_123" }),
});

await paygentContext({ userId: "user_123" }, async () => {
  const { text } = await generateText({
    model,
    prompt: "Summarize the French Revolution in two sentences.",
  });
  console.log(text);
});

The middleware throws PaygentLimitExceeded on a hard gate (before the model call), and meters cost + tokens after a successful generation or stream. If you pass userId to aiMiddleware(), it takes precedence; otherwise it falls back to the active paygentContext() user.

Next steps