CIRCLE
AI and billing

Call the LiteLLM gateway

Call LLMs and MCP tools through the gateway with the user's own token, handle empty wallets and rate limits, and attribute spend to a session.

Read Credit wallet and LiteLLM gateway first for the model. In short: the user's access token for the gateway audience is the API key, the gateway debits the wallet, and your application never handles a debit.

Prerequisites

  • LiteLLM support is on for the application in auth-service, and the plan of the user grants access to models (see the auth-service guide LiteLLM gateway and credit wallet in an application).
  • gatewayUrl is set in createAuthClient, and an AiClient exists: createAiClient({ auth, grants }).
  • The user signed in after gatewayUrl was set, so their grant covers the gateway and wallet audiences.

With the SDK's own calls

UserAi.gateway.postJson(path, body) posts JSON and returns the parsed answer as unknown; you validate it. A refusal throws an AuthAiError.

import type { FastifyInstance } from "fastify";
import { z } from "zod";
import { AuthAiError, createAiClient, isInsufficientCredit } from "@circle-rd/auth-ai";
import type { AuthClient, GrantManager } from "@circle-rd/auth-ai";
import { requireAuth } from "@circle-rd/auth-ai/fastify";
import type { AuthHandlers } from "@circle-rd/auth-ai/web";

declare const auth: AuthClient;
declare const grants: GrantManager;
declare const handlers: AuthHandlers;
declare const app: FastifyInstance;

const ai = createAiClient({ auth, grants });

const chatBody = z.object({ model: z.string().min(1), message: z.string().min(1).max(2000) });
const chatAnswer = z.object({
  choices: z.array(z.object({ message: z.object({ content: z.string().nullable() }) })),
});

app.post("/api/chat", { preHandler: requireAuth(handlers) }, async (request, reply) => {
  const body = chatBody.parse(request.body);
  const session = request.authSession!;
  const user = ai.forUser(session.userId);
  try {
    const answer = await user.gateway.postJson("/v1/chat/completions", {
      model: body.model,
      messages: [{ role: "user", content: body.message }],
      metadata: { session_id: session.id },
    });
    return { reply: chatAnswer.parse(answer).choices[0]?.message.content ?? "" };
  } catch (err) {
    if (!(err instanceof AuthAiError)) throw err;
    if (isInsufficientCredit(err)) {
      return reply.status(402).send({ error: err.toPublicJSON(), topupUrl: user.wallet.topupUrl() });
    }
    if (err.code === "GW_002") {
      const retryAfter = err.details?.["retryAfterSeconds"];
      if (typeof retryAfter === "number") void reply.header("retry-after", retryAfter);
      return reply.status(429).send({ error: err.toPublicJSON() });
    }
    return reply.status(502).send({ error: err.toPublicJSON() });
  }
});

The gateway maps to the SDK's errors by status: 402 is GW_001 (empty wallet), 429 is GW_002 (rate limit, with details.retryAfterSeconds when the gateway sends Retry-After), 401 and 403 are GW_003, 503 is GW_004. Return the public JSON to your users and keep err.message for logs.

With an OpenAI-compatible SDK

gateway.fetch signs each request with a fresh user token, replacing whatever Authorization header the SDK set, and refuses any origin other than the gateway's so the token cannot leak. Give it to the SDK together with baseURL and a placeholder key:

import OpenAI from "openai";
import { createAiClient, isInsufficientCreditStatus } from "@circle-rd/auth-ai";
import type { AuthClient, GrantManager } from "@circle-rd/auth-ai";

declare const auth: AuthClient;
declare const grants: GrantManager;
declare const userId: string; // the signed-in user's `sub`
declare const sessionId: string; // any stable id of your choice

const { gateway } = createAiClient({ auth, grants }).forUser(userId);

const openai = new OpenAI({
  baseURL: gateway.openAiBaseUrl, // <gatewayUrl>/v1
  apiKey: "unused", // replaced by gateway.fetch
  fetch: gateway.fetch,
});

try {
  const completion = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: "Hello" }],
    metadata: { session_id: sessionId },
  });
  console.log(completion.choices[0]?.message.content);
} catch (err) {
  if (isInsufficientCreditStatus(err)) {
    // 402: the wallet is empty or negative. Offer a top-up.
  } else if (typeof err === "object" && err !== null && "status" in err && err.status === 429) {
    // Rate limited: back off.
  } else {
    throw err;
  }
}
This snippet is a sketch: it relies on the documented SDK API and on the fetch and baseURL options of the openai package, but it was not compiled against that package here. Model names depend on your gateway: list them with gateway.listModels().

A 401 from the gateway drops the cached token, so retrying the call mints a new one. Tokens live one hour and are never stored.

isInsufficientCreditStatus(err) tests for status === 402 on any error object, which is how OpenAI-compatible SDKs expose the HTTP status. For AuthAiErrors use isInsufficientCredit(err).

Attribute spend to a session

Put metadata: { session_id: "<id>" } in the request body. The wallet then keeps the spend of that session, and you can read it back (for the calling application only):

import type { UserAi } from "@circle-rd/auth-ai";

declare const user: UserAi;

const usage = await user.wallet.getSessionUsage("chat-7f3a", { days: 7 });
console.log(usage.calls, usage.total, usage.byModel);

usage.total is a bigint of micro-euros; byModel lists model, amount, count and token counts per model (model is mcp for tool calls and other when the gateway reported none). Billing is asynchronous: a call may take a few seconds to appear, so poll if you need the figure right after the call. The example app's self-test does exactly that.

Models, agents and MCP servers

import type { GatewayClient } from "@circle-rd/auth-ai";

declare const gateway: GatewayClient;

const models = await gateway.listModels(); // /v1/models, only what the user's plan may call
const agents = await gateway.listAgents(); // /v1/agents
const servers = await gateway.listMcpServers(); // /v1/mcp/server

const url = gateway.mcpUrl(); // <gatewayUrl>/mcp
const oneServer = gateway.mcpUrl("my-server"); // <gatewayUrl>/my-server/mcp
const token = await gateway.getAccessToken(); // for an MCP client's Authorization: Bearer header

mcpUrl(name) rejects names outside [A-Za-z0-9][A-Za-z0-9_.-]{0,127} with CFG_001. Mint the token per connection: it expires after an hour.

Show the balance first

Check wallet.getAccount() before an expensive action and use canSpend, never the raw balance: it is true for an unlimited account or a strictly positive balance. A call that finishes after the balance reached zero is still billed, so a balance can go negative until the next top-up.

Copyright © 2026