Proxy

The proxy sits between your app and your AI provider. Your app sends each AI request to an ai-tally URL instead of the provider’s URL. ai-tally passes the request to the provider, sends the answer back to your app, and notes how many tokens the call used. It never saves what you asked or what came back.

The proxy is in the path of every call

Each call takes one extra network hop, so it is a little slower. If the proxy is down, calls sent through it fail. If that is not acceptable for your product, use the Python SDK or OpenTelemetry instead. Both send their records separately from your calls.

URLs

Replace your provider’s base URL with one of these:

Provider	Proxy URL
OpenAI	https://ingest.ai-tally.com/openai/v1
Anthropic	https://ingest.ai-tally.com/anthropic
Gemini	https://ingest.ai-tally.com/gemini

Everything after the URL stays the same as the provider’s own API. Only these three providers are supported.

Keep sending your provider key the way you do today: Authorization for OpenAI, x-api-key for Anthropic, x-goog-api-key or ?key= for Gemini. ai-tally forwards it unchanged.

Here is a whole call for OpenAI and for Anthropic. These are the same examples the dashboard shows you when you create a key, so they stay in step with the product. Put your own key where they say YOUR_TALLY_KEY.

OpenAI

# Point the OpenAI SDK at the ai-tally proxy
export OPENAI_BASE_URL="https://ingest.ai-tally.com/openai/v1"

# ai-tally identifies your org by this ingest key, sent as the X-Tenant-Key header:
#   X-Tenant-Key: YOUR_TALLY_KEY
# e.g. a raw request:
curl "https://ingest.ai-tally.com/openai/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "X-Tenant-Key: YOUR_TALLY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}]}'

A brand-new key can take a few seconds to attribute while the proxy picks it up. Your provider key is unchanged and never sent to ai-tally.

Anthropic

# Point the Anthropic SDK at the ai-tally proxy
export ANTHROPIC_BASE_URL="https://ingest.ai-tally.com/anthropic"

# Send your ai-tally ingest key as the X-Tenant-Key header:
#   X-Tenant-Key: YOUR_TALLY_KEY
# e.g. a raw request (your x-api-key and anthropic-version pass through untouched):
curl "https://ingest.ai-tally.com/anthropic/v1/messages" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "X-Tenant-Key: YOUR_TALLY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-haiku-4-5","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}'

A brand-new key can take a few seconds to attribute while the proxy picks it up. Your provider key is unchanged and never sent to ai-tally.

Headers

Header	Needed?	What it is
X-Tenant-Key	Yes	Your ai-tally API key. It needs write scope.
X-Tally-Feature-Tag	No	A name for the part of your product making the call, like support-chat. Lets you see cost per feature.
X-Tally-Account-Id-Hash	No	Which of your customers the call is for, as a hashed customer id that cannot be traced back. Lets you see cost per customer.

ai-tally removes these three headers before passing the call on, so your AI provider never sees them.

curl https://ingest.ai-tally.com/openai/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "X-Tenant-Key: $TALLY_KEY" \
  -H "X-Tally-Feature-Tag: support-chat" \
  -H "X-Tally-Account-Id-Hash: $CUSTOMER_HASH" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Say hi"}]}'

Getting the customer hash

The proxy never sees your real customer ids. You turn an id into its hash yourself, with the ai-tally Python SDK installed and TALLY_KEY set:

python -m tally.hash_account your-customer-id

It prints a 64-character code. The same id always gives the same code in your organization, so you can compare customers. Nobody can turn the code back into the id.

Streaming

Streamed answers pass through as they arrive. The proxy does not hold them back.

• OpenAI: add "stream_options": {"include_usage": true} to your request. Without it, OpenAI does not say how many tokens a stream used, and the cost shows as a blank. The proxy never changes your request, so it cannot add this for you.
• Anthropic and Gemini: nothing to add.

Turning the proxy on

The proxy is off for every new organization. An admin turns it on with Zero-code proxy on the API Keys page. While it is off, the proxy refuses your keys. The SDK and OpenTelemetry keep working either way.

How long changes take

The proxy keeps its own copy of your keys and the on/off switch, so it does not have to ask ai-tally during each call. It updates that copy about every 45 seconds. So allow about a minute for:

• a new key to start working
• a revoked key to stop working
• the switch to turn on or off

What the proxy records

For each call, the proxy records:

• the provider and model
• token counts
• the reply code, sizes and duration
• the URL path
• your feature tag and customer hash, if you sent them

It never records the request or the answer.

Next

• Something failing? The error replies are explained in Check it works.