Python SDK

The Python SDK is a small library you add to your app. It records your AI calls in the background and sends ai-tally the counts, never the text. It never makes your calls fail, and it never waits on ai-tally.

Install

Install the SDK with this command. It needs Python 3.10 or later.

pip install "git+https://github.com/jain-aanchal/ai-tally#subdirectory=sdk/python"

Do not run pip install tally. That installs a different, unrelated package.

Use it

Two lines connect your app. This is the same example the dashboard shows you when you create a key:

# pip install "git+https://github.com/jain-aanchal/ai-tally#subdirectory=sdk/python"
import tally

tally.init("YOUR_TALLY_KEY")
# From here your unmodified openai / anthropic calls are auto-instrumented:
# model, tokens, and cost land in ai-tally with no per-call record_* code.

Set with_account(...) once per request to unlock cost per customer; start_trace(feature_tag=...) for cost per feature.

In your own code, leave the key out and let it come from TALLY_KEY, and say which customer and which feature each call is for:

import tally
from openai import OpenAI

tally.init(feature_tag="assistant")  # once, at startup; reads your key from TALLY_KEY
client = OpenAI()

def handle(request):
    # Every AI call inside this block counts toward this customer.
    with tally.with_account(request.customer_id):
        # And toward this feature, instead of the default above.
        with tally.start_trace(feature_tag="summarize"):
            return client.chat.completions.create(model="gpt-4o-mini", messages=request.messages)

That is all most apps need. tally.init() records calls made with the official openai and anthropic libraries automatically.

tally.init

Call it once when your app starts.

tally.init(key: str | None = None, *, endpoint: str | None = None, feature_tag: str | None = None, instrument: bool = True, instrument_stream_usage: bool = False, flush_interval_s: float = 1.0, catalog: PriceCatalog | None = None) -> TallyClient

Connect the process to ai-tally in one line. Idempotent, non-blocking, never raises.

Option	Default	What it does
key	None	Your ai-tally API key. If left out, it is read from TALLY_KEY.
endpoint	None	Where to send records. If left out, it is read from TALLY_ENDPOINT, or else https://ingest.ai-tally.com.
feature_tag	None	The feature name used when a call has no other.
instrument	True	Record openai and anthropic calls automatically.
instrument_stream_usage	False	For streamed OpenAI chat calls, ask OpenAI to report token counts. Without it, those calls have a blank cost unless you set stream_options={"include_usage": True} yourself.
flush_interval_s	1.0	How often, in seconds, records are sent.
catalog	None	Your own price list for the SDK’s local estimate. ai-tally works out the final cost itself either way.

Without a key, init logs a warning and records nothing. Calling it a second time does nothing.

What is recorded automatically

• openai: chat.completions.create, responses.create and embeddings.create.
• anthropic: messages.create and messages.stream.

Both normal and async clients are covered. Clients made before or after init both count. Calls through other libraries need record_llm_call, below.

Cost per customer: with_account

tally.with_account(account_id: str | None, *, label: str | None = None) -> Iterator[TraceContext]

Scope an account to a block without touching the trace (CTO-181).

Every call inside the block counts toward that customer. Set it once per request, for example in your web framework’s middleware.

• The id is protected. The SDK turns your customer id into a hashed customer id that cannot be traced back, before anything leaves your app. The real id is never sent.
• Clearing it. Pass None to clear it, for example in a background job that should not count toward any customer.
• Labels. label is an optional readable name. It is not stored with your call records.

Cost per feature: start_trace

tally.start_trace(*, feature_tag: str | None = None, session_id: str | None = None, account_id: str | None = None, account_label: str | None = None) -> AbstractContextManager[TraceContext]

Begin a fresh trace (always a new trace_id). Returns the context manager.

Starts a new trace, which is one traced unit of work, like handling one request. Calls inside it carry its feature name. A new trace does not pick up the customer from an outer with_account, so pass account_id if you need it.

For work that moves between processes, like Celery tasks, use this instead. It takes the same options plus trace_id.

tally.with_trace_context(*, trace_id: str | None = None, feature_tag: str | None = None, session_id: str | None = None, account_id: str | None = None, account_label: str | None = None, inherit: bool = True) -> Iterator[TraceContext]

Set the trace context for the duration of the block, then restore prior values.

Recording other calls: record_*

Use these for calls the SDK does not record on its own. All options are named. None of them raise errors.

tally.record_llm_call(*, provider: str, model: str, usage: Usage, signals: TraceSignals | None = None, at: date | None = None, account_id: str | None = None, account_label: str | None = None) -> LlmCallResult

Record an LLM call end-to-end. Never raises.

tally.record_embedding_call(*, provider: str, model: str, input_tokens: int, at: date | None = None, account_id: str | None = None, account_label: str | None = None) -> EmbeddingCallResult

Record an embedding call so the span lands in the gateway's embeddings bucket.

tally.record_tool_call(*, provider: str, tool: str, cost_micro_usd: int | None = None, input_tokens: int | None = None, output_tokens: int | None = None, latency_ms: int | None = None, call_id: str | None = None, account_id: str | None = None, account_label: str | None = None) -> None

Record a tool call so the span lands in the gateway's tools cost-layer bucket.

tally.record_vector_call(*, provider: str, index: str, operation: str, cost_micro_usd: int | None = None, record_count: int | None = None, latency_ms: int | None = None, account_id: str | None = None, account_label: str | None = None) -> None

Record a vector-DB call so the span lands in the gateway's vector cost-layer bucket.

tally.pricing.Usage(input_tokens: int = 0, output_tokens: int = 0, cached_input_tokens: int = 0)

Usage(input_tokens: 'int' = 0, output_tokens: 'int' = 0, cached_input_tokens: 'int' = 0)

What each one is for:

• record_llm_call: an AI model call through any other library. For Gemini, use provider="google".
• record_embedding_call: an embedding call, priced on its input tokens.
• record_tool_call: a paid tool or API your feature calls, like a web search API.
• record_vector_call: a vector database call, like a Pinecone query.

Costs are whole micro-dollars. 1_000_000 is $1.

Each one also takes account_id to set the customer for just that call.

Other functions

tally.flush(timeout: float = 5.0) -> None

Drain buffered spans to the gateway synchronously (bounded). Safe no-op before init.

tally.uninstrument() -> None

Reverse all provider patches and tear down the process-global client (CTO-260 §4.1).

tally.get_client() -> TallyClient | None

The process-global client, or None before init.

tally.hash_account(account_id: str, *, key: str | None = None, endpoint: str | None = None) -> str

Return the HMAC-SHA256 hex of account_id under the tenant's active key.

• tally.flush: sends everything waiting right now. A short script should call it before it exits. A long-running app does not need to.
• tally.uninstrument: turns the SDK off and removes its hooks.
• tally.get_client: returns the running SDK client, or None before init.
• tally.hash_account: returns the hashed customer id for the proxy’s X-Tally-Account-Id-Hash header. It fetches your organization’s hashing key, so unlike everything else here, it can raise an error, for example when there is no key.

Why it never breaks your app

• Nothing is sent while your call runs. Records go into a queue in memory, and a background thread sends them.
• Errors inside the SDK are caught. Your AI call goes ahead as normal, and errors from the AI provider reach your code unchanged.
• Failed sends are retried. When ai-tally is slow or down, sends are retried up to five times, and a record is never counted twice.
• The queue is capped. If ai-tally stays unreachable, the queue holds up to 10,000 records, then drops the oldest and logs a warning.
• A rejected key is not retried. A wrong key or a read key is refused, and the SDK logs a warning.

Next

• Nothing showing up? See Check it works.