What gets counted

Each AI call your app makes becomes one span in ai-tally. A span is one traced unit of work: which model was called, how many tokens it used, and what it cost. ai-tally adds spans up to show where your money goes.

Six kinds of cost

An AI feature costs more than the model call. ai-tally sorts spend into six kinds, called layers:

Layer	What it is	Where the numbers come from
LLM	Calls to AI models, like a chat completion.	Your calls, through the proxy, the SDK or OpenTelemetry.
Embeddings	Turning text into vectors for search.	Your calls, through the SDK or OpenTelemetry.
Tool calls	Paid APIs your feature calls, like a web search API.	Your calls, recorded with the SDK or OpenTelemetry.
Vector DB	Searches against a vector database.	Your calls, recorded with the SDK or OpenTelemetry.
Compute	The servers and functions your AI feature runs on.	Your cloud bill, through a connection on the Connectors page.
Egress	Data sent out of your cloud to the internet.	Your cloud bill, through a connection on the Connectors page.

Cost Explorer shows all six, over time.

Cost per feature

A feature is a part of your product that uses AI, like support-chat or summarize. You name it with a feature tag:

• Proxy: the X-Tally-Feature-Tag header.
• Python SDK: feature_tag in tally.init or tally.start_trace.
• OpenTelemetry: the gen_ai.feature_tag attribute.

Calls without a tag are counted as untagged. To see cost by feature, open Cost Explorer and group by feature.

Cost per customer

A customer here means one of your customers, the people or companies using your product. ai-tally shows what each one costs you on the Cost per Account page under Analyze.

To count a call toward a customer, you attach a hashed customer id that cannot be traced back. It works like this:

• Each organization has its own secret key. Your customer id is turned into a 64-character code with that key.
• The same id always gives the same code in your organization, so all of a customer’s calls add up together.
• Nobody can turn the code back into the id. Codes from two different organizations never match.
• ai-tally never stores your real customer ids. The Python SDK makes the code inside your app. With the proxy or OpenTelemetry, you make it yourself with python -m tally.hash_account your-customer-id.

Shared costs

Compute and egress come from your cloud bill as one total per day. They cannot be tied to a single customer’s calls, so ai-tally shares them out. Cost per Account shows these shared amounts separately from each customer’s direct cost, and names the rule it used:

• Pro rata on direct spend (the default): each customer’s share of compute and egress matches their share of direct AI cost.
• Even split across accounts: compute and egress are split equally across every customer and the untagged calls.

Next

• See how each call is priced, and why some costs show a blank: Prices and blanks.