Cloud billing resolves to a resource. Your consumers don't.

You put a gateway in front of your models, issue every internal team its own key, and now you know exactly who is calling. Then someone in finance asks which team spent what, and you discover the two halves of that sentence live in different universes.

Cloud billing resolves to a resource. One line for the model resource, inside one subscription, and no dimension anywhere in that data corresponds to a team, a person, or a key. Every call from every consumer aggregates into the same meter.

Meanwhile the gateway emits token metrics dimensioned by subscription id, which is exactly the identity you want. It just isn't in the billing data and never will be.

So the join doesn't exist and you have to build it: read token counts out of telemetry keyed by subscription, price them yourself, and store the result per consumer per period. That's the whole mechanism, and it's much less obvious than it sounds.

1. Gateway policy emits a token metric. dimensioned by subscription id, api id, model
2. Application Insights logger, workspace-based.
3. Log Analytics. the AppMetrics table — not the classic customMetrics projection
4. Rollup job sums by subscription, model, and day.
5. Priced usage rows, one per consumer per period. each row references the exact rate row applied to it, from a versioned rate table

• Cloud billing resolves to the resource only — and never joins any of the above.

Where the identity actually lives

This is the part I'd have wanted written down, because I lost real time to it.

The token counts come from an LLM token-metric policy applied at the product scope, emitting a custom metric dimensioned by subscription id, API id, and model. That flows through an Application Insights logger attached to the gateway, into a workspace-based App Insights instance, and lands in Log Analytics.

It is worth being precise about what it is not:

• Not the gateway's resource diagnostic logs. Those describe requests, not token usage.
• Not the dedicated LLM log table. That table was empty on our instance, because it requires a Log Analytics diagnostic setting on the gateway resource rather than an App Insights logger, and because the APIs had been imported as generic HTTP rather than as typed LLM APIs.

Three reproduction details will save you an afternoon each.

Workspace-based App Insights changes the schema. Telemetry lands in the Log Analytics App* tables — AppMetrics, AppRequests — with dimensions inside a Properties JSON column. It does not land in the classic customMetrics/requests projection. Querying the classic schema returns intermittently empty results that look exactly like a dead pipeline. I spent real time convinced the metric emission was broken when the metrics were arriving fine into a table I wasn't reading. Standardize on the Log Analytics schema, and allow a few minutes after traffic before asserting absence — one of our paths has shown ingest latency well past seven minutes.

Sampling doesn't apply. The service diagnostic sampling that governs request and trace telemetry does not govern custom metrics emitted through the logger. "Are my metrics sampled?" is the first question anyone building cost attribution should ask, and here the answer is no.

The query is short. Sum in long format, pivot in application code rather than in KQL:

AppMetrics
| where TimeGenerated >= bin(ago(3d), 1d)
| where Name in ('Prompt Tokens','Prompt Cached Tokens','Completion Tokens',
                 'Prompt Audio Tokens','Completion Audio Tokens','Completion Reasoning Tokens')
| extend sid   = tostring(Properties['SubscriptionId']),
         model = tostring(Properties['Model'])
| where isnotempty(sid) and sid != 'master'
| summarize value = sum(Sum), watermark = max(TimeGenerated)
    by sid, model, Name, bucket = bin(TimeGenerated, 1d)

That SubscriptionId dimension is the gateway subscription id, which is the same identifier your system of record uses. That single equality is the entire join. It's what lets a dollar figure reach a team.

Two things that emit nothing

The built-in all-access subscription is excluded explicitly, since its traffic isn't attributable to any consumer.

More interesting: subscriptions scoped to an individual API, or to all APIs, bypass product-scope policies entirely. They emit no token metric at all. This one is documented, and worth knowing before you discover it:

If you're using an API-scoped subscription, an all-APIs subscription, or the built-in all-access subscription, policies configured at the product scope aren't applied to requests from that subscription.

Microsoft Learn — Subscriptions in Azure API Management

We proved it with a negative test anyway — an API-scoped subscription reached the backend and got a 401, because the product policy that injects the upstream credential never ran. Same root cause, two symptoms: no credential injection and no metering.

If your metering lives in a product-scope policy, then any subscription issued outside that scope is invisible to it. That's not pedantry. It's a hole in your cost data shaped exactly like whoever asked for a special-case key.

Why the price list is a table with a trigger on it

The tempting implementation is a config value. Rates in a settings file, multiply, done.

A config value answers "what does this cost?" A chargeback number has to answer a different question: "what did this cost on the twentieth of June, and can you prove nobody changed the rate since?" A config value is a single mutable cell. Restating history and correcting an error look identical, and there's no artifact showing which rate produced which number.

So rates live in a versioned, effective-dated table, with at most one open rate per meter, and each stored usage row references the exact rate row applied to it. A database trigger makes the rows immutable: deletes are refused, updates are refused except for closing the open period, and once closed a period can't be reopened or slid. Changing a price means inserting a new row and closing the old one. Rewriting history is a database error rather than a policy someone might forget.

That's the difference between a number you can defend in a conversation with finance and a number you can only assert.

Two smaller decisions in the same spirit. Meters are keyed by the billable model, not by the gateway's routing alias, because an alias is renameable and isn't unique across pricing tiers, so pricing on it means a rename silently re-prices history. And a usage bucket that can't be priced is stored anyway, with real token counts and an explicit unpriced marker, rather than dropped. A skipped meter and a consumer who used nothing look identical in a report. That's the failure mode you least want in a cost system.

Showback now, chargeback later, and the column name that keeps you honest

Today this is showback. Every consumer's usage is attributed and costed, the numbers are visible in an admin console, and nobody is billed.

Chargeback is the end goal and the groundwork is deliberate. The rate basis on each row already distinguishes retail from negotiated from provisioned-throughput from a locked internal rate. Cost center is captured at intake, so attribution exists before anyone asks for it. Each usage row points at the rate that produced it. Moving from showback to chargeback means inserting a price row with a different basis, not rebuilding the pipeline.

The honest limit is in the name of the column. It's called estimated retail on purpose. Tokens multiplied by list price is an allocation basis, not an invoice. Cloud billing charges the resource, and provisioned-throughput, negotiated, and batch pricing all diverge from list. It's the right instrument for apportioning cost across teams and the wrong one for reconciling against a bill. Naming it precisely is the cheapest possible defense against someone three quarters from now reading it as revenue.

Four ways I got the cost math wrong

This is the part I'd want to read, so it's the part I'll tell honestly.

Streamed responses report no usage at all

The streaming API omits the usage object entirely unless you explicitly ask for it. Rate limiting still worked, because the throttling policy estimates rather than measures. The provider still billed every token. Only attribution went to zero, which meant a team could stream all month and show nothing.

A gateway policy that injects the usage flag on streaming requests fixes it, and the fix has to leave non-streaming bodies untouched byte for byte, because a body you re-serialize is a body you might subtly alter on the way to the model.

I did not find this in production data. It came out of an adversarial review of the design before any of it was built, which is the only reason it was never wrong in the first place. Design review catching your worst bug is a better outcome than production catching it, and a less flattering story, so it's worth saying which one happened.

Cached tokens are not the same shape across providers

This is the one to internalize, because it is a documented difference between two vendors' public APIs and it will bite anyone brokering both.

• Azure OpenAI. prompt_tokens includes the cached tokens reported in prompt_tokens_details.cached_tokens. Cached is a subset. Billable input is prompt minus cached.
• Anthropic Messages API. input_tokens and the cache-read count are disjoint. Billable input is input_tokens as it stands.

[receipts] Azure OpenAI prompt caching · Anthropic prompt caching

The original formula assumed the first semantic for everything. Applied to Anthropic, subtracting a large cache-read count from a small fresh-input count clamps the fresh input to zero and drops it entirely.

The error is asymmetric, which is the part worth designing around. Applying the OpenAI rule to Anthropic under-bills. Applying the Anthropic rule to OpenAI double-counts. So unknown models default to the subset semantic: it preserves existing behavior and cannot over-bill a newly onboarded model. Model matching looks for provider markers anywhere in the identifier rather than at the start, so a provider-qualified or registry-pathed model id can't silently fall back to the wrong semantic.

The live data made it obvious once anyone looked. On a single day, one OpenAI model showed roughly two million prompt tokens against about 1.7 million cached — plausible as a subset. One Anthropic model showed 525 prompt tokens against 8.4 million cache reads. Cached cannot be a subset of a number sixteen thousand times smaller.

Two honest qualifications on that second figure. It came from a single agentic consumer whose workload replays a large cached context against very small fresh inputs, so the ratio is a property of that usage pattern rather than a property of Anthropic traffic in general. And the defect was never dependent on the ratio being extreme — the formula was semantically wrong for every Anthropic call regardless. The extreme ratio is only why it became visible.

That's the actual lesson, and it's a worse one than "we found a bug." On a consumer with modest caching, the same wrong formula produces numbers that look entirely plausible. Detectability depended on someone's traffic shape, not on the severity of the error. So the fix ships a tripwire: the subset branch now logs a warning whenever cached exceeds prompt, because that shape is the signature of a misclassified model and it shouldn't need a pathological consumer to surface.

A rolling window that ate the first hours of every day

The rollup ran nightly, re-querying and overwriting the trailing few daily buckets. Its lower bound was a relative offset evaluated when the job fired, while results were grouped into calendar days. So the oldest day in range was always partial — it began at the timer's hour rather than at midnight.

Because each pass overwrites the buckets it touches, the last pass to see a given day rewrote it missing its first few hours, permanently.

On one measured day, 12.6% of that day's tokens fell inside the clipped window. That window was about 14.6% of wall-clock time, so the loss ran slightly under what a uniform traffic distribution would predict, which is what you'd expect with lighter overnight usage. Treat it as one representative day, not as a constant.

The structural claim is stronger than the measurement anyway: every settled day was missing its first few hours. The magnitude varies with traffic distribution. The defect does not.

The fix is one line of intent. Floor the window to the same boundary the grouping uses, so every bucket you query is a whole bucket.

And one I haven't fixed

At the time of writing this is still open. The stored total-token count excludes cached tokens while the cost includes them, so any panel dividing cost by total tokens reports an inflated per-token rate. It's documented in the code with a pointer to the ticket. It's the least dangerous of the four, because the attribution and the dollar figures are both correct and only a derived ratio is wrong.

It's still wrong. I shipped fixes for the others and wrote this one down, which is an ordinary engineering tradeoff, and I'd rather describe it than imply a clean sweep.

What I'd check first, in your system

• Does your metering policy sit at a scope that every issued credential actually passes through? Anything issued outside that scope is invisible, and it will be the special-case key someone asked you for as a favor.
• Are you reading the schema your telemetry actually writes to? Workspace-based instances moved the tables. An empty result and a broken pipeline look identical.
• Does anything in your pricing path treat a rate as configuration? If a rate is a mutable cell, you have a number you can state and can't defend.
• Does your rollup window align to the same boundary as your grouping? If it's a relative offset over calendar buckets, you are losing the edge of every bucket, quietly and permanently.