LLMs, GenAI, RAG, agents, evaluation · insight-1641

Tag every model call with feature, tenant tier and prompt version so spend can be attributed

Situation
The monthly model invoice jumps 40% and nobody can say which feature, customer segment or release caused it.
Action
Record provider-reported input, output and cached tokens on each call with feature, route, tenant tier, prompt version and model as dimensions. Compute spend from usage times the price table, reconcile with the invoice monthly, and alert on per-feature budgets.
Reason
OpenTelemetry's GenAI conventions define request model and token-usage attributes, and your own attributes make the same data sliceable by the owners who control cost.
Boundary
Any application with more than one model-backed feature or customer segment.

Symptom or risk

Cost control becomes a blanket cut, budgets cannot be enforced per owner, and a prompt change that doubled tokens goes unattributed.

A single-feature prototype with one owner and one invoice line.

High-cardinality tags (per-user ids) inflate telemetry cost; use tiers and hashed buckets.

Allocating spend by request count, which hides that a few features use most tokens.

OpenTelemetry's GenAI span conventions (status Development) define attributes for the request model and for input and output token usage.
OpenTelemetry semantic conventions for generative AI (spans and metrics)