Engineering, production, governance, cost, communication · insight-0287

Track unit cost per decision, not just the monthly cloud bill

Situation
Monthly ML infrastructure cost rose 40%, and nobody can say whether it is growth, waste or a new model.
Action
Tag resources by model, environment and team; report cost per thousand predictions, per training run and per business decision alongside volume; and flag idle resources and failed-run spend.
Reason
Unit costs separate growth from inefficiency and connect spend to the value each model produces.
Boundary
Training clusters, inference endpoints, LLM API usage, data pipelines.

Symptom or risk

Aggregate bills hide which model, team or pipeline drives cost, so cost reviews become arguments instead of decisions.

Very small deployments may not justify tagging effort; revisit when spend becomes material.

Allocating shared costs needs a rule that some teams will dispute; publish it.

Ignoring development and experimentation spend, which can exceed production.