Concept map: AI cost

Taught and used in

MLOps and LLMOps, Business judgment

Insights 34

• Budget attention cost by sequence length before raising the context limit
• Run the first training step on the largest batch and longest sequence you will see
• Trade compute for memory with activation checkpointing before shrinking the model
• Measure GPU memory with the framework's allocator counters, not nvidia-smi alone
• Profile the input pipeline when GPU utilization stays low
• Separate total from active parameters when sizing mixture-of-experts models
• Estimate key-value cache memory before promising long contexts on self-hosted models
• Use QLoRA to fit a larger base model, then evaluate the quantized result honestly

Projects 6

• Tool-calling agent vs deterministic workflow for rebooking proposals after a cancellation
• Cost and latency routing study: small model, large model, router and cascade on checkable answers
• Intent classification for banking support: LLM zero/few-shot vs a TF-IDF baseline on cost, accuracy and failure modes
• Embedding drift monitoring for a virtual assistant, evaluated on simulated traffic with known injected shifts
• PII redaction pipeline evaluation before documents cross an LLM trust boundary
• LLM question-answering service: versioned golden set, regression gate and request observability