Cut LLM cost without losing quality

A retail bank (simulated) uses an LLM to classify and summarise customers' secure messages so agents can triage them. Replies are due within 4 business hours, so the workload is not latency-critical. A separate nightly job summarises closed conversations for quality assurance.

Workload (simulated): 1,000,000 requests a month. Each request has 3,000 input tokens, of which 2,000 are a static prefix (instructions and policy excerpts, identical for every request) and 1,000 are the customer conversation; output is 200 tokens.

All prices are hypothetical, in USD per million tokens:

Model	Input	Output	Cached input read
Large	3.00	15.00	0.30
Small	0.25	1.25	not used here

Real providers price caching, cache writes and batch discounts differently; check current documentation before using any of this structure.