LLMs, GenAI, RAG, agents, evaluation · insight-1525

Enforce hard output limits in code, because length instructions in prompts are only targets

Situation
A delay push notification must fit 120 characters in the app, and the prompt says at most 120 characters.
Action
Treat the limit as a validated constraint: measure length in the channel's units after generation, request a shorter rewrite on failure, fall back to a template built from structured fields, and log the overflow rate per prompt and model version.
Reason
A model generates tokens rather than characters and typically does not count exactly while writing, so a prompt states a target and only code can guarantee a limit; overflow rates also shift between model versions.
Boundary
Notifications, SMS, titles, form fields and any channel with a hard length limit.

Symptom or risk

A share of messages overflow, the client cuts them mid-word or mid-gate-number, and the truncated text hides the one fact that mattered.

Soft length preferences, such as a short paragraph, where an occasional overrun is harmless.

Retries add latency and cost, and template fallbacks read less naturally.

Counting characters in code while the channel limits bytes or encoded units, which differ for non-English text.