LLMs, GenAI, RAG, agents, evaluation · insight-1617
Inventory every model call site so a deprecation notice answers what breaks and by when
- Evidence type
- practitioner heuristic
- Difficulty
- easy
- Roles
- ML engineer, Applied AI
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A vendor announces that a model version will be retired, and prompts are spread across services, notebooks and no-code tools.
Action
Generate an inventory from gateway logs, provider usage reports and code search: feature, owner, model id, prompt version, endpoint and eval suite. Alert on calls to any model with an announced shutdown, and open one ticket per row when a notice arrives, with the replacement tested on that row's own eval suite.
Reason
Vendors publish shutdown dates per model with notice periods that differ by model class, so the date alone is not a plan. As read in October 2026, Anthropic documents at least 60 days' notice for publicly released models and OpenAI lists a shutdown date per model.
Boundary
Organisations with more than a handful of LLM features or more than one team calling providers.
Symptom or risk
Teams spend weeks searching, and a forgotten batch job fails after the shutdown date.
Exceptions
A single feature with one owner and one call site.
Trade-offs
Keeping the inventory current needs a gateway or a convention; shadow keys defeat it.
Pitfalls
Listing aliases that silently move to a new version, and omitting scheduled jobs and notebooks.
Lessons and concepts
Claims and their sources
Anthropic's deprecation page, as read on 2026-10-11, states at least 60 days' notice for publicly released models and advises testing replacements well before retirement.
OpenAI's deprecations page, as read on 2026-10-12, lists a shutdown date per model, notifies active customers by email and names a replacement for some models.