LLMs, GenAI, RAG, agents, evaluation · insight-1556
Hand work across sessions through a fixed checklist the agent may tick but not rewrite
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Applied AI, ML engineer
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A multi-day task spans many context windows, and each new session must know what is done and what remains.
Action
Create the full checklist once up front, store it in a structured file, allow sessions to change only each item's status field plus a short note, and have each session pick one unfinished item.
Reason
The long-running agent report used a JSON feature list that agents could change only by pass or fail status, one feature per session, together with a progress file.
Boundary
Multi-session builds, migrations, large data clean-ups.
Symptom or risk
The agent edits the plan to match what it managed to do, drops hard items, or redoes finished work, so scope quietly shrinks.
Exceptions
Genuinely exploratory work needs a plan that can change, with the changes reviewed.
Trade-offs
An up-front list can be wrong, so allow changes only through a reviewed path.
Pitfalls
Marking an item passing without the evidence the checklist requires.
Lessons and concepts
Claims and their sources
Anthropic's long-running agent report describes an initializer-written JSON feature list where later agents may change only a feature's pass or fail status, with one feature per session and a progress file for handoff.