LLMs, GenAI, RAG, agents, evaluation · insight-1555
Design long agent tasks as resumable jobs with checkpoints, not one long session
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- ML engineer, Applied AI
- Industries
- logistics
Summary: situation, action, reason, boundary
Situation
A research or reconciliation agent needs tens of minutes and dozens of tool calls, and a timeout or deploy kills it midway.
Action
Run the task as a background job that saves state after each completed step, keyed by a run id, so a retry or a new worker resumes from the last checkpoint and the user can poll status.
Reason
Checkpointing frameworks exist for this reason, saving graph state per thread to support fault tolerance and human-in-the-loop pauses.
Boundary
Runs longer than a request timeout, or runs that wait on people or slow systems.
Symptom or risk
The whole run restarts from zero, repeating cost and side effects, or the half-done work is lost with no record of where it stopped.
Exceptions
Short tasks gain little from the machinery.
Trade-offs
Checkpoint storage, versioning of saved state and resume logic add complexity.
Pitfalls
Resuming into side effects that already happened without idempotency keys; an in-memory checkpointer also loses state on restart.
Lessons and concepts
Claims and their sources
LangGraph documentation presents checkpointers that save a thread's state as checkpoints for fault tolerance and human-in-the-loop use, and notes that an in-memory checkpointer loses data on restart.