Neural networks, deep learning, NLP · insight-1432
Shuffle the training loader explicitly, because the default does not
- Evidence type
- documented fact
- Difficulty
- easy
- Roles
- ML engineer, Data scientist
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A DataLoader is built from rows exported sorted by route, date or label, and training loss is jagged with a repeating pattern of the same length as the epoch.
Action
Set shuffle to true for the training loader and false for evaluation, print the class counts of the first few batches, and reshuffle every epoch with a seeded generator.
Reason
In PyTorch the shuffle option is off by default; setting it to true reshuffles the data at the start of every epoch.
Boundary
Map-style datasets built from files or tables with any inherent order.
Symptom or risk
Consecutive batches contain one class or one period, gradients swing between them, batch statistics are unrepresentative and the final epoch's content dominates what the model remembers.
Exceptions
Time-series models that need ordered windows; shuffle windows, not rows inside them.
Trade-offs
Shuffling destroys locality for sequential file reads; use a shuffle buffer for large files.
Pitfalls
Shuffling the validation loader, which makes metrics order-dependent if code later assumes alignment.
Lessons and concepts
Claims and their sources
The PyTorch DataLoader's shuffle argument is off by default, and setting it to True reshuffles the data at every epoch.