Neural networks, deep learning, NLP · insight-0448
Use causal masking when the prediction must not see later tokens or events
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- ML engineer, Applied AI, Data scientist
- Industries
- e-commerce, saas
Summary: situation, action, reason, boundary
Situation
You use a bidirectional encoder or an unmasked Transformer over a customer's event stream to predict their next action in real time.
Action
For next-step or streaming prediction, apply a causal mask (or a unidirectional model) so position t attends only to positions up to t, and validate with features cut at each prediction time.
Reason
Decoder self-attention masks future positions to preserve the autoregressive property, whereas BERT-style encoders are trained to use context on both sides.
Boundary
Next-event prediction, token-level forecasting, real-time scoring of streams.
Symptom or risk
During training each position attends to later events, so offline accuracy is excellent and the model collapses when only past events exist at serving time.
Exceptions
Classifying a completed sequence, such as a closed support ticket, can use bidirectional context.
Trade-offs
Causal models see less context per position, which can lower accuracy on whole-sequence tasks.
Pitfalls
Building training sequences from the complete session but serving on partial sessions.
Lessons and concepts
Claims and their sources
Vaswani et al. mask future positions in decoder self-attention so predictions depend only on known earlier outputs.
Devlin et al. pretrain BERT as a deep bidirectional encoder that conditions on left and right context.