Neural networks, deep learning, NLP · insight-1443

Choose which segment of a text pair to truncate instead of accepting the default strategy

Situation
A BERT-style pair classifier takes a customer message and a long policy clause with max_length=128 and truncation=True.
Action
Pick only_first or only_second for the segment that is safe to shorten, or pre-trim by sentence, and log the share of pairs truncated per segment on the dev set.
Reason
Hugging Face documents that longest_first removes tokens one at a time from the longest sequence of a pair, while only_first and only_second shorten only that segment.
Boundary
Cross-encoders, NLI-style pairs, question and context pairs.

Symptom or risk

Tokens are removed from whichever segment is longer, so the cut can fall on the exact words the label depends on, and nothing signals it.

If under about 1 percent of pairs are truncated the choice hardly matters.

Truncating one side hard can lose more of it than a balanced cut would.

Tuning max_length on the test set; comparing runs that truncated differently.

Hugging Face documents truncation=True or longest_first as removing tokens one at a time from the longest sequence in a pair, and only_first and only_second as truncating just that sentence of the pair.
Padding and truncation (Hugging Face Transformers documentation)