Neural networks, deep learning, NLP · insight-0451
Start text classification with TF-IDF and a linear model
- Evidence type
- empirical finding
- Difficulty
- easy
- Roles
- Data scientist, Applied AI, Data analyst
- Industries
- aviation
Summary: situation, action, reason, boundary
Situation
You need to route airline customer complaints into 8 queues and the first idea is to fine-tune a Transformer.
Action
Fit TF-IDF on word and character n-grams with logistic regression or a linear SVM on the same split and metric. Move to a Transformer only if it beats this baseline by a margin that matters for the queue's cost of error.
Reason
Linear models on n-gram features are fast, interpretable and often close to deep classifiers on topic-style tasks.
Boundary
Topic, intent and routing classification with keyword-rich text.
Symptom or risk
Without a lexical baseline you cannot tell whether the Transformer's cost, latency and maintenance buy anything; many routing tasks are driven by keywords.
Exceptions
Tasks that depend on word order, negation, long-range context or cross-lingual transfer usually favour pretrained models.
Trade-offs
The baseline needs no GPU and is easy to explain; it degrades on paraphrases and unseen vocabulary.
Pitfalls
Tuning the Transformer extensively and leaving the baseline at default regularization.
Lessons and concepts
Claims and their sources
Joulin et al. report that the fastText linear classifier is often on par with deep learning classifiers in accuracy while orders of magnitude faster.
Inverse document frequency weighting originates with Sparck Jones's term-specificity work.