Neural networks, deep learning, NLP · insight-1460

Keep invariance and directional tests for a text classifier next to its accuracy report

Situation
A complaint-priority classifier passes its test set and then fails on trivial rewrites in the first demo.
Action
Maintain small templated groups: invariance (names, airports, typos must not change the label), directional (adding "not" must move the score the expected way) and minimum functionality (obvious cases), and run them on every candidate.
Reason
The CheckList paper proposes behavioural testing with a matrix of linguistic capabilities and test types, and reports critical failures in commercial and state-of-the-art models and more bugs found by practitioners using it.
Boundary
Classifiers, extractors and similarity models in text products.

Symptom or risk

Aggregate accuracy hides brittle behaviour on typos, swapped names and negation, and regressions between model versions go unseen.

Capabilities your product never meets.

Templates are synthetic and written by you; passing them proves little beyond those cases.

Writing tests from the model's own outputs; treating a pass rate as accuracy.

CheckList (Ribeiro et al., ACL 2020) proposes behavioural testing of NLP models using a matrix of linguistic capabilities and test types, and reports critical failures in commercial and state-of-the-art models (abstract, arXiv 2005.04118).
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList