Neural networks, deep learning, NLP · insight-1451
Report metrics per detected language when messages mix Turkish and English
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Data scientist, Data analyst, Applied AI
- Industries
- aviation, e-commerce
Summary: situation, action, reason, boundary
Situation
Customer messages for an airline are Turkish, English and mixed, and one classifier reports a single accuracy.
Action
Add language identification per message with a mixed flag, report metrics per slice with intervals, and test a shared multilingual model against per-language models on the weakest slice.
Reason
Slicing exposes differences that averages hide, and cross-lingual results vary widely by language, so a single number is not safe for a mixed population.
Boundary
Multilingual support, search and social text.
Symptom or risk
The aggregate follows the majority language, a weak minority or mixed slice is hidden, and a vocabulary split by language dilutes shared evidence.
Exceptions
Traffic that is effectively one language.
Trade-offs
Language-ID errors on short messages; more slices mean wider intervals.
Pitfalls
Running language ID on messages like "ok" or codes; dropping mixed messages from the test set.
Lessons and concepts
Claims and their sources
XTREME reports that cross-lingual transfer results vary widely across languages.