Introduced on day 41
TF-IDF
A weighting of bag-of-words counts that multiplies how often a word occurs in a message by how rare the word is across messages, so distinctive words dominate.
In detail
Inverse document frequency is roughly the log of the number of messages divided by the number containing the word; implementations add smoothing and normalise each row. TF-IDF gives no relation between different words. With logistic regression on top it is the standard strong lexical baseline for text classification and the ancestor of BM25 scoring in search.