Concept map: Neural vs classical baseline comparison

Taught and used in

Deep learning and neural networks, Validation and metrics

Insights 16

• Use Adam to iterate fast, but test SGD with momentum before final training
• Beat a tuned gradient-boosting baseline before shipping a neural net on tabular data
• Give the baseline and the neural model the same tuning budget
• Report results over several seeds before claiming a neural improvement
• Start text classification with TF-IDF and a linear model
• Judge custom-trained embeddings by downstream lift, not by nearest-neighbour anecdotes
• Prefer a small deep ensemble over MC dropout when uncertainty must hold up
• Vary the initialisation seed and the data-order seed separately to see which one moves the metric

Projects 8

• Will this flight fill? A from-scratch numpy MLP vs gradient boosting on airline booking snapshots
• Catalogue thumbnails: a small CNN vs pixel baselines on Fashion-MNIST
• Day-ahead household load: a 1-D CNN or GRU against seasonal naive and lag regression
• Few labels, foreign domain: frozen ImageNet features vs a small CNN on EuroSAT land cover
• Same product, different shop: TF-IDF vs sentence embeddings for product matching on Abt-Buy
• Routing airline complaints: TF-IDF + logistic regression vs a small neural text classifier