Skip to content
ML Academy

DL, NLP and LLMs

Interview question

  • midlevel
  • 8 minsuggested time
  • 3Applied AI, ML engineer, Data scientist

Explain self-attention to an engineer who knows RNNs. Why did Transformers largely replace RNNs for language tasks, and what does attention cost?

Tests: Mechanism of scaled dot-product attention, multi-head attention and positional information; comparison with recurrence; computational complexity.

day 37day 38day 39day 42

Sign in to practise: answers are saved before model answers are shown.