Multi-Head Attention

Medium

llms Google DeepMind Meta

What advantage does multi-head attention provide over single-head attention in transformers?

Multiple Choice

It reduces the computational cost by parallelizing attention across heads It allows the model to capture different types of relationships by attending to different representation subspaces simultaneously It increases the context window length proportionally to the number of heads It eliminates the need for positional encoding

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations

Multi-Head Attention

Multiple Choice

Related problems

Deep-dive guides