Attention Mechanism in Seq2Seq

Medium

neural-networks Google Meta ByteDance

In sequence-to-sequence models, the attention mechanism was introduced to address a specific limitation of the basic encoder-decoder architecture. What is this limitation?

Multiple Choice

The fixed-size context vector creates an information bottleneck that loses detail from long input sequences The decoder cannot generate outputs longer than the input sequence The encoder processes tokens left-to-right only, ignoring right-to-left context The encoder and decoder cannot share vocabulary embeddings

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations

Attention Mechanism in Seq2Seq

Multiple Choice

Related problems

Deep-dive guides