Self-Attention Mechanism

Medium

llms Google DeepMind OpenAI

In the transformer self-attention mechanism, what are the roles of the Query, Key, and Value vectors?

Multiple Choice

Query encodes position, Key encodes semantics, and Value encodes syntax Query and Key compute compatibility scores via dot product, and the resulting weights are used to aggregate Value vectors Query is the input, Key is the output, and Value is the loss function All three are identical projections used for redundancy

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations

Self-Attention Mechanism

Multiple Choice

Related problems

Deep-dive guides