Transformer and Attention Interview Questions (With Answers)

Why Attention Dominates This Category

Attention is the mechanism transformers are built from, and it's also one of the few deep learning concepts interviewers expect you to explain mechanically, not just describe at a high level. Whether the role is LLM-focused or general neural networks, you should be able to walk through what attention actually computes, why multiple heads help, where the idea came from before transformers existed, and why modern implementations look nothing like the naive version on paper. This guide covers all four.

The reason interviewers lean on attention so heavily is that it rewards genuine understanding and punishes memorization almost immediately. It's easy to recite "queries, keys, and values" without being able to explain what a query actually represents, or to say "multiple heads capture different relationships" without being able to explain what breaks if you only used one. The sections below are written to close exactly those gaps.

Self-Attention: The Mechanism Itself

"Walk me through what self-attention actually computes, step by step."

Self-attention lets every token in a sequence look at every other token — including itself — and decide how much to weight each one when building its own updated representation. Mechanically, each token's embedding is projected into three vectors: a query, a key, and a value. The query of one token is compared against the keys of every token (via a dot product), scaled, and passed through softmax to produce attention weights that sum to one. Those weights are used to compute a weighted sum of the value vectors, which becomes that token's new representation.

The detail that separates a memorized answer from an understood one: the scaling by the square root of the key dimension exists specifically to keep the dot products from growing too large and pushing softmax into regions with near-zero gradients. Without that scaling, as the dimensionality of the query and key vectors grows, dot products tend to grow with it, and a softmax fed very large or very unevenly sized inputs saturates — most of the weight collapses onto one token and gradients through the rest vanish.

It's also worth being able to explain, unprompted, why this is called "self" attention: the queries, keys, and values are all derived from the same input sequence, as opposed to cross-attention, where queries come from one sequence (say, a decoder) and the keys and values come from a different sequence (an encoder's output). That distinction resurfaces constantly in encoder-decoder architectures and in multimodal models that attend from text tokens to image features. Practice: Self-Attention Mechanism.

Multi-Head Attention: Why One Head Isn't Enough

"If self-attention already lets every token see every other token, why do transformers use eight or more attention heads instead of one?"

A single attention head produces one weighted combination of the sequence per token, which forces it to compromise between different kinds of relationships — say, tracking which noun a pronoun refers to versus tracking overall sentence structure. Multi-head attention runs several smaller attention operations in parallel, each on its own learned linear projection of the queries, keys, and values, then concatenates the results and projects them back to the model's dimension.

Interviewers are listening for the intuition that heads can specialize — some attend locally, some attend to long-range dependencies, some track syntactic patterns — without you needing to name exactly what any specific head learns, since that's an empirical, model-specific detail.

A common trick question: "does adding more heads always help?" It doesn't. Splitting the model's dimensionality across more heads means each head operates in a smaller subspace, so beyond a certain point, adding heads gives each one less room to represent anything useful, and you also pay a real computational and memory cost for every additional head. The number of heads is a hyperparameter tuned like any other, not a lever you maximize blindly. It's also worth knowing the shape mechanics: if the model dimension is d and there are h heads, each head typically operates on d/h dimensions, and the concatenated output of all heads is projected back to dimension d before continuing to the next layer. Practice: Multi-Head Attention.

Attention Before Transformers: The Seq2Seq Origin Story

"Attention existed before transformers. What problem was it originally solving?"

This question checks historical grounding. Before transformers, sequence-to-sequence models used an RNN encoder to compress an entire input sequence into a single fixed-length vector, which the decoder then had to unpack to generate the whole output — a severe bottleneck for long inputs. Bahdanau-style attention fixed this by letting the decoder, at each output step, compute a weighted combination over all of the encoder's hidden states rather than relying on one compressed vector, so the decoder could dynamically focus on the most relevant part of the input for whatever it was generating next.

Transformers kept the core idea — a weighted combination over relevant states — and removed the recurrence entirely, applying the same idea to every pair of tokens in parallel instead of one decoder step at a time.

This history matters for a subtler reason too: it explains why transformers needed positional encodings while RNNs never did. An RNN processes tokens one at a time in order, so position is implicit in the computation itself; a transformer's self-attention treats the sequence as an unordered set of vectors to compare, so without an explicit signal injected into each token's embedding, "the dog bit the man" and "the man bit the dog" would look identical to the attention mechanism. Being able to connect that consequence back to the removal of recurrence is a strong signal that you understand the architecture's history, not just its current form. Practice: Attention Mechanism in Seq2Seq.

Flash Attention: Making the Same Math Faster

"Standard attention is quadratic in sequence length. How does Flash Attention make it practical for long contexts without changing the output?"

Flash Attention doesn't change what attention computes — it's mathematically exact, not an approximation. What it changes is how the computation is scheduled on GPU hardware. Standard implementations materialize the full attention score matrix in GPU high-bandwidth memory (HBM), which is slow to read and write and scales quadratically with sequence length. Flash Attention instead tiles the computation into blocks that fit in fast on-chip SRAM, fusing the matrix multiplications, softmax, and masking so intermediate results never have to round-trip through HBM.

The result is the same output as standard attention, computed faster and with far less memory overhead, which is a big part of why long-context models became practical. Interviewers want to hear "exact, not approximate" and "the bottleneck was memory movement, not floating-point operations" — that's the whole insight.

It's worth contrasting this explicitly with approaches that do approximate attention, such as sparse or linear attention variants that skip or approximate parts of the computation to reduce the quadratic cost mathematically. Flash Attention takes a different path entirely: it doesn't reduce the number of operations, it reduces wasted time moving data that GPUs would otherwise spend idle waiting on. Being able to name that distinction — an algorithmic-complexity fix versus a memory-access fix — is what separates candidates who've read the abstract from candidates who understand why it mattered enough to become the default. Practice: Flash Attention.

How to Prepare

1. Be able to draw the query-key-value flow of self-attention on a whiteboard, including where the softmax and scaling happen.
2. Know the one-sentence version of why seq2seq attention was invented — the fixed-length bottleneck — before you talk about transformers at all.
3. For Flash Attention specifically, lead with "same output, different memory access pattern" — that single distinction answers most of the follow-ups.

For fine-tuning, inference, and evaluation questions that build on these mechanics, see the full LLM engineer interview questions hub.