Flash Attention

Hard

llms NVIDIA Meta Google

What is the key innovation of Flash Attention that makes it faster than standard attention?

Multiple Choice

It approximates attention with random sampling to reduce computation It uses tiling to compute attention in blocks within fast SRAM, avoiding materializing the full attention matrix in HBM It replaces softmax with a linear function to achieve O(n) complexity It reduces the number of attention heads to speed up computation

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations

Flash Attention

Multiple Choice

Related problems

Deep-dive guides