Flash Attention

Hard

llms NVIDIA Meta Google

Multiple Choice

It approximates attention with random sampling to reduce computation It uses tiling to compute attention in blocks within fast SRAM, avoiding materializing the full attention matrix in HBM It replaces softmax with a linear function to achieve O(n) complexity It reduces the number of attention heads to speed up computation

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations