KV Cache in Autoregressive Generation

Hard

llms NVIDIA Google Microsoft

What is the KV cache and why is it critical for efficient autoregressive generation?

Multiple Choice

It caches the model weights to speed up layer computation It stores previously computed Key and Value vectors to avoid redundant recomputation during sequential token generation It caches the tokenizer's vocabulary for faster encoding It pre-computes and stores the output logits for all possible next tokens

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations

KV Cache in Autoregressive Generation

Multiple Choice

Related problems

Deep-dive guides