Speculative Decoding

Hard

llms Google DeepMind NVIDIA

How does speculative decoding speed up LLM inference?

Multiple Choice

It uses multiple GPUs to generate different parts of the response simultaneously A small draft model proposes tokens that the large model verifies in parallel, accepting multiple tokens per forward pass It skips attention computation for tokens with high predicted confidence It pre-computes all possible continuations and caches them

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations

Speculative Decoding

Multiple Choice

Related problems

Deep-dive guides