Adam Optimizer Internals

Medium

neural-networks Google DeepMind NVIDIA

The Adam optimizer combines ideas from two earlier optimization methods. Which two techniques does Adam unify, and how?

Multiple Choice

Momentum (exponential moving average of gradients) and RMSProp (exponential moving average of squared gradients) L1 regularization (Lasso) and L2 regularization (Ridge) applied to the gradient updates Nesterov accelerated gradient and AdaGrad's cumulative squared gradient scaling Batch normalization of gradients and learning rate warmup scheduling

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations

Adam Optimizer Internals

Multiple Choice

Related problems

Deep-dive guides