Adam Optimizer Internals

Medium

neural-networks Google DeepMind NVIDIA

Multiple Choice

Momentum (exponential moving average of gradients) and RMSProp (exponential moving average of squared gradients) L1 regularization (Lasso) and L2 regularization (Ridge) applied to the gradient updates Nesterov accelerated gradient and AdaGrad's cumulative squared gradient scaling Batch normalization of gradients and learning rate warmup scheduling

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations