Exploding Gradients and Gradient Clipping

Hard

neural-networks Google DeepMind Meta

Multiple Choice

Repeated multiplication by large weight matrices causes exponential gradient growth; clipping rescales the gradient when its norm exceeds a threshold, preserving direction but bounding magnitude Large input values overflow the activation functions; clipping truncates inputs before they enter each layer The loss function has discontinuities that produce infinite gradients; clipping smooths the loss landscape Batch normalization layers amplify gradients at each step; clipping disables normalization when gradients are too large

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations