Vanishing Gradient Problem

Easy

neural-networks Google DeepMind Meta

In a deep network using sigmoid activations, gradients can become extremely small in early layers during backpropagation. Which of the following best explains why this happens?

Multiple Choice

The sigmoid function clips large values, discarding gradient information The chain rule multiplies many small sigmoid derivatives, causing gradients to shrink exponentially with depth The bias terms in each layer absorb most of the gradient signal Sigmoid outputs are always positive, which causes gradients to oscillate and cancel out

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations

Vanishing Gradient Problem

Multiple Choice

Related problems

Deep-dive guides