He Initialization for ReLU

Medium

neural-networks DeepMind Google NVIDIA

Multiple Choice

ReLU zeroes out roughly half the activations, halving the variance, so the factor of 2 compensates for this reduction ReLU outputs are unbounded, so larger initial weights are needed to prevent underflow He initialization accounts for the bias terms that Xavier initialization ignores ReLU networks always have more layers, requiring proportionally larger initial weights

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations