Xavier Weight Initialization

Medium

neural-networks DeepMind Google Meta

Xavier (Glorot) initialization sets weights with variance 2/(n_in + n_out). Why is this preferred over initializing all weights to a small constant for networks with sigmoid or tanh activations?

Multiple Choice

It keeps the variance of activations and gradients roughly constant across layers, preventing signal decay or explosion It ensures all weights are positive, which is required for sigmoid and tanh to operate correctly It initializes weights to the global optimum of the loss surface for linear networks It makes the network invariant to the number of layers, so depth has no effect on training

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations

Xavier Weight Initialization

Multiple Choice

Related problems

Deep-dive guides