Layer Normalization vs Batch Normalization

Medium

neural-networks Google OpenAI Meta

Layer normalization normalizes across features within a single sample, while batch normalization normalizes across samples in a mini-batch. In which scenario is layer normalization strongly preferred?

Multiple Choice

Sequence models like transformers, because layer norm does not depend on batch statistics and handles variable-length sequences Large-batch image classification, because layer norm is more computationally efficient than batch norm Single-sample inference only, because layer norm requires a batch size of exactly 1 Generative adversarial networks, because layer norm stabilizes the discriminator's gradients

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations

Layer Normalization vs Batch Normalization

Multiple Choice

Related problems

Deep-dive guides