Distributed Training: DDP

Medium

deep-learning-libraries Meta NVIDIA Amazon

Multiple Choice

It splits the model layers across GPUs, with each GPU computing one layer It replicates the model on each GPU, partitions the data, and synchronizes gradients across GPUs using all-reduce after each backward pass It trains different models on different GPUs and picks the best one It uses a single GPU but simulates multiple GPUs in software

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations