Mixture of Experts Architecture

Hard

llms Google DeepMind Meta

Multiple Choice

It trains multiple complete models and averages their outputs It routes each token to a small subset of expert networks, keeping per-token compute low while total parameters are high It uses a single expert for all tokens but switches experts between layers It reduces model parameters by sharing weights across all experts

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations