A/B Test Duration

Medium

statistics Airbnb Netflix Uber

In A/B testing, why is it problematic to stop the test as soon as the p-value drops below 0.05?

Multiple Choice

The p-value is unreliable before the test ends It inflates the false positive rate due to repeated testing (peeking problem) It violates the assumption of normality The sample size becomes too small to be meaningful

Correct!

Incorrect — try again next time!

Upgrade to Pro to unlock solutions and explanations

A/B Test Duration

Multiple Choice

Related problems

Deep-dive guides