Mixed Precision Training

Training with a mix of labeled and unlabeled data Using float16 for most computations and float32 for critical accumulations, reducing memory and increasing speed on compatible hardware Alternating between training and evaluation every other batch Using different learning rates for different layers