#training

Wiki 1

• BF16 vs FP16 BF16 keeps FP32's 8-bit exponent and shrinks the mantissa; FP16 shrinks the exponent and keeps the mantissa — for neural training, range beats precision