Number formats — Linear algebra for LLMs

A float is sign × mantissa × $2^{\text{exponent}}$

\displaystyle x = (-1)^{s} \times 1.m \times 2^{e}

A floating-point number spends its bits on three things: a sign, an exponent (how big: the range) and a mantissa (how many significant digits: the precision). Every format below is a different split.

Write

Exponent bits → range. Mantissa bits → precision.

The formats you'll meet

Format	s/e/m bits	Bytes	Max
FP32	1/8/23	4	$3.4 \times 10^{38}$
BF16	1/8/7	2	$3.4 \times 10^{38}$
FP16	1/5/10	2	65,504
FP8 E4M3	1/4/3	1	448
FP8 E5M2	1/5/2	1	57,344
INT8	—	1	127
INT4	—	0.5	7

Write

Bytes: FP32 4, BF16/FP16 2, FP8/INT8 1, INT4 ½.

BF16 vs FP16: range vs precision

\displaystyle \text{BF16: max} \approx 3.4 \times 10^{38}, \; 8 \text{ bits}

\displaystyle \text{FP16: max } 65{,}504, \; 11 \text{ bits}

BF16 keeps FP32’s range but only 8 significant bits (counting the implicit leading 1): between 256 and 512 its numbers are 2 apart, so $256 + 1$ rounds back to 256. FP16 is about 8× finer but tops out at 65,504, so FP16 training needs loss scaling to keep gradients in range. BF16 doesn’t, which is why it became the default for training.

Write

BF16 = FP32’s range, low precision. FP16 = more precision, small range.

Accumulate in higher precision

Adding thousands of small products in BF16 loses digits: once the running sum is large, small terms round away. Matrix units multiply BF16 inputs but accumulate in FP32, and optimizers keep an FP32 master copy of the weights for the same reason (Part 8’s 16 bytes per parameter).

Weight memory = parameters × bytes

\displaystyle 8.03 \times 10^9 \text{ params}

\displaystyle \text{FP32: } 32.1 \text{ GB} \quad \text{BF16: } 16.1 \text{ GB}

\displaystyle \text{FP8: } 8.0 \text{ GB} \quad \text{INT4: } 4.0 \text{ GB}

Llama 3 8B in each format, weights only. Halving the bytes halves the memory and, as the next deck shows, roughly doubles how fast you can generate.

Write

Weight memory =

N

× bytes per parameter.

✏️ Your turn

Size a 70B model

\displaystyle N = 70 \times 10^9

How much memory do the weights need in BF16? In INT4?

Work it out on paper, then go to the next card to check.

Answer

\displaystyle 70 \times 10^9 \times 2 = 140 \text{ GB}

\displaystyle 70 \times 10^9 \times 0.5 = 35 \text{ GB}

In BF16 it needs at least two 80 GB GPUs before any KV cache; in INT4 it fits on one.

In PyTorch

torch.finfo(torch.bfloat16)      # eps=0.0078125, max≈3.39e38
torch.finfo(torch.float16)       # eps≈0.000977,  max=65504
x.to(torch.bfloat16)

📌 Recap

Number formats in one card

Exponent bits set the range; mantissa bits set the precision.
BF16: FP32’s range in 2 bytes; FP16: finer but max 65,504; FP8: 1 byte.
Accumulate in FP32.
Weight memory = parameters × bytes.

Write

The four lines above.