Flagship module · where every byte goes · 7 beats

Training Memory

A 70B model in BF16 is 141 GB of weights — and about 1.13 TB of training state before a single activation exists. This module derives all five terms from the update rule that requires them, then hands you a calculator to size real runs. It ends deliberately at the point where parameter sharding stops helping, which is where distributed training has to begin.

Core path ~31 min: 01 → 04 then 07 · Systems deep dive +24 min: 05 → 06

Beat 01 / 07CONCEPT~5 min

What problem existed?

After this beat you can: You can explain why a model that fits in memory still cannot be trained in memory.

A 70B-parameter model in BF16 is about 141 GB of weights. That already exceeds one 80 GB GPU, and inference people stop the story there.

Training does not. The weights turn out to be the smallest of the four persistent terms.

The reason is mixed precision. Forward and backward run in BF16 for speed, but the parameter update cannot.

Adding a small update to a large weight in BF16 loses the update entirely to rounding — BF16 has roughly 8 mantissa bits, so an update smaller than about 2⁻⁸ of the weight rounds away. The loss curve stalls.

So an FP32 master copy of every parameter is kept. And Adam keeps two more FP32 buffers per parameter for its moment estimates.

Each piece exists to prevent a specific failure under that recipe. This is the classic mixed-precision Adam accounting, not an invariant of training — low-bit optimizer states, different optimizer designs, and alternative precision schemes all change the constant.

Then there is a fifth term that behaves differently from the other four.

Activations are not parameter-scaled. You cannot get them from P. They are set by runtime shape and architecture: sequence length, micro-batch, depth, width, head count.

They are therefore the term you control at launch time, and the term that surprises people when a context-length experiment OOMs a run that trained fine yesterday with the same weights.

Getting this wrong is expensive in a specific way: you discover it minutes into a job that took an hour to schedule. Engineers who can size a run on paper first are doing arithmetic, not magic — and by the end of this module you will have done it for a 70B run yourself.

FIVE TERMS, TWO SCALING LAWS MODEL-STATE MEMORY ACTIVATION MEMORY primarily parameter-scaled runtime + architecture-scaled P × bytes/state f(L, h, heads, s, micro-batch, arch) ──────────────────────── ────────────────────────────────── weights (BF16) stored forward tensors per layer gradients (BF16/FP32) master weights (FP32) ← not obtainable from P alone, optimizer moments (FP32 × 2) though L and h do grow with size Confusing which term a knob moves is the root of most sizing mistakes.

Two different scaling laws under one memory budget. Every lever touches one of them, not both.

How this comes up in interviews

Try answering out loud before revealing.

Prove it before continuing

Which memory term cannot be computed from parameter count alone?

Answer the question above to continue — committing unlocks, right or wrong.
See it in production

You have the mechanism. These teams had to decide with it under a production constraint — and you commit to an architecture before you are shown theirs.