Welcome

This course brings back the linear algebra you need to read, write and debug transformer code. You may have learned it years ago: vectors, matrices, eigenvalues. Here each idea comes back the way it’s used in a transformer: as shapes, matrix products, attention scores, low-rank updates and FLOP counts.

Who it’s for: engineers who learned linear algebra once and work with LLMs now. No proofs. Every idea ends with where it shows up in a transformer or in PyTorch.

How it works

Every topic is a deck of flashcards. Each card shows one idea, one operation or one step of a worked example. Press Next and the next card builds on it: a matrix, then its transpose, then the rule, then one for you to try.

1. Get paper and a pen. On each card, copy what the ✍️ Write box says. Writing it out is what makes it stick.
2. On ✏️ Your turn cards, work it out before you press Next. The next card has the answer.
3. 🔗 In a transformer boxes link the idea to the model and to PyTorch.
4. Each part ends with a check: 5 or 6 questions, with new numbers each time you try. You can skip a question, or the whole check. Get 80% right on the first try for the ✓.

To	On a phone	On a computer
Go to the next card	Next, swipe left, or tap the right side of the card	→, Space or Enter
Go back	←, or swipe right	←
See the whole deck at once	All cards	All cards

Your place in each deck is saved in this browser, so you can stop at any card and pick up later.

The map

Part	You’ll be able to	Time
0 · Start here	read the notation; find your gaps with a quick diagnostic	10 min
1 · Vectors and embedding space	compute dot products, norms and cosine similarity by hand; say why attention divides by $\sqrt{d_k}$	35 min
2 · Matrices and matrix multiplication	multiply matrices by hand four ways; predict shapes; count FLOPs	45 min
3 · Matrices as transformations	read a matrix as a map; use rotations, rank and projections	35 min
4 · Tensors, shapes and einsum	predict any tensor’s shape; split heads without scrambling tokens	35 min
5 · Attention, one operation at a time	trace one attention head on paper, end to end	50 min
6 · The rest of the transformer	explain every matrix in a block; count a model’s parameters and FLOPs	45 min
7 · Decompositions and low rank	read an SVD; size a LoRA adapter	40 min
8 · Gradients through matrices	derive a linear layer’s gradients by matching shapes; size a training run	35 min
9 · Numbers on real hardware	estimate memory, arithmetic intensity and quantization error	35 min
10 · Capstone and reference	size a real model end to end; keep the cheat sheet	30 min

Each part fits in one sitting. Start with the notation, then take the optional diagnostic to see which parts you can go through quickly.