Projections

Drop $x = [2, 3]$ straight down onto the horizontal axis and you land at $[2, 0]$. You kept the part of $x$ along $e_1$ and threw away the rest.

With $|u| = 1$: $x \cdot u$ is the shadow length (Part 1), and multiplying by $u$ points it the right way. Here $x \cdot u = 1.2 + 0.8 = 2$, so the projection is $2u = [1.2, 1.6]$.

Subtract the projection and what’s left has nothing along $u$: it’s orthogonal to $u$. Every vector splits into a part along $u$ plus a part at 90° to it.

$(x \cdot u)u = u(u^\top x) = (uu^\top)x$, so projecting is multiplying by $P = uu^\top$: an outer product, rank 1. Projecting twice changes nothing more: $P^2 = P$. Removing the direction is $I - P$.

Write $\operatorname{proj}_u x$ and $x - \operatorname{proj}_u x$, then check the second is orthogonal to $u$.

Stack $k$ orthonormal directions as the columns of $U$ ($d \times k$). Then $UU^\top x$ keeps exactly the part of $x$ inside their span. With $k = 1$ it’s the $uu^\top$ you just used.

• $\operatorname{proj}_u x = (x \cdot u)u = uu^\top x$ for unit $u$.
• $x - (x \cdot u)u$ is orthogonal to $u$: the direction removed.
• $P = uu^\top$ (or $UU^\top$) has $P^2 = P$.
• Ablation, steering, probes and mean-centering are all projections.