Soft thresholding

Idea. An orthonormal design decouples the coefficients, and for each one the lasso shrinks the least-squares value towards zero by $\lambda$, setting it to zero if it is smaller than $\lambda$ in size. This is soft thresholding.

Step 1: decouple. Expanding with $X^{\top }X=I$ and $z=X^{\top }y$, $$\frac{1}{2}\Vert y-Xb{\Vert }^2=\frac{1}{2}\Vert b-z{\Vert }^2+\text{const},$$ so each $b_j$ minimises $\frac{1}{2}(b_j-z_j{)}^2+\lambda |b_j|$ on its own.

Step 2: one coefficient. For $b>0$ the derivative is $b-z+\lambda$, which vanishes at $b=z-\lambda$; that is allowed only if $z>\lambda$. Symmetrically for $b<0$. Otherwise the minimum is at $b=0$. So $$b_j=\mathrm{sign}(z_j)(|z_j|-\lambda {)}_{+}.$$

Step 3: the numbers. With $\lambda =1$: $3\to 2$, $-0.5\to 0$, $1.2\to 0.2$. The sum is $$2+0+0.2=2.2=\frac{11}{5}.$$

Compare with ridge. With the penalty $\lambda \Vert b{\Vert }^2$ on the full residual sum of squares, ridge divides every coefficient by $1+\lambda$: it shrinks all of them and sets none to zero. The lasso's kink at zero is what produces exact zeros. Note also the convention: without the $\frac{1}{2}$ in front of the squared error, the threshold becomes $\frac{\lambda }{2}$.