See the Bessel inequality for what we'll prove.
Since we're approximating x, let's write down the exact approximation and error term:
yn = ∑i=1n〈x,ei〉ei
rn = x - yn
Now a bit of simple magic
is x without its projections on e1
, so it should be orthogonal
to all of them
Indeed, for j=1,...,n, we have that
〈rn,ej〉 = 〈x,ej〉 - ∑i=1n〈〈x,ei〉ei,ej〉 =
(All elements of the sum disappear except for i=j, since 〈ei,ej〉=0...)
〈x,ej〉 - 〈x,ej = 0.
Thus we have a shorter orthogonal sequence
of elements e1
; in particular, rn
But (applying what is essentially the Pythagorean theorem...) we know that
||x||2 = ||rn+yn||2 =
The first term ||rn
||yn||2 ≤ ||x||2.
yields the inequality.