Orthogonal Complements and Projections

1. Definition of the Orthogonal Complement

Given a subspace \(M\) of a Hilbert space \(H\), we define its orthogonal complement \(M^\perp\) to be the collection of all those vectors \(y \in H\) such that \(\left<x,y\right> = 0\) for all \(x \in M\).

Theorem

The orthogonal complement \(M^\perp\) of a closed subspace \(M\) satisfies:

\(M^\perp\) is a closed subspace.
\(M^\perp \cap M = \{0\}\).
\((M^\perp)^{\perp} = M\).
\(M + M^{\perp} = H\).

Proof

Meta (Main Idea)

All are routine except showing that \((M^\perp)^{\perp} \subset M\) and the last conclusion. Proofs depend on the results below.

2. A Lemma: Minimizing Distance to Closed, Convex Sets

Lemma

Suppose \(E\) is a closed convex set in a Hilbert space \(H\) and \(v\) is a vector in \(H\) not belonging to \(E\). Then there exists a unique \(e_* \in E\) such that \(||e_* - v|| = \inf_{e \in E} ||e - v||\) (i.e., the infimum is attained).

Proof

Meta (Main Idea)

Let \(\{e_n\}_{n=1}^\infty\) be a sequence of vectors in \(E\) such that \(\lim_{n \rightarrow \infty} ||e_n - v|| = \inf_{e \in E} ||e - v||.\) We use the parallelogram law:

\[{}\frac{||e_n - e_m||^2}{4}{}\]

\[{}= \frac{||e_n - v||^2}{2} + \frac{||e_m - v||^2}{2}{}\]

\[{}- \left| \left| \frac{e_n + e_m}{2} - v \right| \right|^2. {}\]

Fix \(\epsilon > 0\) and let \(N\) be sufficently large that \(n\geq N\) implies that

\[{}||e_n - v||^2{}\]

\[{}< (\inf_{e} ||e - v||)^2 + \frac{\epsilon^2}{4}{}\]

and likewise for all \(m \geq N\).

Because \(E\) is convex, \((e_n+e_m)/2 \in E\) and therefore

\[{}\left|\left| \frac{e_n+e_m}{2} - v\right|\right|{}\]

\[{}\geq \inf_{e \in E} ||e-v||.{}\]

The parallelogram law implies that \(||e_n - e_m|| < \epsilon\), which guarantees that the sequence \(\{e_n\}_{n=1}^\infty\) is Cauchy. It converges to a point \(e_* \in E\) because \(E\) is closed, and continuity of the norm implies that

\[{}||e_* - v||{}\]

\[{}= \lim_{n \rightarrow \infty} ||e_n - v||{}\]

\[{}= \inf_{e \in E} ||e - v||. {}\]

If there are two points at which the infimum is attained, the parallelogram law applied in the same way guarantees that the two points must be equal.

3. Orthogonal Projections

Theorem (Projections)

Let \(M\) be a closed subspace of a Hilbert space \(H\). Then the following are true:

Every \(x \in H\) has a unique decomposition \(x = P x + Q x\) where \(Px \in M\) and \(Q x\) is orthogonal to every \(y \in M\).
The mappings \(x \mapsto P x\) and \(x \mapsto Q x\) are both linear.
\(||x||^2 = ||Px||^2 + ||Qx||^2\).
\(P^2 x = P x\), \(Q^2 x = Q x\), \(P Q x = Q P x = 0\).

Proof

Meta (Idea of Proof)

To show existence of \(Px\) and \(Q x\), let \(y\) be the vector in \(M\) which is closest to \(H\).

Let \(P x =: y\). This clearly maps into \(M\). If \(x - P x\) is not orthogonal to \(M\), then \(|| x - P x - t h||^2\) does not have a critical point at \(t = 0\).
For linearity, observe that
\[{}P (c x + y) - c P x - P y{}\]
\[{}= - Q (cx + y) + c Q x + Q y{}\]
Both sides are in \(M\) and \(M^\perp\) and so are both zero.
Uniqueness follows by examining a similar difference of decompositions.
The Pythagorean law follows by orthogonality.

Theorem (Continued)

For any closed subspace \(M\), \((M^\perp)^{\perp} \subset M\).

Proof

Meta (Main Idea)

Suppose \(z \in (M^\perp)^{\perp}\). Write as \(z = P z + Q z\). This \(z\) is orthogonal to everything in \(M^{\perp}\), as is \(P z\). But \(Q z \in M^{\perp}\), so \(\left<z - Pz,Qz\right> = 0\), meaning \(Qz = z - P z = 0\).

Proposition

Suppose that \(M_1\) and \(M_2\) are closed subspaces. Then

\(M_1 \cap M_2\) is closed.
\((\overline{M_1 + M_2})^\perp = M_1^\perp \cap M_2^\perp\)
\((M_1 \cap M_2)^\perp = \overline{M_1^\perp + M_2^\perp}\)

Proof

Meta (Main Idea)

For the first equality, argue containment in both directions. For the second, use \((M^{\perp})^{\perp} = M\). For closedness, if \(z = x + y\), write \(y = P y + Q y\) for \(P\) orthogonal projection onto \(M_1 \cap M_2\). Then \(y - P y \in M_2 \cap (M_1 \cap M_2)^\perp\) because both are in \(M_2\).

Important (Sums of closed subspaces aren't always closed)

Let \(H := \ell^2({\mathbb N})\). Let \(M\) be the closed subspace of all sequences

\[ (a_1,0,a_3,0,a_5,0,\ldots) \in \ell^2({\mathbb N}). \]

Let \(N\) be the closed subspace of all sequences

\[ \left(b_1,\frac{b_1}{2},b_3,\frac{b_3}{4},b_5,\frac{b_5}{6},\ldots \right) \in \ell^2({\mathbb N}). \]

The intersection \(M \cap N\) is trivial, so every vector in \(M + N\) has a unique decomposition. For the vector

\[{}v_n{}\]

\[{}:={}\]

\[{}\Big(0,1,0,\frac{1}{2},0,\frac{1}{4},{}\]

\[{}\ldots,0,\frac{1}{2n},0,0,0,\ldots \Big){}\]

it is

\[{}(-1,0,-1,0, \ldots,-1,0,0,0,\ldots){}\]

\[{} + \left(1,1,1,\frac{1}{2},1,\frac{1}{4},\ldots,1,\frac{1}{2n},0,\ldots \right).{}\]

But \(\lim_{n \rightarrow \infty} v_n\) exists but the limit vector \(v\) does not belong to the sum \(M + N\) because every sequence \((c_1,c_2,\ldots)\) in \(M + N\) necessarily has \(2n c_{2n} - c_{2n-1} \rightarrow 0\) as \(n \rightarrow \infty\).