Advanced Analysis

Hilbert Space Orthonormal Bases

1. Definition of ONB

Definition
Given a Hilbert space \(H\), a set \(\mathcal B\) of vectors in \(H\) is called an orthonormal basis when \(||e_i|| = 1\) for all \(e_i \in {\mathcal B}\), \(\left<e_i, e_j\right> = 0\) when \(e_i \neq e_j\) both belong to \(\mathcal B\), and \(H\) is the closure of all finite linear combinations of elements of \(\mathcal B\).

We will focus specifically on the situation when \(\mathcal B\) is finite or countably infinite.

2. Orthonormal Sets and Best Approximations

Proposition
Given any mutually orthogonal vectors \(v_1,\ldots,v_n\),
\[ ||v_1 + \cdots + v_n||^2 = ||v_1||^2 + \cdots ||v_n||^2. \]
Corollary
Given any vector \(x \in H\) and any orthonormal \(e_1,\ldots,e_n\),
\[{}\left| \left| x - \sum_{i=1}^n c_i e_i \right| \right|^2{}\]
\[{}= \sum_{i=1}^n | c_i - \left<x,e_i\right>|^2{}\]
\[{}+ \left| \left| x - \sum_{i=1}^n \left<x,e_i\right> e_i \right|\right|^2.{}\]
In particular, \(\sum_{i=1}^n \left<x,e_i\right> e_i\) is the unique linear combination of \(e_i\)'s which is closest in norm to \(x\).
Proof
Meta (Main Idea)
Recall that we have seen this computation before in the context of Parseval's Identity. Use the fact that
\[ x - \sum_{i=1}^n \left<x,e_i\right> e_i \]
is orthogonal to each \(e_i\).

3. Characterization and Existence of Bases

Theorem
Suppose \(\mathcal B = \{e_n\}_n\) is a countable orthonormal set. Then the following are equivalent:
  1. \(\mathcal B\) is a basis of \(H\).
  2. For each \(x \in H\),
    \[ x = \sum_{n} \left<x,e_n\right>e_n \]
  3. For each \(x \in H\),
    \[ ||x||^2 = \sum_{n} |\left<x,e_n\right>|^2. \]
(In each case, note that the sum is taken over \(\{1,\ldots,\# {\mathcal B}\}\) when \({\mathcal B}\) is finite and over the natural numbers when \(\mathcal B\) is infinite.)
Proof
Meta (Main Idea)
For I implies II, we observe that if there is some linear combination of \(e_1,\ldots,e_N\) with distance less than \(\epsilon\) to \(e\), then the best approximation property implies that \(\sum_{n=1}^M \left<x,e_n\right>e_n\) must also be within distance \(\epsilon\) whenever \(M \geq N\). This implies that \(||x - \sum_{n=1}^M \left<x,e_n\right>e_n|| \rightarrow 0\) as \(M \rightarrow \infty\), which is what is meant by the equality in property II.

For II implies III, use orthogonality to say that \(|| \sum_{n=1}^M \left<x,e_n\right>e_n||^2 = \sum_{n=1}^M |\left<x,e_n\right>|^2\) and then take limits as \(M \rightarrow \infty\).

For III implies I, if the closure of the span is not everything, there must be a nonzero vector \(x\) in its complement. For this \(x\), the Parseval identity cannot possibly be true.
Theorem
A Hilbert space \(H\) has a countable (finite or countably infinite) basis if and only if it has a countable dense subset (such a Hilbert space is called separable.)
Proof
Meta (Main Idea)
To prove the existence of a basis, write the countable dense set in order and apply the Gram-Schmidt process to it.

More specifically, we let \(v_1,v_2,\ldots\) be an enumeration of the vectors in the dense set. For each \(n\), let \(k_n\) be the smallest \(k\) such that the span of \(v_1,\ldots,v_k\) is \(n\)-dimensional (if the process terminates at some finite \(n\), that's actually fine and one just argues that \(H\) must be finite-dimensional). We then apply the Gram-Schmidt process to the vectors \(v_{k_1},v_{k_2},v_{k_3},\ldots\). We're guaranteed that no \(v_{k_n}\) is in the span of the previous \(v_{k_i}\), so projecting \(v_{k_n}\) onto the orthogonal complement of the span of the vectors \(e_1,\ldots,e_{n-1}\) generated from \(v_{k_1},\ldots,v_{k_{n-1}}\) always gives a nonzero result; normalize this projection and label it \(e_n\).

A key observation is that every element of the dense set will be a finite linear combination of the orthonormal vectors produced by the Gram-Schmidt process. This means that finite linear combinations of the basis elements are dense, and consequently that the process does indeed generate a basis.

On the converse side, if \(H\) is known to have a countable orthonormal basis, we can approximate any vector \(v \in H\) by a finite linear combination of \(e_1,\ldots,e_n\) within distance less than \(\epsilon/2\) for some \(n\). We can replace the coefficients with rational numbers (or complex numbers with rational real and imaginary parts) and assume that the distance is now no greater than \(\epsilon\). But the finite linear combinations over \(\mathbb{Q}\) (or rational real and imaginary parts) of basis elements are themselves a countable set and we've just shown it to be dense.