Hilbert Spaces

See Sections 1-5 of these lecture notes for a gentle introduction to the notion of Hilbert spaces.

1. Preliminaries: Inner Product Spaces

Video. Hilbert Space Introduction

Definition of an Inner Product Suppose \(V\) is a vector space over a field \(F\) which is either real or complex. We say that a pairing \(\left< \cdot , \cdot \right> \ : V \times V \rightarrow F\) is an inner product when it satisfies the following criteria:

(Sesquilinearity) For any \(f,g,h \in V\) and \(c \in F\),
- \(\left< c f + g, h\right> = c \left<f,h\right>+ \left<g,h\right>\)
- \(\left<f, h\right> = \left< h , f \right>\) when the field \(F\) is real and \(\left< f, h \right> = \overline{\left< h , f \right>}\) when \(F\) is complex.
(Nonnegativity) For any \(f \in V\), \(\left<f,f\right>\) is real and nonnegative. Moreover, \(\left<f,f\right> = 0\) if and only if \(f = 0\).

A vector field \(V\) with an inner product \(\left<\cdot,\cdot\right>\) are together referred to as an inner product space.

Theorem (Cauchy-Schwarz Inequality)

Any inner product \(\left< \cdot , \cdot \right>\) satisfies

\[ \left| \left<f , g \right> \right| \leq \sqrt{ \left< f,f\right> \left<g,g\right> }\]

with equality if and only if \(f\) and \(g\) are linearly dependent.

Proof

Meta (Main Idea)

We're going to look at \(\left<f + t g, f + t g\right>\) and minimize it as a function of \(t\). That is easy because as a function of \(t\), it's at most quadratic. The Cauchy-Schwarz inequality will be a consequence of explicitly computing the minimum value of the quadratic and observing that it can't be negative because the nonnegativity property of inner products forbids it.

Corollary

If \(||f|| := \sqrt{\left<f,f\right>}\), then \(||\cdot||\) is a norm on \(V\). In particular, it satisfies the triangle inequality:

\[ || f + g || \leq ||f|| + ||g|| \]

for all \(f,g \in V\), with equality if and only if \(f\) and \(g\) point in the same direction (i.e., if there exists \(h\) such that \(f\) and \(g\) are both nonnegative scalar multiples of \(h\).)

Proof

Meta (Main Idea)

Expand \(||f+g||^2\) and use Cauchy-Schwarz to estimate the cross terms.

2. Hilbert Spaces: Definition and Examples

Definition of a Hilbert Space A Hilbert space \(H\) is an inner product space \((H,\left<\cdot,\cdot\right>)\) which is complete with respect to the associated metric \(d(f,g) := ||f-g||\).

Video. Hilbert Space Examples

Example Hilbert Space: \({\mathbb R}^n\) Suppose that vectors \(x\) and \(y\) in \({\mathbb R}^n\) have standard coordinates \((x_1,\ldots,x_n)\) and \((y_1,\ldots,y_n)\), respectively. Let \(\left<x,y\right> := x_1 y_1 + \cdots + x_n y_n\). This \(\left<\cdot,\cdot\right>\) defines an inner product on \({\mathbb R}^n\); the topology induced by the norm is the standard topology, and in particular \({\mathbb R}^n\) is a Hilbert space when equipped with this inner product.

Meta (Main Idea)

The proof that \({\mathbb R}^n\) is a Hilbert space has two parts. The first is to verify that \(\left<\cdot,\cdot\right>\) really is an inner product. Because it's all real, sesquilinearity is a consequence of linearity and symmetry: \(\left<x,y\right> = \left<y,x\right>\). We have \(\left<x,x\right> = x_1^2 + \cdots + x_n^2\), so it's clearly nonnegative and zero only for the zero vector.

The distance function defined by \(d(x,y) := ||x-y||\) is exactly the so-called Euclidean distance. To show completeness, argue that vectors which are Cauchy in this metric must have coordinates which are also Cauchy (e.g., fixing an index \(j\) and taking the \(j\)-th coordinate of each vector results in a Cauchy sequence of real numbers.) Then the problem largely reduces to one of working with single-variable Cauchy sequences.

Example: Incompleteness Let \(V\) be the vector space of continuous, complex-valued functions \(f\) on the unit interval \([-1,1]\). The functional

\[ \left<f,g\right> := \int_{-1}^1 f(t) \overline{g(t)} dt \]

defines an inner product on \(V\), but \(V\) is not complete and consequently \((V,\left<\cdot,\cdot\right>)\) is an inner product space but not a Hilbert space.

Meta (Main Idea)

Here the difficult part is showing that this is not a Hilbert space. One must construct a Cauchy sequence \(\{f_n\}_{n=1}^\infty\) which has no limit in the specific sense that there is no element \(g \in V\) such that \(||f_n - g|| \rightarrow 0\). An example is the sequence

\[ f_n(t) := \begin{cases} -1 & t < -\frac{1}{n} \\ nt & t \in \left[-\frac{1}{n},\frac{1}{n} \right] \\ 1 & t > \frac{1}{n} \end{cases}\]

(for technical reasons, they have to be continuous so that they belong to \(V\).) This sequence can be seen to converge pointwise to \(1\) on \(t > 0\), \(-1\) on \(t < 0\) and \(0\) at \(t=0\), which isn't a continuous function. However, this is not yet a proof because the sense in which the limit is taken has changed (in norm versus pointwise). It's a bit more direct to show that for any continuous function \(g\) on \([-1,1]\), the quantity \(|f_n(t) - g(t)|^2\) equals \(|1+g(t)|^2\) when \(t < 1/n\) and \(|1-g(t)|^2\) when \(t > 1/n\). That means

\[{}||f_n - g||^2{}\]

\[{}\geq \int_{-1}^{-\frac{1}{n}} |1+g(t)|^2 dt{}\]

\[{}+ \int_{\frac{1}{n}}^1 |1-g(t)|^2 dt.{}\]

At least one of \(|1+g(0)|\) and \(|1-g(0)|\) must be nonzero, so by continuity at least one of \(|1+g(t)|^2\) and \(|1-g(t)|^2\) must be nonzero on a neighborhood of \(t=0\). This allows us to show that the limit as \(n \rightarrow \infty\) cannot be zero.

Exercises

Suppose that \(\{x_n\}_{n=1}^\infty\) and \(\{y_n\}_{n=1}^\infty\) are convergent sequences in a Hilbert space \(H\). Show that \(\lim_{n \rightarrow \infty} \left<x_n,y_n\right>\) exists and
\[ \lim_{n \rightarrow \infty} \left<x_n,y_n\right> = \left<\lim_{n \rightarrow \infty} x_n, \lim_{n \rightarrow \infty} y_n\right>. \]
Hint
Begin by rewriting the difference \(\left<x_n,y_n\right>-\left<x,y\right>\) to more prominently feature the differences \(x_n - x\) and \(y_n - y\).
We say that a sequence \(\{x_n\}_{n=1}^\infty\) in a Hilbert space converges weakly to \(x \in H\) when \(\lim_{n \rightarrow \infty} \left<x_n,y\right> = \left<x,y\right>\) for each \(y \in H\). Prove that \(x_n \rightarrow x\) in the standard topology of \(H\) if and only if \(\{x_n\}_{n=1}^\infty\) converges weakly to \(x\) and \(||x_n|| \rightarrow ||x||\) as \(n \rightarrow \infty\).
Hint
For the reverse direction, expand \(||x_n - x||^2\) and use the facts that \(\left<x_n,x\right> \rightarrow ||x||^2\) and \(||x_n||^2 \rightarrow ||x||^2\) as \(n \rightarrow \infty\) to show that \(||x_n - x||^2 \rightarrow 0\) as \(n \rightarrow \infty\).