Mean Square Convergence and Parseval's Identity
1. Notation and Computations for Continuous Functions
To begin, let's introduce new notation to simplify computations that will follow. Given two complex-valued Riemann-integrable functions \(f\) and \(g\) on \([0,1]\), let
\[ \left<f,g\right> := \int_0^1 f(x) \overline{g(x)} dx. \]
Let's also define \(e_n\) for each \(n \in {\mathbb Z}\) to be the function \(e_n(x) := e^{2 \pi i n x}\). Using this new notation, the \(n\)-th Fourier coefficient of \(f\), previously denoted \(\hat{f}(n)\), can be written as \(\left<f,e_n\right>\) and the \(N\)-th symmetric partial sum of the Fourier series equals
\[ S_Nf := \sum_{n=-N}^N \left<f,e_n\right> e_n. \]
Important
The most important fact about the exponentials and this bracket operation
(called the inner product) is that there is orthogonality:
\[ \left<e_n,e_m\right> = \begin{cases} 1 & n = m, \\ 0 & n \neq m. \end{cases}\]
The proof of this fact is carried out by directly evaluating the integrals. When \(n = m\), \(\left<e_n,e_m\right>\) is the integral of \(e_n \overline{e_n}\), which is identically \(1\). When \(n \neq m\), the inner product equals the integral of \(e^{2 \pi i (n-m) x}\) on \([0,1]\), which one can verify by direct computation must equal zero.
Let's also define
\[ ||f|| := \sqrt{\int_0^1 |f(x)|^2 dx}\]
for any Riemann-integrable complex-valued function \(f\) on \([0,1]\). Note that the square root is well-defined because the integral is of a nonnegative real-valued function and so will always be a nonnegative real number. With bracket notation, it's equivalent to write \(||f||^2 := \left<f,f\right>\). We can think of \(||f||\) as measuring the size of \(f\) in some averaged sense.
The main result which will be proved in this section is the following:
Theorem (Mean Square Convergence and Parseval's Identity)
Let \(f\) be any complex-valued \(1\)-periodic function on the real line which is Riemann integrable on \([0,1]\). Then \(||f - S_N f|| \rightarrow 0\) as \(N \rightarrow \infty\) and moreover
\[ \int_0^1 |f(x)|^2 dx = \sum_{n=-\infty}^\infty |\hat{f}(n)|^2. \]
At its core, this theorem rests on the following proposition, which characterizes the best possible approximation of a function by finitely many complex exponentials, where “best” is measured by the integral of the square of the difference.
Proposition
For any complex numbers \(a_{-N},\ldots,a_{N}\) and any complex-valued Riemann-integrable function \(f\) on \([0,1]\),
\[{}\left|\left| f - \sum_{n=-N}^N a_n e_n \right| \right|^2{}\]
\[{}= \left| \left| f - S_N f \right| \right|^2{}\]
\[{}+ \sum_{n=-N}^N |a_n - \left<f,e_n\right>|^2. {}\]
Proof
Define \(g := f - S_N f\) and \(b_n := a_n - \left<f,e_n\right>\). Then
\[ \left|\left| f - \sum_{n=-N}^N a_n e_n \right| \right|^2 = \left|\left| g - \sum_{n=-N}^N b_n e_n \right| \right|^2 \]
and the goal is to prove that this must equal
\[ ||g||^2 + \sum_{n=-N}^N |b_n|^2. \]
We can now write
\[ \left<g - \sum_{n=-N}^N b_n e_n,g - \sum_{n=-N}^N b_n e_n\right>\]
as a double sum, using the fact that the bracket is linear in the first argument and conjugate linear in the second, i.e.,
\[{}\left<g - \sum_{n=-N}^N b_n e_n,g - \sum_{n=-N}^N b_n e_n\right>{}\]
\[{}= ||g||^2 + \sum_{n=-N}^N \sum_{m=-N}^{N} b_n \overline{b_m} \left<e_n,e_m\right>{}\]
\[{}- \sum_{m=-N}^N \overline{b_m}\left<g,e_m\right>- \sum_{n=-N}^N b_n \left<e_n,g\right>.{}\]
Now all the terms in the double sum vanish when \(n \neq m\); the terms on the diagonal give exactly the sum of \(|b_n|^2\) over \(n=-N,\ldots,N\). As for the remaining terms, it must be the case that \(\left<g,e_m\right> = 0 = \left<e_n,g\right>\) when \(n,m \in \{-N,\ldots,N\}\). In the first case, it's because
\[{}\left<g,e_m\right>{}\]
\[{}= \left<f,e_m\right> - \sum_{n=-N}^N \left<f,e_n\right> \left<e_n,e_m\right>{}\]
\[{}= \left<f,e_m\right> - \left<f,e_m\right>{}\]
by virtue of the fact that all terms in the sum vanish when \(n \neq m\) and the term \(n=m\) is exactly equal to \(\left<f,e_m\right>\). Showing that \(\left<g,e_m\right> = 0\) is similar.
Notice an important corollary: among all linear combinations of \(e_{-N},\ldots,e_N\), the one which is closest to \(f\) in the mean square sense is exactly the one where the coefficient \(a_n\) of each exponential \(e_n\) is taken to be the Fourier coefficient \(\left<f,e_n\right>\).
Corollary
If \(T_N f\) is the \(N\)-th Cesàro mean of the symmetric partial sums of the Fourier series (i.e., \(T_N f := (S_0 f + \cdots + S_{N-1} f)/N\)), then
\[ || f - S_N f|| \leq || f - T_N f||. \]
Proof
\(T_N f\) is itself a linear combination of \(e_{-N},\ldots,e_N\) for some coefficients \(a_n\), and so the proposition yields this inequality.
Corollary
If \(f\) is a continuous \(1\)-periodic complex-valued function, then \(||f - S_N f|| \rightarrow 0\) as \(N \rightarrow \infty\). Informally, this means that the Fourier series of \(f\) converges to \(f\) in some average sense even if not at every point. Moreover,
\[ ||f||^2 = \sum_{n=-\infty}^\infty |\hat{f}(n)|^2. \]
Proof
We have already seen that \(T_N f \rightarrow f\) uniformly, so this forces \(||f - T_N f|| \rightarrow 0\) as \(N \rightarrow \infty\). To prove the identity, use the proposition again with each \(a_n := 0\). Since \(||f - S_N f|| \rightarrow 0\), this implies that the doubly-infinite series must have a limit as \(N \rightarrow \infty\).
2. Arbitrary Riemann-integrable Functions
A key consequence of the proposition above (in the case when each \(a_n = 0\)) is that \(||f - S_N f||\) is monotonically decreasing as \(N \rightarrow \infty\), simply because the left-hand side is constant in \(N\) and the sum on the right-hand side increases as a function of \(N\). Thus it converges to something as \(N \rightarrow \infty\). We also see that
\[ ||f||^2 \geq \sum_{n=-\infty}^\infty |\hat{f}(n)|^2 \]
for any complex-valued Riemann integrable function on \([0,1]\). Our goal is to prove that this inequality is actually an equality, known as Parseval's identity. It turns out that this is equivalent to showing that \(||f - S_N f|| \rightarrow 0\) as \(N \rightarrow \infty\).
Lemma
For any real numbers \(a\) and \(b\),
\[ |a|^2 \leq 2 |a-b|^2 + 2 |b|^2. \]
Proof
Regard \(a\) as fixed; the right-hand side is quadratic in \(b\) with positive second derivative, so it suffices to prove the inequality at the critical point, i.e.,
\[ 0 = 4 (b-a) + 4 b \Rightarrow b = \frac{a}{2}. \]
But when \(b = a/2\), the inequality is simply an equality. As a corollary, \(|a|^2 \leq 2 |a-b|^2 + 2|b|^2\) for all complex \(a\) and \(b\) as well, which can be achieved by simply summing the inequalities for the real and imaginary parts. Integrating this pointwise inequality gives that
\[ ||h||^2 \leq 2||h-h'||^2 + 2 ||h||^2 \]
for any complex-valued Riemann-integrable functions \(h,h'\) on \([0,1]\). In particular, when \(h := f - S_N f\) and \(h' := g - S_N g\),
\[{}||f - S_N f||^2{}\]
\[{}\leq 2 || (f-g) - S_N(f-g)||^2{}\]
\[{}+ 2 ||g - S_N g||^2{}\]
\[{}\leq 2 || f-g||^2 + 2 || g - S_N g||^2,{}\]
where the second line follows from the proposition above, which implies that \(||h - S_N h||^2\) can never exceed \(||h||^2\). So to prove that \(||f - S_N f|| \rightarrow 0\), it suffices to show that for any \(\epsilon\), there is a continuous function \(g\) such that \(||f - g|| < \epsilon\), since for this fixed \(g\), it is always the case that \(||g - S_N g|| < \epsilon\) for all \(N\) sufficiently large. This would, by the inequality just established, show that \(||f - S_N f|| < 2 \epsilon\) for all \(N\) sufficiently large.
Theorem
For any 1-periodic complex-valued function \(f\) which is Riemann integrable on \([0,1]\) and any \(\epsilon > 0\), there is a \(1\)-periodic continuous function \(g\) such that \(||f - g|| < \epsilon\).
Proof
3. Corollary: Uniqueness of Fourier Series
Corollary
Suppose \(f\) and \(g\) are 1-periodic complex-valued functions on \({\mathbb R}\) which are Riemann integrable on \([0,1]\), if \(\hat{f}(n) = \hat{g}(n)\) for all \(n \in {\mathbb Z}\), then \(\int_0^1 |f(x)-g(x)|^2 dx = 0\). Informally, this means that \(f\) and \(g\) can differ only in ways that are “invisible” to integration. For example, it means that \(f(x) = g(x)\) at all points of continuity of \(f-g\).
Proof
Apply Parseval's identity to the difference \(f - g\).