Power Series and Taylor Series
1. Power Series
Definition
An infinite series of the form
\[ \sum_{n=0}^\infty a_n (x - c)^n \]
for constant \(c \in {\mathbb R}\) and a sequence \(\{a_n\}_{n=0}^\infty\) of real (and later, complex) numbers is called a power series. The constant \(c\) is called the center, and the \(a_n\)'s are called the coefficients of the power series.
The basic questions regarding power series are:
- For which \(x\) do they converge?
- When the series converges, how regular (i.e., smooth) is the limit function?
Theorem (Hadamard Criterion)
Given a power series as defined above, let
\[ \frac{1}{R} := \limsup_{n \rightarrow \infty} |a_n|^{1/n}. \]
Note that we define \(R := \infty\) if the limsup is zero and \(R := 0\) if the limsup is infinity. The power series must converge for all \(x\) with \(|x-c| < R\) and must diverge for all \(x\) with \(|x-c| > R\). If the set \(\{x \in {\mathbb R} \ : \ |x-c| < R\}\) is nonempty (i.e., if \(R>0\)), then \(f\) is infinitely differentiable there.
Proof
(Step 1: Divergence Outside) Suppose \(|x-c| := \rho > R\). By assumption, \(\rho > 0\), so \(1/\rho\) is well-defined and has \(\rho^{-1} < R^{-1}\). Because the limsup equals
\[ R^{-1} = \inf_{N} \sup_{n \geq N} |a_n|^{1/n}, \]
it follows that for each \(N\),
\[ R^{-1} \leq \sup_{n \geq N} |a_n|^{1/n}. \]
This means that \(\rho^{-1} < \sup_{n \geq N} |a_n|^{1/n}\) and so for every \(N\), there must exist some \(n \geq N\) such that \(|a_n|^{1/n} \geq \rho^{-1}\). By letting \(N\) tend to infinity, we see that there must exist a subsequence \(a_{n_k}\) such that \(|a_{n_k}|^{1/n_k} \geq \rho^{-1}\). But then
\[ |a_{n_k} (x-c)^{n_k}| \geq |\rho^{-1} \rho|^{n_k} \geq 1 \]
for each \(k\), which means that the series cannot converge because its terms do not tend to zero.
(Step 2: Convergence Inside) Fix any positive \(\rho < R\) and suppose \(|x-c| \leq \rho\). (Note that any \(x\) satisfying \(|x-c| < R\) admits such a choice of \(\rho\).) By definition of the limsup,
\[ \inf_{N} \sup_{n \geq N} |a_n|^{1/n} = R^{-1}. \]
Let \(r\) be any finite real number strictly between \(\rho\) and \(R\). Since \(r^{-1} > R^{-1}\), there must be some \(N\) such that
\[ \sup_{n \geq N} |a_n|^{1/n} < r^{-1}. \]
This forces \(|a_n|^{1/n} < r^{-1}\) for each \(n \geq N\), and in turn forces \(|a_n| \leq r^{-n}\) for all \(n \geq N\). As a consequence, every \(x\) with \(|x-c| \leq \rho\) has \(|a_n (x-c)^n| \leq (\rho/r)^n\) for all \(n\) sufficiently large. But then
\[ \sum_{n=N_1}^{N_2} |a_n (x-c)^{n}| \leq \sum_{n=N_1}^{N_2} \left(\frac{\rho}{r}\right)^n \]
for all \(N_1 < N_2\) larger than \(N\). Since \(\rho / r < 1\), the geometric series with ratio \(\rho/r\) converges, and the Cauchy criterion (both for numerical sequences and uniform convergence) then gives uniform convergence because the right-hand side can be made as small as desired as long as \(N_1\) and \(N_2\) are large enough.
As a side note, if the radius of convergence \(R\) is neither zero nor infinity, then the interval \((c-R,c+R)\) has finite length and the Hadamard criterion does not specify whether there is convergence or divergence at the endpoints. Resolving convergence at the endpoints in this case requires plugging in \(x = c \pm R\) and directly analyzing the numerical series that results.
Theorem
If the radius of convergence of a power series is strictly positive or infinite, then the function to which the series converges on \((c-R,c+R)\) is infinitely differentiable.
Proof
In part 2 of the proof of Hadamard's criterion, we actually showed that the series converges uniformly on intervals \(|x-c| < \rho\) for any \(\rho < R\). This means the limit function must be continuous (because the partial sums are). So the limit function is continuous on all intervals \((c-\rho,c+\rho)\). Because \(\rho\) is arbitrary (aside from being smaller than \(R\)), this forces the limit function to be continuous on all of \((c-R,c+R)\), since any point of discontinuity would have to belong to \((c-\rho,c+\rho)\) for some \(\rho < R\).
We can complete the theorem using induction on the number of derivatives and by differentiating term-by-term (provided we know there is uniform convergence of the derivatives). We need to make two observations:
- The term-by-term derivative of a power series is another power series:\[{}\sum_{n=0}^{\infty} a_n(x-c)^n{}\]\[{}= \sum_{n=0}^{\infty} n a_n (x-c)^{n-1}{}\]\[{}= \sum_{n=0}^\infty (n+1) a_{n+1} (x-c)^n.{}\]Here the last line holds by observing that \(n a_n = 0\) when \(n=0\) and then shifting the index of summation \(n \mapsto n-1\).
- The radius of convergence of the differentiated series is at least as large as radius of the original series, i.e.,\[{}\limsup_{n \rightarrow \infty} |(n+1) a_{n+1}|^{1/n}{}\]\[{}\leq \limsup_{n \rightarrow \infty} |a_n|^{1/n}.{}\]There are two parts. The first is to observe that, for any \(\delta > 0\),\[ 1 \leq (n+1)^{1/n} \leq 1 + \delta\]for all \(n\) sufficiently large, which is easily established by induction (see the lemma below). Thus \(\limsup_{n \rightarrow \infty} |(n+1) a_{n+1}|^{1/n}|\) must exceed \(\limsup_{n \rightarrow \infty} |a_{n+1}|^{1/n}\) but is also bounded by \((1+\delta) \limsup_{n \rightarrow \infty} |a_{n+1}|^{1/n}\) for any \(\delta > 0\). This can only be the case if they're equal.
But now observe that
\[ \sum_{n=0}^\infty a_{n+1} (x-c)^n \]
converges whenever \(\sum_{n=0}^\infty a_n(x-c)^n\) does because we can multiply the first series by \((x-c)\) and it will simply equal the second series minus its constant term. Thus the radius of convergence of the first series must be at least as big as the radius of our original series,
i.e.
\[ \limsup_{n \rightarrow \infty} |a_{n+1}|^{1/n} \leq \limsup_{n \rightarrow \infty} |a_{n}|^{1/n}. \]
Combining these two observations, we see that when a power series is differentiated term-by-term, the result is a new power series with
a radius which is at least as large as the original (in fact, it will always be equal, but we do not need to prove this now). In particular, this means that the differentiated series converges uniformly on all compact subsets of the interval \((c-R,c+R)\), and thus the original function is differentiable and
\[{}\frac{d}{dx} \sum_{n=0}^\infty a_n (x-c)^n{}\]
\[{}= \sum_{n=0}^\infty (n+1) a_{n+1} (x-c)^n.{}\]
But if we know that every power series is \(k\)-times differentiable on its interval of convergence, then the right-hand side is \(k\)-times differentiable on the interval, which makes the left-hand side \(k+1\)-times differentiable. By induction, we must have that all power series with positive or infinite radius of convergence are infinitely differentiable on their (open) interval of convergence.
Lemma
For any \(\delta > 0\) and any natural number \(n \geq 1\),
\[ (1 + \delta)^n \geq 1 + \delta n + \frac{n(n-1)\delta^2}{2}.\]
In particular, if \(n > 1 + 2 \delta^{-2}\), then it is also true that
\[ (1 + \delta)^n \geq 1 + n. \]
Proof
When \(n=1\), the inequality asserts that \(1+\delta \geq 1 + \delta\), which is certainly true. When \(n=2\), it states that \((1+\delta)^2 \geq 1 + 2 \delta + \delta^2\), which is, once again, an identity. In general, if the inequality is true for some \(n\), then
\[{}(1 + \delta)^{n+1}{}\]
\[{}\geq (1 + \delta) \left[ 1 + \delta n + \frac{n(n-1)\delta^2}{2} \right]{}\]
\[{}\geq 1 + \delta n + \frac{n(n-1)\delta^2}{2}{}\]
\[{}+ \delta + \delta^2 n + \frac{n(n-1)\delta^3}{2}{}\]
\[{}\geq 1 + \delta(n+1) + \frac{n(n+1) \delta^2}{2}.{}\]
2. Taylor Series Preliminaries
Lemma
Suppose that \(f\) is \(k\)-times differentiable on \((a,b)\) and that each of \(f,f',\ldots,f^{(k-1)}\) extend continuously to \([a,b]\) such that \(f(a) = \cdots = f^{(k-1)}(a) = 0\). Then there exists \(c \in (a,b)\) such that
\[ f(b) = f^{(k)}(c) \frac{(b-a)^k}{k!}. \]
Proof
The proof proceeds by induction on \(k\). By the Generalized Mean Value Theorem applied to the functions \(f\) and \((x-a)^k/k!\), there exists \(c_1 \in (a,b)\) such that
\[{}f(b) \frac{(c_1-a)^{k-1}}{(k-1)!}{}\]
\[{}= (f(b)-f(a)) \left[ \left. \frac{d}{dx} \frac{(x-a)^k}{k!} \right|_{x=c_1} \right]{}\]
\[{}= \left[ \frac{(b-a)^k}{k!} - \frac{(a-a)^k}{k!} \right] f'(c_1){}\]
\[{}= \frac{(b-a)^k}{k!} f'(c_1). {}\]
If \(k = 1\), setting \(c := c_1\) for the \(c_1\) just used gives the desired equality. Otherwise, we may apply case \(k-1\) of this lemma to the function \(f'\) on the interval \([a,c_1]\) to conclude the existince of \(c \in (a,c_1)\) such that
\[ f'(c_1) = f^{(k)}(c) \frac{(c_1-a)^{k-1}}{(k-1)!}. \]
Substituting this formula into the previous one gives that
\[{}f(b) \frac{(c_1-a)^{k-1}}{(k-1)!}{}\]
\[{}= \frac{(b-a)^k}{k!}f^{(k)}(c) \frac{(c_1-a)^{k-1}}{(k-1)!}.{}\]
Since \(c_1 \neq a\), both sides can be divided by \((c_1-a)^{k-1}/(k-1)!\) to reach the desired conclusion.
Corollary
Suppose that \(f\) is \(k\)-times differentiable on \((b,a)\) and that each of \(f,f',\ldots,f^{(k-1)}\) extend continuously to \([b,a]\) such that \(f(a) = \cdots = f^{(k-1)}(a) = 0\). Then there exists \(c \in (b,a)\) such that
\[ f(b) = f^{(k)}(c) \frac{(b-a)^k}{k!}. \]
Proof
Apply the previous lemma to \(f(-x)\) on the interval \([-a,-b]\) to conclude that
\[{}f(- (-b)) {}\]
\[{}= (-1)^k f^{(k)}(-c) \frac{(-b - (-a))^k}{k!}{}\]
for some \(c \in (-a,-b)\).
3. Taylor's Theorem with Remainder
Taylor's Theorem is, in some sense, a generalization of the Mean Value Theorem to functions of higher orders of differentiability. One can see this to be the case by recognizing that when \(k = 1\) in the corollary below, the identity for \(f(b)\) simplifies to \(f(b) = f(a) + f'(c)(b-a)\), which is simply a rewriting of the Mean Value Theorem.
Corollary (Taylor's Theorem with Lagrange Remainder)
Suppose that \(a,b \in {\mathbb R}\) are two unequal numbers and that \(f\) is \(k\) times differentiable on the open interval joining \(a\) and \(b\) and that \(f,\ldots,f^{(k-1)}\) extend continuously to the closure of that interval. Then there exists \(c\) strictly between \(a\) and \(b\) such that
\[{}f(b){}\]
\[{}= \left[ \sum_{n=0}^{k-1} \frac{f^{(n)}(a) (b-a)^n}{n!} \right]{}\]
\[{}+ \frac{f^{(k)}(c)(b-a)^k}{k!}.{}\]
Proof
Depending on the relative ordering of \(a\) and \(b\), apply the relevant result above to the function \(g(x) = f(x) - \sum_{n=0}^{k-1} \frac{f^{(n)}(a) (x-a)^n}{n!}\). This function is designed specifically so that \(g(a) = \cdots = g^{(k-1)}(a) = 0\) and that \(g^{(k)}(x) = f^{(k)}(x)\) for all relevant \(x\).
It is natural to consider what happens for infinitely differentiable functions in the limit \(k \rightarrow \infty\). Since the finite sum in Taylor's formula above is simply a partial sum of what's traditionally called the Taylor Series, convergence of the Taylor series to \(f(b)\) happens exactly when the remainder term
\[ \frac{f^{(k)}(c)(b-a)^k}{k!} \rightarrow 0 \]
as \(k \rightarrow \infty\). If, for example, there is a constant \(M\) such that
\[ |f^{(k)}(x)| \leq C k! M^k \]
for each \(k \geq 0\) and each \(x\) in some open interval \(I\), then it is simple to see that
\[ f(b) = \sum_{n=0}^\infty \frac{f^{(n)}(a) (b-a)^n}{n!}\]
provided that \(a, b \in I\) satisfy \(|a-b| < M^{-1}\).
Example
Let \(f(x) := \sqrt{x}\). For each \(n \geq 1\),
\[ \frac{d^n}{dx^n} \sqrt{x} = x^{-n+\frac{1}{2}} \prod_{\ell=1}^n \frac{3-2\ell}{2}. \]
Since \(|(2 \ell-3)/2| \leq \ell\) for each \(\ell \geq 1\), it follows that
\[ \left| \frac{d^n}{dx^n} \sqrt{x} \right| \leq \sqrt{x} n! x^{-n}\]
for each \(x > 0\) and each \(n \geq 1\). If \(x \in [a,2a)\), \(\sqrt{x} n! x^{-n} \leq \sqrt{a} n! a^{-n}\) for each \(n \geq 1\), so Taylor's formula with remainder implies that the Taylor series for \(\sqrt{x}\) at the point \(x=a\) will converge on \([a,2a)\) to \(\sqrt{x}\), i.e.,
\[{}\sqrt{x}{}\]
\[{}= \sqrt{a} + {}\]
\[{}\frac{1}{2 \sqrt{a}} (x-a){}\]
\[{}- \frac{1}{4 \cdot 2! \cdot a^{\frac{3}{2}}}(x-a)^2{}\]
\[{}+ \frac{1 \cdot 3}{8 \cdot 3! \cdot a^\frac{5}{2}}(x-a)^3{}\]
\[{}- \frac{1 \cdot 3 \cdot 5}{16 \cdot 4! \cdot a^\frac{7}{2}}(x-a)^4{}\]
\[{}+ \cdots{}\]
\[{}+ (-1)^{n-1} \frac{1 \cdot 3 \cdots (2n-3)}{2 \cdot 4 \cdots (2n)}\frac{(x-a)^n}{a^{n-\frac{1}{2}}}{}\]
\[{}+ \cdots{}\]
provided \(a \leq x < 2a\). Because Taylor series converge on symmetric intervals, the series must also converge to something on the whole interval \((0,2a)\). A modification of our argument above (using the fact that \(x^{-n+\frac{1}{2}} \leq 2^{n-\frac{1}{2}}a^{-n+\frac{1}{2}}\) when \(x \in (a/2,a]\)) shows that the limit function must still equal \(\sqrt{x}\) on \((a/2,a]\).
To understand the behavior of the Taylor series on the full interval \((0,2a)\), some additional cleverness is needed. Let \(\{c_n\}_{n=0}^\infty\) denote the Taylor series coefficients of \(\sqrt{x}\) with center \(a\). We can see relatively easily that \(c_0 := \sqrt{a}\) and \(c_{n+1} = c_{n} \frac{1 - 2n}{2(n+1)a}\) for \(n > 0\). Let \(f\) denote the sum of the Taylor series on \((0,2a)\). We know that it is some smooth function. It follows by term-by-term differentiation that
\[{}a f'(x) = \sum_{n=0}^\infty a (n+1) c_{n+1}(x-a)^n = \sum_{n=0}^\infty \frac{c_n(1-2n)}{2} (x-a)^n{}\]
\[{}(x-a)f'(x) = \sum_{n=0}^\infty n c_n (x-a)^n{}\]
Summing these identities on \((0,2a)\) gives that \(x f'(x) = \frac{1}{2}f(x)\). It follows that
\[ \frac{d}{dx} \frac{f(x)}{\sqrt{x}} = \frac{x f'(x) - \frac{1}{2}f(x)}{\sqrt{x}} = 0 \]
which means that \(f(x)/\sqrt{x}\) is constant on this interval. Because we know that the ratio is \(1\) at \(x=a\), it must be \(1\) everywhere, i.e., \(f(x) = \sqrt{x}\) on \((0,2a)\).
Exercises (Further Results)
- Show that when \(\sum_{n=0}^\infty a_n\) is a convergent series,\[ \lim_{r \rightarrow 1^{-}} \sum_{n=0}^\infty a_n r^n = \sum_{n=0}^\infty a_n. \]HintFirst let \(A_N := \sum_{n=0}^N a_n\) and use summation by parts to prove that\[ \sum_{n=0}^\infty a_n r^n = (1-r) \sum_{N=0}^\infty A_N r^N. \]Then show that the sum on the right-hand side tends to \(\lim_{N \rightarrow \infty} A_N\) as \(r \rightarrow 1^{-}\) by breaking into terms \(N > N_0\) for which \(|A_N - \sum_{n=0}^\infty a_n|\) is small and finitely many terms \(N \leq N_0\) whose contribution tends to \(0\) as \(r \rightarrow 1^{-}\).
- Use the previous exercise to show that when a Taylor series converges converges at an endpoint of its radius of convergence, the function that the series sums to is always continuous there.
- Prove that \(\frac{2n-2}{2n-3} \geq \sqrt{\frac{n-\frac{1}{2}}{n-\frac{3}{2}}}\) for each \(n \geq 2\).HintWrite both ratios as \(1 +\) stuff and square both sides.
- Using the previous part, show that \(\frac{1}{2} \cdot \frac{3}{4} \cdots \frac{2n-3}{2n-2} \leq \sqrt{\frac{1}{2n-1}}\) for all \(n \geq 2\).
- Using the previous part and question 2, prove that the Taylor series for \(\sqrt{x}\) centered at \(x=a\) converges to \(\sqrt{x}\) for all \(x \in [0,2a]\) (i.e., that the series converges at the endpoints of the interval to the correct value.)