1. Definition and Derivative Laws
Definition
Suppose \(f\) is a real-valued function on some interval \(I\) and that \(p \in I\). We say that \(f\) is differentiable at \(p\) and has derivative equal to \(f'(p)\) when
\[ \lim_{x \rightarrow p} \frac{f(x) - f(p)}{x-p} = f'(p). \]
The fraction inside the limit is called the difference quotient. Also be aware that there are many different and reasonably common notations for the derivative, including
\[ f', \frac{d f}{dx}, D f, \dot f, \text{ etc.}\]
Basic limit laws and the linearity of the difference quotient as a function of \(f\) imply the following “simple” differentiation laws:
Theorem (Simple Differentiation Laws)
Suppose \(f\) and \(g\) are defined on a neighborhood of \(p\) and differentiable at \(p\). Then
- For any constant \(c\), \(c f\) and \(cg\) are differentiable at \(p\) and \((cf)'(p) = c (f'(p))\) (and likewise \((cg)'(p) = c (g'(p))\). )
- The function \(f + g\) is differentiable at \(p\) and \((f+g)'(p) = f'(p) + g'(p)\).
2. Differentiability and Continuity
A critically important feature of differentiability is that it is a strictly stronger notion than continuity, as evidenced by the following theorem:
Theorem (Differentiability Implies Continuity)
If \(f\) is a real function defined on some open interval \(I\) containing a point \(p\) and if \(f\) is differentiable at \(p\), then \(f\) must be continuous at \(p\).
Proof
Fix any \(\epsilon > 0\). Let \(\delta_1\) be chosen so that
\[ \left| \frac{f(x)-f(p)}{x-p} - f'(p) \right| < 1 \]
for all \(x \in I\) with \(0 < |x-p| < \delta_1\). Then let \(\delta := \min \{\delta_1, \frac{\epsilon}{1 + |f'(p)|} \}\) and suppose that \(x \in I\) satisfies \(0 < |x-p| < \delta\). By virtue of the fact that
\[{}\left| f(x) - f(p) \right|{}\]
\[{}\leq |x-p| \left| \frac{f(x)-f(p)}{x-p} - f'(p) \right|{}\]
\[{}+ |f'(p)| |x-p|,{}\]
when \(0 < |x-p| < \delta\), it must be true that \(|x-p| < \delta_1\) and consequently
\[ \left| \frac{f(x)-f(p)}{x-p} - f'(p) \right| < 1. \]
Thus
\[ |f(x) - f(p)| \leq (|f'(p)| + 1) |x-p|. \]
Because \(|x-p| < \epsilon / (|f'(p)| + 1)\), it follows that \(|f(x) - f(p)| < \epsilon\). Note that if \(x = p\), \(|f(x) - f(p)| = 0 < \epsilon\), so for any \(x \in I\) satisfying \(|x-p| < \delta\), it follows that \(|f(x) - f(p)| < \epsilon\) as desired.
Note that the converse is certainly not true: there exist plenty of continuous functions which are not differentiable. In fact, we will eventually see that there exist functions that are continuous at every point and differentiable at no points.
Two slightly less elementary observations are the product and quotient rules. These rely on the observation just made that differentiability implies continuity:
Theorem (Product and Quotient Rules)
Suppose \(f\) and \(g\) are defined on a neighborhood of \(p\) and differentiable at \(p\). Then
- (Product Rule) The pointwise product function \(fg\) is differentiable at \(p\) and\[(fg)'(p) = f'(p) g(p) + f(p) g'(p). \]
- (Quotient Rule) If \(g(p) \neq 0\), then the quotient \(f/g\) is differentiable at \(p\) and\[ (f/g)'(p) = \frac{f'(p)g(p) - f(p)g'(p)}{(g(p))^2}. \]
Proof
3. Little \(o\) Notation and the Chain Rule
Definition
Given two functions \(f\) and \(g\) on some common domain \(I \subset {\mathbb R}\), we say that \(f(x)\) is \(o(g(x))\) and write \(f(x) = o(g(x))\) as \(x \rightarrow p\) when for all \(\epsilon > 0\), there is some neighborhood \(N_\delta(p)\) such that \(|f(x)| \leq \epsilon |g(x)|\) for all \(x \in N_\delta(p)\setminus \{p\}\). Likewise, the statement \(f(x) = h(x) + o(g(x))\) as \(x \rightarrow p\) is to be interpreted as \(f(x) - h(x) = o(g(x))\).
The most important application of little \(o\) notation is the following observation:
Fact
For a function \(f\) defined on some open interval containing the point \(p\), \(f\) is differentiable at \(p\) with derivative \(f'(p)\) if and only if
\[{}f(x){}\]
\[{}= f(p) + f'(p)(x-p){}\]
\[{}+ o(x-p){}\]
as \(x \rightarrow p\).
Proof
This is a direct consequence of the observation that for any \(x \neq p\)
\[{}\left| \frac{f(x) - f(p)}{x-p} - f'(p) \right| \leq \epsilon{}\]
\[{}\Leftrightarrow{}\]
\[{}|f(x) - f(p) - f'(p) (x-p)|{}\]
\[{}\leq \epsilon |x-p|.{}\]
Recall also that since \(\epsilon\) is any positive real number, there is no logical distinction between the quantity being less than \(\epsilon\) versus less than or equal to \(\epsilon\).
Theorem (Chain Rule)
Suppose that \(g\) is defined on some open interval \(I\) containing \(p\) and is differentiable at \(p\). Suppose also that \(f\) is defined on some open interval \(U\) containing \(g(p)\) and is differentiable at \(g(p)\). Then \(f ( g(x))\) is defined on some open interval containing \(p\), is differentiable at \(p\), and has
\[ \left. \frac{d}{dx} f(g(x)) \right|_{x=p} = f'(g(p))g'(p). \]
Proof
Exercise
Suppose that \(f\) is a real-valued function on some open interval \(I\) containing the point \(p\). Show that \(f\) is differentiable at \(p\) if and only if there exists a real number \(m\) such that for any \(m' \neq m\), there is some open interval \(I'\) containing \(p\) such that
\[{}|f(x) - f(p) - m(x-p)|{}\]
\[{}\leq \frac{99}{100} |f(x) - f(p) - m'(x-p)|{}\]
for all \(x \in I \cap I'\). In other words, \(f\) is differentiable at \(p\) if and only if there is a distinguished linear function \(m(x-p)+f(p)\) passing through the graph of \(f\) at \((p,f(p))\) which, when compared to any other linear function \(m'(x-p)+f(p)\), is at least a 1% better approximation of \(f\) on some (presumably small) neighborhood of \(p\).
Hint
Use the inequality
\[{}|f(x) - f(p) - m'(x-p)|{}\]
\[{}\leq |f(x) - f(p) - m(x-p)|{}\]
\[{}+ |m-m'||x-p|{}\]
when showing that \(f\) must be differentiable to conclude that for any \(m' \neq m\), there is a small interval of points \(x\) near \(p\) on which
\[ |f(x) - f(p) - m'(x-p)| \leq 100 |m-m'| |x-p|. \]
To prove differentiability, fix \(\epsilon > 0\) and choose an appropriate value of \(m'\) in this inequality to establish that \(|f(x) - f(p) - m (x-p)|\) is less than \(\epsilon |x-p|\).
For the reverse direction, use the inequality
\[{}|f(x) - f(p) - m'(x-p)|{}\]
\[{}\geq |m-m'||x-p|{}\]
\[{}- |f(x) - f(p) - m(x-p)|{}\]
and differentiability to show that this approximation property holds.