Total Derivatives
1. Definition of Differentiability
Definition
Suppose \(U \subset {\mathbb R}^n\) is open and \(f : U \rightarrow {\mathbb R}^m\). We say that \(f\) is differentiable at \(p \in U\) when there exists a linear map \(D_p f : {\mathbb R}^n \rightarrow {\mathbb R}^m\) such that
\[ \lim_{x \rightarrow p} \frac{f(x) - f(p) - (D_p f) (x-p)}{||x-p||} = 0. \]
Theorem
If \(U \subset {\mathbb R}^n\) is open and \(f : U \rightarrow {\mathbb R}^m\) is differentiable at \(p \in U\), then the derivative is unique.
Proof
Suppose there were two linear maps \(L_1, L_2\) such that
\[ \lim_{x \rightarrow p} \frac{f(x) - f(p) - L_i (x-p)}{||x-p||} = 0. \]
for \(i = 1,2\). Taking the difference of the limits gives that
\[ \lim_{x \rightarrow p} \frac{(L_1-L_2) (x-p)}{||x-p||}\]
However, if there is any vector \(v\) such that \(L_1v \neq L_2 v\), then by letting \(x_n := p + t_n v\) for a sequence of positive \(t\)'s tending to zero, it follows that
\[ \frac{(L_1 - L_2)(x_n-p)}{||x_n-p||} = \frac{(L_1 - L_2)(v)}{||v||}\]
which does not tend to infinity as \(n \rightarrow \infty\). The sequential characterization of limits then dictates that the original limit could not possibly equal zero.
2. Differentiability, Continuity, and Partial Differentiability
Theorem
If \(U \subset {\mathbb R}^n\) is open and \(f : U \rightarrow {\mathbb R}^m\) is differentiable at \(p \in U\), then \(f\) is continuous at \(p\).
Proof
Theorem
If \(U \subset {\mathbb R}^n\) is open and \(f : U \rightarrow {\mathbb R}^m\) is differentiable at \(p \in U\), then directional derivatives of \(f\) exist in all directions and the derivative matrix of \(f\) is the Jacobian matrix
\[ \begin{bmatrix} \frac{\partial f_1}{\partial x_1} & \cdots & \frac{\partial f_1}{\partial x_n} \\
\vdots & \ddots & \vdots \\
\frac{\partial f_m}{\partial x_1} & \cdots & \frac{\partial f_m}{\partial x_n}
\end{bmatrix}\]
Proof
Theorem
If \(U \subset {\mathbb R}^n\) is open and \(f : U \rightarrow {\mathbb R}^m\) is partially differentiable in all coordinate directions on a neighborhood of \(p\) and the partial derivatives of \(f\) are continuous at \(p\), then \(f\) is differentiable at \(p\).
Proof
Meta (Main Idea)
Consider the case \(m=1\). Let \(z_0 := p\) and let \(z_n = x\). For each \(j \in \{1,\ldots,n-1\}\), let \(z_j\) be the vector whose first \(j\) coordinates agree with the first \(j\) coordinates of \(x\) and whose remaining coordinates agree with \(p\). Then write
\[{}f(x) - f(p){}\]
\[{}= (f(z_n) - f(z_{n-1})) {}\]
\[{}+ \cdots{}\]
\[{}+ (f(z_1) - f(z_{0})).{}\]
The points \(z_j\) and \(z_{j-1}\) have coordinates which agree everywhere except in the \(j\)-coordinate position. Use the Mean Value Theorem to express each difference in terms of a partial derivative of \(f\) times a coordinate of \(x-p\) (both in the same direction): for each \(j=1,\ldots,n\)
\[{}f(z_j) - f(z_{j-1}){}\]
\[{}= \left. \frac{\partial f}{\partial x_j} \right|_{\xi_j} (x_j - p_j){}\]
for some point \(\xi_j\) on the line segment joining \(z_{j-1}\) to \(z_j\).
Continuity of the partial derivatives will then yield that \(f(x)-f(p)\) equals the dot product \((x-p) \cdot \nabla f(p)\) plus a term belonging to \(o(x-p)\) as \(x \rightarrow p\):
\[{}f(z_j) - f(z_{j-1}){}\]
\[{}= \left. \frac{\partial f}{\partial x_j} \right|_{\xi_j} (x_j - p_j){}\]
\[{}= \left. \frac{\partial f}{\partial x_j} \right|_{p} (x_j - p_j){}\]
\[{}+ \left[ \left. \frac{\partial f}{\partial x_j} \right|_{\xi_j} - \left. \frac{\partial f}{\partial x_j} \right|_{p}\right] (x_j - p_j){}\]
\[{}= \left. \frac{\partial f}{\partial x_j} \right|_{p} (x_j - p_j) + o(x-p).{}\]
In this argument, it's important that the distance from \(p\) to each \(\xi_j\) is never greater than the distance from \(p\) to \(x\). One also needs to assume that \(x\) belongs to a small neighborhood of \(p\) having the form \((p_1-\epsilon,p_1+\epsilon) \times \cdots \times (p_n - \epsilon,p_n+\epsilon)\) so that all the line segments belong to \(U\).
When \(m \neq 1\), we apply the argument above coordinate-wise to establish that \(f(x) - f(p) = D_p f (x-p) + o(x-p)\) as \(x \rightarrow p\).
3. Algebraic Differentiation Laws
Theorem
If \(U \subset {\mathbb R}^n\) is open and \(f,g : U \rightarrow {\mathbb R}^m\) are both differentiable at \(p\), then
- \(f + g\) is differentiable at \(p\) and \(D_p (f+g) = D_p f + D_p g\)
- \(\alpha f\) is differentiable at \(p\) for any \(\alpha \in {\mathbb R}\) and \(D_p (\alpha f) = \alpha D_p f\).
- \(f \cdot g\) is differentiable at \(p\) and\[{}D_p(f \cdot g)(h){}\]\[{}= (D_p f (h)) \cdot g(p) + f(p) \cdot (D_p g(h)){}\]for all \(h \in {\mathbb R}^n\), where \(\cdot\) denotes the usual dot product on \({\mathbb R}^m\). In other words, regarding \(f\) and \(g\) as column vectors gives that\[ D_p (f \cdot g) = (g(p))^T D_p f + (f(p))^T D_p g. \]
Proof