1. Vector Space Preliminaries
Suppose that \(V\) is some finite-dimensional real vector space. Recall that the dual space of \(V\), usually denoted \(V^*\) consists of linear functionals on \(V\), i.e., elements of \(V^*\) are linear functions \(\ell \ : \ V \rightarrow R\). The key property that we will need is that the double dual \((V^*)^*\) is canonically isomorphic to \(V\), meaning that there is a natural way to regard elements of \((V^*)^*\) as simply being vectors in \(V\):
Theorem
If \(V\) is a finite-dimensional real vector space, then every element \(u \in (V^*)^*\) admits a unique element \(e \in V\) such that \(u(\ell) = \ell(e)\) for all \(\ell \in V^*\). The mapping \(u \mapsto e\) is linear and an isomorphism of \((V^*)^*\) and \(V\).
Proof
Let \(\{e_i\}_{i=1}^n\) be a basis of \(V\). We know that every element \(x \in V\) admits unique real coefficients \(\{c_i(x) \}_{i=1}^n\) such that \(x = c_1(x) e_1 + \cdots + c_n(x) e_n\). By uniqueness of the coefficients, we can see that for every fixed \(i\), the mapping \(x \mapsto c_i(x)\) must be linear, so we can think of each \(c_i\) as a linear functional on \(V\). If \(\ell\) is any linear functional on \(V\), then for each \(x \in V\), linearity guarantees that
\[ \ell(x) = \sum_{i=1}^n c_i(x) \ell(e_i). \]
This means that the functional \(\ell\) must lie in the span of the \(\{c_i\}_{i=1}^n\) because it's expressible as a linear combination of these \(c_i\)'s with coefficients not depending on the argument (i.e., \(\ell(e_i)\) does not depend on \(x\)). It also means that
\[ \ell = \sum_{i=1}^n \ell(e_i) c_i, \]
because when the linear functionals on both sides of this formula always agree when evaluated on any \(x \in V\).
Now suppose that \(u \in (V^*)^*\). Consider the vector
\[ x := \sum_{i=1}^n u(c_i) e_i. \]
Now take the formula for \(x\) and apply any linear functional \(\ell \in V^*\) to both sides:
\[{}\ell(x){}\]
\[{}= \ell \left( \sum_{i=1}^n u(c_i) e_i \right){}\]
\[{}= \sum_{i=1}^n u(c_i) \ell(e_i){}\]
\[{}= u \left( \sum_{i=1}^n \ell(e_i) c_i \right) = u(\ell).{}\]
Thus every \(u \in (V^*)^*\) admits some \(x \in V\) such that \(u(\ell) = \ell(x)\) for all \(\ell \in V^*\). There couldn't possibly be two distinct vectors \(x,x' \in V\) with this property since \(c_i(x-x') \neq 0\) for at least one \(i\) (because the coefficients of \(x\) and \(x'\) in the basis \(\{e_i\}_{i=1}^n\) can't be equal when \(x \neq x'\)), which means that \(u(c_i)\) can't equal both \(c_i(x)\) and \(c_i(x')\) because \(c_i(x) \neq c_i(x')\). Because this \(x\) is unique, it must be the case that the mapping \(u \mapsto x\) is linear (simply because the formula \((c u_1 + u_2)(\ell) = \ell(c x_1 + x_2)\) would be true for any \(c \in {\mathbb R}\) and any \(\ell \in V^*\) if \(u_1(\ell) = \ell(x_1)\) and \(u_2(\ell) = \ell(x_2)\)).
2. Integration of Vector-Valued Functions
Definition
Suppose that \(R \subset {\mathbb R}^n\) is a box and \(V\) is a finite-dimensional
real vector space. A function \(f \ : \ R \rightarrow V\) is called Riemann integrable when the function \(x \mapsto \ell(f(x))\) is bounded and Riemann integrable for each \(\ell \in V^*\). When such an \(f\) is Riemann integrable, we define
\[ \int_R f(x) dx \]
to be the unique vector \(v \in V\) with the property that
\[ \ell(v) = \int_R \ell(f(x)) dx \]
for each \(\ell \in V^*\), which must exist because \(\ell \mapsto \int_R \ell(f(x)) dx\) is a linear map and therefore belongs to \((V^*)^*\).
Note that because everything is canonical, there are lots of other ways to define integration of vector-valued functions which give the same answer. In particular, one way to do it is to choose a basis \(\{e_i\}_{i=1}^n\) of \(V\), write \(f\) in terms of this basis as \(f(x) := \sum_{i=1}^n f_i(x) e_i\) for some functions \(f_i\), and then define
\[ \int_R f(x) dx := \sum_{i=1}^n \left[ \int_R f_i(x) dx \right] e_i. \]
Exercises
- Use the definition to show that the function \(f : [0,2 \pi] \rightarrow {\mathbb R}^2\) given by \(f(t) = (\cos(t),\sin(t))\) has the property that\[ \int_0^{2 \pi} f(t) dt \]is the zero vector in \({\mathbb R}^2\).
- Use the definition to show that if \(f\) and \(g\) are two \(V\)-valued Riemann integrable functions on \(R\), then \(c f + g\) is as well for any constant \(c \in {\mathbb R}\) and\[{}\int_{R} (c f(x) + g(x)) dx{}\]\[{}= c \int_{R} f(x) dx + \int_{R} g(x) dx. {}\]In other words, integration is still linear.
- Use the properties of Riemann integration to show that when \(f\) is a \(V\)-valued Riemann integrable function on \(R\) and when \(w : R \rightarrow {\mathbb R}\) is any traditional Riemann integrable function, \(x \mapsto w(x) f(x)\) is also Riemann integrable.
A important property of vector-valued Riemann integrable functions is that it's not just linear functionals which can be applied to them: if \(\varphi\) is any suitable (presumably nonlinear) Lipschitz function defined on a sufficient subset of \(V\), then \(\varphi(f)\) is also Riemann integrable. The precise statement is as follows:
Proposition
Suppose that \(R \subset {\mathbb R}^n\) is a box and \(f \ : \ R \rightarrow V\) is Riemann integrable for some finite-dimensional real vector space \(V\). Let \(||\cdot||\) be any norm on \(V\) and suppose that \(\varphi\) is any function defined on the closure of the image of \(f\) which is Lipschitz there, i.e., there is a finite constant \(C\) such that \(| \varphi(v_1) - \varphi(v_2)| \leq C ||v_1 - v_2||\) whenever \(v_1,v_2\) belong to the closure of the image of \(f\). Then \(\varphi(f)\) is itself a Riemann integrable function.
Proof
This argument runs in essentially the same way that it appeared in the context of Lipschitz images of real-valued functions. Suppose that \(\mathcal P\) is a partition of \(R\) and let \(R'\) be a box in \(\mathcal P\). Let \(\{e_i\}_{i=1}^n\) be any basis of \(V\) and let \(\{c_i\}_{i=1}^n\) be the elements of \(V^*\) such that \(x = \sum_{i=1}^n c_i(x) e_i\) for each \(x \in V\). By the triangle inequality,
\[ || v_1 - v_2|| \leq \sum_{i=1}^n |c_i(v_1) - c_i(v_2)| ||e_i|| \]
for each \(v_1,v_2 \in V\). This implies that
\[{}\sup_{x \in R'} \varphi(f(x)) - \inf_{x \in R'} \varphi(f(x)){}\]
\[{}= \sup_{x,y \in R'} \left[ \varphi(f(x)) - \varphi(f(y)) \right]{}\]
\[{}\leq \sup_{x,y \in R'} C || f(x) - f(y)||{}\]
\[{}\leq \sup_{x,y \in R'} \!C \sum_{i=1}^n |c_i(f(x)) - c_i(f(y))| || e_i||{}\]
\[{}\leq \sum_{i=1}^n C ||e_i|| \sup_{x,y \in R'} \left[ c_i(f(x)) - c_i(f(y)) \right].{}\]
In particular, it must follow that
\[{}\mathcal{U}(\varphi(f),\mathcal{P}) - \mathcal{L}(\varphi(f),\mathcal{P}){}\]
\[{}\leq \sum_{i=1}^n C ||e_i|| \Big[\mathcal{U}(c_i(f),\mathcal{P}){}\]
\[{}- \mathcal{L}(c_i(f),\mathcal{P}) \Big].{}\]
Because each \(c_i (f)\) is by definition Riemann integrable, it is possible to make each term on the right-hand side smaller than any fixed threshold \(\epsilon\) (taking common refinements as usual). Thus \(\varphi(f)\) must indeed be Riemann integrable.
Exercise
Prove the following corollary: if \(R\) is a box in \({\mathbb R}^n\), \(f\) is a \(V\)-valued Riemann integrable function, and \(\varphi\) is a Lipschitz map from \(V\) into some finite-dimensional real vector space \(W\), then \(\varphi(f)\) is also Riemann integrable.
3. Jensen's Inequality
Using the deep properties of convex functions that have been established, it is possible to prove the following fundamental theorem known as Jensen's inequality. It can be regarded as a sort of integral extension of the convexity property (replace the integral by a sum and the function \(w(x)\) by coefficient summing to one, and the inequality simply reduces to the familiar convexity inequality).
Theorem
Suppose \(R \subset {\mathbb R}^d\) is a box, \(E \subset R\) is a Jordan region, \(w\) is a nonnegative Riemann integrable function on \(E\) such that \(\int_E w(x) dx = 1\). Suppose also that \(f\) is a Riemann integrable \(V\)-valued function on \(E\) and that \(\varphi\) is any real-valued convex function defined on some open convex set containing the image \(f(E)\). Then \(\varphi(f(x)) w(x)\) is Riemann integrable on \(E\) and
\[{}\varphi \left( \int_E f(x) w(x) dx \right){}\]
\[{}\leq \int_E \varphi(f(x)) w(x) dx. {}\]
Proof
Note that \(\overline{O}\) is convex. The Separating Hyperplane Theorem guarantees that \(\overline{O}\) may be written as an intersection of half spaces \(\{ y \in V \ : \ \ell(y) \geq c \}\) for some appropriate set of pairs \((\ell,c) \in V^* \times {\mathbb R}\). Since \(f(x)\) belongs to the interior of \(\overline{O}\), \(\ell(f(x)) > c\) for each \(x \in E\). Consequently (see the exercises on Riemann integration comparison theorems) \(\int_E \ell(f(x)) w(x) dx > c\) as well. This means that \(\int_E f(x) w(x) dx\) belongs to any closed convex set \(K\) which contains \(f(x)\) for all \(x \in E\). In particular, it belongs to \(\overline{O}\) and is never on the boundary (since every boundary point \(y\) of \(\overline{O}\) satisfies \(\ell(y) = c\) for a suitable pair \((\ell,c)\) but \(\ell(\int_E f(x)w(x)dx)\) is always strictly greater than \(c\)) the vector \(\int_E f(x) w(x) dx\) belongs to \(O\).
Let \(y_0 := \int_E f(x) w(x) dx\); since \(\varphi\) is defined on an open set containing \(y_0\), there is some \(c_0 \in {\mathbb R}\) and \(\ell_0 \in V^*\) such that \(\varphi(y) \geq c_0 + \ell_0(y)\) for all \(y\) in the domain of \(\varphi\), with equality when \(y = y_0\).
\[{}\varphi \left( \int_E f(x) w(x) dx \right){}\]
\[{}= c_0 + \ell_0 \left( \int_E f(x) w(x) dx \right){}\]
\[{}= c_0 + \int_E \ell_0(f(x) w(x)) dx{}\]
\[{}= \int_E (c_0 + \ell_0(f(x))) w(x) dx{}\]
where on the last line we have used the fact that \(c_0 = \int_E c_0 w(x) dx\) because \(\int_E w(x) dx = 1\). Because \(w\) is nonnegative and \(c_0 + \ell_0(f(x)) \leq \varphi(f(x))\), it would immediately follow that
\[{}\int_E (c_0 + \ell_0(f(x))) w(x) dx{}\]
\[{}\leq \int_E \varphi(f(x)) w(x) dx {}\]
if one knew that \(\varphi(f(x))w(x)\) was Riemann integrable.
By the proposition above and the fact that convex functions are Lipschitz on compact subsets of open domains, it suffices to show that \(\overline{f(E)}\) is compact. Because it's closed, one only needs to show that the image \(f(E)\) is bounded. But this is a direct consequence of the fact that \(\ell(f(x))\) is bounded by definition for any \(\ell \in V^*\), since we can always write \(f(x) = \sum_{i=1}^n \ell_i(f(x)) e_i\) for any basis \(\{e_i\}_{i=1}^n\) of \(V\) and some appropriate choice of \(\ell_1,\ldots,\ell_n \in V^*\). Since the functions \(\ell_i(f(x))\) are bounded on \(E\) and the vectors \(e_i\) are constant, \(f(E)\) must be bounded.
Corollary (The Triangle Inequality)
Suppose \(f\) is any Riemann integrable \(V\)-valued function on a Jordan region \(E \subset {\mathbb R}^d\) and \(\tilde w\) is a nonnegative Riemann integrable function on \(E\). Then for any norm \(|| \cdot ||\) on \(V\),
\[ \left| \left| \int_{E} f(x) \tilde w(x) dx \right| \right| \leq \int_{E} || f(x)|| \tilde w(x) dx. \]
Proof
Suppose first that \(\int_E \tilde w(x) dx\) is strictly positive. Apply Jensen's inequality to the function \(f\) and weight \(w := \tilde w / \int_E \tilde w(x) dx\). Multiply both sides of the resulting inequality by \(\int_E \tilde w(x) dx\) to obtain the desired conclusion.
If it happens that \(\int_E \tilde w(x) dx = 0\), then any Riemann integrable function \(h\) on \(E\) admits some finite constant \(M\) such that \(-M \tilde w(x) \leq h(x) \tilde w(x) \leq M \tilde w(x)\) for all \(x \in E\), which (after integrating in \(x\)) implies that \(\int_{E} h(x) \tilde w(x) dx = 0\). This forces \(\int_E f(x) \tilde w(x) dx = 0\) when \(f\) is a Riemann integrable \(V\)-valued function on \(E\), and it also forces \(\int_E ||f(x)|| \tilde w(x) dx = 0\), so the inequality is still true when \(\int_E \tilde w(x) dx = 0\).