Calculus/Inverse function theorem, implicit function theorem

In this chapter, we want to prove the inverse function theorem (which asserts that if a function has invertible differential at a point, then it is locally invertible itself) and the implicit function theorem (which asserts that certain sets are the graphs of functions).

Banach's fixed point theorem

Theorem:

Let $(M,d)$ be a complete metric space, and let $f:M\to M$ be a strict contraction; that is, there exists a constant $0\leq \lambda <1$ such that

$\forall m,n\in M:d(f(m),f(n))\leq \lambda d(m,n)$.

Then $f$ has a unique fixed point, which means that there is a unique $x\in M$ such that $f(x)=x$. Furthermore, if we start with a completely arbitrary point $y\in M$, then the sequence

$y,f(y),f(f(y)),f(f(f(y))),\ldots$

converges to $x$.

Proof:

First, we prove uniqueness of the fixed point. Assume $x,y$ are both fixed points. Then

$d(x,y)=d(f(x),f(y))\leq \lambda d(x,y)\Rightarrow (1-\lambda )d(x,y)=0$.

Since $0\leq \lambda <1$, this implies $d(x,y)=0\Rightarrow x=y$.

Now we prove existence and simultaneously the claim about the convergence of the sequence $y,f(y),f(f(y)),f(f(f(y))),\ldots$. For notation, we thus set $z_{0}:=y$ and if $z_{n}$ is already defined, we set $z_{n+1}=f(z_{n})$. Then the sequence $(z_{n})_{n\in \mathbb {N} }$ is nothing else but the sequence $y,f(y),f(f(y)),f(f(f(y))),\ldots$.

Let $n\geq 0$. We claim that

$d(z_{n+1},z_{n})\leq \lambda ^{n}d(z_{1},z_{0})$.

Indeed, this follows by induction on $n$. The case $n=0$ is trivial, and if the claim is true for $n$, then $d(z_{n+2},z_{n+1})=d(f(z_{n+1}),f(z_{n}))\leq \lambda d(z_{n+1},z_{n})\leq \lambda \cdot \lambda ^{n}d(z_{1},z_{0})$.

Hence, by the triangle inequality,

${\begin{aligned}d(z_{n+m},z_{n})&\leq \sum _{j=n+1}^{n+m}d(z_{j},z_{j-1})\\&\leq \sum _{j=n+1}^{n+m}\lambda ^{j-1}d(z_{1},z_{0})\\&\leq \sum _{j=n+1}^{\infty }\lambda ^{j-1}d(z_{1},z_{0})\\&=d(z_{1},z_{0})\lambda ^{n}{\frac {1}{1-\lambda }}\end{aligned}}$.

The latter expression goes to zero as $n\to \infty$ and hence we are dealing with a Cauchy sequence. As we are in a complete metric space, it converges to a limit $x$. This limit further is a fixed point, as the continuity of $f$ ($f$ is Lipschitz continuous with constant $\lambda$) implies

$x=\lim _{n\to \infty }z_{n}=\lim _{n\to \infty }f(z_{n-1})=f(\lim _{n\to \infty }z_{n-1})=f(x)$.$\Box$

A corollary to this important result is the following lemma, which shall be the main ingredient for the proof of the inverse function theorem:

Lemma:

Let $g:{\overline {B_{r}(0)}}\to {\overline {B_{r}(0)}}$ (${\overline {B_{r}(0)}}\subset \mathbb {R} ^{n}$ denoting the closed ball of radius $r$) be a function which is Lipschitz continuous with Lipschitz constant less or equal $1/2$ such that $g(0)=0$. Then the function

$f:{\overline {B_{r}(0)}}\to \mathbb {R} ^{n},f(x):=g(x)+x$

is injective and $B_{r/2}(0)\subseteq f(B_{r}(0))$.

Proof:

First, we note that for $y\in B_{r/2}(0)$ the function

$h:{\overline {B_{r}(0)}}\to \mathbb {R} ^{n},h(z):=y-g(z)$

is a strict contraction; this is due to

$\|y-g(z)-(y-g(z'))\|=\|g(z')-g(z)\|\leq {\frac {1}{2}}\|z-z'\|$.

Furthermore, it maps ${\overline {B_{r}(0)}}$ to itself, since for $z\in {\overline {B_{r}(0)}}$

$\|y-g(z)\|\leq \|y\|+\|g(z-0)\|\leq {\frac {r}{2}}+{\frac {1}{2}}\|z\|\leq r$.

Hence, the Banach fixed-point theorem is applicable to $h$. Now $x$ being a fixed point of $h$ is equivalent to

$f(x)=y$,

and thus $B_{r/2}(0)\subseteq f(B_{r}(0))$ follows from the existence of fixed points. Furthermore, if $f(x)=f(x')$, then

${\frac {1}{2}}\|x-x'\|\geq \|g(x)-g(x')\|=\|f(x)-x-(f(x')-x')\|=\|x-x'\|$

and hence $x=x'$. Thus injectivity.$\Box$

The inverse function theorem

Theorem:

Let $f:\mathbb {R} ^{n}\to \mathbb {R} ^{n}$ be a function which is continuously differentiable in a neighbourhood $x_{0}\in \mathbb {R} ^{n}$ such that $f'(x_{0})$ is invertible. Then there exists an open set $U\subseteq \mathbb {R} ^{n}$ with $x_{0}\in U$ such that $f|_{U}$ is a bijective function with an inverse $f^{-1}:f(U)\to U$ which is differentiable at $x_{0}$ and satisfies

$(f^{-1})'(f(x_{0}))=(f'(x_{0}))^{-1}$.

Proof:

We first reduce to the case $f(x_{0})=0$, $x_{0}=0$ and $f'(x_{0})={\text{Id}}$. Indeed, suppose for all those functions the theorem holds, and let now $h$ be an arbitrary function satisfying the requirements of the theorem (where the differentiability is given at $x_{0}$). We set

${\tilde {h}}(x):=h'(x_{0})^{-1}(h(x_{0}-x)-h(x_{0}))$

and obtain that ${\tilde {h}}$ is differentiable at $0$ with differential ${\text{Id}}$ and ${\tilde {h}}(0)=0$; the first property follows since we multiply both the function and the linear-affine approximation by $h'(x_{0})^{-1}$ and only shift the function, and the second one is seen from inserting $x=0$. Hence, we obtain an inverse of ${\tilde {h}}$ with it's differential at ${\tilde {h}}(0)=0$, and if we now set

$h^{-1}(y):=({\tilde {h}}^{-1}(h'(x_{0})^{-1}(y-h(x_{0})))-x_{0})$,

it can be seen that $h^{-1}$ is an inverse of $h$ with all the required properties (which is a bit of a tedious exercise, but involves nothing more than the definitions).

Thus let $f$ be a function such that $f(0)=0$, $f$ is invertible at $0$ and $f'(0)={\text{Id}}$. We define

$g(x):=f(x)-x$.

The differential of this function is zero (since taking the differential is linear and the differential of the function $x\mapsto x$ is the identity). Since the function $g$ is also continuously differentiable at a small neighbourhood of $0$, we find $\delta >0$ such that

${\frac {\partial g}{\partial x_{j}}}(x)<{\frac {1}{2n^{2}}}$

for all $j\in \{1,\ldots ,n\}$ and $x\in B_{\delta }(0)$. Since further $g(0)=f(0)-0=0$, the general mean-value theorem and Cauchy's inequality imply that for $k\in \{1,\ldots ,n\}$ and $x\in B_{\delta }(0)$,

$|g_{k}(x)|=|\langle x,{\frac {\partial g}{\partial x_{j}}}(t_{k}x)\rangle |\leq \|x\|n{\frac {1}{2n^{2}}}$

for suitable $t_{k}\in [0,1]$. Hence,

$\|g(x)\|\leq |g_{1}(x)|+\cdots +|g_{n}(x)|\leq {\frac {1}{2}}\|x\|$ (triangle inequality),

and thus, we obtain that our preparatory lemma is applicable, and $f$ is a bijection on ${\overline {B_{\delta }(0)}}$, whose image is contained within the open set ${\overline {B_{\delta /2}(0)}}$; thus we may pick $U:=f^{-1}(B_{\delta /2}(0))$, which is open due to the continuity of $f$.

Thus, the most important part of the theorem is already done. All that is left to do is to prove differentiability of $f^{-1}$ at $0$. Now we even prove the slightly stronger claim that the differential of $f^{-1}$ at $x_{0}$ is given by the identity, although this would also follow from the chain rule once differentiability is proven.

Note now that the contraction identity for $g$ implies the following bounds on $f$:

${\frac {1}{2}}\|x\|\leq \|f(x)\|\leq {\frac {3}{2}}\|x\|$.

The second bound follows from

$\|f(x)\|\leq \|f(x)-x\|+\|x\|=\|g(x)\|+\|x\|\leq {\frac {3}{2}}\|x\|$,

and the first bound follows from

$\|f(x)\|\geq |\|f(x)-x\|-\|x\||=\left|\|g(x)\|-\|x\|\right|\geq {\frac {1}{2}}\|x\|$.

Now for the differentiability at $0$. We have, by substitution of limits (as $f$ is continuous and $f(0)=0$):

${\begin{aligned}\lim _{\mathbf {h} \to 0}{\frac {\|f^{-1}(\mathbf {h} )-f^{-1}(0)-\operatorname {Id} (\mathbf {h} -0)\|}{\|\mathbf {h} \|}}&=\lim _{\mathbf {h} \to 0}{\frac {\|f^{-1}(f(\mathbf {h} ))-f(\mathbf {h} )\|}{\|f(\mathbf {h} )\|}}\\&=\lim _{\mathbf {h} \to 0}{\frac {\|\mathbf {h} -f(\mathbf {h} )\|}{\|f(\mathbf {h} )\|}},\end{aligned}}$

where the last expression converges to zero due to the differentiability of $f$ at $0$ with differential the identity, and the sandwhich criterion applied to the expressions

${\frac {\|\mathbf {h} -f(\mathbf {h} )\|}{{\frac {3}{2}}\|\mathbf {h} \|}}$

and

${\frac {\|\mathbf {h} -f(\mathbf {h} )\|}{{\frac {1}{2}}\|\mathbf {h} \|}}$.$\Box$

The implicit function theorem

Theorem:

Let $f:\mathbb {R} ^{n}\to \mathbb {R}$ be a continuously differentiable function, and consider the set

$S:=\{(x_{1},\ldots ,x_{n})\in \mathbb {R} ^{n}|f(x_{1},\ldots ,x_{n})=0\}$.

If we are given some $y\in S$ such that $\partial _{n}f(y)\neq 0$, then we find $U\subseteq \mathbb {R} ^{n-1}$ open with $(y_{1},\ldots ,y_{n-1})\in U$ and $g:U\to S$ such that

$y=g(y_{1},\ldots ,y_{n-1})$ and $\{(z_{1},\ldots ,z_{n-1},g(z_{1},\ldots ,z_{n-1}))|(z_{1},\ldots ,z_{n-1})\in U\}\subseteq S$,

where $\{(z_{1},\ldots ,z_{n-1},g(z_{1},\ldots ,z_{n-1}))|(z_{1},\ldots ,z_{n-1})\in U\}$ is open with respect to the subspace topology of $U$.

Furthermore, $g$ is a differentiable function.

Proof:

We define a new function

$F:\mathbb {R} ^{n}\to \mathbb {R} ^{n},F(x_{1},\ldots ,x_{n}):=(x_{1},\ldots ,x_{n-1},f(x_{1},\ldots ,x_{n}))$.

The differential of this function looks like this:

$F'(x)={\begin{pmatrix}1&0&\cdots &&0\\0&1&&&\vdots \\\vdots &&\ddots &&\\0&\cdots &0&1&0\\\partial _{1}f(x)&&\cdots &&\partial _{n}f(x)\end{pmatrix}}$

Since we assumed that $\partial _{n}f(y)\neq 0$, $F'(y)$ is invertible, and hence the inverse function theorem implies the existence of a small open neighbourhood ${\tilde {V}}\subseteq \mathbb {R} ^{n}$ containing $y$ such that restricted to that neighbourhood $F$ is itself invertible, with a differentiable inverse $F^{-1}$, which is itself defined on an open set ${\tilde {U}}$ containing $F(y)$. Now set first

$U:=\{(x_{1},\ldots ,x_{n-1})|(x_{1},\ldots ,x_{n-1},0)\in {\tilde {U}}\}$,

which is open with respect to the subspace topology of $\mathbb {R} ^{n-1}$, and then

$g:U\to \mathbb {R} ,g(x_{1},\ldots ,x_{n-1}):=\pi _{n}(F^{-1}(x_{1},\ldots ,x_{n-1},0))$,

the $n$-th component of $F^{-1}(x_{1},\ldots ,x_{n-1},0)$. We claim that $g$ has the desired properties.

Indeed, we first note that $F^{-1}(x_{1},\ldots ,x_{n-1},0)=(x_{1},\ldots ,x_{n-1},g(x_{1},\ldots ,x_{n-1}))$, since applying $F$ leaves the first $n-1$ components unchanged, and thus we get the identity by observing $F(F^{-1}(x))=x$. Let thus $(z_{1},\ldots ,z_{n-1})\in U$. Then

${\begin{aligned}f(z_{1},\ldots ,z_{n-1},g(z_{1},\ldots ,z_{n-1}))&=(\pi _{n}\circ F)(F^{-1}(z_{1},\ldots ,z_{n-1},0))\\&=\pi _{n}((F\circ F^{-1})(z_{1},\ldots ,z_{n-1},0))=0\end{aligned}}$.

Furthermore, the set

$\{(z_{1},\ldots ,z_{n-1},g(z_{1},\ldots ,z_{n-1}))|(z_{1},\ldots ,z_{n-1})\in U\}$

is open with respect to the subspace topology on $S$. Indeed, we show

$\{(z_{1},\ldots ,z_{n-1},g(z_{1},\ldots ,z_{n-1}))|(z_{1},\ldots ,z_{n-1})\in U\}=S\cap {\tilde {V}}$.

For $\subseteq$, we first note that the set on the left hand side is in $S$, since all points in it are mapped to zero by $f$. Further,

$F(z_{1},\ldots ,z_{n-1},g(z_{1},\ldots ,z_{n-1}))=(z_{1},\ldots ,z_{n-1},0)\in {\tilde {U}}$

and hence $\subseteq$ is completed when applying $F^{-1}$. For the other direction, let a point $(x_{1},\ldots ,x_{n})$ in $S\cap {\tilde {V}}$ be given, apply $F$ to get

$F((x_{1},\ldots ,x_{n}))=(x_{1},\ldots ,x_{n-1},0)\in {\tilde {U}}$

and hence $(x_{1},\ldots ,x_{n-1})\in U$; further

$(x_{1},\ldots ,x_{n-1},g(x_{1},\ldots ,x_{n-1}))=(x_{1},\ldots ,x_{n})$

by applying $F$ to both sides of the equation.

Now $g$ is automatically differentiable as the component of a differentiable function.$\Box$

Informally, the above theorem states that given a set $\{x\in \mathbb {R} ^{n}|f(x)=0\}$, one can choose the first $n-1$ coordinates as a "base" for a function, whose graph is precisely a local bit of that set.