Partial Differential Equations/Calculus of variations

Calculus of variations is a method for proving existence and uniqueness results for certain equations; in particular, it can be applied to some partial differential equations. The method works as follows: Let's say we have an equation which is to be solved for the variable $x$ (this variable can also be a function). We look for a function whose minimizers satisfy the equation, and then prove that there exists a minimizer. We have thus obtained an existence result.

In some cases, we will additionally be able to show that values $x$ satisfying the equation are minimizers of the function. If we now find out about the number of minimizers of the function, we will also know the numbers of solutions to the equation. If then the function has only one minimizer, we have obtained a uniqueness result.

Sometimes, calculus of variations also works ‘the other way round’: We have a function whose minimizers are difficult to find. Then we show that the minimizers of this function are exactly the solutions of a partial differential equation, which is easy to solve. We then solve the partial differential equation in order to obtain the minimizers of the function.

"Normal" equations

Consider the equation system

$(*){\begin{cases}f_{1}(x)&=0\\f_{2}(x)&=0\\~~~~\vdots \\f_{d}(x)&=0\\\end{cases}}$

for functions $f_{n}:\mathbb {R} ^{d}\to \mathbb {R} ,n\in \{1,\ldots ,d\}$. If there exists a function $f\in {\mathcal {C}}^{1}(\mathbb {R} ^{d})$ such that

$\nabla f={\begin{pmatrix}f_{1}\\f_{2}\\\vdots \\f_{d}\end{pmatrix}}$

we find that the equation system $(*)$ is satisfied if and only if

$\nabla f(x)=0$

If $f$ satisfies the right conditions, we have $\nabla f(x)=0$ at exactly one point $x\in \mathbb {R} ^{d}$:

Definition 13.1:

Let $f\in {\mathcal {C}}^{2}(\mathbb {R} ^{d})$, and let's denote the Hessian matrix of $f$ at $x$ by $H_{f}(x)$. $f$ is called strongly convex iff

$\exists c>0:\forall x\in \mathbb {R} ^{d},\mathbf {v} \in \mathbb {R} ^{d}:\mathbf {v} ^{T}H_{f}(x)\mathbf {v} \geq c\|\mathbf {v} \|^{2}$

Theorem 13.2:

Let $f\in {\mathcal {C}}^{2}(\mathbb {R} ^{d})$ be strongly convex. Then $f$ has exactly one critical point (i. e. a point $x\in \mathbb {R} ^{d}$ where $\nabla f(x)=0$).

Proof:

From $f$ being strongly convex it follows that for all $x\in \mathbb {R} ^{d}$, $H_{f}(x)$ is positively definite. Therefore, every critical point is a local minimum (this is due to the sufficient condition for local minima). Thus, it suffices to prove that there is exactly one local minimum.

1.

We show that there exists a local minimum.

We take Taylor's formula around $0$:

$\forall x\in \mathbb {R} ^{d}:\exists \lambda \in [0,1]:f(x)=f(0)+x^{T}\nabla f(0)+{\frac {1}{2}}x^{T}H_{f}(\lambda x)x$

Thus,

${\begin{aligned}\forall x\in \mathbb {R} ^{d}:f(x)&=f(0)+x^{T}\nabla f(0)+{\frac {1}{2}}x^{T}H_{f}(\lambda x)x&{\text{ for a }}\lambda \in [0,1]\\&\geq f(0)+x^{T}\nabla f(0)+{\frac {c}{2}}\|x\|^{2}&f{\text{ is strongly convex}}\\&\geq f(0)-\|x\|\|\nabla f(0)\|+{\frac {c}{2}}\|x\|^{2}&{\text{Cauchy-Schwarz inequality}}\end{aligned}}$

for a $c\in \mathbb {R} _{>0}$. Therefore, there exists an $R\in \mathbb {R} _{>0}$ such that

$\forall x\notin B_{R}(0):f(x)>f(0)~~~~~(**)$

By the extreme value theorem, there exists a minimum $y$ of $f$ in ${\overline {B_{R}(0)}}$. It can not be attained on the border, because if $f(y)\in \partial B_{R}(0)$, then $y\notin B_{R}(0)$ and thus by $(**)$ $f(y)>f(0)$, which would imply that $y$ is not a minimum. Therefore it is attained in the interior and is thus a local minimum. In fact, from $(**)$ and from $y$ being a minimum on ${\overline {B_{R}(0)}}$ even follows that it is a global minimum of $f:\mathbb {R} ^{d}\to \mathbb {R}$.

2.

We show that there is only one local minimum.

Let $x$ and $y$ be two local minima. We show that $y=x$, thereby excluding the possibility of two different minima. We define a function $\mu :\mathbb {R} \to \mathbb {R}$ as follows:

$\mu (\xi ):=f(\xi y+(1-\xi )x)$

Let's calculate the first and second derivative of $\mu$:

${\begin{aligned}\mu '(\xi )&={\begin{pmatrix}\partial _{x_{1}}f(\xi y+(1-\xi )x)&\cdots &\partial _{x_{d}}f(\xi y+(1-\xi )x)\end{pmatrix}}(y-x)&{\text{ multi-dimensional chain rule}}\\&=\sum _{k=1}^{d}\partial _{x_{k}}f(\xi y+(1-\xi )x)(y_{k}-x_{k})&\\\mu ''(\xi )&=\sum _{k=1}^{d}{\begin{pmatrix}\partial _{x_{1}}\partial _{x_{k}}f(\xi y+(1-\xi )x)&\cdots &\partial _{x_{d}}\partial _{x_{k}}f(\xi y+(1-\xi )x)\end{pmatrix}}(y-x)&{\text{ multi-dimensional chain rule}}\\&=(y-x)^{T}H_{f}(\xi y+(1-\xi )x)(y-x)&\end{aligned}}$

Since $x$ and $y$ are local minima, $\nabla f(x)=0$ and $\nabla f(y)=0$. Therefore,

$\mu '(0)=\nabla f(x)\cdot (y-x)=0$

and

$\mu '(1)=\nabla f(y)\cdot (y-x)=0$

Therefore, by the mean value theorem, there exists a $\lambda \in (0,1)$ such that

$\mu ''(\lambda )=0$

But since

$\mu ''(\lambda )=(y-x)^{T}H_{f}(\lambda y+(1-\lambda )x)(y-x){\overset {{\text{strong convexity of }}f}{\geq }}c\|y-x\|^{2}$

, $\mu ''(\lambda )=0$ implies $x=y$.$\Box$

Corollary 13.3:

Suppose we have an equation system

$(*){\begin{cases}f_{1}(x)&=0\\f_{2}(x)&=0\\~~~~\vdots \\f_{d}(x)&=0\\\end{cases}}$

If there is a function $f\in {\mathcal {C}}^{2}(\mathbb {R} ^{d})$ which is strongly convex and

$\nabla f={\begin{pmatrix}f_{1}\\f_{2}\\\vdots \\f_{d}\end{pmatrix}}$

, then the equation system $(*)$ has exactly one solution.

Proof: See exercise 1.

Example 13.4:

Another example is given in exercise 2.