Extreme value theorem – "Math for Non-Geeks"

In the following we are going to deal with continuous functions on compact intervals. These are intervals that are closed and bounded, i.e. have the form $[a,b]$. We will see that such functions are always bounded and attain a minimum and maximum. This theorem is called the Extreme Value Theorem. It is used in mathematics to prove the existence of relative extrema, i.e. points of a function that are "at the extreme" of being the lowest point in the graph (the minimum) or the highest point in the graph (the maximum).

Motivation

Motivation and intuition behind the extreme value theorem (in German)

Let's take a continuous function $f$ which is defined on a compact interval $[a,b]$. I.e. we are considering a function $f:[a,b]\to \mathbb {R}$. This function has the value $f(a)$ at the point $a$ and the value $f(b)$ at the point $b$.

The function values f(a) and f(b)

Now $f$ is defined for every intermediate point between $a$ and $b$. Intuitively, for functions without gaps in the domain of definition, continuity means that we are able to draw the graph without lifting the pen from the paper. Hence, the graph of $f$ connects the points $(a,f(a))$ and $(b,f(b))$ by a continuous path without jumps. The following graph provides an example for such a function $f$:

The fraph of the function f

We note that the function $f$ above is bounded. And it attains a maximum and minimum value:

The function f ist bounded and attans a maximum and a minimum

Is it always that way? Try for yourself to connect the points $(a,f(a))$ and $(b,f(b))$ by different graphs without lifting the pen. Could you imagine to draw the graph of an unbounded function - even if your paper was infinitely large?

Intuitively, the answer is no. No matter how far your graph goes up or down, you need to return to the end points at $(a,f(a))$ or $(b,f(b))$. Going to infinity "forces you to lift the pen" and is therefore not allowed. However, the function $f$ can attain very large values (like $10^{100}$ or more) while staying bounded, as long as you return in order to reach the end point. This situation is illustrated in the following figure:

Another bounded function with values f(a) and f(b), attaining a maximum and a minimum

So our first intuition tells us that when connecting the two points $(a,f(a))$ and $(b,f(b))$ without lifting the pen, our function stays bounded. And it attains a maximum and a minimum. Now let's think about what could go wrong when phrasing this intuition in a mathematical way. The end points $a$ and $b$ of the domain of definition $[a,b]$ could become problematic: For an open domain of definition $(a,b)$, the function could run towards $\pm \infty$ at $a$ or $b$, or it could converge towards a value without attaining it. Including the boundary points $\{a,b\}$ in the domain of definition excludes these cases as it "catches the function" at the end points $(a,f(a))$ and $(b,f(b))$. If we move a boundary to infinity, let's say by considering the domain of definition $[a,\infty )$, the function could have "infinitely much time" and might run towards infinity while being continuous. This happens for instance for $f(x)=x^{2}$. So we also expect problems with unbounded domains of definition. A statement like "the maximum and minimum are attained" can only be expected to hold true on a compact interval $[a,b]$. Now, in between $a$ and $b$, the function could also "break out" and tend towards $\pm \infty$ (like $f(x)={\tfrac {1}{x}}$ near $x=0$). This scenario will be prevented by assuming continuity of the function $f$.

In the following, we will mathematically verify that our intuition is true. That means we prove a statement like "the maximum and minimum are attained" for continuous functions defined on a compact interval $[a,b]$ and discuss what may go wrong if we choose other domains of definition.

Extreme value theorem

Theorem (Extreme value theorem)

Explanation of the extreme value theoremm (in German). (YouTube-Video published by Quatematik)

Every continuous function defined on a compact interval $[a,b]$ is bounded and attains a maximum and a minimum (extreme values). That means, if $f:[a,b]\to \mathbb {R}$ with $a,b\in \mathbb {R}$ and $a<b$ is a continuous function, then there are arguments ${\tilde {x}},{\hat {x}}\in [a,b]$, such that for all arguments $x\in [a,b]$ the inequality $f({\hat {x}})\leq f(x)\leq f({\tilde {x}})$ holds.

Example (Extreme value theorem)

Graph of a function $f:[0,1]\to \mathbb {R}$ with $f(x)=x^{2}\cdot \cos(x)\cdot e^{\cos(x)}-\ln(x+1)$. This function attains a maximum and a minimum, which is attained in form of a function value.

Consider $f:[0,1]\to \mathbb {R}$ with $f(x)=x^{2}\cdot \cos(x)\cdot e^{\cos(x)}-\ln(x+1)$. The domain of definition $[0,1]$ is a compact interval. In addition, $f$ is continuous as it is composed out of continuous functions $x\mapsto x^{2}$, $x\mapsto e^{x}$, $x\mapsto \cos(x)$ and $x\mapsto \ln(x+1)$ with domains of definition $[0,1]$. Hence, $f$ must attain a maximum and a minimum.

Proof (Extreme value theorem)

Proof of the extreme value theorem (in German)

Let $f:[a,b]\to \mathbb {R}$ be a continuous function with $a,b\in \mathbb {R}$ and $a<b$. We will only prove explicitly that $f$ is bounded from above and attains a maximum. The analogous statement that $f$ is bounded from below and attains a minimum can be shown the same way.

So let us consider the image $f([a,b])$. This is the the set of all function values, which are attained by $f$. Let us take the supremum $\sup f([a,b])$ of the set $f([a,b])$, where we explicitly allow for use of the extended definition of the supremum $\sup f([a,b])=\infty$ . If $f$ is bounded from above, then $\sup f([a,b])=\infty$ and else, $\sup([a,b])\in \mathbb {R}$ (since $f([a,b])\neq \emptyset$ , the case $\sup \emptyset =-\infty$ cannot occur).

Now, we know that there is a sequence in $f([a,b])$ , which tends towards the supremum $\sup f([a,b])$ (for each nonempty set $M$ there is a subsequence in $M$, tending towards $\sup M$). Hence there is a sequence $(x_{n})_{n\in \mathbb {N} }$ of arguments in $[a,b]$ with $\lim _{n\to \infty }f(x_{n})=\sup f([a,b])$.

We now make use of the Bolzano-Weierstraß theorem. This theorem tells us that each sequence in a compact interval $[a,b]$ with $a,b\in \mathbb {R}$ and $a<b$ has a converging subsequence. Hence, $(x_{n})_{n\in \mathbb {N} }$ also has a convergent subsequence $\left(x_{n_{k}}\right)_{k\in \mathbb {N} }$. Let ${\tilde {x}}$ be the limit of the convergent subsequence $\left(x_{n_{k}}\right)_{k\in \mathbb {N} }$. Since $a\leq x_{n_{k}}\leq b$ for all $k\in \mathbb {N}$ , there is also $a\leq {\tilde {x}}\leq b$ and therefore ${\tilde {x}}\in [a,b]$. So, ${\tilde {x}}$ must be an argument of the function $f$. Since $f$ is continuous, we can make use of the sequential definition of continuity

$f\left({\tilde {x}}\right)=f\left(\lim _{k\to \infty }x_{n_{k}}\right)=\lim _{k\to \infty }f\left(x_{n_{k}}\right)=\sup f([a,b])$

$\sup f([a,b])$ is a function value of $f$ and hence a real number. Therefore we know that $f$ is bounded from above. And we have shown that $f$ attains its upper bound $\sup f([a,b])$ at the argument ${\tilde {x}}$ . Therefore, $f(x)\leq f({\tilde {x}})$ for all $x\in [a,b]$ and indeed, $f({\tilde {x}})$is the maximum among all function values of $f$.

Assumptions of the theorem

Assumptions of the extreme value theorem (in German)

Let's take a look at the assumptions made within the extreme value theorem:

• $f$ is a continuous function
• $f$ is defined on a compact interval $[a,b]$

Are those assumptions really necessary or can we relax them without losing validity of the extreme value theorem?

Assumption of continuity

Function $f(x)={\tfrac {1}{x}}$ on the interval $[-1,1]$ with $f(0)=0$

First, we note that continuity prevents the function $f$ from "breaking out" to $+\infty$ or $-\infty$ within its domain of definition. Ij we just allow any function $f:[a,b]\to \mathbb {R}$ , no matter whether it is continuous or not, we will find non-continuous functions which are violating the extreme value theorem. The following function is unbounded (so it does not attain any extrema) and non-continuous at $x=0$ :

$f:[-1,1]\to \mathbb {R} :x\mapsto {\begin{cases}{\frac {1}{x}}&;x\neq 0\\0&;x=0\end{cases}}$

So we cannot simply drop the assumption that $f$ is continuous.

Interval-assumption

The function $f(x)={\tfrac {1}{x}}$ on $(0,1]$

The domain of definition is also important. It must include its boundary (i.e. be closed). This way we "catch" the function at the interval boundary and make sure it does not "run away" towards infinity. The function $g:(0,1]\to \mathbb {R} :x\mapsto {\tfrac {1}{x}}$ is an example which "runs away" as we approach $x=0$.

Unboundedly large domains are also problematic, since the function has "infinitely much time" to run away. An easy example is the function $h:[0,\infty )\to \mathbb {R} :x\mapsto x^{2}$. And there are functions, which are defined on a bounded domain of definition, continuous and do not "run away" towards infinity, but do not attain an extremum. This happens if there are open boundaries or gaps within the domain of definition. The extremum would then be attained at the boundary (or the gap) - but this argument has been removed from the domain of definition.

Question: Does the continuous function $f$ always attain a maximum or a minimum, if it is defined on a bounded interval?

No. For instance, the function $f:(0,1)\to \mathbb {R}$, $f(x)=x$ does not. There is: $f((0,1))=(0,1)$. So the infimum and maximum of the image are 0 and 1, which would be attained at the boundary ($x=0$ and $x=1$), if $f$ was defined there. However, we removed 0 and 1 from the domain of definition, so $f$ does no longer attain an extremum.

The same may happen when removing a maximum or minimum from the interior of the domain of definition instead of the boundary - which creates a gap. Of course, the function may also "run away" at such a gap. An example for this effect is the continuous but unbounded function $j:[-1,1]\setminus \{0\}\to \mathbb {R} :x\mapsto {\tfrac {1}{x}}$. The argument $x=0$ is excluded from the domain of definition $[-1,1]\setminus \{0\}=[-1,0)\cup (0,1]$ . So this function is well defined and continuous, but it "runs away" at the gap. In a mathematical language, the function is unbounded and hence violates the conclusion of the extreme value theorem.

Outlook: Generalization of the theorem

So far, we only considered intervals (possibly with gaps) as candidates for the domain of definition. Is this restriction really necessary? This time, the answer is no. For instance, we can take the union of two intervals $D=[a,b]\cup [c,d]$ with $a<b<c<d$ and define some continuous and real-valued function $j$ on $D$. If we restrict $j$ to only $[a,b]$ or $[c,d]$ , we can apply the extreme value theorem. Both the functions $j_{1}=j|_{[a,b]}$ and $j_{2}=j|_{[c,d]}$ with restricted domain of definition are bounded and hence attain a maximum and a minimum. The function $j$ must therefore also be bounded. Its maximum is the larger of the both maxima of $j_{1}$ and $j_{2}$ , so $j$ also attains a maximum (the same holds for the minimum). Therefore, every continuous function defined on the union of two closed intervals $[a,b]\cup [c,d]$ fulfills the conclusion of the extreme value theorem. The same holds if we consider three or more closed intervals - or an even larger class of domains of definition. In fact, we can precisely state what this larger class of domains of definition is:

If we take a second look at the proof, we note that the domain of definition is only mentioned at one point: where we make use of the Bolzano-Weierstraß theorem. We used it to show that any sequence from the domain of definition contains a convergent subsequence. Hence, the proof arguments hold true, as long as the domain of definition allows for the usage of the Bolzano-Weierstraß theorem.

So we can generalize the above theorem. It will hold not only on closed intervals $[a,b]$, but on all sets satisfying the Bolzano-Weierstraß theorem. We will call these sets satisfying the Bolzano-Weierstraß theorem sequentially closed sets:

Definition (Sequential closedness)

A subset of the real numbers is called sequentially closed iff any sequence out of this set has a convergent subsequence.

If the domain of definition $D$ of a continuous function $f:D\to \mathbb {R}$ is sequentially compact, then the function $f$ must fulfill the extreme value theorem. The generalization of sequential compactness from real numbers to other sets of mathematical objects is one of the topics dealt with in topology.

Exercise: Image of polynomials of even degree

Exercise: Image of polynomials of even degree (in German)

Exercise: Continuous functions on [0,1]

Exercise: Continuous functions on [0,1] (in German)

In addition, the statement of the above exercise can be generalized in multiple ways:

1. We have shown that on a compact interval $[a,b]$ , there is no continuous function attaining each value twice.
2. Similarly, one may show that there is no continuous function $f:\mathbb {R} \to \mathbb {R}$attaining each of its values twice.
3. And for each given number $n\in \mathbb {N}$, $n>1$ , one can show that there is no continuous function $f:[a,b]\to \mathbb {R}$ attaining each of its values exactly $n$ times.

Exercise for understanding: Give an example for:

1. A continuous function $f:[0,1]\to \mathbb {R}$, attaining each of its function values exactly once.
2. A function $f:(0,1]\to \mathbb {R}$ (non-continuous) attaining all of its function values exactly twice.

Possible solution:

1. $f:[0,1]\to \mathbb {R}$ with $f(x)=x$
2. $f:(0,1]\to \mathbb {R}$ with $f(x)={\begin{cases}2x&0<x\leq {\frac {1}{2}}\\2x-1&{\frac {1}{2}}<x\leq 1\end{cases}}$