Derivatives of higher order – "Math for Non-Geeks"

Motivation

The derivative $f'$ describes the current rate of change of the function $f$. Now the derivative function $f'$ can be differentiated again, provided that it is again differentiable. The obtained derivative of the derivative is called second derivative or derivative of second order and is called $f''$ or $f^{(2)}$. This can be done arbitrarily often. If the second derivative is again differentiable, a third derivative $f^{(3)}$ can be constructed, then a fourth derivative $f^{(4)}$ and so on.

These higher derivatives allow statements about the course of a function graph. The second derivative tells us whether a graph is curved upwards ("convex") or curved downwards ("concave"). If a function has a convex graph, its gradient increases continuously. For this convexity, $f''(x)>0$ is a sufficient condition. If the second derivative is always positive, then the first derivative must grow continuously. Analogously, it follows from $f''(x)<0$ that the graph is concave and the derivative falls monotonically.

Higher-order derivatives do not only tell us more about abstract functions, they can also have a physical meaning. Consider the function $f:[a,b]\to \mathbb {R}$ with $f(t)=t^{3}+2$, which shall describe the location $f(t)$ of a car at the time $t$. We already know that we can calculate the speed of the car at the time $t$ with the first derivative: $f'(t)=f^{(1)}(t)=3t^{2}$. What does the derivative $f''(t)$ of $f'$ say? This is the instantaneous rate of change of speed and thus the acceleration of the car. It accelerates with $f''(t)=f^{(2)}(t)=6t$. So second derivatives describe accelerations.

Now we can derive this second derivative again, whereby we get the rate of change of acceleration $f'''(t)=f^{(3)}(t)=6$. This is called jerk in vehicle dynamics and indicates how fast a car increases acceleration or how fast it initiates braking. For example, a big jerk occurs during emergency braking. Since $f^{(3)}(t)<0$ is in an emergency stop, the graph of the speed $f'$ is convex - the speed decreases more and more. The fourth derivative $f^{4}(t)=0$ again tells us that the jerk has no instantaneous rate of change.

Definition

Definition (Derivatives of higher order)

Let $f:D\to W$ with $D,W\subseteq \mathbb {R}$ be a real function. We set $f^{(0)}:=f$ and in the case of differentiability $f^{(1)}=f'$. We define the second derivative via $f^{(2)}=\left(f^{(1)}\right)'$, the third derivative via $f^{(3)}=\left(f^{(2)}\right)'$ etc., if these higher derivatives exist. Overall, we define recursively for $k\in \mathbb {N} _{0}$:

$f^{(k+1)}:=\left(f^{(k)}\right)'$

We say $f$ that is $k$ times differentiable, if the $k$-th derivative $f^{(k)}$ of $f$ exists. $f$ is called $k$ times continuously differentiable, if $f^{(k)}$ is continuous (which is a stronger statement).

The set of all $k$ times continuously differential functions with domain of definition $D$ and range $W$ is denoted $C^{k}(D,W)$. In particular $C^{0}(D,W)=C(D,W)$ consists of the continuous functions. If we can derive the function $f$ arbitrarily often, we write $f\in C^{\infty }(D,W)$. If $W=\mathbb {R}$, then we can write $C^{k}(D)$ or $C^{\infty }(D)$ in short. Those sets of functions satisfy the inclusion chain:

$C(D,W)\supseteq C^{1}(D,W)\supseteq C^{2}(D,W)\supseteq \ldots \supseteq C^{k}(D,W)\supseteq C^{k+1}(D,W)$

Question: Are the following statements true of false?

1. $|x|\in C(\mathbb {R} )$
2. $|x|\in C^{1}(\mathbb {R} )$
3. $\operatorname {sgn}(x)\in C(\mathbb {R} )$
4. $\operatorname {sgn}(x)\in C^{1}(\mathbb {R} )$
5. $x\in C(\mathbb {R} )$
6. $x\in C^{1}(\mathbb {R} )$

Solutions:

1. true
2. false
3. false
4. false
5. true
6. true

Examples for higher derivatives

Derivatives of the power function

Example (Derivatives of the power function)

We consider the function $f:\mathbb {R} \to \mathbb {R} ,x\mapsto x^{2}$. This function is infinitely often differentiable, since there is for all $x\in \mathbb {R}$ and all $k\in \mathbb {N} ,k\geq 3$:

${\begin{aligned}&\ f'(x)=2x\\[0.3em]&\ f''(x)=2\\[0.3em]&\ f^{(k)}=0\\[0.3em]\end{aligned}}$

In general, for $f:\mathbb {R} \to \mathbb {R} ,x\mapsto x^{n}$ with $n\in \mathbb {N}$ there is:

${\begin{aligned}&\ f'(x)=nx^{n-1}\\[0.3em]&\ f^{(k)}(x)=n(n-1)\cdot \ldots \cdot (n-k+1)x^{n-k}{\text{ for }}k\leq n\\[0.3em]&\ f^{(k)}=0{\text{ for }}k>n\\[0.3em]\end{aligned}}$

Derivatives of the exponential function

Example (Derivatives of the exponential function)

For the exponential function $\exp :\mathbb {R} \to \mathbb {R}$ since $\exp '(x)=\exp(x)$ for all $x\in \mathbb {R}$ we have infinite differentiability $\exp \in C^{\infty }(\mathbb {R} ,\mathbb {R} )$. In addition there is for all $k\in \mathbb {N}$:

$\exp ^{(k)}(x)=\exp(x)$

Derivatives of the sine function

Example (Derivatives of the sine function)

The function $\sin :\mathbb {R} \to \mathbb {R}$ is infinitely often continuously differentiable. For all $x\in \mathbb {R}$ there is:

${\begin{aligned}&\sin '(x)=\cos(x)\\[0.3em]&\sin ''(x)=\cos '(x)=-\sin(x)\\[0.3em]&\sin '''(x)=(-\sin )'(x)=-\cos(x)\\[0.3em]&\sin ^{(4)}(x)=(-\cos )'(x)=\sin(x)\\[0.3em]\end{aligned}}$

In general, for all $n\in \mathbb {N} _{0}$ there is:

${\begin{aligned}&\sin ^{(4n+1)}(x)=\cos(x)\\[0.3em]&\sin ^{(4n+2)}(x)=\cos '(x)=-\sin(x)\\[0.3em]&\sin ^{(4n+3)}(x)=(-\sin )'(x)=-\cos(x)\\[0.3em]&\sin ^{(4n)}(x)=(-\cos )'(x)=\sin(x)\\[0.3em]\end{aligned}}$

Question: What are the derivatives von $\cos :\mathbb {R} \to \mathbb {R}$?

We use that $\sin '(x)=\cos(x)$. For $x\in \mathbb {R}$ there is

${\begin{aligned}&\cos '(x)=-\sin(x)\\[0.3em]&\cos ''(x)=-\cos(x)\\[0.3em]&\cos '''(x)=\sin(x)\\[0.3em]&\cos ^{(4)}(x)=\cos(x)\end{aligned}}$

In general, for all $n\in \mathbb {N} _{0}$ there is:

${\begin{aligned}&\cos ^{(4n+1)}(x)=-\sin(x)\\[0.3em]&\cos ^{(4n+2)}(x)=-\cos(x)\\[0.3em]&\cos ^{(4n+3)}(x)=\sin(x)\\[0.3em]&\cos ^{(4n)}(x)=\cos(x)\end{aligned}}$

Exercises: higher derivatives

Exactly once differentiable function

Additional question: Is $f'$ continuous at $0$?

Nope. Take the two sequences:

${\begin{aligned}&\ (a_{n})_{n\in \mathbb {N} }=({\tfrac {1}{2n\pi }})_{n\in \mathbb {N} }\\[0.3em]&\ (b_{n})_{n\in \mathbb {N} }=({\tfrac {1}{(2n+1)\pi }})_{n\in \mathbb {N} }\\[0.3em]\end{aligned}}$

For these sequences, there is: $\lim _{n\to \infty }a_{n}=\lim _{n\to \infty }b_{n}=0$. However

${\begin{aligned}&\ f'(a_{n})=2{\frac {1}{2n\pi }}\cdot \underbrace {\sin \left(2n\pi \right)} _{=0}-\underbrace {\cos \left(2n\pi \right)} _{=1}=-1\\[0.3em]&\ f'(b_{n})=2{\frac {1}{(2n+1)\pi }}\cdot \underbrace {\sin \left((2n+1)\pi \right)} _{=0}-\underbrace {\cos \left((2n+1)\pi \right)} _{=-1}=1\end{aligned}}$

So $\lim _{x\to 0}f'(x)$ doesn't exist. By means of the sequence criterion, $f'$ is hence not continuous at $0$.

Remark: Therefore, $f'$ is also not differentiable at $0$.

Computation rules for higher derivatives

Linearity

The linearity of derivatives is also "inherited" to higher derivatives: If $f$ and $g$ are differentiable, for $a,b\in \mathbb {R}$ the function $af+bg$ is also differentiable with

$(af+bg)'=af'+bg'$

If $f$ and $g$ are now even twice differentiable, then there is

$(af+bg)''=(af'+bg')'=af''+bg''$

If we continue to do so, we will get

Theorem (Linearity of higher derivatives)

Let $a,b\in \mathbb {R}$ and $f,g:D\to \mathbb {R}$ be $n$ times differentiable. Then also $af+bg:D\to \mathbb {R}$ is $n$ times differentiable, and for all $n\in \mathbb {N} _{0}$ there is:

$(af+bg)^{(n)}=af^{(n)}+bg^{(n)}$

Example (Linearity of higher derivatives)

Since $\sin ^{(4n+1)}=\cos$ and $\cos ^{(4n+1)}=-\sin$ for $n\in \mathbb {N}$ there is

$(3\sin(x)+4\cos(x))^{(1001)}=3\sin ^{(1001)}(x)+4\cos ^{(1001)}(x)=3\sin ^{(4\cdot 250+1)}(x)+4\cos ^{(4\cdot 250+1)}(x)=3\cos(x)-4\sin(x)$

Proof (Linearity of higher derivatives)

Theorem whose validity shall be proven for $n\in \mathbb {N}$ bewiesen werden soll:

$(af+bg)^{(n)}=af^{(n)}+bg^{(n)}$

1. Base case:

$(af+bg)^{(0)}=af+bg=af^{(0)}+bg^{(0)}$

2. Inductive step:

2a. Inductive hypothesis:

$(af+bg)^{(n)}=af^{(n)}+bg^{(n)}$

2b. Induction theorem:

$(af+bg)^{(n+1)}=af^{(n+1)}+bg^{(n+1)}$

2c. Proof of induction step:

${\begin{aligned}(af+bg)^{(n+1)}&=((af+bg)^{(n)})'\\[0.3em]&{\color {OliveGreen}\left\downarrow \ {\text{induction assumption}}\right.}\\[0.3em]&=(af^{(n)}+bg^{(n)})'\\[0.3em]&{\color {OliveGreen}\left\downarrow \ {\text{linearity of the derivative}}\right.}\\[0.3em]&=af^{(n+1)}+bg^{(n+1)}\end{aligned}}$

Leibniz rule for product functions

We now try to determine a general formula for the $n$-th derivative of the product function $fg$ of two arbitrarily often differentiable functions $f$ and $g$. By applying the factor-, sum- and product rule several times we obtain for $n=1,2,3$

${\begin{aligned}(fg)'&=f'g+fg'\\(fg)''&=(f'g+g'f)'=(f'g)'+(fg')'\\&=f''g+f'g'+f'g'+fg''\\&=f''g+2f'g'+fg''\\(fg)'''&=(f''g+2f'g'+fg'')'\\&=(f''g)'+(2f'g')'+(fg'')'\\&=f'''g+f''g'+2f''g'+2f'g''+f'g''+fg'''\\&=f'''g+3f''g'+3f'g''+fg'''\end{aligned}}$

If we plug in $f=x$ and $g=y$, and instead of the derivatives of $f$ and $g$ the corresponding powers of $x$ and $y$, we see a clear analogy to the binomial theorem:

${\begin{aligned}(x+y)^{1}&=x^{1}y^{0}+x^{0}+y^{1}\\(x+y)^{2}&=x^{2}+2xy+y^{2}\\&=x^{2}y^{0}+2x^{1}y^{1}+x^{0}y^{2}\\(x+y)^{3}&=x^{3}+3x^{2}y+3xy^{2}+y^{3}\\&=x^{3}y^{0}+3x^{2}y^{1}+3x^{1}y^{2}+3x^{0}y^{3}\end{aligned}}$

This analogy can be made clear as follows:

We assign for every $k\in \mathbb {N} _{0}$ the derivative $f^{(k)}$ to the power $x^{k}$, and the derivative $g^{(k)}$ to the power $y^{k}$. The $0$-th derivative $f^{(0)}=f$ corresponds to the $0$-th power $x^{0}=1$. The derivative of the term $f^{(k)}g^{(l)}$ is by means of the product rule

$(f^{(k)}g^{(l)})'=f^{(k+1)}g^{(l)}+f^{(k)}g^{(l+1)}$

The expression $f^{(k+1)}g^{(l)}+f^{(k)}g^{(l+1)}$ now corresponds in our analogy to the sum $x^{k+1}y^{l}+x^{k}y^{l+1}$. We get this term from $x^{k}y^{l}$ by multiplication with $x+y$. For our polynomials, the distributive law yields

$(x^{k}y^{l})(x+y)=x^{k+1}y^{l}+x^{k}y^{l+1}$

Therefore, the application of the product rule corresponds to the multiplication with the sum $x+y$. Thus the $n$-th derivative $(fg)^{(n)}$ corresponds to the power $(x+y)^{n}$. From the binomial theorem

$(x+y)^{n}=\sum _{k=0}^{n}{\binom {n}{k}}x^{k}y^{n-k}$

we hence get the

Theorem (Leibniz rule for derivatives)

Let $f,g:D\to \mathbb {R}$ be $n$ times differentiable functions. Then, $fg:D\to \mathbb {R}$ is $n$ times differentiable, and for all $n\in \mathbb {N} _{0}$, there is:

$(fg)^{(n)}=\sum _{k=0}^{n}{\binom {n}{k}}f^{(k)}g^{(n-k)}$

Example (Leibniz rule for derivatives)

Using the Leibniz rule we calculate $(x^{3}e^{x})^{(2016)}$. The rule is applicable because $x\mapsto x^{3}$ and $x\mapsto e^{x}$ are arbitrarily often differentiable on $\mathbb {R}$. There is

${\begin{aligned}(x^{3}e^{x})^{(2016)}&{\underset {\text{rule}}{\overset {\text{Leibniz}}{=}}}\sum _{k=0}^{2016}{\binom {2016}{k}}(x^{3})^{(k)}(e^{x})^{(2016-k)}\\[0.3em]&\color {Gray}\left\downarrow \ (x^{3})'=3x^{2},\ (x^{3})''=6x,\ (x^{3})^{(3)}=6,\ (x^{3})^{(k)}=0{\text{ for }}k\geq 4,\ (e^{x})^{(k)}=e^{x}{\text{ for all }}k\in \mathbb {N} \right.\\[0.3em]&={\binom {2016}{0}}x^{3}e^{x}+{\binom {2016}{1}}\cdot 3x^{2}e^{x}+{\binom {2016}{2}}\cdot 6xe^{x}+{\binom {2016}{3}}\cdot 6e^{x}\\[0.3em]&=x^{3}e^{x}+2016\cdot 3x^{2}e^{x}+{\frac {2016\cdot 2015}{2}}\cdot 6xe^{x}+{\frac {2016\cdot 2015\cdot 2014}{3\cdot 2}}\cdot 6e^{x}\\[0.3em]&=x^{3}e^{x}+2016\cdot 3x^{2}e^{x}+2016\cdot 2015\cdot 3xe^{x}+2016\cdot 2015\cdot 2014e^{x}\end{aligned}}$

Proof (Leibniz rule for derivatives)

Theorem whose validity shall be proven for $n\in \mathbb {N}$ bewiesen werden soll:

$(fg)^{(n)}=\sum _{k=0}^{n}{\binom {n}{k}}f^{(k)}g^{(n-k)}$

1. Base case:

$(fg)^{(0)}=fg=\sum _{k=0}^{0}{\binom {0}{k}}f^{(k)}g^{(0-k)}$

2. Inductive step:

2a. Inductive hypothesis:

$(fg)^{(n)}=\sum _{k=0}^{n}{\binom {n}{k}}f^{(k)}g^{(n-k)}$

2b. Induction theorem:

$(fg)^{(n+1)}=\sum _{k=0}^{n+1}{\binom {n+1}{k}}f^{(k)}g^{(n+1-k)}$

2c. Proof of induction step:

${\begin{aligned}(fg)^{(n+1)}&=\left[(fg)^{(n)}\right]'\\[0.3em]&\color {Gray}\left\downarrow \ {\text{ induction assumption}}\right.\\[0.3em]&=\left[\sum _{k=0}^{n}{\binom {n}{k}}f^{(k)}g^{(n-k)}\right]'\\[0.3em]&\color {Gray}\left\downarrow \ {\text{ linearity of the derivative}}\right.\\[0.3em]&=\sum _{k=0}^{n}{\binom {n}{k}}\left[f^{(k)}g^{(n-k)}\right]'\\[0.3em]&\color {Gray}\left\downarrow \ {\text{ product rule}}\right.\\[0.3em]&=\sum _{k=0}^{n}{\binom {n}{k}}\left[f^{(k+1)}g^{(n-k)}+f^{(k)}g^{(n-k+1)}\right]\\[0.3em]&=\sum _{k=0}^{n}{\binom {n}{k}}f^{(k+1)}g^{(n-k)}+\sum _{k=0}^{n}{\binom {n}{k}}f^{(k)}g^{(n-k+1)}=\\[0.3em]&\color {Gray}\left\downarrow \ {\text{ index shift}}\right.\\[0.3em]&=\sum _{k=1}^{n+1}{\binom {n}{k-1}}f^{(k)}g^{(n-(k-1))}+\sum _{k=0}^{n}{\binom {n}{k}}f^{(k)}g^{(n-k+1)}\\[0.3em]&\color {Gray}\left\downarrow \ {\binom {n}{-1}}=0{\text{ and }}{\binom {n}{n+1}}=0\right.\\[0.3em]&=\sum _{k=0}^{n+1}{\binom {n}{k-1}}f^{(k)}g^{(n-k+1)}+\sum _{k=0}^{n+1}{\binom {n}{k}}f^{(k)}g^{(n-k+1)}\\[0.3em]&=\sum _{k=0}^{n+1}\left[{\binom {n}{k-1}}+{\binom {n}{k}}\right]f^{(k)}g^{(n-k+1)}\\[0.3em]&\color {Gray}\left\downarrow \ {\text{ sum of binomial coefficients}}\right.\\[0.3em]&=\sum _{k=0}^{n+1}{\binom {n+1}{k}}f^{(k)}g^{((n+1)-k)}\end{aligned}}$