2.1 Differentiation of Real-valued Functions

This theorem motivates the following definition.

Thus by Theorem 3, any $C^1$ function is differentiable everywhere in $S$.

Example 5.

Let $f(x,y)=(x-1{)}^3(y+2)$. At which points of ${\mathbb{R}}^2$ is $f$ differentiable?

Example 6.

Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be a polynomial of total degree $d$. At which points of ${\mathbb{R}}^n$ is $f$ differentiable?

Contrast this with the example using a naive, incorrect definition for differentiable. The correct definition of differentiable functions eventually shows that polynomials are differentiable, and leads us towards other concepts that we might find useful, like $C^1$. The incorrect naive definition leads to $f(x,y)=x$ not being differentiable. Although it looks more complicated, the correct version does two important things that we look for from mathematical definitions: it includes the functions that we intuitively believe it should, and it leads us to new interesting properties.

Directional derivatives and the meaning of the gradient

A direction in ${\mathbb{R}}^n$ is naturally represented by a unit vector. In general a vector has a direction and a magnitude; if we are only interested in directions, we can just consider vectors with magnitude equal to $1$, i.e. unit vectors.

Thus, given a unit vector $\mathbf{u}$ and a point $\mathbf{x}\in {\mathbb{R}}^n$, the point $\mathbf{x}+h\mathbf{u}$ is the point reached by starting at $\mathbf{x}$ and traveling a distance $h$ in the direction $\mathbf{u}$. So $f(\mathbf{x}+h\mathbf{u})-f(\mathbf{x})$ represents the change in $f$ if we start at $\mathbf{x}$ and move a distance $h$ in the direction $\mathbf{u}$. This motivates the following definition:

Based on our knowledge of first-year calculus, we can see that ${\partial }_{\mathbf{u}}f(\mathbf{x})$ represents the instantaneous rate of change of $f$ as we move in the direction $\mathbf{u}$ through the point $\mathbf{x}$.

By comparing the definitions of directional derivative and partial derivative, we see that for any $j\in \{1,\dots ,n\}$, $$\frac{\partial f}{\partial x_j}={\partial }_{{\mathbf{e}}_j}f$$ where ${\mathbf{e}}_j$ denotes the unit vector in the $j$th coordinate direction.

The proof of this is almost exactly like the proof of Theorem 2. This is not surprising, since partial derivatives are just a special case of directional derivatives.

The proof does not require $\mathbf{u}$ to be a unit vector; this is only needed for the interpretation of ${\partial }_{\mathbf{u}}f$ as a directional derivative. For any vector $\mathbf{v}$, whether or not it is a unit vector, it is generally true that if $f$ is differentiable at a point $\mathbf{x}$, then $$\begin{array}{}\text{(13)}& \underset{h\to 0}{lim}\frac{f(\mathbf{x}+h\mathbf{v})-f(\mathbf{x})}{h}=\mathbf{v}\cdot \mathrm{\nabla }f(\mathbf{x}).\end{array}$$ The proof again is essentially the same as that of Theorem 2. It is left as an exercise.

The converse of Theorem 4 is not true. One can find functions $f$ such that at some point $\mathbf{x}$, the directional derivative ${\partial }_{\mathbf{u}}f(\mathbf{x})$ exists for every unit vector $\mathbf{u}$, but $f$ is not differentiable at $\mathbf{x}$. See the exercises below for some examples.

Example 7.

Assume that $S$ is an open subset of ${\mathbb{R}}^n$ and that $f:S\to \mathbb{R}$ is differentiable. At a point $\mathbf{x}\in S$, determine the direction ${\mathbf{u}}^{\ast }$ in which $f$ is increasing most rapidly, in the sense that $${\partial }_{{\mathbf{u}}^{\ast }}f(\mathbf{x})=max\{{\partial }_{\mathbf{u}}f(\mathbf{x}):\mathbf{u}\text{ a unit vector}\}.$$

This example tells us what the gradient of a real-valued function means. If you remember only one thing from MAT 237, it should be this:

This is the principle that allows machine learning by gradient descent, determines seam carving for image resizing, and will play an important role in the rest of this course.

Example 8.

Let $f(x,y,z)=\frac{1}{2}{x}^2+\frac{1}{4}{y}^4+\frac{1}{6}{z}^6$. Find the direction in which $f$ has the greatest increase at the point $(1,1,1)$.

Example 9.

Let $f(x,y,z)=xy{z}^2$. Find the direction in which $f$ is decreasing most rapidly at the point $(1,1,1)$.

Problems

Basic

1. Determine all points where a function $f$ is differentiable, and determine $\mathrm{\nabla }f$ at those points.

Problems like this are normally solved by using Theorem 3 and properties of continuous functions which allow us to recognize partial derivatives as continuous. Examples might be

• $f(x,y)=xy\cos (y^2-2x)$ for $(x,y)\in {\mathbb{R}}^2$.
• $f(x,y,z)=x^2{e}^{y/z}$ for $(x,y,z)\in {\mathbb{R}}^3$ such that $z\ne 0$.
• $f(x,y,z)=|(x,y,z)|=\sqrt{x^2+y^2+z^2}$ for $(x,y,z)\in {\mathbb{R}}^3.$
• $f(x,y)=({\sin }^{2}x+{\sin }^{2}y{)}^{1/2}$ for $(x,y)\in {\mathbb{R}}^2$.

2. Given a function $f$, find the direction in which $f$ is increasing/decreasing most rapidly at the point $\mathbf{x}$. For example, consider any of the functions in the above question, at the point $(1,1)$ or $(1,1,1)$, depending on the dimension.

3. If $f$ is a complicated function like Example 4, determining whether $f$ is differentiable at a given point using the defintion rather than using Theorem 3 is difficult. But determining the directional derivatives at a point using their definition is not. For example

   • Let $$f(x,y)=\{\begin{array}{ll}\frac{x^{2}y}{x^2+y^2}& \text{ if }(x,y)\ne (0,0)\\ 0& \text{ if }(x,y)=(0,0).\end{array}$$ For the unit vector $\mathbf{u}=\frac{1}{\sqrt{2}}(1,-1)$, determine ${\partial }_{\mathbf{u}}f(\mathbf{0})$.
   • Let $$f(x,y,z)=\{\begin{array}{ll}\frac{xyz}{x^2+y^2+z^2}& \text{ if }(x,y,z)\ne (0,0,0)\\ 0& \text{ if }(x,y,z)=(0,0,0).\end{array}$$ For the unit vector $\mathbf{u}=\frac{1}{3}(1,2,-2)$, determine ${\partial }_{\mathbf{u}}f(\mathbf{0})$.

Advanced

4. Let $$f(x,y)=\{\begin{array}{ll}\frac{x^{2}y}{x^2+y^2}& \text{ if }(x,y)\ne (0,0)\\ 0& \text{ if }(x,y)=(0,0).\end{array}$$

   • For any unit vector $\mathbf{u}=(u_1,u_2)$, determine ${\partial }_{\mathbf{u}}f(\mathbf{0})$.

   • Prove that $f$ is not differentiable at the origin.

     Hint

     If $f$ were differentiable at the origin, then necessarily ${\partial }_{\mathbf{u}}f(\mathbf{0})=u_1{\partial }_{1}f(\mathbf{0})+u_2{\partial }_{2}f(\mathbf{0})$.

5. Define $f:{\mathbb{R}}^2\to \mathbb{R}$ by $$f(x,y)=\{\begin{array}{ll}\frac{x^{3}y}{x^4+y^2}& \text{ if }(x,y)\ne (0,0)\\ 0& \text{ if }(x,y)=(0,0)\end{array}$$

• Is $f$ continuous at $(0,0)$? Determine the answer using material from Section 1.2.

• Show that for every unit vector $\mathbf{u}=(u_1,u_2)$, the directional derivative ${\partial }_{\mathbf{u}}f(0,0)$ exists and equals zero.

• Prove that $f$ is not differentiable at $(0,0)$.

  Hint

  Consider ${\displaystyle \frac{f(t,t^2)-f(0,0)}{|(t,t^2)|}}$ for small values of $t$. Also, note that if $|t|<1$, then $|t|\le |(t,t^2)|\le \sqrt{2}|t|$.

6. Consider the function $f:{\mathbb{R}}^2\to \mathbb{R}$ defined by $$f(x,y)=\{\begin{array}{ll}\frac{y^3-x^{8}y}{x^6+y^2}& \text{ if }(x,y)\ne (0,0)\\ 0& \text{ if }(x,y)=(0,0).\end{array}$$ Prove that
   • all partial derivatives of $f$ exist everywhere in ${\mathbb{R}}^2$,
   • at least one partial derivative of $f$ is not continuous at $(0,0)$,
   • $f$ is differentiable at $(0,0)$.
     Hint

     See Example 4.
7. A function $f:{\mathbb{R}}^n\to \mathbb{R}$ is said to be homogeneous of degree $d$ if $$f(\lambda \mathbf{x})={\lambda }^{d}f(\mathbf{x})\text{ for every nonzero }\mathbf{x}\in {\mathbb{R}}^n\text{ and every }\lambda >0.$$ The same definition holds if the domain of $f$ is ${\mathbb{R}}^n\setminus \{\mathbf{0}\}$.

Prove that if $f$ is differentiable away from the origin and homogeneous of degree $d$, then for every unit vector $\mathbf{u}$, the directional derivative ${\partial }_{\mathbf{u}}f$ is homogeneous of degree $d-1$.

In particular this imples that all partial derivatives are homogeneous of degree $d-1$, since partial derivatives are a special case of directional derivatives.

8. Give a detailed proof of $\text{(13)}$, and hence of Theorem 4 (by small modifications of the proof of Theorem 2.)

9. Suppose that $S$ is an open subset of ${\mathbb{R}}^n$ that contains the origin, and that $g$ is a function $S\to \mathbb{R}$. Prove that if $\underset{\mathbf{x}\to \mathbf{0}}{lim}g(\mathbf{x})=L,$ then ${\displaystyle \underset{h\to 0}{lim}g(h\mathbf{u})=L}$ for every unit vector $\mathbf{u}$.

This was used in the proof of Theorem 2. It is good practice to write out the proof. You should be able to supply all relevant definitions from memory. This would be considered an easy proof question for a test.

10. Suppose that $a<0<b$, and that $g:(a,b)\to \mathbb{R}$. Prove that $$\underset{h\to 0}{lim}\frac{g(h)}{|h|}=0⟺\underset{h\to 0}{lim}\frac{g(h)}{h}=0.$$ This was also used in the proof of Theorem 2.

$\Leftarrow$  $\Uparrow$  $\Rightarrow$

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.0 Canada License.