Alexis Clairaut (France, 1713-1765) was taught mathematics by his father at a young age, and later became interested in astronomy and geophysics. He took part in a trip to the arctic circle to show the Earth was a flattened sphere (an oblate spheroid) and wrote a book explaining the mathematical arguments, “Théorie de la figure de la Terre” (“The Theory of the Shape of The Earth”).
For notational simplicity, we will prove this for a function of \(2\) variables. This implies the general case, since when we compute \(\frac{\partial^2 f}{\partial x_i \partial x_j}\) or \(\frac{\partial^2 f}{\partial x_j \partial x_i}\) at a particular point, all the variables except \(x_i\) and \(x_j\) are “frozen”, so that \(f\) can be considered (for that computation) as a function of \(x_i\) and \(x_j\) alone.
Given \(\mathbf x = (x,y) \in S\subseteq\R^2\), we will show that \[\begin{equation}\label{clair} \lim_{h\to 0} \frac 1 {h^2} \big[ f(x+h, y+h) -f(x, y+h) - f(x+h, y) +f(x, y) \big] = \frac{\partial^2 f}{\partial x \partial y} (x,y) \end{equation}\] and that the limit also equals \(\frac{\partial^2 f}{\partial y \partial x} (x,y)\). This will both prove the theorem and tell us what mixed second-order partial derivatives mean.
Let \(h\in \R\) such that \(B(\mathbf x;2h)\subseteq S\), and let \[ \phi(s) = f(x+h, s) -f(x, s). \] Then, using the \(1\)-dimensional Mean Value Theorem twice, \[\begin{align*} f(x+h, y+h) -&f(x, y+h) - f(x+h, y) +f(x, y) & \\ &= \phi(y+h) - \phi(y)& \\ &= h\phi'(y+\theta_1 h) \end{align*}\] for some \(\theta_1\in (0,1)\), by the single variable MVT. Then \[\begin{align*} h\phi'(y+\theta_1 h)&= h \left[ \frac{\partial f}{\partial y}(x+h, y+\theta_1 h) - \frac{\partial f}{\partial y}(x, y+\theta_1 h) \right] \\ &= h \left[ h\left( \frac{\partial }{\partial x} (\frac{\partial f}{\partial y}(x+\theta_2 h, y+\theta_1 h)\right) \right] \end{align*}\]for some \(\theta_2\in (0,1)\), again applying the single variable MVT. Finally, this simplifies to \[ h^2 \frac{\partial^2 f}{\partial x \partial y}(x+\theta_2 h, y+\theta_1 h). \] Since this holds for all sufficiently small \(h\) (with \(\theta_1\) and \(\theta_2\) depending on \(h\), but always in the interval \((0,1)\)) and \((x+\theta_2 h, y+\theta_1 h)\to (x,y)\) as \(h\to 0\) and \(\frac{\partial^2 f}{\partial x \partial y}\) is continuous, this implies \(\eqref{clair}\). On the other hand, by going through the same argument, but with the roles of the \(x\) and \(y\) reversed, we find that the limit in \(\eqref{clair}\) also equals \(\frac{\partial^2 f}{\partial y \partial x} (x,y)\).Note: It can happen that a function \(f(x,y)\) has second-order partial derivatives at every point, but that at some points, \(\frac{\partial^2 f}{\partial x \, \partial y} \ne \frac{\partial^2 f}{\partial y \, \partial x}\).
We will not worry about this much, because in this class we will mostly encounter functions of class \(C^2\). If you are interested in an example, consider \[ f(x,y) = \begin{cases} \frac{x^3y}{x^2+y^2}&\text{ if }(x,y)\ne (0,0)\\ 0&\text{ if }(x,y)=(0,0) . \end{cases} \]Higher-order partial derivatives
In general, we can keep on differentiating partial derivatives as long as successive partial derivatives continue to exist. We define the classes of functions that have continuous higher order partial derivatives inductively.
If \(f\) is \(C^k\), then when computing partial derivatives up to order \(k\), we do not need to worry about the order in which the partial derivatives are computed.
It suffices to show that \[\begin{equation}\label{rearr} \frac{\partial}{\partial x_{i_k} }\cdots \frac{\partial}{\partial x_{i_1} }f \ = \ \left(\frac{\partial}{\partial x_n }\right)^{\alpha_n}\cdots \left(\frac{\partial}{\partial x_1 }\right)^{\alpha_1}f \end{equation}\] where \(\alpha_j\ge 0\) denotes the number of times that \(\frac\partial{\partial x_j}\) appears on the right-hand side of \(\eqref{rearr}\), for \(j=1,\ldots, n\). Thus \(\alpha_1+\ldots + \alpha_n = k\). In other words, we aim to show that we can rearrange the indices \(i_1,\ldots, i_k\) in increasing order, so that the rearranged indices \(j_1,\ldots, j_k\) satisfy \(j_1\le j_2\le \cdots \le j_k\).
We prove \(\eqref{rearr}\) by induction. For \(k=2\), it follows from Theorem 1. Suppose it holds for \(k-1\), and consider \[\frac{\partial}{\partial x_{i_k} }\frac{\partial}{\partial x_{i_{k-1}} }\cdots \frac{\partial}{\partial x_{i_1} }f.\] We can reorder \(\dfrac{\partial}{\partial x_{i_{k-1}} }\cdots \dfrac{\partial}{\partial x_{i_1} }f\) so that the indices increase (right-to-left) by the inductive hypothesis. Then apply to inductive hypothesis to \(\dfrac{\partial}{\partial x_{i_1}}f\) to reorder the indices \(i_2, \ldots, i_k\). Now the sequence will be in order, except possibly the two terms closest to \(f\). By Clairaut’s Theorem, we can switch them. So the right and left sides of \(\eqref{rearr}\) are equal.Example 1.
If \(f:\R^3\to \R\) is \(C^5\), then \[ \frac{\partial}{\partial x } \frac{\partial}{\partial y } \frac{\partial}{\partial z } \frac{\partial}{\partial y } \frac{\partial}{\partial x }f \ = \ \frac{\partial^2}{\partial x^2 } \frac{\partial^2}{\partial y^2 } \frac{\partial}{\partial z } f \ = \ \frac{\partial}{\partial z } \frac{\partial^2}{\partial y^2 } \frac{\partial^2}{\partial x^2 } f. \]
Example 2.
Let \(f(x,y,z) = x^2 \arctan( y^2+z \cos(e^{\sin (yz)}))\). Compute \[ \frac{\partial^3}{\partial x^3 } \frac{\partial^{17}}{\partial y^{17} } \frac{\partial^{41}}{\partial z^{41} } f(x,y,z) \]Solution
Note that \(f\) is a composition of sums and products of infinitely differentiable functions, so \(f\) is \(C^k\) on all of \(\R^3\), for every \(k\). We have \[\frac{\partial^2 f}{\partial x^2}=2\arctan( y^2+z \cos(e^{\sin (yz)})),\] so \(\dfrac{\partial^3 f}{\partial x^3}=0\). Theorem 2 allows us to compute \[ \frac{\partial^3}{\partial x^3 } \frac{\partial^{17}}{\partial y^{17} } \frac{\partial^{41}}{\partial z^{41} } f(x,y,z) = \frac{\partial^{17}}{\partial y^{17} } \frac{\partial^{41} }{\partial z^{41} } \frac{\partial^3}{\partial x^3 } f(x,y,z) = 0. \]Remark. Just like in the case of second-order partial derivatives, the order might matter if \(f\) is not of class \(C^k\), that is, if the \(k\)th order derivatives exist but aren’t all continuous.
Chain Rule with Higher Derivatives
Suppose that \(f:\R^n\to \R\) and \(\mathbf g:\R^m\to \R^n\) are functions of class \(C^2\), and consider the composite function \(\phi = f\circ \mathbf g:\R^m\to \R\).
The chain rule implies that \(\phi\) is \(C^2\). We can write all second partial derivatives of \(\phi\) in terms of first and second partial derivatives of \(f\) and \(\mathbf g\), but it is easy to make notational mistakes, so one has to be careful.
Example 3.
Suppose that \(f:\R^3\to\R\) and \(\mathbf g:\R^2\to \R^3\) are both \(C^2\). Compute \(\frac{\partial^2\phi}{\partial x^2}\), for \(\phi = f\circ \mathbf g\).
Solution
We start by writing down what the chain rule says about \(\partial_x (w\circ \mathbf g)\), where \(w:\R^3\to \R\) is \(C^1\): \[\begin{equation}\label{wd}
\partial_x (w\circ \mathbf g) =
\partial_1 w \ \partial_x g_1
+
\partial_2 w \ \partial_x g_2
+
\partial_3 w \ \partial_x g_3,
\end{equation}\] where partial derivatives of \(w\) are evaluated at \(\mathbf g(\mathbf x)\). That is, \(\partial_j w\) really means \((\partial_j w)\circ \mathbf g\). Also, \(g_1,g_2, g_3\) denote the components of \(\mathbf g\), as usual.
To compute a second derivative of \(\phi\), we must start by differentiating once. We use the general rule \(\eqref{wd}\), with \(w\) replaced by \(f\). We will carefully keep track of which functions are composite functions, since we need to pay attention to this when we differentiate a second time. This gives \[\begin{equation}\label{phi1} \partial_x \phi = \partial_x (f\circ \mathbf g) = ((\partial_1 f)\circ\mathbf g) \ \partial_x g_1 + ((\partial_2 f)\circ\mathbf g) \ \partial_x g_2 + ((\partial_3 f)\circ\mathbf g) \ \partial_x g_3. \end{equation}\] Now we have to differentiate again. The product rule gives us
\[\begin{align*} \partial_x\partial_x \phi &=(\partial_x\partial_x g_1) (\partial_1 f) + (\partial_x g_1)\left[\partial_x \left(\partial_1 f\right)\right] \\ &+(\partial_x\partial_x g_2) (\partial_2 f) + (\partial_x g_2)\left[\partial_x \left(\partial_2 f\right)\right] \\ &+(\partial_x\partial_x g_3) (\partial_3 f) + (\partial_x g_3)\left[\partial_x \left(\partial_3 f\right)\right] \end{align*}\]
Applying \(\partial_x\) to a function of \(f\) requires the chain rule, so the bracketed terms can be (similarly) expanded to \[ \left[\partial_x \left(\partial_1 f\right)\right]=(\partial_x g_1) (\partial_1 \partial_1 f) + (\partial_x g_2) (\partial_2 \partial_1 f) + (\partial_x g_3) (\partial_3 \partial_1 f) \] Combining these and condensing them with sigma notation, we get \[ \partial_x\partial_x \phi = \sum_{i=1}^3\sum_{j=1}^3 \partial_i\partial_j f \ \partial_x g_i \, \partial_x g_j + \sum_{i=1}^3 \partial_i f \, \partial_x \partial_x g_i, \] where first and second derivatives of \(f\) are evaluated at \(\mathbf g(\mathbf x)\).Although we have called it an example, this is close to the general case. Note that we used different variables for the domains of \(g\) and \(f\): \((x,y)\) and \((x_1, x_2, x_3)\). This is a good practice to keep track of where chain rule needs to be applied. When we took \(\partial_x (\partial_1 f)\), we had to consider the hidden \(\mathbf g(x,y)\) in the function \(\partial_1 f\), because \(x\) is not a variable for \(f\). You can use the variety of notations for vectors to your advantage here to provide reminders.
Remark. In carrying out computations like this, we often write \(\partial_j f\) instead of \((\partial_j f)\circ \mathbf g\) in formulas like \(\eqref{phi1}\). This is okay, as long as
- you remember that \(\partial_j f\) really means \(\partial_j f\circ \mathbf g\),
- you correctly take this into account when differentiating a second time, as we did, and
- for any potential reader, you also write something like “where derivatives of \(f\) are evaluated at \(\mathbf g(\mathbf x)\).”
Example 4: Polar Coordinates.
Suppose that \(f:\R^2\to \R\) is a \(C^2\) function, and define \(\mathbf g: \R^2\to \R^2\) by \[ g(r,\theta) = (r\cos \theta, r\sin \theta) \] Let \(\phi = f\circ \mathbf g\), so that \(\phi(r,\theta)= f(r\cos\theta, r\sin\theta)\). Compute \(\partial_r\partial_\theta \phi\) in terms of \(r,\theta\), and partial derivatives of \(f\).Solution
In general, if \(w(x,y)\) is a \(C^2\) function of 2 variables, then \[\begin{align} \partial_r (w\circ \mathbf g) &= \partial_x w \, \cos\theta + \partial_y w \ \sin \theta \label{pr} \\ \partial_\theta (w\circ \mathbf g) &= -\partial_x w \, r\sin\theta + \partial_y w \ r\cos \theta \label{ptheta} \end{align}\] where derivatives of \(w\) are evaluated at \((r\cos\theta, r\sin \theta)\). Thus \(\eqref{pr}\) implies that \[ \partial_r \phi = \partial_x f \, \cos\theta + \partial_y f \ \sin \theta \] where derivatives of \(f\) are evaluated at \((r\cos\theta, r\sin \theta)\). Now we differentiate with respect to \(\theta\). We have to use \(\eqref{ptheta}\) twice (with \(w\) replaced by \(\partial_x f\) and \(\partial_y f\)) when differentiating derivatives of \(f\), since they are composed with \(\mathbf g\). Doing this, and assembling the results, we end up with \[ \partial_\theta\partial_r \phi = (-\partial_x \partial_x f \ + \ \partial_y \partial_y f) r\sin \theta \, \cos \theta \ +\partial_x \partial_y f (r\cos^2\theta - r\sin^2\theta) -\partial_x f \sin\theta + \partial_y f \cos\theta, \] where derivatives of \(f\) are evaluated at \((r\cos\theta,r\sin \theta)\).%This section may be unnecessary.
Multi-index notation for higher derivatives.
If \(f\) is a \(C^k\) function of \(n\) variables, then a \(k\)th-order partial derivative of \(f\) can always be written in the form \[ \left(\frac{\partial}{\partial x_1 }\right)^{\alpha_1}\left(\frac{\partial}{\partial x_{2} }\right)^{\alpha_{2}}\cdots \left(\frac{\partial}{\partial x_n }\right)^{\alpha_n}f \]where \(\alpha_j\ge 0\) for every \(j\), and \(\alpha_1+\ldots+\alpha_n = k\).
It is sometimes convenient to introduce the more concise notation \[ \partial^\alpha f =\left(\frac{\partial}{\partial x_1 }\right)^{\alpha_1}\left(\frac{\partial}{\partial x_{2} }\right)^{\alpha_{2}}\cdots \left(\frac{\partial}{\partial x_n }\right)^{\alpha_n}f, \]where \(\alpha = (\alpha_1,\ldots, \alpha_n)\) with every \(\alpha_j\) a nonnegative integer. This is called a multi-index. For a multi-index \(\alpha\), the sum \(\alpha_1+\ldots +\alpha_n\) is called the order of \(\alpha\). Thus, if \(\alpha\) is a multi-index of order \(k\), then \(\partial^\alpha f\) is a partial derivative of order \(k\).
The order of a multi-index is sometimes denoted \(|\alpha|\). Note that this is not the same as the Euclidean norm.
Example 5.
For \(f(x,y,z) = x^7 e^{yz}\), compute \(\partial^\alpha f\) for \(\alpha = (4,1,5)\).
Solution
This is another way of asking us to compute \(\frac{\partial^4}{\partial x^4}\frac{\partial}{\partial y}\frac{\partial^5}{\partial z^5}f\). We can differentiate in any order we please. We have \[ \frac{\partial^5}{\partial z^5}f = x^7 y^5e^{yz}, \] So \[ \frac{\partial^4} {\partial x^4} \frac{\partial^5} {\partial z^5} f = 7\cdot6\cdot5\cdot4\cdot x^3 y^5 e^{yz} = 840 x^3 y^5e^{yz}. \] Then finally \[ \partial^\alpha f = \frac{\partial}{\partial y}\frac{\partial^4}{\partial x^4}\frac{\partial^5}{\partial z^5}f = 840x^3( 5y^4 e^{yz} + y^5 ze^{yz}) = 840x^3y^4e^{yz}(5+yz). \]The Hessian matrix
If we write \(H(\mathbf a)\), it means that all of the entries of \(H\) are evaluated at the point \(\mathbf a\), which must be a point in the domain of \(f\): \[ H(\mathbf a) = Hf(\mathbf a)= \left(\begin{array}{ccc} \partial_1\partial_1 f(\mathbf a) & \cdots & \partial_1\partial_n f(\mathbf a) \\ \vdots & \ddots & \vdots \\ \partial_n\partial_1 f(\mathbf a) & \cdots & \partial_n\partial_n f(\mathbf a) \end{array} \right). \]
Note that \(H\) is always a symmetric \(n\times n\) matrix. Recall that an \(n\times n\) matrix \(A\) is symmetric if \(A^T=A\), or equivalently if \(a_{ij}=a_{ji}\) for \(1\leq i,j \leq n\) where \(a_{ij}\) is the \((i,j)\) entry of \(A\). In the case of the Hessian matrix, the symmetry condition \(H_{ij}=H_{ji}\) becomes \(\partial_i\partial_j f=\partial_j\partial_i f\) which is true by Clairaut’s theorem, since the Hessian is only defined if \(f\) is \(C^2\).
Problems
Basic problems
Combining the chain rule and higher-order partial derivatives. Questions of this sort will appear on one or more tests.
Suppose that \(f:\R^2\to \R\) is a \(C^2\) function, and define \(\mathbf g: \R^2\to \R^2\) by \[ \mathbf g(s,t) = (st, \frac 12[s^2-t^2]) \] Let \(\phi = f\circ \mathbf g\), so that \(\phi(s,t)= f(st, \frac 12[s^2-t^2])\). Compute \(\partial_s\partial_s \phi\) in terms of \(s, t\), and derivatives of \(f\).
Suppose that \(f:\R^2\to \R\) is a \(C^2\) function, and define \(\mathbf g: \R^2\to \R^2\) by \[ \mathbf g(r,t) = (r \cosh t, r\sinh t) \] Let \(\phi = f\circ \mathbf g\), and compute \(\frac{\partial^2 \phi}{\partial r\, \partial t}\)in terms of \(r, t\), and derivatives of \(f\).
Recall that the hyberbolic trigonometric functions are \[ \cosh t = \frac 12(e^t+e^{-t}), \quad \sinh t = \frac 12(e^t-e^{-t}). \]
Suppose that \(f:\R^2\to \R\) is a \(C^2\) function, and define \(\phi:\{(r,s)\in \R^2 : s\ne 0\}\to \R\) by \(\phi = f(rs, \frac rs)\). Compute \(\frac{\partial^2 \phi}{\partial r^2}\) in terms of \(r,s\), and derivatives of \(f\).
Suppose that \(f:\R^2\to \R\) is a \(C^2\) function, and for \((x,y,z)\in \R^3\) with \(z\ne 0\), define \(\phi:\R^3\to \R\) by \(\phi = f(xz, \frac yz)\). Compute \(\frac{\partial^2 \phi}{\partial z^2}\) in terms of \(x,y,z\), and derivatives of \(f\).
Suppose that \(g:\R^n\to \R\) and \(f:\R\to \R\) are \(C^2\) functions, and let \(\phi = f\circ g:\R^n\to \R\). Express an arbitrary second-order partial derivative \(\partial_i\partial_j \phi\) in terms of derivatives of \(f\) and \(g\). Since \(f\) is a function of a single variable, you can write \(f'\) and \(f''\) to denote its first and second derivatives.
Suppose that \(f:\R^2\to \R\) is a \(C^2\) function. Fix \(\mathbf a\) and \(\mathbf h\) in \(\R^2\), and define \(\phi:\R\to \R\) by \[ \phi(s) = f(\mathbf a + s \mathbf h). \] Express \(\phi''(s)\) in terms of \(f, s,\mathbf a\), and \(\mathbf h\).
You can also consider the same question for \(f:\R^n\to \R\), with \(\mathbf a, \mathbf h\in \R^n\). The extra difficulties are mainly notational.
- Suppose that \(f:\R^2\to \R\) is a \(C^3\) function. Fix \(\mathbf a\) and \(\mathbf h\) in \(\R^2\), and define \(\phi:\R\to \R\) by \[ \phi(s) = f(\mathbf a + s \mathbf h). \] Express \(\phi'''(s)\) in terms of \(f, s,\mathbf a\), and \(\mathbf h\).
You also should be sufficiently familiar with multi-index notation to answer questions like the following:
For \(f(x,y,z) = e^{2x+3y+4z}\) and multi-index \(\alpha= (2,0,1)\), compute \(\partial^\alpha f\).
For \(f(x,y,z) = y^2\cos(x^2 z)\) and\(\alpha = (1,2,1)\), compute \(\partial^\alpha f\).
Use multi-index notation to express your answer to the question 7.
Advanced
Let \(f(x,y) =(x^4+x^2y^2)^{1/2}\). Compute all second-order partial derivatives of \(f\) at \((0,0)\), if they exist, and determine whether \(\partial^2 f/\partial x\, \partial y = \partial^2 f/\partial y\, \partial x\) at \((0,0)\).
Suppose that \(f:\R^n\to \R\) and \(\mathbf g = (g_1,\ldots, g_n):\R^m\to \R^n\) are both \(C^2\), and let \(\phi = f\circ \mathbf g\). Prove that for all \(i,j\in \{1,\ldots, m\}\), \[\begin{equation}\label{D2cr} \partial_i \partial_j \phi = \sum_{k=1}^n \sum_{l=1}^n(\partial_k \partial_l f) (\partial_i g_k)( \partial_j g_l) + \sum_{k=1}^n(\partial_k f) (\partial_i \partial_j g_k) \end{equation}\] where the first and second-order partial derivatives of \(f\) are evaluated at \(\mathbf g(\mathbf x)\).
In problem 6, you computed \(\phi''(s)\), where \(\phi(s) = f(\mathbf a+s\mathbf h)\). Express the answer using matrix-vector notation, using \(H\) for the Hessian matrix of \(f\).
\(\Leftarrow\) \(\Uparrow\) \(\Rightarrow\)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.0 Canada License.