\(\renewcommand{\R}{\mathbb R }\)
\(\Leftarrow\) \(\Uparrow\) \(\Rightarrow\)
Our first main theorem about compactness is the following:
Remark 1. Although “compact” is the same as “closed and bounded” for subsets of Euclidean space, it is not always true that “compact means closed and bounded.”
In practice, this only becomes important when you study more advanced mathematics. In \(\R^n\), it will always be true that compact is the same as closed and bounded.
Remark 2. We have called this the Bolzano-Weierstrass Theorem. That name is sometimes given to what we called the Bounded Sequence Theorem in the previous section. The two theorems are very closely related.
Below we will present the proof of Theorem 1. One ingredient in the proof is the following proposition, which we present separately, since it is a basic and useful fact.
Relying in part on Lemma 2, we can now give the
Compact \(\Leftarrow\) Closed and Bounded. Assume that \(S\) is closed and bounded, and let \(\{ \mathbf x_j\}_j\) be any sequence in \(S\). Since \(S\) is bounded, the Bounded Sequence Theorem implies that there is a convergent subsequence, say \(\{ \mathbf x_{k_j}\}_j\). Since \(S\) is closed, Proposition 1 above implies that the limit of this subsequence must belong to \(S\). Thus every sequence in \(S\) has a a subsequence that converges to a limit in \(S\). This is \(\Leftarrow\).
Note that the hardest part of the proof was contained in the Bounded Sequence Theorem from the previous section.
Compact \(\Rightarrow\) Closed and Bounded. (sketch) We will show the contrapositive: that if \(S\) is not closed and bounded, there exists a sequence in \(S\) with no converging subsequence.
If \(S\) is not closed and bounded, then it is either not closed or not bounded. We will consider these two cases separately:
Case 1: \(S\) is not bounded. This means that there does not exist any \(r>0\) such that \(S \subseteq B({\mathbf 0}; r)\).
How can we use this assumption to define a sequence \(\{ \mathbf x_j\}_j\) in \(S\) that does not converge? Figure out the idea, and fill in the details if you wish.
Notation. We often use the abbreviations \[ \inf_S f = \inf \{ f(\mathbf x) : \mathbf x \in S \}, \qquad \sup_S f = \sup \{ f(\mathbf x) : \mathbf x \in S \}. \]
We will discuss the proof below, after first seeing what the theorem is good for.
A major application of calculus is to optimization problems: figure out a way to do something that maximizes good outcomes, minimizes bad outcomes, and respects certain constraints.
The Extreme Value Theorem is useful because it can sometimes guarantee that an optimization problem must have a solution. Its weakness is that it does not give any indication how to find the solution, when it exists; but at least it can potentially spare us from wasting time and effort that we might otherwise spend looking for a nonexistent solution.
Consider the following two problems:
Design a rectangular box in a way that maximizes the volume of the box, while respecting the constraint that the length \(+\) width \(+\) height \(\le 3.\)
Design a rectangular box in a way that maximizes length \(+\) width \(+\) height, while respecting the constraint that the volume of the box \(\le 3.\)
By formulating these as problems of maximizing a continuous function in a set \(S\subseteq \R^3\), one can see that the Extreme Value Theorem guarantees that one of the problems has a solution. It does not tell us anything about the other problem, which does not have a solution.
You are asked to consider these in the Problem 3.
Consider the problem \[ \text{ minimize } f(x,y) = x^2+y^2 + \cos(x^3 + e^y) \text{ on }\R^2. \] Using our new notation, we could also write “find \((\mathbf x_*)\in \R^2\) such that \(f(\mathbf x_*) = \inf_{\R^2} f\).”
Does a solution exist?
Find out
Here the Extreme Value Theorem cannot be used immediately, since \(\R^2\) is not compact. To use it, we have to find a compact set that will contain the infimum.
The proof of this is an exercise in the practice problems .
We will also need
This may be familiar from MAT 137. The proof is outlined below.
You may also remember that when \(\sup S\) belongs to \(S\), then we say that the supremum is a maximum, and we have the right to refer to it as \(\max S\) instead of \(\sup S\), if we want to. Similarly with \(\inf S\) and \(\min S\), a minimum.
We will prove only that \(\sup S\in S\). The proof for \(\inf\) is similar.
First, since \(S\) is compact, \(S\) is bounded, so \(S\) has a real number supremum. Let’s call it \(x = \sup S\). We have to show that \(x \in S\). Since \(S\) is compact and hence closed, this is the same as proving that \(x \in \overline S\), which is the same as showing that \[\begin{equation}\label{p3} \forall \varepsilon >0, \quad (x-\varepsilon, x+\varepsilon)\cap S \ne \emptyset. \end{equation}\] Recall that in \(1\)-dimension, the “open ball” \(B(x; \varepsilon)\) is just the interval \((x-\varepsilon, x+\varepsilon)\). So our task is to prove \(\eqref{p3}\). But if \((x-\varepsilon, x]\cap S=\emptyset\), then \(x-\varepsilon\) is an upper bound for \(S\), which contradicts the assumption that \(x\) is the least upper bound.Using these lemmata, we can now present the
Suppose that \(K\) is compact and that \(f:K\to \R\) is continuous.
Let \(S = \{ f(\mathbf x) : \mathbf x\in K\}\subseteq\R\) be the image of \(K\) under \(f\). Our plan is to show that \(S\) is compact, then apply the Extreme Value Theorem for one variable to \(S\), and pull the results back to \(K\).
Proof that \(S\) is compact. To show that \(S\) is compact, we will prove that any sequence in \(S\) has a subsequence that converges to a limit in \(S\). Consider a sequence \(\{y_j\}_j\) in \(S\). By definition of \(S\), for each such point there is some \(\mathbf x_j\) in \(K\) such that \(y_j = f(\mathbf x_j)\). Because \(K\) is compact, there exists a subsequence \(\{ \mathbf x_{j_k}\}_j\) that converges to a limit \(\mathbf x\in K\). According to Lemma above, since \(f\) is continuous, \[ y_{j_k} = f(\mathbf x_{j_k}) \to f(\mathbf x) \quad\text{ as }k\to \infty. \] Since \(f(\mathbf x)\in S\), this proves that \(\{y_j\}_j\) has a subsequence that converges to a limit in \(S\). Since this argument applies to any sequence in \(S\), it follows that \(S\) is compact.
Proof that \(f\) attains its supremum. Since \(S\) is compact, it follows from Lemma that \(\sup S \in S\). Let \(y^* = \sup S = \sup\{ f(\mathbf x) : \mathbf x\in K\}\). Then the statement that \(y^*\in S\) says exactly that \(y^* = f(\mathbf x^*)\) for some \(\mathbf x^*\in K\). In other words, \(\sup_K f\) is attained at \(\mathbf x^*\in K\).
Proof that \(f\) attains its infimum. This is parallel to the proof that \(f\) attains its supremum.Remark. The first part of the proof of the Extreme Value Theorem can be easily modified to show that if \(K\) is a compact subset of \(\R^n\) and \(\mathbf f:K\to \R^k\) is continuous, then \(\mathbf f(K)=\{ \mathbf f(\mathbf x) : \mathbf x\in K\}\) is a compact subset of \(\R^k\). That is, the continuous image of a compact set is compact.
Give an example of
Determine whether a set \(S\subseteq \R^n\) is compact. For example:
Determine whether a sequence \(\{\mathbf x_j\}\) has a subsequence that converges to a limit in a set \(S\).
Does the Extreme Value Theorem guarantee that a given function with some constraints must achieve a minimum/maximum? For example:
Consider the problem of designing a rectangular box in a way that maximizes the volume while respecting the constraints length+width+height \(\le 3\). Does the Extreme Value Theorem guarantee that this problem must have a solution?
Consider the problem of designing a rectangular box in a way that maximizes length+width+height while respecting the constraint that the volume \(\le 3\). Does the Extreme Value Theorem guarantee that this problem must have a solution?
When the domain is not compact, can you modify the problem so that Extreme Value Theorem can be applied?
Let \(f(x,y) = x+y + e^{x^2-5x -y^2 -2y}\), and let \(S = \{(x,y)\in \R^2: x\ge 0, y\ge 0\}\). Prove that \(f\) attains its minimum in \(S\).
Let \(f(x,y,z ) = x^2 + y^2 + z^2 +\dfrac{\sin (xy)}{1+z^2}\). Prove that there is a point in \(\R^3\) that minimizes \(f\).
Hint
See Example 3 for a model of how to address this type of problem.
The previous question has many variants that have appeared on previous MAT 237 tests.
Prove that a closed subset of a compact set in \(\R^n\) is compact.
Give a different proof of Proposition 1, by showing that if \(S\) is a closed subset of \(\R^n\), and \(\{ \mathbf x_j\}_j\) is any sequence in \(\R^n\) that converges to a limit \(\mathbf x \not\in S\), then \(\{\mathbf x_j\}_j\) is not a sequence in \(S\) (in other words, that \(\mathbf x_j\not \in S\) for some \(j\).)
Hint
\(\mathbf x\not\in S\) means that \(\mathbf x\in S^c\). If \(S\) is closed, what do we know about \(S^c\)?
Fill in the details in the proof of Theorem 1 that in any set \(S\) that is not closed, there exists a sequence with no subsequence that converges to a limit in \(S\).
Fill in the details in the proof of Theorem 1 that in any unbounded set \(S\), there exists a sequence with no convergent subsequence.
Fill in the last missing details in the proof of Proposition 3.
Assume that \(K_1\) is a compact subset of \(\R^n\) and \(K_2\) is a compact subset of \(\R^m\). Let \[ K_1\times K_2 = \{(\mathbf x, \mathbf y)\in \R^n\times \R^m = \R^{n+m} : \mathbf x \in K_1, \mathbf y\in K_2 \}. \]Prove that \(K_1\times K_2\) is a compact subset of \(\R^{m+n}\).
(In short, prove that a Cartesian Product of two compact sets is compact.) There are at least two different possible proofs, using two different characterizations of compact sets.
Guide for proving Theorem 6.
\(\Leftarrow\) \(\Uparrow\) \(\Rightarrow\)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.0 Canada License.