1.4: Compactness and Applications

\(\renewcommand{\R}{\mathbb R }\)

1.4: Compactness and Applications

  1. Compactness
  2. The Extreme Value Theorem
  3. Uniformly continuous functions
  4. Problems

\(\Leftarrow\)  \(\Uparrow\)  \(\Rightarrow\)

Compactness

A set \(S\subseteq \R^n\) is said to be compact if every sequence in \(S\) has a subsequence that converges to a limit in \(S\).
A technical remark, safe to ignore. In more advanced mathematics courses, what we have defined above is called sequentially compact, and the word compact is reserved for something a little different.

Our first main theorem about compactness is the following:

A set \(S \subseteq \R^n\) is compact \(~\iff~\) \(S\) is closed and bounded.

Remark 1. Although “compact” is the same as “closed and bounded” for subsets of Euclidean space, it is not always true that “compact means closed and bounded.”

How can this be? There are vast realms of mathematics, none of which we will discuss in this class, that take place in settings more general and much “bigger” than finite-dimensional Euclidean space.

In practice, this only becomes important when you study more advanced mathematics. In \(\R^n\), it will always be true that compact is the same as closed and bounded.

Remark 2. We have called this the Bolzano-Weierstrass Theorem. That name is sometimes given to what we called the Bounded Sequence Theorem in the previous section. The two theorems are very closely related.

Below we will present the proof of Theorem 1. One ingredient in the proof is the following proposition, which we present separately, since it is a basic and useful fact.

If \(\{ \mathbf x_j\}_j\) is a convergent sequence in a \(closed\) set \(S\subseteq \R^n\), then the limit of the sequence must belong to \(S\).
This is a straightforward deduction from the definitions, try to prove it yourself before reading the details.
Details
Let \(\{ \mathbf x_j\}_j\) be a sequence in a closed set \(S\subseteq \R^n\), and assume that \(\mathbf x_j\to \mathbf x\) as \(j\to \infty\). We will show that \(\mathbf x \in S\). Since \(S\) is closed, this is the same as proving that \(\mathbf x \in \overline S\), which is the same as showing that \[\begin{equation}\label{l1} \forall \varepsilon >0, \quad B(\mathbf x; \varepsilon) \cap S \ne \emptyset. \end{equation}\] To do this, note that by the definition of limit, and the convergence of the sequence, we know that \[ \forall \varepsilon >0, \quad \exists J>0\text{ such that }\mathbf x_{j}\in B(\mathbf x; \varepsilon) \quad\text{ for all }j\ge J. \] Since \(\mathbf x_{_j} \in S\), this implies that \(\eqref{l1}\) holds.

Relying in part on Lemma 2, we can now give the

Details of Theorem 1 (with some details left to you).

Compact \(\Leftarrow\) Closed and Bounded. Assume that \(S\) is closed and bounded, and let \(\{ \mathbf x_j\}_j\) be any sequence in \(S\). Since \(S\) is bounded, the Bounded Sequence Theorem implies that there is a convergent subsequence, say \(\{ \mathbf x_{k_j}\}_j\). Since \(S\) is closed, Proposition 1 above implies that the limit of this subsequence must belong to \(S\). Thus every sequence in \(S\) has a a subsequence that converges to a limit in \(S\). This is \(\Leftarrow\).

Note that the hardest part of the proof was contained in the Bounded Sequence Theorem from the previous section.

Compact \(\Rightarrow\) Closed and Bounded. (sketch) We will show the contrapositive: that if \(S\) is not closed and bounded, there exists a sequence in \(S\) with no converging subsequence.

If \(S\) is not closed and bounded, then it is either not closed or not bounded. We will consider these two cases separately:

Case 1: \(S\) is not bounded. This means that there does not exist any \(r>0\) such that \(S \subseteq B({\mathbf 0}; r)\).

How can we use this assumption to define a sequence \(\{ \mathbf x_j\}_j\) in \(S\) that does not converge? Figure out the idea, and fill in the details if you wish.

HintYour argument could look like this: “For each \(j\), let \(\mathbf x_j\) be a point in \(S\) such that …. We know that such a point exists because …. This procedure for choosing points \(\mathbf x_j\) defines a sequence \(\{ \mathbf x_j\}_j\). The sequence cannot have a limit because the way that we chose it implies that …”
Case 2: \(S\) is not closed. This means (see Theorem 4 in Section 1.1) that there exists \(\mathbf x\in \partial S\) such that \(\mathbf x \not\in S\). The definition of boundary then implies for every \(\varepsilon>0\) \[ B(\mathbf x; \varepsilon) \cap S \ne \emptyset. \] Since \(\mathbf x\not\in S\), it follows that \[ \forall \varepsilon >0, \exists \mathbf y\in S\text{ such that }\mathbf y\ne\mathbf x \text{ and }|\mathbf y - \mathbf x|<\varepsilon. \] The idea is to use this assumption todefine a sequence \(\{\mathbf y_j\}\) in \(S\) that converges to \(\mathbf x\). Every subsequence of this sequence will also converge to \(\mathbf x\). Thus no subsequence can converge to a limit in \(S\), since \(\mathbf x \not\in S\). Try to figure out how to choose the sequence; then if you wish, you can write down the details, which should look like the missing details from Case 1 above.

The Extreme Value Theorem

Suppose that \(K\) is a compact subset of \(\R^n\), and that \(f:K\to \R\) is continuous. Then the set \(f(K)=\{ f(\mathbf x) : \mathbf x \in K \}\) is compact, and there exists \(\mathbf x^*\) and \(\mathbf x_*\) in \(K\) such that \[\begin{equation}\label{evt1} f(\mathbf x^*) = \sup \{ f(\mathbf x) : \mathbf x \in K \}, \qquad f(\mathbf x_*) = \inf \{ f(\mathbf x) : \mathbf x \in K \}. \end{equation}\]
When there exists \(\mathbf x^*\) as in \(\eqref{evt1}\), we say that \(f\) attains its supremum, and when there exists \(\mathbf x_*\), we say that \(f\) attains its infimum.

Notation. We often use the abbreviations \[ \inf_S f = \inf \{ f(\mathbf x) : \mathbf x \in S \}, \qquad \sup_S f = \sup \{ f(\mathbf x) : \mathbf x \in S \}. \]

We will discuss the proof below, after first seeing what the theorem is good for.

Applications of the Extreme Value Theorem

A major application of calculus is to optimization problems: figure out a way to do something that maximizes good outcomes, minimizes bad outcomes, and respects certain constraints.

The Extreme Value Theorem is useful because it can sometimes guarantee that an optimization problem must have a solution. Its weakness is that it does not give any indication how to find the solution, when it exists; but at least it can potentially spare us from wasting time and effort that we might otherwise spend looking for a nonexistent solution.

Consider the following two problems:

Example 1.

Design a rectangular box in a way that maximizes the volume of the box, while respecting the constraint that the length \(+\) width \(+\) height \(\le 3.\)

Example 2.

Design a rectangular box in a way that maximizes length \(+\) width \(+\) height, while respecting the constraint that the volume of the box \(\le 3.\)

By formulating these as problems of maximizing a continuous function in a set \(S\subseteq \R^3\), one can see that the Extreme Value Theorem guarantees that one of the problems has a solution. It does not tell us anything about the other problem, which does not have a solution.

You are asked to consider these in the Problem 3.

Example 3.

Consider the problem \[ \text{ minimize } f(x,y) = x^2+y^2 + \cos(x^3 + e^y) \text{ on }\R^2. \] Using our new notation, we could also write “find \((\mathbf x_*)\in \R^2\) such that \(f(\mathbf x_*) = \inf_{\R^2} f\).”

Does a solution exist?

Find outHere the Extreme Value Theorem cannot be used immediately, since \(\R^2\) is not compact. To use it, we have to find a compact set that will contain the infimum.

We see that \(f((0,0)) = \cos 1 < 1\), while if \(x^2+y^2 >2\), then \[f(x,y) > 2 + \cos(x^3+e^y) \ge 1.\] Since we are trying to minimize \(f\), we can safely ignore all points where \(x^2+y^2 > 2\). In other words, for this particular function \(f\), \[ \inf_{\R^2} f = \inf_K f \quad\text{ where }K = \{ (x,y) : x^2+y^2\le 2\}. \] Now \(K\) is compact: it is bounded, since it is contained in \(B(\mathbf 0; 1.42)\), and closed, since it is defined by nonstrict equalities. The function \(f\) is continuous, because it is built from elementary functions, so the Extreme Value Theorem guarantees that there exists \((x_*, y_*)\in K\) such that \(f(x_*, y_*) = \inf_{K} f\). Since \(\inf_{\R^2} f = \inf_K f\), this shows that \((x_*, y_*)\) solves the minimization problem.

Proof of the Extreme Value Theorem

Assume that \(\{ {\mathbf z}_j\}_j\) is a sequence in a set \(S\subseteq \R^k\) for some \(k\ge 1\), and that \({\mathbf z}_j \to {\mathbf z}\in S\) as \(j\to \infty\). Assume also that \(f\) is a continuous function on \(S\). Then \(f({\mathbf z}_j ) \to f({\mathbf z})\) as \(j\to \infty\).

The proof of this is an exercise in the practice problems .

We will also need

If \(S\) is a compact subset of \(\R\), then \(\sup S \in S\) and \(\inf S\in S\).

This may be familiar from MAT 137. The proof is outlined below.

You may also remember that when \(\sup S\) belongs to \(S\), then we say that the supremum is a maximum, and we have the right to refer to it as \(\max S\) instead of \(\sup S\), if we want to. Similarly with \(\inf S\) and \(\min S\), a minimum.

Details

We will prove only that \(\sup S\in S\). The proof for \(\inf\) is similar.

First, since \(S\) is compact, \(S\) is bounded, so \(S\) has a real number supremum. Let’s call it \(x = \sup S\). We have to show that \(x \in S\). Since \(S\) is compact and hence closed, this is the same as proving that \(x \in \overline S\), which is the same as showing that \[\begin{equation}\label{p3} \forall \varepsilon >0, \quad (x-\varepsilon, x+\varepsilon)\cap S \ne \emptyset. \end{equation}\] Recall that in \(1\)-dimension, the “open ball” \(B(x; \varepsilon)\) is just the interval \((x-\varepsilon, x+\varepsilon)\). So our task is to prove \(\eqref{p3}\). But if \((x-\varepsilon, x]\cap S=\emptyset\), then \(x-\varepsilon\) is an upper bound for \(S\), which contradicts the assumption that \(x\) is the least upper bound.

Using these lemmata, we can now present the

Details of the Extreme Value Theorem.

Suppose that \(K\) is compact and that \(f:K\to \R\) is continuous.

Let \(S = \{ f(\mathbf x) : \mathbf x\in K\}\subseteq\R\) be the image of \(K\) under \(f\). Our plan is to show that \(S\) is compact, then apply the Extreme Value Theorem for one variable to \(S\), and pull the results back to \(K\).

Proof that \(S\) is compact. To show that \(S\) is compact, we will prove that any sequence in \(S\) has a subsequence that converges to a limit in \(S\). Consider a sequence \(\{y_j\}_j\) in \(S\). By definition of \(S\), for each such point there is some \(\mathbf x_j\) in \(K\) such that \(y_j = f(\mathbf x_j)\). Because \(K\) is compact, there exists a subsequence \(\{ \mathbf x_{j_k}\}_j\) that converges to a limit \(\mathbf x\in K\). According to Lemma above, since \(f\) is continuous, \[ y_{j_k} = f(\mathbf x_{j_k}) \to f(\mathbf x) \quad\text{ as }k\to \infty. \] Since \(f(\mathbf x)\in S\), this proves that \(\{y_j\}_j\) has a subsequence that converges to a limit in \(S\). Since this argument applies to any sequence in \(S\), it follows that \(S\) is compact.

Proof that \(f\) attains its supremum. Since \(S\) is compact, it follows from Lemma that \(\sup S \in S\). Let \(y^* = \sup S = \sup\{ f(\mathbf x) : \mathbf x\in K\}\). Then the statement that \(y^*\in S\) says exactly that \(y^* = f(\mathbf x^*)\) for some \(\mathbf x^*\in K\). In other words, \(\sup_K f\) is attained at \(\mathbf x^*\in K\).

Proof that \(f\) attains its infimum. This is parallel to the proof that \(f\) attains its supremum.

Remark. The first part of the proof of the Extreme Value Theorem can be easily modified to show that if \(K\) is a compact subset of \(\R^n\) and \(\mathbf f:K\to \R^k\) is continuous, then \(\mathbf f(K)=\{ \mathbf f(\mathbf x) : \mathbf x\in K\}\) is a compact subset of \(\R^k\). That is, the continuous image of a compact set is compact.

Problems

Basic

  1. Give an example of

  2. Determine whether a set \(S\subseteq \R^n\) is compact. For example:

  3. Determine whether a sequence \(\{\mathbf x_j\}\) has a subsequence that converges to a limit in a set \(S\).

  4. Does the Extreme Value Theorem guarantee that a given function with some constraints must achieve a minimum/maximum? For example:

  5. When the domain is not compact, can you modify the problem so that Extreme Value Theorem can be applied?

Advanced

  1. Prove that a closed subset of a compact set in \(\R^n\) is compact.

  2. Give a different proof of Proposition 1, by showing that if \(S\) is a closed subset of \(\R^n\), and \(\{ \mathbf x_j\}_j\) is any sequence in \(\R^n\) that converges to a limit \(\mathbf x \not\in S\), then \(\{\mathbf x_j\}_j\) is not a sequence in \(S\) (in other words, that \(\mathbf x_j\not \in S\) for some \(j\).)

    Hint\(\mathbf x\not\in S\) means that \(\mathbf x\in S^c\). If \(S\) is closed, what do we know about \(S^c\)?

  3. Fill in the details in the proof of Theorem 1 that in any set \(S\) that is not closed, there exists a sequence with no subsequence that converges to a limit in \(S\).

  4. Fill in the details in the proof of Theorem 1 that in any unbounded set \(S\), there exists a sequence with no convergent subsequence.

  5. Fill in the last missing details in the proof of Proposition 3.

  6. Assume that \(K_1\) is a compact subset of \(\R^n\) and \(K_2\) is a compact subset of \(\R^m\). Let \[ K_1\times K_2 = \{(\mathbf x, \mathbf y)\in \R^n\times \R^m = \R^{n+m} : \mathbf x \in K_1, \mathbf y\in K_2 \}. \]Prove that \(K_1\times K_2\) is a compact subset of \(\R^{m+n}\).

(In short, prove that a Cartesian Product of two compact sets is compact.) There are at least two different possible proofs, using two different characterizations of compact sets.

Guide for proving Theorem 6.

  1. Assume that \(K\) is a compact subset of \(\R^n\) and that \(\mathbf f:K\to \R^k\) is continuous. Fix \(\varepsilon>0\) and consider the set \[ S = \{(\mathbf x, \mathbf y)\in K\times K : |\mathbf f(\mathbf x) - \mathbf f(\mathbf y)|\ge \varepsilon\}. \]Also, define \(g:S\to \R\) by \(g(\mathbf x, \mathbf y) = |\mathbf x - \mathbf y|\), and define \(\delta = \inf \{ g(\mathbf x, \mathbf y) : (\mathbf x, \mathbf y)\in S\}\).

\(\Leftarrow\)  \(\Uparrow\)  \(\Rightarrow\)

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.0 Canada License.