1.3: Sequences

\(\renewcommand{\R}{\mathbb R }\) \(\renewcommand{\N}{\mathbb N }\) \(\renewcommand{\Z}{\mathbb Z }\)

1.3: Sequences and Completeness

  1. Basics of sequences
  2. Review of completeness from first-year calculus
  3. Subsequences and convergent subsequences
  4. Completeness in \(\R^n\)
  5. Problems

\(\Leftarrow\)  \(\Uparrow\)  \(\Rightarrow\)

Introduction:

A common technique for solving a problem (think for example of minimizing a function \(\mathbf f: \R^n\to \R\)) is iteration:

To carry out the last point, we would like to have theorems that allow us to conclude that a sequence has a limit under certain hypotheses.

If we want such a theorem, the best thing we can imagine would be a theorem saying that “Every sequence in \(\R^n\) converges to a limit.” But this is false.
ExampleConsider for example the sequence \(\mathbf x_j = j {\mathbf e}_1\).
The next best thing we can imagine would be a theorem saying that “Every bounded sequence in \(\R^n\) converges to a limit.” But this is also false.
ExampleConsider for example the sequence \(\mathbf x_j = (-1)^j {\mathbf e}_1\).

However, if we change the statement a little, we get a true statement that enables us to find convergent sequences. To do this, we will need to introduce the notion of a subsequence. With this, we will prove

Every bounded sequence in \(\R^n\) has a subsequence that converges to a limit.

This is an excellent theorem if you like convergent sequences. It is also very useful for proving that certain kinds of problems (for example, minimization problems) have solutions, as we will see.

Just like in the one dimensional case, this theorem is connected to the completeness property of \(\R^n\). But it is not as intutive as applying the least upper bound property, as we will discuss below.

Basics of sequences

You are familiar with sequences of real numbers from MAT 137. A sequence may be written in various ways, including \[ \{ a_j \}_{j=1}^\infty,\qquad \{ a_j \}_j,\qquad \{ a_j \}, \quad \ldots \]

There are many notations for a sequence in \(\R^n\). It may be written as \[ \{ \mathbf a_j \}_{j=1}^\infty,\qquad \{ \mathbf a_j \}_j,\qquad \{ \mathbf a_j \}, \quad \ldots \]It is exactly like a sequence of real numbers, except that the terms \(\mathbf a_1,\mathbf a_2, \ldots\) in the sequence are all elements of \(\R^n\) rather than real numbers. Note that, even though it uses curly braces \(\{, \},\) like a set, the order of a sequence matters. You may use other notations in your other courses, to indicate that it is ordered. This notation in calculus is preferred in single variable calculus because brackets \([,],\) and parentheses \((,),\) are used frequently for intervals, and we will use it to establish continuity with your previous experience.

If we like, we can mimic the formal definition from MAT 137 and say that a sequence in \(\R^n\) is a function \(\N\to \R^n\), or more generally, a function whose domain has the form \(\{ j\in \Z : j\ge j_0\}\) and whose range is a subset of \(\R^n\).

We say that a sequence \(\{ \mathbf a_j \}\) in \(\R^n\) converges to the limit \({\mathbf L}\in \R^n\) if \[\begin{equation}\label{conv.seq} \forall \varepsilon>0,\exists J>0 \ \ \text{ such that if } j \ge J \mbox{ then } |\mathbf a_j - {\mathbf L}|< \varepsilon. \end{equation}\]

This is exactly like the definition of the limit of a sequence of real numbers from MAT 137 except that we replace the absolute value in the old definition by the Euclidean norm in the new one. Note also that we can reformulate definition \(\eqref{conv.seq}\) in terms of the sequence of real numbers \(\{ |\mathbf a_j - {\mathbf L}|\}_j\), as follows: \[ \lim_{j\to \infty}\mathbf a_j ={\mathbf L} \qquad\iff \qquad \lim_{j\to\infty} |\mathbf a_j - {\mathbf L}| = 0. \]

This being the case, you already understand many aspects of limits of sequences in \(\R^n\), including limit laws for sequences. Nice work!

If we have a sequence \(\{ \mathbf a_j\}_j\) of elements of \(\R^n\), then as usual we will write the components of a typical element \(\mathbf a_j\) of the sequence as \((a_{j1}, \ldots, a_{jn})\). If we consider some \(k\in \{1,\ldots, n\}\), then the sequence of \(k^{\text{th}}\) components, that is, \(\{ a_{jk} \}_{j=1}^\infty\) is a sequence of real numbers.

Let \(\{ \mathbf a_j\}\) be a sequence in \(\R^n\), and let \({\mathbf L} = (L_1,\ldots, L_n)\in \R^n\). Then \[ \lim_{j\to \infty}\mathbf a_j = {\mathbf L} \quad \iff \quad \lim_{j\to \infty}a_{jk} = L_k \text{ for all }k=1,\ldots, n. \]
Details This is almost exactly like the proof of the corresponding theorem about limits of functions. You can fill out the details.

One potentially confusing aspect of this theorem is the notation. Considering a concrete example may help make it digestible.

Example Consider the sequence in \(\R^3\), \[ \mathbf a_j = \left( \frac 1j , \frac{j^3-1}{j^3+j^2}, \frac{\sin(j^2)}{2j}\right), \qquad j = 1,2,3, \ldots \] The sequence of first components is \(\{a_{j1}\}_{j=1}^\infty\), where \(a_{j1} = \frac{1}{j}.\)

The sequence of second components is \(\{a_{j2}\}_{j=1}^\infty\), where \[a_{j2} = \dfrac {j^3-1}{j^3+j^2} = 1 - \dfrac{j^2+1}{j^3+j^2} .\]

The sequence of third components is \(\{a_{j3}\}_{j=1}^\infty\), where \(a_{j3} =\dfrac{\sin(j^2)}{2j}.\)

We can check using first-year calculus that these sequences converge to \(0\), \(1\), and \(0\), respectively.

So the theorem implies that \(\mathbf a_j \to {\mathbf L} = (0,1,0)\) as \(j\to \infty\).

Review of completeness from first-year calculus

Recall the following facts from MAT137.

A basic property of the real numbers is the Completeness Axiom, also known as the Least Upper Bound Property, which states that

Every nonempty set of real numbers that is bounded above has a least upper bound.

Recall:

  1. A number \(a\) is an upper bound for a set \(S\subseteq \R\) if \(a\ge x\) for every \(x\in S\). If \(S\) has an upper bound it is said to be bounded above.

  2. A number \(a\) is the least upper bound for \(S\) if \(a\) is an upper bound for \(S\), and \[ \text{ for every upper bound $b$ for $S$}, \qquad a\le b. \]

The least upper bound of a set \(S\) is often called the supremum of \(S\), written \(\sup S\).

The Completeness axiom also implies that every nonempty set \(S\) that is bounded below has a greatest lower bound, or infimum, denoted \(\inf S\).

Using the Completeness Axiom, one can prove the following:

Every nondecreasing sequence of real numbers that is bounded above converges to a limit.

(Recall that a sequence \(\{ x_j\}_j\) is nondecreasing if \(x_k \ge x_i\) whenever \(k\ge i\).)

A presentation of the proof with a lot of discussion of ideas and strategies can be found in this MAT 137 video. A concise proof is below.

Details

Let \(\{ x_j\}_j\) be a bounded nondecreasing sequence. Let \(\ell = \sup \{ x_j\}\). The idea of the proof is to show that \(x_j \rightarrow \ell\) as \(j\to \infty\).

To do this, fix \(\varepsilon>0\). Since \(\ell - \varepsilon <\ell\), and since \(\ell\) is the least upper bound for the sequence, then \(\ell - \varepsilon\) cannot be an upper bound. Thus, there exists some \(J\) such that \(x_J>\ell-\varepsilon\). Then, since the sequence is nondecreasing, it follows that \(x_j>\ell-\varepsilon\) for all \(j\ge J\). Also, since \(\ell\) is an upper bound, it is clear that \(x_j \le \ell\) for all \(j\).

We have thus shown that \[ \ell-\varepsilon < x_j \le \ell \qquad\text{ for all }j\ge J. \] Since \(\varepsilon>0\) was arbitrary, this shows that \(x_j\to \ell\) as \(j\to \infty\).

Subsequences and convergent sequences

What is a subsequence?

Assume that \(\{ \mathbf a_j\}_j\) is a sequence in \(\R^n\), for \(n\ge 1\).

A subsequence of \(\{ \mathbf a_j\}_j\) is a new sequence, obtained by deleting some terms from the original sequence.

For example, consider the sequence of real numbers defined by \[ a_j = \left(1- \frac 1j\right) \sin\left( \frac{ j \pi}4\right), \qquad j\ge 1. \] The first few terms in this sequence are \[ 0, \quad \frac 12 , \quad \frac{\sqrt 2}3, \quad 0,\quad \frac{-2\sqrt 2}5, \quad -\frac 56,\quad \frac{-3\sqrt 2}7, \quad 0,\quad \frac{4\sqrt 2}9, \quad \frac 9{10},\quad \ldots \] This sequence does not converge. But suppose we take a subsequence of every eighth term, \(j = 2, 10, 18, 26, \ldots\). The first few terms in the resulting sequence would be \[ \frac 12, \quad \frac 9{10},\quad \frac {17}{18}, \quad \frac {25}{26}, \quad \frac {33}{34}, \quad \frac {41}{42}, \quad \ldots \] We can write the subsequence as \(\{ a_{8k-6}\}_k\), since the \(k^{\text{th}}\) term of the new sequence is the \((8k-6)^{\text{th}}\) term of the original sequence. We can also write it as \(\{ a_{j_k}\}_{k\ge 1}\) where \(j_k = 8k-6\).

This is an example of a subsequence. It illustrates how, starting with a sequence that does not converge, it may be possible to find a subsequence that does converge to a limit. We can see by substituting \(j_k = 8k-6\) into the definition of the sequence that \(a_{j_k} = 1 - \frac 1{8k-6}\), which we know converges to \(1\). Note also in this example we could have chosen a different subsequence, for example, the sequence of all \(0\)s, by setting \(j_k = 4k-3\). The fact that two subsequences had different limits implies that the original sequence does not converge.

We now give the formal definition of subsequence, which you will need when you want to prove anything about subsequences.

A subsequence of \(\{ \mathbf a_j \}_{j}\) in \(\R^n\), is a new sequence, denoted \(\{ \mathbf a_{j_k}\}_k\), where \(\{ j_k \}\) is an increasing sequence of integers. (Increasing means that \(j_{k+1} > j_k\) for every \(k\).) Thus, the \(k\)th term \(\mathbf a_{j_k}\) of the subsequence is the \(j_k\)th term of the original sequence.

Think of a subsequence as taking some terms of the original sequence, and possibly omitting others, without changing their order. We can also generalize to sequences starting at an arbitrary integer \(j_0\) by requiring each \(j_k\geq j_0\).

Fact. If \(\{ \mathbf a_j \}_j\) is a sequence in \(\R^n\) that converges to a limit \({\mathbf L}\in \R^n\), then any subsequence of \(\{ \mathbf a_j \}_j\) converges to the same limit.

This says that if we have a sequence that converges to a limit, and we delete some terms, then the terms that are left will converge to the same limit. The proof is one of the practice problems. It is short – you may be able to do it in your head.

The 1-dimensional Bounded Sequence Theorem

Next we prove a version of the Bounded Sequence Theorem for sequences of real numbers. We will use this to later prove the general case of the theorem.

Every bounded sequence of real numbers has a convergent subsequence.

There are several possible proofs. A proof by bisecting intervals can be found in Folland (Section 1.5, Theorem 1.18). Two different proofs can be found below.

First proof using least upper bound

How can we deduce from the LUB principle that a bounded real valued sequence \(\left(a_k\right)_k\) admits a convergent subsequence \(\left(a_{\varphi(j)}\right)_j\)

We know that \(\exists M>0,\,\forall k,\,|a_k|<M\). Then \[L=\sup\left\{x\in[-M,M]\,:\,x<a_k\text{ for infinitely many $k$}\right\}\] exists by the LUB principle, since the set is bounded from above by \(M\) and contains \(-M\).

Taking \(\varphi(1)=1\), suppose that we have constructed all terms in the subsequence up to \(\varphi(j-1)\) and we want to construct the next term \(a_{\varphi(j)}\). Since \(L-\frac1j\) is less than \(L\), it is not an upper bound of the above set. Hence there exist infinitely many \(k\) such that \[L-\frac1j<a_k<L+\frac1j .\]

We pick \(\varphi(j)\) to be such a \(k\) which is greater than \(\varphi(j-1)\), which is possible since there are infinitely many such \(k\). Hence, we construct a subsequence \(\left(a_{\varphi(j)}\right)_j\) such that \[\forall j,\,L-\frac1j<a_{\varphi(j)}<L+\frac 1j.\] Therefore this subsequence converges to \(L\).
Second proof using monotone sequence theorem

Let \(\{ a_j\}_j\) be a bounded sequence of real numbers.

For each \(j\), let \(b_j = \inf \{ a_k : k > j\}\).

. \(\{ b_j\}_j\) is a bounded, nondecreasing sequence.

To check this, let’s name the sets \(S_j = \{ a_k : k > j \}\). Thus \(b_j = \inf S_j\). Note that if \(j_2 > j_1\) then \(S_{j_2}\subseteq S_{j_1}\). So any lower bound for \(S_{j_1}\) is also a lower bound for \(S_{j_2}\). In particular, \(b_{j_1} = \inf S_{j_1}\) is a lower bound for \(S_{j_1}\) and hence \(S_{j_2}\). It is thus less than or equal to \(b_{j_2}=\inf S_{j_2} =\) the greatest lower bound for \(S_{j_2}\). This shows that \(b_{j_2} \ge b_{j_1}\) whenever \(j_2> j_1\), and hence proves that \(\{ b_j\}_j\) is nondecreasing. It is bounded because by assumption there exists some \(M\in \R\) such that \(|a_j|\le M\) for every \(j\), so \(|b_j| \le M\) for every \(j\). This proves the claim.

Remark: by exactly the same argument, we could show that if \(A \subseteq B \subseteq \R\), then \(\inf A \ge \inf B\) and \(\sup A \le \sup B\).

In view of Lemma 5 and the Monotone Sequence Theorem, \(\{ b_j\}_j\) is a convergent sequence. Let \(\ell\) denote its limit.

\(\{ a_j \}_j\) has a subsequence that converges to \(\ell\).

To prove this, we will describe how to choose the subsequence.

Let \(a_{j_1} = a_1\).

Assume that we have chosen \(j_1, j_2, \ldots, j_{k}\). We will describe how to choose \(j_{k+1}\). The idea is to choose it so that \(a_{j_{k+1}}\) is (nearly) the smallest term, among all those in the sequence after \(a_{j_k}\).

Note that \(b_{j_k} = \inf \{ a_j : j > j_k \}\) and since it is the infimum, we can choose \(j_{k+1} > j_k\) such that \[ b_{j_k} \le a_{j_{k+1}} < b_{j_k} + \frac 1k \le \ell+ \frac 1k. \] This describes how to choose \(j_{k+1}\). By repeating this procedure indefinitely, we generate the subsequence \(\{ a_{j_k} \}_k\).

Finally, we know that \(\{ b_{j_k}\}_k\) converges to the same limit \(\ell\) as \(\{ b_j\}_j\) (by the Fact mentioned above), and \(\displaystyle\lim_{k\to\infty} \left(\ell+ \frac 1k\right)=\ell\), so \(\{ a_{j_k} \}_k\) converges to the same limit, by the Squeeze Theorem for sequences.

The Bounded Sequence Theorem - the general case

Finally, we consider the Bounded Sequence Theorem, which is our main goal for this discussion. We recall the statement.

Bounded Sequence Theorem. Every bounded sequence in \(\R^n\) has a subsequence that converges to a limit.

Before giving the proof, we illustrate the idea by considering an example.

Example For a real number \(x\), let \(\lfloor x\rfloor\) denote the largest integer less than or equal to \(x\), and let \(\text{frac}(x)\) denote the “fractional part” of \(x\), that is, \[ \text{frac}(x) = x - \lfloor x \rfloor. \] Let \(\{ \mathbf a_j\}_j\) be the sequence in \(\R^2\) defined by \[ \mathbf a_j = ( \text{frac} (j/3), \text{frac}(j/7)). \] Thus the first few terms in the sequence (starting with \(\mathbf a_1\)) are \[\begin{equation}\label{ss1} \left(\frac 13, \frac 17\right), \ \left(\frac 23, \frac 27\right), \ \left(0, \frac 37\right), \ \left(\frac 13, \frac 47\right), \ \left(\frac 23, \frac 57\right), \ \left(0, \frac 67\right), \ \ldots \end{equation}\] We can easily find a subsequence such that the first components converge. For example, we can consider the subsequence \(\{ \mathbf a_{j_k} \}\) for \(j_k = 3k\). This amounts to discarding all \(\mathbf a_j\) in the original sequence where \(j\) is not divisible by \(3\). The resulting subsequence is \[\begin{equation}\label{ss11} \left(0, \frac 37\right), \ \left(0, \frac 67\right), \ \left(0, \frac 27\right), \ \left(0, \frac 57\right), \ \left(0, \frac 17\right), \ \left(0, \frac 47\right), \ (0, 0), \ \ldots \end{equation}\]For this subsequence the first component definitely converges (since it is always equal to zero) but the second component does not.

Similarly, we could consider the subsequence \(\{ \mathbf a_{j_k} \}\) for \(j_k = 7k\). For this subsequence, the second component is always equal to zero, but the first component does not converge. In order to construct a convergent subsequence, we need both of the components to converge at the same time, not just one or the other.

However, there is an easy way out of this difficulty.

The proof of the theorem follows this same strategy of choosing subsequences of subsequences.

Details for Bounded Sequence Theorem

Assume that \(\{ \mathbf a_j\}_j\) is a bounded sequence in \(\R^n\). This means that there exists some \(R>0\) such that \[ |\mathbf a_j| < R \qquad\text{ for every }j. \] It follows that for every \(i\in \{ 1,\ldots, n\}\), and every \(j\), \[ |a_{ji}| = |\mathbf a_j \cdot {\mathbf e}_i| \le |\mathbf a_j|\, |{\mathbf e}_i| < R. \] Here \(a_{ji}\) denotes the \(i\)th component of \(\mathbf a_j\).

Thus for every \(i\), the sequence \(\{ a_{ji}\}_j\) of \(i\)th components is a bounded sequence of real numbers. We now proceed as follows:

  1. Using the one-dimensional case of the Bounded Sequence Theorem, we can choose a subsequence of \(\{ \mathbf a_j\}_j\) such that the first components of the subsequence converge to a limit \(L_1\). The other components remain bounded.

  2. Again using the one-dimensional case of the Bounded Sequence Theorem, we can choose a subsequence of the first subsequence, such that the second components converge to a limit \(L_2\). This is still a subsequence of \(\{ \mathbf a_j\}_j\), the first components still converge to \(L_1\), and the other components remain bounded.

  3. Repeat this procedure \(n-2\) more times. At the end we will arrive at a subsequence of \(\{ \mathbf a_j\}_j\) such that the \(i\)th components converge to a limit \(L_i\) for \(i = 1,\ldots, n\). According to Theorem 2, this implies that the subsequence is convergent.

Completeness in \(\R^n\)

Recall that the set of real numbers is complete - if we have a sequence of real numbers where the terms get closer together, then the sequence has a real number as a limit. This property is not true in the set of rational numbers, for example, which originally motivated the construction of the reals. Unfortunately, the characterization of completeness from first-year calculus, that nonempty subsets of real numbers which are bounded above have least upper bounds, does not translate easily to \(\R^n\).

The fundamental issue is that since \(\R\) is a line it makes sense to ask which of any two elemets is bigger. In other words, given any two numbers \(x,y\in \R\), it is always the case that either \(x\leq y\) or \(y\leq x\). That is, \(\R\) is totally ordered.

On the other hand, given two vectors \({\mathbf x},{\mathbf y}\in\R^2\) (or more generally in \(\R^n\) for \(n\geq 2\)), it doesn’t make sense to ask which of the two vectors is bigger. We could try defining an order by saying \({\mathbf x}\leq {\mathbf y}\) whenever \(| {\mathbf x}|\leq|{\mathbf y} |\) but this turns out to not satisfy some nice properties that we would expect of an order, for example there are many nonequal vectors of the same magnitude, so with this definition \({\mathbf x}\leq {\mathbf y}\) and \({\mathbf y}\leq {\mathbf x}\) does not imply that \({\mathbf x}={\mathbf y}\).

There is no way of ordering elements of \(\R^n\) for our purposes, so we can’t generalize the definition of least upper bound to \(\R^n\). This means the first-year calculus characterization of completeness does not help us directly.

When trying to generalize a concept to \(\R^n\), we will try to find equivalent formulations of the statement until we have a statement that makes sense in \(\R^n\). Then we will use this as the definition of the concept in \(\R^n\).

For example, in \(\R\) we saw above that the Bounded Sequence Theorem follows from the Monotone Convergence Theorem, and the Monotone Convergence Theorem follows from the completeness of \(\R\): \[\text{Completeness of } \R \implies \text{MCT} \implies \text{BST}\]

You can check that these are all actually equivalences, in other words, that you can prove the completeness of \(\R\) (the least upper bound property) from the Monotone Convergence Theorem and that you can prove the Monotone Convergence Theorem from the Bounded Sequence Theorem: \[\text{Completeness of } \R \iff\text{MCT} \iff\text{BST}\] Hence in \(\R\) the notion of completeness is equivalent to the Bounded Sequence Theorem. Finally notice that the statement of the Bounded Sequence Theorem no longer requires the definition of a least upper bound (or an order between vectors) and has a generalization to the Bounded Sequence Theorem in \(\R^n\) as above.

This allows us to define completeness of \(\R^n\) in terms of the bounded sequence theorem. The analogue of the completeness axiom in higher dimensions thus becomes

Every bounded sequence in \(\R^n\) has a convergent subsequence.

For \(n=1\) this notion of completeness is equivalent to the usual single-variable calculus completeness axiom so we have arrived at a generalization of completeness which now works in any dimension, since it does not rely on a least upper bound property in higher dimensions. We could also show that completeness is equivalent to the Intermediate Value Theorem, or to the statement that every absolutely convergent sequence converges.

Problems

The material in this section introduces important definitions and theorems that generalize what you know from single variable calculus. It does not involve any computational methods that can be applied in a staightforward way to solve problems. However, we could ask a question testing whether you are familiar with the main theorems and how to apply them, such as

  1. Consider the sequence \(\{ \mathbf a_j\}_j\) in \(\R^2\), defined by \[ \mathbf a_j = (\cos j, \sin j). \]Does this have a convergent subsequence? Justify your answer. (“Justify your answer” could mean: give the name or content of a relevant theorem, and check that its hypotheses are satisfied.)

Questions about the formal definitions in this section, for example, the definition of limit of a sequence in \(\R^n\), are also possible.

Advanced

  1. Assume that \(\{ \mathbf a_j \}_j\) is a convergent sequence in \(\R^n\). Prove that any subsequence \(\{ \mathbf a_{k_j} \}_j\) converges to the same limit.

  2. Assume that \(\{\mathbf a_j\}\) and \(\{\mathbf b_j\}\) are sequences in \(\R^n\) such that \(\mathbf a_j\to \mathbf a\) and \(\mathbf b_j\to \mathbf b\).

  3. Assume that \(\{ \mathbf a_j \}_j\) is a sequence in \(\R^n\) with \(\displaystyle\lim_{j\to \infty}\mathbf a_j = \mathbf a\), and that \(\mathbf f:\R^n\to \R^k\) is a function that is continuous at \(\mathbf a\). Prove that \(\lim_{j\to \infty} \mathbf f(\mathbf a_j) = \mathbf f(\mathbf a)\).

  4. Find a sequence in \(\R^n\) with no convergent subsequences. Prove that your answer is correct.

  5. Review of calculus: Assume that \(\{ x_j \}_j\) is a sequence of real numbers, and let \(d_j = |x_{j+1} - x_j|\), i.e. the distance between consecutive terms. Is it true that if \(d_j\to 0\) as \(j\to \infty\), then \(x_j\) must converge to a limit as \(j\to \infty\)?

  6. Assume that \(\{ \mathbf a_j \}_j\) is a sequence of points in \(\R^n\), and let \(d_j := |\mathbf a_{j+1} - \mathbf a_j|\), i.e. the Euclidean distance between consecutive points. Is it true that if \(d_j\to 0\) as \(j\to \infty\), then \(\mathbf a_j\) must converge to a limit as \(j\to \infty\)?

HintThis looks a lot like the previous problem.

\(\Leftarrow\)  \(\Uparrow\)  \(\Rightarrow\)

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.0 Canada License.