$\renewcommand{\Re}{\operatorname{Re}}$ $\newcommand{\erf}{\operatorname{erf}}$ $\newcommand{\dag}{\dagger}$ $\newcommand{\const}{\mathrm{const}}$

Orthogonal systems

  1. Examples
  2. Abstract orthogonal systems: definition
  3. Orthogonal systems: approximation
  4. Orthogonal systems: approximation. II
  5. Orthogonal systems: completeness


All systems we considered in the previous lecture Lecture 13 were orthogonal i.e. \begin{equation} (X_n, X_m)=0\qquad \forall m\ne n \label{eq-1} \end{equation} with \begin{equation} (X,Y):=\int_0^l X(x)\bar{Y} (x)\,dx,\qquad \|X|^2:=(X,X). \label{eq-2} \end{equation} where $\bar{Y}$ means complex-conjugate to $Y$.

Exercise. Prove it by direct calculation.

Instead however we show that this nice property (and the fact that eigenvalues are real) is due to self-adjointness (the notion which we do not want to formulate at this time at least)

Consider $X,Y$ satisfying Robin boundary conditions \begin{align} &X'(0)-\alpha X(0)=0,\\ &X'(l)+\beta X(l)=0 \end{align} with $\alpha,\beta\in \mathbb{R}$ (so $Y$ satisfies the same conditions). Note that \begin{multline} (X'',Y)=\int X''(x)\bar{Y}(x)\,dx = \\ -\int X'(x)\bar{Y}'(x)\,dx + X' (l)\bar{Y}(l)- X' (0)\bar{Y}(0)= \\ -(X',Y') -\beta X (l)\bar{Y}(l)-\alpha X(0)\bar{Y}(0).\qquad \label{eq-5} \end{multline} Therefore if we plug $Y=X\ne 0$ an eigenfunction, $X''+\lambda X=0$ we get $-\lambda \|X\|^2$ in the left-hand expression (with obviously real $\|X\|^2\ne 0$) and also we get the real right expression (since $\alpha,\beta\in \mathbb{R}$); so $\lambda$ must be real: all eigenvalues are real.

Further, for $(X,Y'')$ we obtain the same equality albeit with $\alpha,\beta$ replaced by $\bar{\alpha},\bar{\beta}$ and therefore due to assumption $\alpha,\beta\in \mathbb{R}$ \begin{equation} (X'',Y)= (X,Y''). \label{eq-6} \end{equation} But then if $X,Y$ are eigenfunctions corresponding to different eigenvalues $\lambda$ and $\mu$ we get from (\ref{eq-6}) that $-\lambda(X,Y)=-\mu (X,Y)$ and $(X,Y)=0$ due to $\lambda\ne \mu$.

Remark 1. For periodic boundary conditions we cannot apply these arghuments to prove that $\cos(2\pi nx/l)$ and $\cos(2\pi nx/l)$ are orthogonal since they correspond to the same eigenvalue; we need to prove it directly.

Abstract orthogonal systems: definition

Consider linear space $\mathsf{H}$, real or complex. From linear algebra course standard definition

  1. $u+v=v+u\qquad \forall u,v\in \mathsf{H}$;
  2. $(u+v)+w=u+(v+w)\qquad \forall u,v,w\in \mathsf{H}$;
  3. $\exists 0\in \mathsf{H}: \ 0+u=u\qquad \forall u\in \mathsf{H}$;
  4. $\forall u\in \mathsf{H}\ \exists (-u): u+(-u)=0 $;
  5. $\alpha(u+v)=\alpha u+ \alpha v \qquad \forall u,v\in \mathsf{H}\forall\alpha \in \mathbb{R}$;
  6. $(\alpha+\beta)u+v)=\alpha u+ \beta u \qquad \forall u\in \mathsf{H}\forall\alpha,\beta \in \mathbb{R}$;
  7. $\alpha(\beta u)=(\alpha \beta)u \qquad \forall u\in \mathsf{H}\forall\alpha,\beta \in \mathbb{R}$;
  8. $1u=u \forall u \in \mathsf{H}$.

For complex linear space replace $\mathbb{R}$ bv $\mathbb{C}$.

Assume that on $\mathsf{H}$ inner product is defined:

  1. $(u+v,w)=(u,w)+(v,w)\qquad \forall u,v,w\in \mathsf{H}$;
  2. $(\alpha u,v)=\alpha (u,v) \qquad\forall u,v\in \mathsf{H}\forall \alpha\in \mathbb{R}$;
  3. $(u,v)=\bar{(v,u)} \qquad \forall u,v\in \mathsf{H}$;
  4. $\|u\|^2:=(u,u)\ge 0 \qquad \forall u\in \mathsf{H}$ (it means that it is real--if we consider complex spaces) and $\|u\|=0 \iff u=0$.

Definition 1.

  1. Finite dimensional real linear space with an inner product is called Euclidean space.
  2. Finite dimensional complex linear space with an inner product is called Hermitian space.
  3. Infinite dimensional linear space (real or complex) with an inner product is called pre-Hilbert space.

For Hilbert space we will need another property (completeness) which we add later.

Definition 2.

  1. System $\{u_n\}$, $0\ne u_n\in \mathsf{H}$ (finite or infinite) is orthogonal if $(u_m,u_n)=0$ $\forall m\ne n$;
  2. Orthogonal system is orthonormal if $\|u_n\|=1$ $\forall n$, i.e. $(u_m,u_n)=\delta_{mn}$ -- Kronecker symbol.

Orthogonal systems: approximation

Consider finite orthogonal system $\{u_n\}$. Let $\mathsf{K}$ be its linear hull: the set of linear combinations $\sum_n \alpha_n u_n$. Obviously $\mathsf{K}$ is a linear subspace of $\mathsf{H}$. Let $v\in \mathsf{H}$ and we try to find the best approximation of $v$ by elements of $\mathsf{K}$, i.e. we are looking for $w\in \mathsf{K}$ s.t. $\|v-w\|$ minimal.

Theorem 1.

  1. There exists a unique minimizer;
  2. This minimizer is an orthogonal projection of $f$ to $\mathsf{K}$, i.e. $w\in \mathsf{K}$ s.t. $(v-w)$ is orthogonal to all elements of $\mathsf{K}$;
  3. Such orthogonal projection is unique and $w=\sum_n \alpha_n u_n$ with \begin{equation} \alpha_n= \frac{(v,u_n)}{\|u_n\|^2}.\label{eq-7}\end{equation}
  4. $\|v\|^2=\|w\|^2+\|v-w\|^2$.
  5. $v=w \iff \|v\|^2=\|w\|^2$. Proof. (c) Obviously $(v-w)$ is orthogonal to $u_n$ iff (\ref{eq-7}) holds. If (\ref{eq-7}) holds for all $n$ then $(v-w)$ is orthogonal to all $u_n$ and therefore to all their linear combinations.

(d)-(e) In particular $(v-w)$ is orthogonal to $w$ and then \begin{equation*} \|v\|^2= \|(v-w)+w\|^2=\|v-w\|^2+ 2\Re \underbracket{(v-w,w)}_{=0}+\|w\|^2. \end{equation*}

(a)-(b) Consider $w'\in \mathsf{K}$. Then $\|v-w'\|^2= \|v-w\|^2+\|w-w'\|^2$ because $(w-w')\in \mathsf{K}$ and therefore it is orthogonal to $(v-w)$.

Orthogonal systems: approximation. II

Now let $\{u_n\}_{n=1,2,\ldots,}$ be infinite orthogonal system. Consider its finite subsystem with $n=1,2,\ldots, N$, introduce $\mathsf{K}_N$ for it and consider orthogonal projection $w_N$ of $v$ on $\mathsf{K}_N$. Then \begin{equation*} w_N= \sum_{n=1}^N \alpha_N u_n \end{equation*} where $\alpha_n$ are defined by (\ref{eq-7}). Then according to (d) of Theorem \begin{equation*} \|v\|^2 =\|v-w_N\|^2+\|w_N\|^2\ge \|w_N\|^2=\sum_{n=1}^N |\alpha_n |^2\|u_n\|^2. \end{equation*} Therefore series in the right-hand expression below converges \begin{equation} \|v\|^2 \ge \sum_{n=1}^\infty |\alpha_n |^2\|u_n\|^2 \label{eq-8} \end{equation} Really, recall that non-negative series can either converge or diverge to $\infty$.

Then $w_N$ is a Cauchy sequence. Really, for $M>N$ \begin{equation*} \|w_N-w_M\|^2= \sum_{n=N+1}^M |\alpha_n |^2\|u_n\|^2\le \varepsilon_N \end{equation*} with $\varepsilon_N\to 0$ as $N\to \infty$ because series in (\ref{eq-8}) converges.

Now we want to conclude that $w_N$ converges and to do this we must assume that every Cauchy sequence converges.

Definition 3.

  1. $\mathsf{H}$ is complete if every Cauchy sequence converges in $\mathsf{H}$.
  2. Complete pre-Hilbert space is called Hilbert space.

Remark 2. Every pre-Hilbert space could be completed i.e. extended to complete space. From now on $\mathsf{H}$ is a Hilbert space.

Then we can introduce $\mathsf{K}$-- a closed linear hull of $\{u_n\}_{n=1,2,\ldots}$ i.e. the space of \begin{equation} \sum_{n=1}^\infty \alpha_nu_n \end{equation} with $\alpha_n$ satisfying \begin{equation} \sum_{n=1}^\infty |\alpha_n |^2\|u_n\|^2<\infty. \end{equation} (Linear hull would be a space of finite linear combinations).

Let $v\in \mathsf{H}$. We want to find the best approximation of $v$ by elements of $\mathsf{K}$. But then we get immediately

Theorem 2. If $\mathsf{H}$ is a Hilbert space then Theorem 1 holds for infinite systems as well.

Orthogonal systems: completeness

Definition 4. Orthogonal system is complete if equivalent conditions below are satisfied:

  1. Its closed convex hull coincides with $\mathsf{H}$.
  2. If $v\in \mathsf{H}$ is orthogonal to all $u_n$ then $v=0$.

Remark 3. Don't confuse completeness of spaces and completeness of orthogonal systems.

Our next goal is to establish completeness of some orthogonal systems and therefore to give a positive answer (in the corresponding frameworks) to the question in the end of Lecture 12: can we decompose any function into eigenfunctions? Alternatively: Is the general solution a combination of simple solutions?