Linear Algebra/Gram-Schmidt Orthogonalization/Solutions

Solutions

Problem 1

Perform the Gram-Schmidt process on each of these bases for $\mathbb {R} ^{2}$.

1. $\left\langle {\begin{pmatrix}2\\0\\1\end{pmatrix}},{\begin{pmatrix}3\\-1\\5\end{pmatrix}},{\begin{pmatrix}0\\4\\2\end{pmatrix}}\right\rangle$
2. $\left\langle {\begin{pmatrix}0\\1\end{pmatrix}},{\begin{pmatrix}-1\\3\end{pmatrix}}\right\rangle$
3. $\left\langle {\begin{pmatrix}0\\1\end{pmatrix}},{\begin{pmatrix}-1\\0\end{pmatrix}}\right\rangle$

Then turn those orthogonal bases into orthonormal bases.

Answer

1. ${\begin{array}{rl}{\vec {\kappa }}_{1}&={\begin{pmatrix}1\\1\end{pmatrix}}\\{\vec {\kappa }}_{2}&={\begin{pmatrix}2\\1\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}2\\1\end{pmatrix}})={\begin{pmatrix}2\\1\end{pmatrix}}-{\frac {{\begin{pmatrix}2\\1\end{pmatrix}}\cdot {\begin{pmatrix}1\\1\end{pmatrix}}}{{\begin{pmatrix}1\\1\end{pmatrix}}\cdot {\begin{pmatrix}1\\1\end{pmatrix}}}}\cdot {\begin{pmatrix}1\\1\end{pmatrix}}={\begin{pmatrix}2\\1\end{pmatrix}}-{\frac {3}{2}}\cdot {\begin{pmatrix}1\\1\end{pmatrix}}={\begin{pmatrix}1/2\\-1/2\end{pmatrix}}\end{array}}$

2. ${\begin{array}{rl}{\vec {\kappa }}_{1}&={\begin{pmatrix}0\\1\end{pmatrix}}\\{\vec {\kappa }}_{2}&={\begin{pmatrix}-1\\3\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}-1\\3\end{pmatrix}})={\begin{pmatrix}-1\\3\end{pmatrix}}-{\frac {{\begin{pmatrix}-1\\3\end{pmatrix}}\cdot {\begin{pmatrix}0\\1\end{pmatrix}}}{{\begin{pmatrix}0\\1\end{pmatrix}}\cdot {\begin{pmatrix}0\\1\end{pmatrix}}}}\cdot {\begin{pmatrix}0\\1\end{pmatrix}}={\begin{pmatrix}-1\\3\end{pmatrix}}-{\frac {3}{1}}\cdot {\begin{pmatrix}0\\1\end{pmatrix}}={\begin{pmatrix}-1\\0\end{pmatrix}}\end{array}}$

3. ${\begin{array}{rl}{\vec {\kappa }}_{1}&={\begin{pmatrix}0\\1\end{pmatrix}}\\{\vec {\kappa }}_{2}&={\begin{pmatrix}-1\\0\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}-1\\0\end{pmatrix}})={\begin{pmatrix}-1\\0\end{pmatrix}}-{\frac {{\begin{pmatrix}-1\\0\end{pmatrix}}\cdot {\begin{pmatrix}0\\1\end{pmatrix}}}{{\begin{pmatrix}0\\1\end{pmatrix}}\cdot {\begin{pmatrix}0\\1\end{pmatrix}}}}\cdot {\begin{pmatrix}0\\1\end{pmatrix}}={\begin{pmatrix}-1\\0\end{pmatrix}}-{\frac {0}{1}}\cdot {\begin{pmatrix}0\\1\end{pmatrix}}={\begin{pmatrix}-1\\0\end{pmatrix}}\end{array}}$

The corresponding orthonormal bases for the three parts of this question are these.

$\left\langle {\begin{pmatrix}1/{\sqrt {2}}\\1/{\sqrt {2}}\end{pmatrix}},{\begin{pmatrix}{\sqrt {2}}/2\\-{\sqrt {2}}/2\end{pmatrix}}\right\rangle \qquad \left\langle {\begin{pmatrix}0\\1\end{pmatrix}},{\begin{pmatrix}-1\\0\end{pmatrix}}\right\rangle \qquad \left\langle {\begin{pmatrix}0\\1\end{pmatrix}},{\begin{pmatrix}-1\\0\end{pmatrix}}\right\rangle$

This exercise is recommended for all readers.

Problem 2

Perform the Gram-Schmidt process on each of these bases for $\mathbb {R} ^{3}$.

1. $\left\langle {\begin{pmatrix}2\\2\\2\end{pmatrix}},{\begin{pmatrix}1\\0\\-1\end{pmatrix}},{\begin{pmatrix}0\\3\\1\end{pmatrix}}\right\rangle$
2. $\left\langle {\begin{pmatrix}1\\-1\\0\end{pmatrix}},{\begin{pmatrix}0\\1\\0\end{pmatrix}},{\begin{pmatrix}2\\3\\1\end{pmatrix}}\right\rangle$

Then turn those orthogonal bases into orthonormal bases.

Answer

1. ${\begin{array}{rl}{\vec {\kappa }}_{1}&={\begin{pmatrix}2\\2\\2\end{pmatrix}}\\{\vec {\kappa }}_{2}&={\begin{pmatrix}1\\0\\-1\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}1\\0\\-1\end{pmatrix}})={\begin{pmatrix}1\\0\\-1\end{pmatrix}}-{\frac {{\begin{pmatrix}1\\0\\-1\end{pmatrix}}\cdot {\begin{pmatrix}2\\2\\2\end{pmatrix}}}{{\begin{pmatrix}2\\2\\2\end{pmatrix}}\cdot {\begin{pmatrix}2\\2\\2\end{pmatrix}}}}\cdot {\begin{pmatrix}2\\2\\2\end{pmatrix}}={\begin{pmatrix}1\\0\\-1\end{pmatrix}}-{\frac {0}{12}}\cdot {\begin{pmatrix}2\\2\\2\end{pmatrix}}={\begin{pmatrix}1\\0\\-1\end{pmatrix}}\\{\vec {\kappa }}_{3}&={\begin{pmatrix}0\\3\\1\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}0\\3\\1\end{pmatrix}})-{\mbox{proj}}_{[{\vec {\kappa }}_{2}]}({\begin{pmatrix}0\\3\\1\end{pmatrix}})={\begin{pmatrix}0\\3\\1\end{pmatrix}}-{\frac {{\begin{pmatrix}0\\3\\1\end{pmatrix}}\cdot {\begin{pmatrix}2\\2\\2\end{pmatrix}}}{{\begin{pmatrix}2\\2\\2\end{pmatrix}}\cdot {\begin{pmatrix}2\\2\\2\end{pmatrix}}}}\cdot {\begin{pmatrix}2\\2\\2\end{pmatrix}}-{\frac {{\begin{pmatrix}0\\3\\1\end{pmatrix}}\cdot {\begin{pmatrix}1\\0\\-1\end{pmatrix}}}{{\begin{pmatrix}1\\0\\-1\end{pmatrix}}\cdot {\begin{pmatrix}1\\0\\-1\end{pmatrix}}}}\cdot {\begin{pmatrix}1\\0\\-1\end{pmatrix}}\\&\quad ={\begin{pmatrix}0\\3\\1\end{pmatrix}}-{\frac {8}{12}}\cdot {\begin{pmatrix}2\\2\\2\end{pmatrix}}-{\frac {-1}{2}}\cdot {\begin{pmatrix}1\\0\\-1\end{pmatrix}}={\begin{pmatrix}-5/6\\5/3\\-5/6\end{pmatrix}}\end{array}}$

2. ${\begin{array}{rl}{\vec {\kappa }}_{1}&={\begin{pmatrix}1\\-1\\0\end{pmatrix}}\\{\vec {\kappa }}_{2}&={\begin{pmatrix}0\\1\\0\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}0\\1\\0\end{pmatrix}})={\begin{pmatrix}0\\1\\0\end{pmatrix}}-{\frac {{\begin{pmatrix}0\\1\\0\end{pmatrix}}\cdot {\begin{pmatrix}1\\-1\\0\end{pmatrix}}}{{\begin{pmatrix}1\\-1\\0\end{pmatrix}}\cdot {\begin{pmatrix}1\\-1\\0\end{pmatrix}}}}\cdot {\begin{pmatrix}1\\-1\\0\end{pmatrix}}={\begin{pmatrix}0\\1\\0\end{pmatrix}}-{\frac {-1}{2}}\cdot {\begin{pmatrix}1\\-1\\0\end{pmatrix}}={\begin{pmatrix}1/2\\1/2\\0\end{pmatrix}}\\{\vec {\kappa }}_{3}&={\begin{pmatrix}2\\3\\1\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}2\\3\\1\end{pmatrix}})-{\mbox{proj}}_{[{\vec {\kappa }}_{2}]}({\begin{pmatrix}2\\3\\1\end{pmatrix}})\\&={\begin{pmatrix}2\\3\\1\end{pmatrix}}-{\frac {{\begin{pmatrix}2\\3\\1\end{pmatrix}}\cdot {\begin{pmatrix}1\\-1\\0\end{pmatrix}}}{{\begin{pmatrix}1\\-1\\0\end{pmatrix}}\cdot {\begin{pmatrix}1\\-1\\0\end{pmatrix}}}}\cdot {\begin{pmatrix}1\\-1\\0\end{pmatrix}}-{\frac {{\begin{pmatrix}2\\3\\1\end{pmatrix}}\cdot {\begin{pmatrix}1/2\\1/2\\0\end{pmatrix}}}{{\begin{pmatrix}1/2\\1/2\\0\end{pmatrix}}\cdot {\begin{pmatrix}1/2\\1/2\\0\end{pmatrix}}}}\cdot {\begin{pmatrix}1/2\\1/2\\0\end{pmatrix}}\\&\quad ={\begin{pmatrix}2\\3\\1\end{pmatrix}}-{\frac {-1}{2}}\cdot {\begin{pmatrix}1\\-1\\0\end{pmatrix}}-{\frac {5/2}{1/2}}\cdot {\begin{pmatrix}1/2\\1/2\\0\end{pmatrix}}={\begin{pmatrix}0\\0\\1\end{pmatrix}}\end{array}}$

The corresponding orthonormal bases for the two parts of this question are these.

$\left\langle {\begin{pmatrix}1/{\sqrt {3}}\\1/{\sqrt {3}}\\1/{\sqrt {3}}\end{pmatrix}},{\begin{pmatrix}1/{\sqrt {2}}\\0\\-1/{\sqrt {2}}\end{pmatrix}},{\begin{pmatrix}-1/{\sqrt {6}}\\2/{\sqrt {6}}\\-1/{\sqrt {6}}\end{pmatrix}}\right\rangle \qquad \left\langle {\begin{pmatrix}1/{\sqrt {2}}\\-1/{\sqrt {2}}\\0\end{pmatrix}},{\begin{pmatrix}1/{\sqrt {2}}\\1/{\sqrt {2}}\\0\end{pmatrix}}{\begin{pmatrix}0\\0\\1\end{pmatrix}}\right\rangle$

This exercise is recommended for all readers.

Problem 3

Find an orthonormal basis for this subspace of $\mathbb {R} ^{3}$: the plane $x-y+z=0$.

Answer

The given space can be parametrized in this way.

$\{{\begin{pmatrix}x\\y\\z\end{pmatrix}}\,{\big |}\,x=y-z\}=\{{\begin{pmatrix}1\\1\\0\end{pmatrix}}\cdot y+{\begin{pmatrix}-1\\0\\1\end{pmatrix}}\cdot z\,{\big |}\,y,z\in \mathbb {R} \}$

So we take the basis

$\left\langle {\begin{pmatrix}1\\1\\0\end{pmatrix}},{\begin{pmatrix}-1\\0\\1\end{pmatrix}}\right\rangle$

apply the Gram-Schmidt process

${\begin{array}{rl}{\vec {\kappa }}_{1}&={\begin{pmatrix}1\\1\\0\end{pmatrix}}\\{\vec {\kappa }}_{2}&={\begin{pmatrix}-1\\0\\1\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}-1\\0\\1\end{pmatrix}})={\begin{pmatrix}-1\\0\\1\end{pmatrix}}-{\frac {{\begin{pmatrix}-1\\0\\1\end{pmatrix}}\cdot {\begin{pmatrix}1\\1\\0\end{pmatrix}}}{{\begin{pmatrix}1\\1\\0\end{pmatrix}}\cdot {\begin{pmatrix}1\\1\\0\end{pmatrix}}}}\cdot {\begin{pmatrix}1\\1\\0\end{pmatrix}}={\begin{pmatrix}-1\\0\\1\end{pmatrix}}-{\frac {-1}{2}}\cdot {\begin{pmatrix}1\\1\\0\end{pmatrix}}={\begin{pmatrix}-1/2\\1/2\\1\end{pmatrix}}\end{array}}$

and then normalize.

$\left\langle {\begin{pmatrix}1/{\sqrt {2}}\\1/{\sqrt {2}}\\0\end{pmatrix}},{\begin{pmatrix}-1/{\sqrt {6}}\\1/{\sqrt {6}}\\2/{\sqrt {6}}\end{pmatrix}}\right\rangle$

Problem 4

Find an orthonormal basis for this subspace of $\mathbb {R} ^{4}$.

$\{{\begin{pmatrix}x\\y\\z\\w\end{pmatrix}}\,{\big |}\,x-y-z+w=0{\text{ and }}x+z=0\}$

Answer

Reducing the linear system

${\begin{array}{*{4}{rc}r}x&-&y&-&z&+&w&=&0\\x&&&+&z&&&=&0\end{array}}\;{\xrightarrow[{}]{-\rho _{1}+\rho _{2}}}\;{\begin{array}{*{4}{rc}r}x&-&y&-&z&+&w&=&0\\&&y&+&2z&-&w&=&0\end{array}}$

and parametrizing gives this description of the subspace.

$\{{\begin{pmatrix}-1\\-2\\1\\0\end{pmatrix}}\cdot z+{\begin{pmatrix}0\\1\\0\\1\end{pmatrix}}\cdot w\,{\big |}\,z,w\in \mathbb {R} \}$

So we take the basis,

$\left\langle {\begin{pmatrix}-1\\-2\\1\\0\end{pmatrix}},{\begin{pmatrix}0\\1\\0\\1\end{pmatrix}}\right\rangle$

go through the Gram-Schmidt process

${\begin{array}{rl}{\vec {\kappa }}_{1}&={\begin{pmatrix}-1\\-2\\1\\0\end{pmatrix}}\\{\vec {\kappa }}_{2}&={\begin{pmatrix}0\\1\\0\\1\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}0\\1\\0\\1\end{pmatrix}})={\begin{pmatrix}0\\1\\0\\1\end{pmatrix}}-{\frac {{\begin{pmatrix}0\\1\\0\\1\end{pmatrix}}\cdot {\begin{pmatrix}-1\\-2\\1\\0\end{pmatrix}}}{{\begin{pmatrix}-1\\-2\\1\\0\end{pmatrix}}\cdot {\begin{pmatrix}-1\\-2\\1\\0\end{pmatrix}}}}\cdot {\begin{pmatrix}-1\\-2\\1\\0\end{pmatrix}}={\begin{pmatrix}0\\1\\0\\1\end{pmatrix}}-{\frac {-2}{6}}\cdot {\begin{pmatrix}-1\\-2\\1\\0\end{pmatrix}}={\begin{pmatrix}-1/3\\1/3\\1/3\\1\end{pmatrix}}\end{array}}$

and finish by normalizing.

$\left\langle {\begin{pmatrix}-1/{\sqrt {6}}\\-2/{\sqrt {6}}\\1/{\sqrt {6}}\\0\end{pmatrix}},{\begin{pmatrix}-{\sqrt {3}}/6\\{\sqrt {3}}/6\\{\sqrt {3}}/6\\{\sqrt {3}}/2\end{pmatrix}}\right\rangle$

Problem 5

Show that any linearly independent subset of $\mathbb {R} ^{n}$ can be orthogonalized without changing its span.

Answer

A linearly independent subset of $\mathbb {R} ^{n}$ is a basis for its own span. Apply Theorem 2.7.

Remark. Here's why the phrase "linearly independent" is in the question. Dropping the phrase would require us to worry about two things. The first thing to worry about is that when we do the Gram-Schmidt process on a linearly dependent set then we get some zero vectors. For instance, with

$S=\{{\begin{pmatrix}1\\2\end{pmatrix}},{\begin{pmatrix}3\\6\end{pmatrix}}\}$

we would get this.

${\vec {\kappa }}_{1}={\begin{pmatrix}1\\2\end{pmatrix}}\qquad {\vec {\kappa }}_{2}={\begin{pmatrix}3\\6\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}3\\6\end{pmatrix}})={\begin{pmatrix}0\\0\end{pmatrix}}$

This first thing is not so bad because the zero vector is by definition orthogonal to every other vector, so we could accept this situation as yielding an orthogonal set (although it of course can't be normalized), or we just could modify the Gram-Schmidt procedure to throw out any zero vectors. The second thing to worry about if we drop the phrase "linearly independent" from the question is that the set might be infinite. Of course, any subspace of the finite-dimensional $\mathbb {R} ^{n}$ must also be finite-dimensional so only finitely many of its members are linearly independent, but nonetheless, a "process" that examines the vectors in an infinite set one at a time would at least require some more elaboration in this question. A linearly independent subset of $\mathbb {R} ^{n}$ is automatically finite— in fact, of size $n$ or less— so the "linearly independent" phrase obviates these concerns.

This exercise is recommended for all readers.

Problem 6

What happens if we apply the Gram-Schmidt process to a basis that is already orthogonal?

Answer

The process leaves the basis unchanged.

Problem 7

Let $\left\langle {\vec {\kappa }}_{1},\dots ,{\vec {\kappa }}_{k}\right\rangle$ be a set of mutually orthogonal vectors in $\mathbb {R} ^{n}$.

1. Prove that for any ${\vec {v}}$ in the space, the vector ${\vec {v}}-({\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {v}}\,})+\dots +{\mbox{proj}}_{[{\vec {v}}_{k}]}({{\vec {v}}\,}))$ is orthogonal to each of ${\vec {\kappa }}_{1}$, ..., ${\vec {\kappa }}_{k}$.
2. Illustrate the prior item in $\mathbb {R} ^{3}$ by using ${\vec {e}}_{1}$ as ${\vec {\kappa }}_{1}$, using ${\vec {e}}_{2}$ as ${\vec {\kappa }}_{2}$, and taking ${\vec {v}}$ to have components $1$, $2$, and $3$.
3. Show that ${\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {v}}\,})+\dots +{\mbox{proj}}_{[{\vec {v}}_{k}]}({{\vec {v}}\,})$ is the vector in the span of the set of ${\vec {\kappa }}$'s that is closest to ${\vec {v}}$. Hint. To the illustration done for the prior part, add a vector $d_{1}{\vec {\kappa }}_{1}+d_{2}{\vec {\kappa }}_{2}$ and apply the Pythagorean Theorem to the resulting triangle.

Answer

1. The argument is as in the $i=3$ case of the proof of Theorem 2.7. The dot product

   ${\vec {\kappa }}_{i}\cdot \left({\vec {v}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {v}}\,})-\dots -{\mbox{proj}}_{[{\vec {v}}_{k}]}({{\vec {v}}\,})\right)$

   can be written as the sum of terms of the form $-{\vec {\kappa }}_{i}\cdot {\mbox{proj}}_{[{\vec {\kappa }}_{j}]}({{\vec {v}}\,})$ with $j\neq i$, and the term ${\vec {\kappa }}_{i}\cdot ({\vec {v}}-{\mbox{proj}}_{[{\vec {\kappa }}_{i}]}({{\vec {v}}\,}))$. The first kind of term equals zero because the ${\vec {\kappa }}$'s are mutually orthogonal. The other term is zero because this projection is orthogonal (that is, the projection definition makes it zero: ${\vec {\kappa }}_{i}\cdot ({\vec {v}}-{\mbox{proj}}_{[{\vec {\kappa }}_{i}]}({{\vec {v}}\,}))={\vec {\kappa }}_{i}\cdot {\vec {v}}-{\vec {\kappa }}_{i}\cdot \left(({\vec {v}}\cdot {\vec {\kappa }}_{i})/({\vec {\kappa }}_{i}\cdot {\vec {\kappa }}_{i})\right)\cdot {\vec {\kappa }}_{i}$ equals, after all of the cancellation is done, zero).
2. The vector ${\vec {v}}$ is shown in black and the vector ${\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {v}}\,})+{\mbox{proj}}_{[{\vec {v}}_{2}]}({{\vec {v}}\,})=1\cdot {\vec {e}}_{1}+2\cdot {\vec {e}}_{2}$ is in gray.

   

   The vector ${\vec {v}}-({\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {v}}\,})+{\mbox{proj}}_{[{\vec {v}}_{2}]}({{\vec {v}}\,}))$ lies on the dotted line connecting the black vector to the gray one, that is, it is orthogonal to the $xy$-plane.

3. This diagram is gotten by following the hint.

   

   The dashed triangle has a right angle where the gray vector $1\cdot {\vec {e}}_{1}+2\cdot {\vec {e}}_{2}$ meets the vertical dashed line ${\vec {v}}-(1\cdot {\vec {e}}_{1}+2\cdot {\vec {e}}_{2})$; this is what was proved in the first item of this question. The Pythagorean theorem then gives that the hypoteneuse— the segment from ${\vec {v}}$ to any other vector— is longer than the vertical dashed line.

   More formally, writing ${\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {v}}\,})+\dots +{\mbox{proj}}_{[{\vec {v}}_{k}]}({{\vec {v}}\,})$ as $c_{1}\cdot {\vec {\kappa }}_{1}+\dots +c_{k}\cdot {\vec {\kappa }}_{k}$, consider any other vector in the span $d_{1}\cdot {\vec {\kappa }}_{1}+\dots +d_{k}\cdot {\vec {\kappa }}_{k}$. Note that

   ${\vec {v}}-(d_{1}\cdot {\vec {\kappa }}_{1}+\dots +d_{k}\cdot {\vec {\kappa }}_{k})$

   $={\bigl (}{\vec {v}}-(c_{1}\cdot {\vec {\kappa }}_{1}+\dots +c_{k}\cdot {\vec {\kappa }}_{k}){\bigr )}+{\bigl (}(c_{1}\cdot {\vec {\kappa }}_{1}+\dots +c_{k}\cdot {\vec {\kappa }}_{k})-(d_{1}\cdot {\vec {\kappa }}_{1}+\dots +d_{k}\cdot {\vec {\kappa }}_{k}){\bigr )}$

   and that

   ${\bigl (}{\vec {v}}-(c_{1}\cdot {\vec {\kappa }}_{1}+\dots +c_{k}\cdot {\vec {\kappa }}_{k}){\bigr )}\cdot {\bigl (}(c_{1}\cdot {\vec {\kappa }}_{1}+\dots +c_{k}\cdot {\vec {\kappa }}_{k})-(d_{1}\cdot {\vec {\kappa }}_{1}+\dots +d_{k}\cdot {\vec {\kappa }}_{k}){\bigr )}=0$

   (because the first item shows the ${\vec {v}}-(c_{1}\cdot {\vec {\kappa }}_{1}+\dots +c_{k}\cdot {\vec {\kappa }}_{k})$ is orthogonal to each ${\vec {\kappa }}$ and so it is orthogonal to this linear combination of the ${\vec {\kappa }}$'s). Now apply the Pythagorean Theorem (i.e., the Triangle Inequality).

Problem 8

Find a vector in $\mathbb {R} ^{3}$ that is orthogonal to both of these.

${\begin{pmatrix}1\\5\\-1\end{pmatrix}}\qquad {\begin{pmatrix}2\\2\\0\end{pmatrix}}$

Answer

One way to proceed is to find a third vector so that the three together make a basis for $\mathbb {R} ^{3}$, e.g.,

${\vec {\beta }}_{3}={\begin{pmatrix}1\\0\\0\end{pmatrix}}$

(the second vector is not dependent on the third because it has a nonzero second component, and the first is not dependent on the second and third because of its nonzero third component), and then apply the Gram-Schmidt process.

${\begin{array}{rl}{\vec {\kappa }}_{1}&={\begin{pmatrix}1\\5\\-1\end{pmatrix}}\\{\vec {\kappa }}_{2}&={\begin{pmatrix}2\\2\\0\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}2\\2\\0\end{pmatrix}})={\begin{pmatrix}2\\2\\0\end{pmatrix}}-{\frac {{\begin{pmatrix}2\\2\\0\end{pmatrix}}\cdot {\begin{pmatrix}1\\5\\-1\end{pmatrix}}}{{\begin{pmatrix}1\\5\\-1\end{pmatrix}}\cdot {\begin{pmatrix}1\\5\\-1\end{pmatrix}}}}\cdot {\begin{pmatrix}1\\5\\-1\end{pmatrix}}\\&\quad ={\begin{pmatrix}2\\2\\0\end{pmatrix}}-{\frac {12}{27}}\cdot {\begin{pmatrix}1\\5\\-1\end{pmatrix}}={\begin{pmatrix}14/9\\-2/9\\4/9\end{pmatrix}}\\{\vec {\kappa }}_{3}&={\begin{pmatrix}1\\0\\0\end{pmatrix}}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\begin{pmatrix}1\\0\\0\end{pmatrix}})-{\mbox{proj}}_{[{\vec {\kappa }}_{2}]}({\begin{pmatrix}1\\0\\0\end{pmatrix}})\\&\quad ={\begin{pmatrix}1\\0\\0\end{pmatrix}}-{\frac {{\begin{pmatrix}1\\0\\0\end{pmatrix}}\cdot {\begin{pmatrix}1\\5\\-1\end{pmatrix}}}{{\begin{pmatrix}1\\5\\-1\end{pmatrix}}\cdot {\begin{pmatrix}1\\5\\-1\end{pmatrix}}}}\cdot {\begin{pmatrix}1\\5\\-1\end{pmatrix}}-{\frac {{\begin{pmatrix}1\\0\\0\end{pmatrix}}\cdot {\begin{pmatrix}14/9\\-2/9\\4/9\end{pmatrix}}}{{\begin{pmatrix}14/9\\-2/9\\4/9\end{pmatrix}}\cdot {\begin{pmatrix}14/9\\-2/9\\4/9\end{pmatrix}}}}\cdot {\begin{pmatrix}14/9\\-2/9\\4/9\end{pmatrix}}\\&\quad ={\begin{pmatrix}1\\0\\0\end{pmatrix}}-{\frac {1}{27}}\cdot {\begin{pmatrix}1\\5\\-1\end{pmatrix}}-{\frac {7}{12}}\cdot {\begin{pmatrix}14/9\\-2/9\\4/9\end{pmatrix}}={\begin{pmatrix}1/18\\-1/18\\-4/18\end{pmatrix}}\end{array}}$

The result ${\vec {\kappa }}_{3}$ is orthogonal to both ${\vec {\kappa }}_{1}$ and ${\vec {\kappa }}_{2}$. It is therefore orthogonal to every vector in the span of the set $\{{\vec {\kappa }}_{1},{\vec {\kappa }}_{2}\}$, including the two vectors given in the question.

This exercise is recommended for all readers.

Problem 9

One advantage of orthogonal bases is that they simplify finding the representation of a vector with respect to that basis.

1. For this vector and this non-orthogonal basis for $\mathbb {R} ^{2}$

   ${\vec {v}}={\begin{pmatrix}2\\3\end{pmatrix}}\qquad B=\left\langle {\begin{pmatrix}1\\1\end{pmatrix}},{\begin{pmatrix}1\\0\end{pmatrix}}\right\rangle$

   first represent the vector with respect to the basis. Then project the vector onto the span of each basis vector $[{{\vec {\beta }}_{1}}]$ and $[{{\vec {\beta }}_{2}}]$.
2. With this orthogonal basis for $\mathbb {R} ^{2}$

   $K=\left\langle {\begin{pmatrix}1\\1\end{pmatrix}},{\begin{pmatrix}1\\-1\end{pmatrix}}\right\rangle$

   represent the same vector ${\vec {v}}$ with respect to the basis. Then project the vector onto the span of each basis vector. Note that the coefficients in the representation and the projection are the same.
3. Let $K=\left\langle {\vec {\kappa }}_{1},\ldots ,{\vec {\kappa }}_{k}\right\rangle$ be an orthogonal basis for some subspace of $\mathbb {R} ^{n}$. Prove that for any ${\vec {v}}$ in the subspace, the $i$-th component of the representation ${\rm {Rep}}_{K}({\vec {v}}\,)$ is the scalar coefficient $({\vec {v}}\cdot {\vec {\kappa }}_{i})/({\vec {\kappa }}_{i}\cdot {\vec {\kappa }}_{i})$ from ${\mbox{proj}}_{[{\vec {\kappa }}_{i}]}({{\vec {v}}\,})$.
4. Prove that ${\vec {v}}={\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {v}}\,})+\dots +{\mbox{proj}}_{[{\vec {\kappa }}_{k}]}({{\vec {v}}\,})$.

Answer

1. The representation can be done by eye.

   ${\begin{pmatrix}2\\3\end{pmatrix}}=3\cdot {\begin{pmatrix}1\\1\end{pmatrix}}+(-1)\cdot {\begin{pmatrix}1\\0\end{pmatrix}}\qquad {\rm {Rep}}_{B}({\vec {v}}\,)={\begin{pmatrix}3\\-1\end{pmatrix}}_{B}$

   The two projections are also easy.

   ${\mbox{proj}}_{[{\vec {\beta }}_{1}]}({\begin{pmatrix}2\\3\end{pmatrix}})={\frac {{\begin{pmatrix}2\\3\end{pmatrix}}\cdot {\begin{pmatrix}1\\1\end{pmatrix}}}{{\begin{pmatrix}1\\1\end{pmatrix}}\cdot {\begin{pmatrix}1\\1\end{pmatrix}}}}\cdot {\begin{pmatrix}1\\1\end{pmatrix}}={\frac {5}{2}}\cdot {\begin{pmatrix}1\\1\end{pmatrix}}\qquad {\mbox{proj}}_{[{\vec {\beta }}_{2}]}({\begin{pmatrix}2\\3\end{pmatrix}})={\frac {{\begin{pmatrix}2\\3\end{pmatrix}}\cdot {\begin{pmatrix}1\\0\end{pmatrix}}}{{\begin{pmatrix}1\\0\end{pmatrix}}\cdot {\begin{pmatrix}1\\0\end{pmatrix}}}}\cdot {\begin{pmatrix}1\\0\end{pmatrix}}={\frac {2}{1}}\cdot {\begin{pmatrix}1\\0\end{pmatrix}}$

2. As above, the representation can be done by eye

   ${\begin{pmatrix}2\\3\end{pmatrix}}=(5/2)\cdot {\begin{pmatrix}1\\1\end{pmatrix}}+(-1/2)\cdot {\begin{pmatrix}1\\-1\end{pmatrix}}$

   and the two projections are easy.

   ${\mbox{proj}}_{[{\vec {\beta }}_{1}]}({\begin{pmatrix}2\\3\end{pmatrix}})={\frac {{\begin{pmatrix}2\\3\end{pmatrix}}\cdot {\begin{pmatrix}1\\1\end{pmatrix}}}{{\begin{pmatrix}1\\1\end{pmatrix}}\cdot {\begin{pmatrix}1\\1\end{pmatrix}}}}\cdot {\begin{pmatrix}1\\1\end{pmatrix}}={\frac {5}{2}}\cdot {\begin{pmatrix}1\\1\end{pmatrix}}\qquad {\mbox{proj}}_{[{\vec {\beta }}_{2}]}({\begin{pmatrix}2\\3\end{pmatrix}})={\frac {{\begin{pmatrix}2\\3\end{pmatrix}}\cdot {\begin{pmatrix}1\\-1\end{pmatrix}}}{{\begin{pmatrix}1\\-1\end{pmatrix}}\cdot {\begin{pmatrix}1\\-1\end{pmatrix}}}}\cdot {\begin{pmatrix}1\\-1\end{pmatrix}}={\frac {-1}{2}}\cdot {\begin{pmatrix}1\\-1\end{pmatrix}}$

   Note the recurrence of the $5/2$ and the $-1/2$.
3. Represent ${\vec {v}}$ with respect to the basis

   ${\rm {Rep}}_{K}({\vec {v}}\,)={\begin{pmatrix}r_{1}\\\vdots \\r_{k}\end{pmatrix}}$

   so that ${\vec {v}}=r_{1}{\vec {\kappa }}_{1}+\dots +r_{k}{\vec {\kappa }}_{k}$. To determine $r_{i}$, take the dot product of both sides with ${\vec {\kappa }}_{i}$.

   ${\vec {v}}\cdot {\vec {\kappa }}_{i}=\left(r_{1}{\vec {\kappa }}_{1}+\dots +r_{k}{\vec {\kappa }}_{k}\right)\cdot {\vec {\kappa }}_{i}=r_{1}\cdot 0+\dots +r_{i}\cdot ({\vec {\kappa }}_{i}\cdot {\vec {\kappa }}_{i})+\dots +r_{k}\cdot 0$

   Solving for $r_{i}$ yields the desired coefficient.
4. This is a restatement of the prior item.

Problem 10

Bessel's Inequality. Consider these orthonormal sets

$B_{1}=\{{\vec {e}}_{1}\}\quad B_{2}=\{{\vec {e}}_{1},{\vec {e}}_{2}\}\quad B_{3}=\{{\vec {e}}_{1},{\vec {e}}_{2},{\vec {e}}_{3}\}\quad B_{4}=\{{\vec {e}}_{1},{\vec {e}}_{2},{\vec {e}}_{3},{\vec {e}}_{4}\}$

along with the vector ${\vec {v}}\in \mathbb {R} ^{4}$ whose components are $4$, $3$, $2$, and $1$.

1. Find the coefficient $c_{1}$ for the projection of ${\vec {v}}$ onto the span of the vector in $B_{1}$. Check that $|{\vec {v}}\,|^{2}\geq |c_{1}|^{2}$.
2. Find the coefficients $c_{1}$ and $c_{2}$ for the projection of ${\vec {v}}$ onto the spans of the two vectors in $B_{2}$. Check that $|{\vec {v}}\,|^{2}\geq |c_{1}|^{2}+|c_{2}|^{2}$.
3. Find $c_{1}$, $c_{2}$, and $c_{3}$ associated with the vectors in $B_{3}$, and $c_{1}$, $c_{2}$, $c_{3}$, and $c_{4}$ for the vectors in $B_{4}$. Check that $|{\vec {v}}\,|^{2}\geq |c_{1}|^{2}+\dots +|c_{3}|^{2}$ and that $|{\vec {v}}\,|^{2}\geq |c_{1}|^{2}+\dots +|c_{4}|^{2}$.

Show that this holds in general: where $\{{\vec {\kappa }}_{1},\ldots ,{\vec {\kappa }}_{k}\}$ is an orthonormal set and $c_{i}$ is coefficient of the projection of a vector ${\vec {v}}$ from the space then $|{\vec {v}}\,|^{2}\geq |c_{1}|^{2}+\dots +|c_{k}|^{2}$. Hint. One way is to look at the inequality $0\leq |{\vec {v}}-(c_{1}{\vec {\kappa }}_{1}+\dots +c_{k}{\vec {\kappa }}_{k})|^{2}$ and expand the $c$'s.

Answer

First, $|{\vec {v}}\,|^{2}=4^{2}+3^{2}+2^{2}+1^{2}=50$.

1. $c_{1}=4$
2. $c_{1}=4$, $c_{2}=3$
3. $c_{1}=4$, $c_{2}=3$, $c_{3}=2$, $c_{4}=1$

For the proof, we will do only the $k=2$ case because the completely general case is messier but no more enlightening. We follow the hint (recall that for any vector ${\vec {w}}$ we have $|{\vec {w}}\,|^{2}={\vec {w}}\cdot {\vec {w}}$).

${\begin{array}{rl}0&\leq \left({\vec {v}}-{\bigl (}{\frac {{\vec {v}}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\kappa }}_{1}+{\frac {{\vec {v}}\cdot {\vec {\kappa }}_{2}}{{\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}}}\cdot {\vec {\kappa }}_{2}{\bigr )}\right)\cdot \left({\vec {v}}-{\bigl (}{\frac {{\vec {v}}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\kappa }}_{1}+{\frac {{\vec {v}}\cdot {\vec {\kappa }}_{2}}{{\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}}}\cdot {\vec {\kappa }}_{2}{\bigr )}\right)\\&={\begin{aligned}&{\vec {v}}\cdot {\vec {v}}-2\cdot {\vec {v}}\cdot \left({\frac {{\vec {v}}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\kappa }}_{1}+{\frac {{\vec {v}}\cdot {\vec {\kappa }}_{2}}{{\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}}}\cdot {\vec {\kappa }}_{2}\right)\\&+\left({\frac {{\vec {v}}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\kappa }}_{1}+{\frac {{\vec {v}}\cdot {\vec {\kappa }}_{2}}{{\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}}}\cdot {\vec {\kappa }}_{2}\right)\cdot \left({\frac {{\vec {v}}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\kappa }}_{1}+{\frac {{\vec {v}}\cdot {\vec {\kappa }}_{2}}{{\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}}}\cdot {\vec {\kappa }}_{2}\right)\end{aligned}}\\&={\vec {v}}\cdot {\vec {v}}-2\cdot \left({\frac {{\vec {v}}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot ({\vec {v}}\cdot {\vec {\kappa }}_{1})+{\frac {{\vec {v}}\cdot {\vec {\kappa }}_{2}}{{\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}}}\cdot ({\vec {v}}\cdot {\vec {\kappa }}_{2})\right)+\left(({\frac {{\vec {v}}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}})^{2}\cdot ({\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1})+({\frac {{\vec {v}}\cdot {\vec {\kappa }}_{2}}{{\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}}})^{2}\cdot ({\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2})\right)\end{array}}$

(The two mixed terms in the third part of the third line are zero because ${\vec {\kappa }}_{1}$ and ${\vec {\kappa }}_{2}$ are orthogonal.) The result now follows on gathering like terms and on recognizing that ${\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}=1$ and ${\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}=1$ because these vectors are given as members of an orthonormal set.

Problem 11

Prove or disprove: every vector in $\mathbb {R} ^{n}$ is in some orthogonal basis.

Answer

It is true, except for the zero vector. Every vector in $\mathbb {R} ^{n}$ except the zero vector is in a basis, and that basis can be orthogonalized.

Problem 12

Show that the columns of an $n\!\times \!n$ matrix form an orthonormal set if and only if the inverse of the matrix is its transpose. Produce such a matrix.

Answer

The $3\!\times \!3$ case gives the idea. The set

$\{{\begin{pmatrix}a\\d\\g\end{pmatrix}},{\begin{pmatrix}b\\e\\h\end{pmatrix}},{\begin{pmatrix}c\\f\\i\end{pmatrix}}\}$

is orthonormal if and only if these nine conditions all hold

${\begin{array}{ccc}{\begin{pmatrix}a&d&g\end{pmatrix}}\cdot {\begin{pmatrix}a\\d\\g\end{pmatrix}}=1&{\begin{pmatrix}a&d&g\end{pmatrix}}\cdot {\begin{pmatrix}b\\e\\h\end{pmatrix}}=0&{\begin{pmatrix}a&d&g\end{pmatrix}}\cdot {\begin{pmatrix}c\\f\\i\end{pmatrix}}=0\\{\begin{pmatrix}b&e&h\end{pmatrix}}\cdot {\begin{pmatrix}a\\d\\g\end{pmatrix}}=0&{\begin{pmatrix}b&e&h\end{pmatrix}}\cdot {\begin{pmatrix}b\\e\\h\end{pmatrix}}=1&{\begin{pmatrix}b&e&h\end{pmatrix}}\cdot {\begin{pmatrix}c\\f\\i\end{pmatrix}}=0\\{\begin{pmatrix}c&f&i\end{pmatrix}}\cdot {\begin{pmatrix}a\\d\\g\end{pmatrix}}=0&{\begin{pmatrix}c&f&i\end{pmatrix}}\cdot {\begin{pmatrix}b\\e\\h\end{pmatrix}}=0&{\begin{pmatrix}c&f&i\end{pmatrix}}\cdot {\begin{pmatrix}c\\f\\i\end{pmatrix}}=1\end{array}}$

(the three conditions in the lower left are redundant but nonetheless correct). Those, in turn, hold if and only if

${\begin{pmatrix}a&d&g\\b&e&h\\c&f&i\end{pmatrix}}{\begin{pmatrix}a&b&c\\d&e&f\\g&h&i\end{pmatrix}}={\begin{pmatrix}1&0&0\\0&1&0\\0&0&1\end{pmatrix}}$

as required.

This is an example, the inverse of this matrix is its transpose.

${\begin{pmatrix}1/{\sqrt {2}}&1/{\sqrt {2}}&0\\-1/{\sqrt {2}}&1/{\sqrt {2}}&0\\0&0&1\end{pmatrix}}$

Problem 13

Does the proof of Theorem 2.2 fail to consider the possibility that the set of vectors is empty (i.e., that $k=0$)?

Answer

If the set is empty then the summation on the left side is the linear combination of the empty set of vectors, which by definition adds to the zero vector. In the second sentence, there is not such $i$, so the "if ... then ..." implication is vacuously true.

Problem 14

Theorem 2.7 describes a change of basis from any basis $B=\left\langle {\vec {\beta }}_{1},\ldots ,{\vec {\beta }}_{k}\right\rangle$ to one that is orthogonal $K=\left\langle {\vec {\kappa }}_{1},\ldots ,{\vec {\kappa }}_{k}\right\rangle$. Consider the change of basis matrix ${\rm {Rep}}_{B,K}({\mbox{id}})$.

1. Prove that the matrix ${\rm {Rep}}_{K,B}({\mbox{id}})$ changing bases in the direction opposite to that of the theorem has an upper triangular shape— all of its entries below the main diagonal are zeros.
2. Prove that the inverse of an upper triangular matrix is also upper triangular (if the matrix is invertible, that is). This shows that the matrix ${\rm {Rep}}_{B,K}({\mbox{id}})$ changing bases in the direction described in the theorem is upper triangular.

Answer

1. Part of the induction argument proving Theorem 2.7 checks that ${\vec {\kappa }}_{i}$ is in the span of $\left\langle {\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{i}\right\rangle$. (The $i=3$ case in the proof illustrates.) Thus, in the change of basis matrix ${\rm {Rep}}_{K,B}({\mbox{id}})$, the $i$-th column ${\rm {Rep}}_{B}({\vec {\kappa }}_{i})$ has components $i+1$ through $k$ that are zero.
2. One way to see this is to recall the computational procedure that we use to find the inverse. We write the matrix, write the identity matrix next to it, and then we do Gauss-Jordan reduction. If the matrix starts out upper triangular then the Gauss-Jordan reduction involves only the Jordan half and these steps, when performed on the identity, will result in an upper triangular inverse matrix.

Problem 15

Complete the induction argument in the proof of Theorem 2.7.

Answer

For the inductive step, we assume that for all $j$ in $[1..i]$, these three conditions are true of each ${\vec {\kappa }}_{j}$: (i) each ${\vec {\kappa }}_{j}$ is nonzero, (ii) each ${\vec {\kappa }}_{j}$ is a linear combination of the vectors ${\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{j}$, and (iii) each ${\vec {\kappa }}_{j}$ is orthogonal to all of the ${\vec {\kappa }}_{m}$'s prior to it (that is, with $m<j$). With those inductive hypotheses, consider ${\vec {\kappa }}_{i+1}$.

${\begin{array}{rl}{\vec {\kappa }}_{i+1}&={\vec {\beta }}_{i+1}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({\beta _{i+1}})-{\mbox{proj}}_{[{\vec {\kappa }}_{2}]}({\beta _{i+1}})-\dots -{\mbox{proj}}_{[{\vec {\kappa }}_{i}]}({\beta _{i+1}})\\&={\vec {\beta }}_{i+1}-{\frac {\beta _{i+1}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\kappa }}_{1}-{\frac {\beta _{i+1}\cdot {\vec {\kappa }}_{2}}{{\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}}}\cdot {\vec {\kappa }}_{2}-\dots -{\frac {\beta _{i+1}\cdot {\vec {\kappa }}_{i}}{{\vec {\kappa }}_{i}\cdot {\vec {\kappa }}_{i}}}\cdot {\vec {\kappa }}_{i}\end{array}}$

By the inductive assumption (ii) we can expand each ${\vec {\kappa }}_{j}$ into a linear combination of ${\vec {\beta }}_{1},\ldots ,{\vec {\beta }}_{j}$

$={\vec {\beta }}_{i+1}-{\frac {{\vec {\beta }}_{i+1}\cdot {\vec {\kappa }}_{1}}{{\vec {\kappa }}_{1}\cdot {\vec {\kappa }}_{1}}}\cdot {\vec {\beta }}_{1}$

$-{\frac {{\vec {\beta }}_{i+1}\cdot {\vec {\kappa }}_{2}}{{\vec {\kappa }}_{2}\cdot {\vec {\kappa }}_{2}}}\cdot \left({\text{linear combination of }}{\vec {\beta }}_{1},{\vec {\beta }}_{2}\right)-\dots -{\frac {{\vec {\beta }}_{i+1}\cdot {\vec {\kappa }}_{i}}{{\vec {\kappa }}_{i}\cdot {\vec {\kappa }}_{i}}}\cdot \left({\text{linear combination of }}{\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{i}\right)$

The fractions are scalars so this is a linear combination of linear combinations of ${\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{i+1}$. It is therefore just a linear combination of ${\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{i+1}$. Now, (i) it cannot sum to the zero vector because the equation would then describe a nontrivial linear relationship among the ${\vec {\beta }}$'s that are given as members of a basis (the relationship is nontrivial because the coefficient of ${\vec {\beta }}_{i+1}$ is $1$). Also, (ii) the equation gives ${\vec {\kappa }}_{i+1}$ as a combination of ${\vec {\beta }}_{1},\dots ,{\vec {\beta }}_{i+1}$. Finally, for (iii), consider ${\vec {\kappa }}_{j}\cdot {\vec {\kappa }}_{i+1}$; as in the $i=3$ case, the dot product of ${\vec {\kappa }}_{j}$ with ${\vec {\kappa }}_{i+1}={\vec {\beta }}_{i+1}-{\mbox{proj}}_{[{\vec {\kappa }}_{1}]}({{\vec {\beta }}_{i+1}})-\dots -{\mbox{proj}}_{[{\vec {\kappa }}_{i}]}({{\vec {\beta }}_{i+1}})$ can be rewritten to give two kinds of terms, ${\vec {\kappa }}_{j}\cdot \left({\vec {\beta }}_{i+1}-{\mbox{proj}}_{[{\vec {\kappa }}_{j}]}({{\vec {\beta }}_{i+1}})\right)$ (which is zero because the projection is orthogonal) and ${\vec {\kappa }}_{j}\cdot {\mbox{proj}}_{[{\vec {\kappa }}_{m}]}({{\vec {\beta }}_{i+1}})$ with $m\neq j$ and $m<i+1$ (which is zero because by the hypothesis (iii) the vectors ${\vec {\kappa }}_{j}$ and ${\vec {\kappa }}_{m}$ are orthogonal).