6.2: Orthogonal Complements (2024)

Definition \(\PageIndex{1}\): Orthogonal Complement

Let \(W\) be a subspace of \(\mathbb{R}^n \). Its orthogonal complement is the subspace

\[ W^\perp = \bigl\{ \text{$v$ in $\mathbb{R}^n $}\mid v\cdot w=0 \text{ for all $w$ in $W$} \bigr\}. \nonumber \]

The symbol \(W^\perp\) is sometimes read “\(W\) perp.”

Note \(\PageIndex{1}\)

We now have two similar-looking pieces of notation:

\[ \begin{split} A^{\color{Red}T} \amp\text{ is the transpose of a matrix $A$}. \\ W^{\color{Red}\perp} \amp\text{ is the orthogonal complement of a subspace $W$}. \end{split} \nonumber \]

Try not to confuse the two.

Proposition \(\PageIndex{1}\): The Orthogonal Complement of a Column Space

Let \(A\) be a matrix and let \(W=\text{Col}(A)\). Then

\[ W^\perp = \text{Nul}(A^T). \nonumber \]

Proof

To justify the first equality, we need to show that a vector \(x\) is perpendicular to the all of the vectors in \(W\) if and only if it is perpendicular only to \(v_1,v_2,\ldots,v_m\). Since the \(v_i\) are contained in \(W\text{,}\) we really only have to show that if \(x\cdot v_1 = x\cdot v_2 = \cdots = x\cdot v_m = 0\text{,}\) then \(x\) is perpendicular to every vector \(v\) in \(W\). Indeed, any vector in \(W\) has the form \(v = c_1v_1 + c_2v_2 + \cdots + c_mv_m\) for suitable scalars \(c_1,c_2,\ldots,c_m\text{,}\) so

\[ \begin{split} x\cdot v \amp= x\cdot(c_1v_1 + c_2v_2 + \cdots + c_mv_m) \\ \amp= c_1(x\cdot v_1) + c_2(x\cdot v_2) + \cdots + c_m(x\cdot v_m) \\ \amp= c_1(0) + c_2(0) + \cdots + c_m(0) = 0. \end{split} \nonumber \]

Therefore, \(x\) is in \(W^\perp.\)

To prove the second equality, we let

\[ A = \left(\begin{array}{c}—v_1^T— \\ —v_2^T— \\ \vdots \\ —v_m^T—\end{array}\right). \nonumber \]

By the row-column rule for matrix multiplication Definition 2.3.3 in Section 2.3, for any vector \(x\) in \(\mathbb{R}^n \) we have

\[ Ax = \left(\begin{array}{c}v_1^Tx \\ v_2^Tx\\ \vdots\\ v_m^Tx\end{array}\right) = \left(\begin{array}{c}v_1\cdot x\\ v_2\cdot x\\ \vdots \\ v_m\cdot x\end{array}\right). \nonumber \]

Therefore, \(x\) is in \(\text{Nul}(A)\) if and only if \(x\) is perpendicular to each vector \(v_1,v_2,\ldots,v_m\).

Example \(\PageIndex{5}\)

Compute \(W^\perp\text{,}\) where

\[ W = \text{Span}\left\{\left(\begin{array}{c}1\\7\\2\end{array}\right),\;\left(\begin{array}{c}-2\\3\\1\end{array}\right)\right\}. \nonumber \]

Example \(\PageIndex{6}\)

Find all vectors orthogonal to \(v = \left(\begin{array}{c}1\\1\\-1\end{array}\right).\)

Example \(\PageIndex{7}\)

Compute

\[ \text{Span}\left\{\left(\begin{array}{c}1\\1\\-1\end{array}\right),\;\left(\begin{array}{c}1\\1\\1\end{array}\right)\right\}^\perp. \nonumber \]

Fact \(\PageIndex{1}\): Facts about Orthogonal Complements

Let \(W\) be a subspace of \(\mathbb{R}^n \). Then:

1. \(W^\perp\) is also a subspace of \(\mathbb{R}^n .\)
2. \((W^\perp)^\perp = W.\)
3. \(\dim(W) + \dim(W^\perp) = n.\)

Proof

For the first assertion, we verify the three defining properties of subspaces, Definition 2.6.2in Section 2.6.

1. The zero vector is in \(W^\perp\) because the zero vector is orthogonal to every vector in \(\mathbb{R}^n \).
2. Let \(u,v\) be in \(W^\perp\text{,}\) so \(u\cdot x = 0\) and \(v\cdot x = 0\) for every vector \(x\) in \(W\). We must verify that \((u+v)\cdot x = 0\) for every \(x\) in \(W\). Indeed, we have \[ (u+v)\cdot x = u\cdot x + v\cdot x = 0 + 0 = 0. \nonumber \]
3. Let \(u\) be in \(W^\perp\text{,}\) so \(u\cdot x = 0\) for every \(x\) in \(W\text{,}\) and let \(c\) be a scalar. We must verify that \((cu)\cdot x = 0\) for every \(x\) in \(W\). Indeed, we have \[ (cu)\cdot x = c(u\cdot x) = c0 = 0. \nonumber \]

Next we prove the third assertion. Let \(v_1,v_2,\ldots,v_m\) be a basis for \(W\text{,}\) so \(m = \dim(W)\text{,}\) and let \(v_{m+1},v_{m+2},\ldots,v_k\) be a basis for \(W^\perp\text{,}\) so \(k-m = \dim(W^\perp)\). We need to show \(k=n\). First we claim that \(\{v_1,v_2,\ldots,v_m,v_{m+1},v_{m+2},\ldots,v_k\}\) is linearly independent. Suppose that \(c_1v_1 + c_2v_2 + \cdots + c_kv_k = 0\). Let \(w = c_1v_1 + c_2v_2 + \cdots + c_mv_m\) and \(w' = c_{m+1}v_{m+1} + c_{m+2}v_{m+2} + \cdots + c_kv_k\text{,}\) so \(w\) is in \(W\text{,}\) \(w'\) is in \(W'\text{,}\) and \(w + w' = 0\). Then \(w = -w'\) is in both \(W\) and \(W^\perp\text{,}\) which implies \(w\) is perpendicular to itself. In particular, \(w\cdot w = 0\text{,}\) so \(w = 0\text{,}\) and hence \(w' = 0\). Therefore, all coefficients \(c_i\) are equal to zero, because \(\{v_1,v_2,\ldots,v_m\}\) and \(\{v_{m+1},v_{m+2},\ldots,v_k\}\) are linearly independent.

It follows from the previous paragraph that \(k \leq n\). Suppose that \(k \lt n\). Then the matrix

\[ A = \left(\begin{array}{c}—v_1^T— \\—v_2^T— \\ \vdots \\—v_k^T—\end{array}\right)\nonumber \]

has more columns than rows (it is “wide”), so its null space is nonzero by Note3.2.1in Section 3.2. Let \(x\) be a nonzero vector in \(\text{Nul}(A)\). Then

\[ 0 = Ax = \left(\begin{array}{c}v_1^Tx \\ v_2^Tx \\ \vdots \\ v_k^Tx\end{array}\right)= \left(\begin{array}{c}v_1\cdot x\\ v_2\cdot x\\ \vdots \\ v_k\cdot x\end{array}\right)\nonumber \]

by the row-column rule for matrix multiplication Definition 2.3.3in Section 2.3. Since \(v_1\cdot x = v_2\cdot x = \cdots = v_m\cdot x = 0\text{,}\) it follows from Proposition \(\PageIndex{1}\)that \(x\) is in \(W^\perp\text{,}\) and similarly, \(x\) is in \((W^\perp)^\perp\). As above, this implies \(x\) is orthogonal to itself, which contradicts our assumption that \(x\) is nonzero. Therefore, \(k = n\text{,}\) as desired.

Finally, we prove the second assertion. Clearly \(W\) is contained in \((W^\perp)^\perp\text{:}\) this says that everything in \(W\) is perpendicular to the set of all vectors perpendicular to everything in \(W\). Let \(m=\dim(W).\) By 3, we have \(\dim(W^\perp) = n-m\text{,}\) so \(\dim((W^\perp)^\perp) = n - (n-m) = m\). The only \(m\)-dimensional subspace of \((W^\perp)^\perp\) is all of \((W^\perp)^\perp\text{,}\) so \((W^\perp)^\perp = W.\)

Definition \(\PageIndex{2}\): Row Space

The row space of a matrix \(A\) is the span of the rows of \(A\text{,}\) and is denoted \(\text{Row}(A)\).

Recipe: Shortcuts for Computing Orthogonal Complements

For any vectors \(v_1,v_2,\ldots,v_m\text{,}\) we have

\[ \text{Span}\{v_1,v_2,\ldots,v_m\}^\perp = \text{Nul}\left(\begin{array}{c}—v_1^T— \\—v_2^T— \\ \vdots \\—v_m^T—\end{array}\right) . \nonumber \]

For any matrix \(A\text{,}\) we have

\[ \begin{aligned} \text{Row}(A)^\perp &= \text{Nul}(A) & \text{Nul}(A)^\perp &= \text{Row}(A) \\ \text{Col}(A)^\perp &= \text{Nul}(A^T)\quad & \text{Nul}(A^T)^\perp &= \text{Col}(A). \end{aligned} \nonumber \]

Example \(\PageIndex{8}\): Orthogonal complement of a subspace

Compute the orthogonal complement of the subspace

\[ W = \bigl\{(x,y,z) \text{ in } \mathbb{R}^3 \mid 3x + 2y = z\bigr\}. \nonumber \]

Example \(\PageIndex{9}\): Orthogonal complement of an eigenspace

Find the orthogonal complement of the \(5\)-eigenspace of the matrix

\[A=\left(\begin{array}{ccc}2&4&-1\\3&2&0\\-2&4&3\end{array}\right).\nonumber\]

Theorem \(\PageIndex{1}\)

Let \(A\) be a matrix. Then the row rank of \(A\) is equal to the column rank of \(A\).

Proof

By Theorem2.9.1in Section 2.9, we have

\[ \dim\text{Col}(A) + \dim\text{Nul}(A) = n. \nonumber \]

On the other hand the third fact \(\PageIndex{1}\)says that

\[ \dim\text{Nul}(A)^\perp + \dim\text{Nul}(A) = n, \nonumber \]

which implies \(\dim\text{Col}(A) = \dim\text{Nul}(A)^\perp\). Since \(\text{Nul}(A)^\perp = \text{Row}(A),\) we have

\[ \dim\text{Col}(A) = \dim\text{Row}(A)\text{,} \nonumber \]

as desired.