Orthogonal Projections and Orthonormal Basis

Question

Question #1.

a) For all mxn matrices A such that A^tA is invertible, show that P=A(A^tA)^-1A^t is an orthogonal projection.

b) Explain why, if u is an nx1 unit column vector in Rn, part a) shows that uu^t is an orthogonal projection.

Question #2. Let P be a fixed but arbitrary orthogonal projection in Mn(R) and let I denote the nxn identity matrix.

a) If P^2 = P show that (I-P)^2 = I-P.

b) Additionally show that (I-P)^t = I-P. This shows that if P is an orthogonal projection, so is I-P

Question #3. If {w1,w2,...,wk} is an orthonormal basis for a subspace W of Rn and P = the sum from j to k of (wj)(wj)^t, show that P is an orthogonal projection. Show that Ran(P) = W, so that every subspace in Rn admits an orthogonal projection onto it.

Answer 1

A transformation P is a projection if it is idempotent: P²=P.

P is orthogonal if in addition P^t = P.

So, to show that the given P is an orthogonal projection, we need to verify that both of these are true.

Try applying the transpose to the expression for P given in part 1a, and verify that P^t=P. For this it is helpful to refer to algebraic properties of the transpose operation. Then show that multiplying P by itself gives you an expression that reduces to P.

P² = P*P = (A(A^tA)^-1A^t) * (A(A^tA)^-1A^t)

Can you see anything in the middle that simplifies when you move parentheses around?

Question 2 follows in a similar fashion.

1b) Set A = u and appeal to part 1a. Note that since u is a nx1 column vector, u^t is a 1xn row vector, so u^tu is a 1x1 matrix. Moreover, if we look at this matrix product in coordinates, we see

u^tu = u₁² + ... + u_n²

This is just 1, since we've assumed that u is a unit vector. This tells us that for A=u, the product A^tA = u^tu is invertible. Now apply part (1a).