Lecture4

revised June 28, 2021 1 Math S-21b - Lecture #4 Notes In this lecture we define and study subspaces of R n , the span of a collection of vectors, and what it means for a collection of vectors to be linearly independent. In particular, we'll focus on the kernel and image of a linear transformation - subspaces of the domain and codomain, respectively - to motivate the definitions and methods. We'll trim down a redundant spanning set for a given subspace to get a minimal spanning set or basis for a subspace , and we prove that the number of spanning vectors in any basis for a given subspace is always the same - the dimension of the subspace . We then define the rank and nullity of a matrix (and of the linear transformation that it represents). Subspaces of n R Definition : A subspace V of n R is a subset that is closed under vector addition and scalar multiplication. That is, for any vectors 1 2 , V ∈ v v and scalars 1 2 , c c , it must be the case that 1 1 2 2 c c V + ∈ v v . This extends to all linear combinations of vectors in the subspace V . Proposition : The zero vector 0 must be in any subspace. Proof : If V ∈ v is any vector, then 0 V = ∈ v 0 . Note : Subspaces can be simply visualized as "flat things through the origin". If V ∈ v is any vector, the all scalar multiples of v must also be in V , i.e. a line passing through the origin. If 1 2 , V ∈ v v are nonparallel vectors, then all linear combinations of the form 1 1 2 2 c c V + ∈ v v , i.e. a plane through the origin. In higher dimensions, we continue to understand subspaces of n R to be lines, planes, and higher-dimensional "flat things through the origin". We use the term "affine" to refer to parallel objects that do not pass through the origin. Span of a collection of vectors Definition : Given a collection of vectors { } 1 2 , , , n k ∈ v v v R  , we define the span of these vectors to be the set of all linear combinations of these vectors, i.e. { } { } 1 1 1 1 Span , , where , are scalars k k k k c c c c = + + v v v v    . By its definition, the span of any collection of vectors is automatically a subspace. That is, for appropriate scalars, ( ) ( ) 1 1 1 1 1 1 1 ( ) ( ) k k k k k k k c c d d c d c d α β α β α β + + + + + = + + + + v v v v v v    . Though we often define subspaces by conditions, we usually specify a subspace by producing a collection of vectors that span the subspace. For example, we describe a line through the origin as the span of a single vector, and a plane through the origin as the span of two nonparallel vectors. It's important to note, however, that we could also describe a plane as the span of more than two vectors that all lie in that plane. Eliminating such redundancy will motivate the concept of a basis for a subspace. Image and kernel of a linear transformation Suppose A is an m n × matrix that represents a linear transformation : n m T → R R by ( ) T = x Ax . Its domain is n R and its codomain is m R . We define: { } image( ) image( ) im( ) : codomain( ) n T = = = ∈ ⊂ A A Ax x R A { } kernel( ) kernel( ) ker( ) : domain( ) n T = = = ∈ = ⊂ A A x R Ax 0 A Proposition : (1) im( A ) is a subspace of the codomain m R ; and (2) ker( A ) is a subspace of the domain n R . Proof : (1) Any two vectors in im( A ) must be of the form 1 2 , Ax Ax for some vectors 1 2 , x x in the domain. Therefore ( ) 1 1 2 2 1 1 2 2 im( ) c c c c + = + ∈ Ax Ax A x x A by linearity. (2) If 1 2 , ker( ) ∈ x x A , then 1 = Ax 0 and 2 = Ax 0 . So ( ) 1 1 2 2 1 1 2 2 1 2 c c c c c c + = + = + = A x x Ax Ax 0 0 0 , so 1 1 2 2 ker( c c + ∈ x x A) .

revised June 28, 2021 2 Special case of an n n × (square) matrix A When A is a square matrix, the image and kernel give us a new way of characterizing when a matrix is invertible. Proposition : Let A be an n n × (square) matrix. Then the following statements are equivalent: (1) A is invertible (4) rank( ) n = A (full rank) (2) The system = Ax b has a unique solution x for all n ∈ b R (5) im( ) n = A R (3) rref ( ) n = A I (6) { } ker( ) = A 0 (the zero subspace) The proof of the equivalence of these statements is left to your observations and knowledge of the definitions. Calculation of the image and kernel of an m n × matrix A Proposition : The image of any matrix A is the span of its column vectors. Proof : If the matrix A is expressed in terms of its columns as 1 n   ↑ ↑   =   ↓ ↓     A v v  , we know that: 1 1 = Ae v , 2 2 = Ae v , ..., n n = Ae v , so the column vectors are clearly all in the image of A . However, any vector x in the domain can be written as 1 1 1 n n n x x x x     = = + +     x e e   , so ( ) 1 1 1 1 1 1 n n n n n n x x x x x x = + + = + + = + + Ax A e e Ae Ae v v    , i.e. { } 1 im( ) span , , n = A v v  . This is why the image of a matrix A is also referred to as the "column space" of A . Note : Though it's true that { } 1 im( ) span , , n = A v v  , it is not necessarily the case that all of these column vectors are necessary to span the image. There may be some redundancy. Eliminating this redundancy can be accomplished by understanding the kernel of the matrix. Indeed, every vector in the kernel of a matrix will give a linear interdependency of the columns of the matrix. Example #1 : Determine the image and kernel of the matrix 2 1 0 1 1 4 0 3 8   = −     −   A by providing a spanning set of vector for each of these subspaces. From the proposition above, we know that if 1 2 1 0   =       v , 2 1 1 3   = −       v , and 3 0 4 8   =     −   v are the columns of A , then { } 1 2 3 im( ) span , , = A v v v . That's all there is to it. For the kernel, we solve the homogeneous system = Ax 0 . This is most easily done using row reduction: 2 1 0 0 1 1 4 0 1 1 4 0 1 1 4 0 1 0 4 3 0 1 1 4 0 2 1 0 0 0 3 8 0 0 1 8 3 0 0 1 8 3 0 0 3 8 0 0 3 8 0 0 3 8 0 0 0 0 0 0 0 0 0 − − −           − → → − → − → −                     − − −           If we let 3 3 x t = , we get that 1 2 3 4 8 3 t x t x t x t ∈ = −     =   =     R or 4 8 3 t −   =       x . This says simply that 4 ker( ) span 8 3  −      =             A , so we have our spanning set.

revised June 28, 2021 3 Note that because 4 8 ker( ) 3 −   ∈       A , this means that 1 2 3 1 2 3 4 4 8 8 4 8 3 3 3   − ↑ ↑ ↑ −       = = − + + =           ↓ ↓ ↓       A v v v v v v 0 , so there's a linear dependency among these vectors. In particular, this means that we can solve for any one of these vectors in terms of the remaining vectors. For example, 3 1 2 8 4 3 3 = − v v v . This means that { } 3 1 2 span , ∈ v v v , so we don't need it in our spanning set for the image. We could, of course, have eliminated any one of these three vectors in this manner, but it's good standard practice to solve for the later vectors in terms of its predecessors. There are no other linear interdependencies that can be used to eliminate redundancy, so this is the best we can do. That is, { } 1 2 im( ) span , = A v v . Note, in particular, that we retained in our spanning set for the image exactly those column vectors of the original matrix A that eventually yielded Leading 1's in the reduced row-echelon form of this matrix. We can always choose our spanning set for the image such that this is the case. In order to understand this better, we need one more very important definition. Definition : A set of vectors { } 1 2 , , , n k ∈ v v v R  is called linearly independent if given any linear combination of the form 1 1 k k c c + + = v v 0  , this implies that 1 0 k c c = =  . That is, there is no nontrivial way to combine these vectors to yield the zero vector. Note that this definition also means that it's impossible to solve for any one of these vectors in terms of the others. This is the essential quality of linear independence - there is no redundancy among a linearly independent set of vectors. Test for linear independence Given a collection of vectors { } 1 2 , , , n k ∈ v v v R  , if we write 1 k   ↑ ↑   =   ↓ ↓   A v v  , then the expression 1 1 k k c c + + = v v 0  can be expressed as = Ac 0 where 1 k c c   =       c  . So the statement that 1 1 k k c c + + = v v 0  implies that 1 0 k c c = =  can be restated very succinctly as = Ac 0 implies = c 0 . That is, a collection of vectors { } 1 2 , , , k v v v  will be linearly independent if and only if { } ker( ) = A 0 . In our example above, we found that for 1 2 3   ↑ ↑ ↑   =   ↓ ↓ ↓   A v v v , 4 ker( ) span 8 3  −      =             A . Therefore these column vectors were not linearly independent. Definition : Given a subspace V of n R , a collection of vectors { } 1 2 , , , k V ∈ v v v  is called a basis of V if { } 1 2 Span , , , k V = v v v  and { } 1 2 , , , k v v v  are linearly independent. A basis is a minimal spanning set, and it's important to note that any given subspace can have many different bases. Note : n R is itself a subspace of n R (the whole space). The standard basis { } 1 2 , , , n = e e e  E is familiar to us, but, in fact, any linearly independent collection of n vectors { } 1 2 , , , n v v v  in n R would provide an alternate basis for n R .