Advanced linear algebra (3rd ed)

Graduate Texts in Mathematics 135 Editorial Board S Axler K.A Ribet Graduate Texts in Mathematics TAKEUTI/ZARING Introduction to Axiomatic Set Theory 2nd ed OXTOBY Measure and Category 2nd ed SCHAEFER Topological Vector Spaces 2nd ed HILTON/STAMMBACH A Course in Homological Algebra 2nd ed MAC LANE Categories for the Working Mathematician 2nd ed HUGHES/PIPER Projective Planes J.-P SERRE A Course in Arithmetic TAKEUTI/ZARING Axiomatic Set Theory HUMPHREYS Introduction to Lie Algebras and Representation Theory 10 COHEN A Course in Simple Homotopy Theory 11 CONWAY Functions of One Complex Variable I 2nd ed 12 BEALS Advanced Mathematical Analysis 13 ANDERSON/FULLER Rings and Categories of Modules 2nd ed 14 GOLUBITSKY/GUILLEMIN Stable Mappings and Their Singularities 15 BERBERIAN Lectures in Functional Analysis and Operator Theory 16 WINTER The Structure of Fields 17 ROSENBLATT Random Processes 2nd ed 18 HALMOS Measure Theory 19 HALMOS A Hilbert Space Problem Book 2nd ed 20 HUSEMOLLER Fibre Bundles 3rd ed 21 HUMPHREYS Linear Algebraic Groups 22 BARNES/MACK An Algebraic Introduction to Mathematical Logic 23 GREUB Linear Algebra 4th ed 24 HOLMES Geometric Functional Analysis and Its Applications 25 HEWITT/STROMBERG Real and Abstract Analysis 26 MANES Algebraic Theories 27 KELLEY General Topology 28 ZARISKI/SAMUEL Commutative Algebra Vol I 29 ZARISKI/SAMUEL Commutative Algebra Vol II 30 JACOBSON Lectures in Abstract Algebra I Basic Concepts 31 JACOBSON Lectures in Abstract Algebra II Linear Algebra 32 JACOBSON Lectures in Abstract Algebra III Theory of Fields and Galois Theory 33 HIRSCH Differential Topology 34 SPITZER Principles of Random Walk 2nd ed 35 ALEXANDER/WERMER Several Complex Variables and Banach Algebras 3rd ed 36 KELLEY/NAMIOKA et al Linear Topological Spaces 37 MONK Mathematical Logic 38 GRAUERT/FRITZSCHE Several Complex Variables 39 ARVESON An Invitation to C∗ -Algebras 40 KEMENY/SNELL/KNAPP Denumerable Markov Chains 2nd ed 41 APOSTOL Modular Functions and Dirichlet Series in Number Theory 2nd ed 42 J.-P SERRE Linear Representations of Finite Groups 43 GILLMAN/JERISON Rings of Continuous Functions 44 KENDIG Elementary Algebraic Geometry 45 LOÈVE Probability Theory I 4th ed 46 LOÈVE Probability Theory II 4th ed 47 MOISE Geometric Topology in Dimensions and 48 SACHS/WU General Relativity for Mathematicians 49 GRUENBERG/WEIR Linear Geometry 2nd ed 50 EDWARDS Fermat’s Last Theorem 51 KLINGENBERG A Course in Differential Geometry 52 HARTSHORNE Algebraic Geometry 53 MANIN A Course in Mathematical Logic 54 GRAVER/WATKINS Combinatorics with Emphasis on the Theory of Graphs 55 BROWN/PEARCY Introduction to Operator Theory I: Elements of Functional Analysis 56 MASSEY Algebraic Topology: An Introduction 57 CROWELL/FOX Introduction to Knot Theory 58 KOBLITZ p-adic Numbers, p-adic Analysis, and Zeta-Functions 2nd ed 59 LANG Cyclotomic Fields 60 ARNOLD Mathematical Methods in Classical Mechanics 2nd ed 61 WHITEHEAD Elements of Homotopy Theory 62 KARGAPOLOV/MERIZJAKOV Fundamentals of the Theory of Groups 63 BOLLOBAS Graph Theory 64 EDWARDS Fourier Series Vol I 2nd ed 65 WELLS Differential Analysis on Complex Manifolds 3rd ed 66 WATERHOUSE Introduction to Affine Group Schemes 67 SERRE Local Fields 68 WEIDMANN Linear Operators in Hilbert Spaces 69 LANG Cyclotomic Fields II 70 MASSEY Singular Homology Theory 71 FARKAS/KRA Riemann Surfaces 2nd ed 72 STILLWELL Classical Topology and Combinatorial Group Theory 2nd ed 73 HUNGERFORD Algebra 74 DAVENPORT Multiplicative Number Theory 3rd ed 75 HOCHSCHILD Basic Theory of Algebraic Groups and Lie Algebras (continued after index) Steven Roman Advanced Linear Algebra Third Edition Steven Roman Night Star Irvine, CA 92603 USA sroman@romanpress.com Editorial Board S Axler Mathematics Department San Francisco State University San Francisco, CA 94132 USA axler@sfsu.edu ISBN-13: 978-0-387-72828-5 K.A Ribet Mathematics Department University of California at Berkeley Berkeley, CA 94720-3840 USA ribet@math.berkeley.edu e-ISBN-13: 978-0-387-72831-5 Library of Congress Control Number: 2007934001 Mathematics Subject Classification (2000): 15-01 c 2008 Springer Science+Business Media, LLC All rights reserved This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights Printed on acid-free paper springer.com To Donna and to Rashelle, Carol and Dan Preface to the Third Edition Let me begin by thanking the readers of the second edition for their many helpful comments and suggestions, with special thanks to Joe Kidd and Nam Trang For the third edition, I have corrected all known errors, polished and refined some arguments (such as the discussion of reflexivity, the rational canonical form, best approximations and the definitions of tensor products) and upgraded some proofs that were originally done only for finite-dimensional/rank cases I have also moved some of the material on projection operators to an earlier position in the text A few new theorems have been added in this edition, including the spectral mapping theorem and a theorem to the effect that dim²= ³ dim²= i ³, with equality if and only if = is finite-dimensional I have also added a new chapter on associative algebras that includes the wellknown characterizations of the finite-dimensional division algebras over the real field (a theorem of Frobenius) and over a finite field (Wedderburn's theorem) The reference section has been enlarged considerably, with over a hundred references to books on linear algebra Steven Roman Irvine, California, May 2007 Preface to the Second Edition Let me begin by thanking the readers of the first edition for their many helpful comments and suggestions The second edition represents a major change from the first edition Indeed, one might say that it is a totally new book, with the exception of the general range of topics covered The text has been completely rewritten I hope that an additional 12 years and roughly 20 books worth of experience has enabled me to improve the quality of my exposition Also, the exercise sets have been completely rewritten The second edition contains two new chapters: a chapter on convexity, separation and positive solutions to linear systems (Chapter 15) and a chapter on the QR decomposition, singular values and pseudoinverses (Chapter 17) The treatments of tensor products and the umbral calculus have been greatly expanded and I have included discussions of determinants (in the chapter on tensor products), the complexification of a real vector space, Schur's theorem and Geršgorin disks Steven Roman Irvine, California February 2005 Preface to the First Edition This book is a thorough introduction to linear algebra, for the graduate or advanced undergraduate student Prerequisites are limited to a knowledge of the basic properties of matrices and determinants However, since we cover the basics of vector spaces and linear transformations rather rapidly, a prior course in linear algebra (even at the sophomore level), along with a certain measure of “mathematical maturity,” is highly desirable Chapter contains a summary of certain topics in modern algebra that are required for the sequel This chapter should be skimmed quickly and then used primarily as a reference Chapters 1–3 contain a discussion of the basic properties of vector spaces and linear transformations Chapter is devoted to a discussion of modules, emphasizing a comparison between the properties of modules and those of vector spaces Chapter provides more on modules The main goals of this chapter are to prove that any two bases of a free module have the same cardinality and to introduce Noetherian modules However, the instructor may simply skim over this chapter, omitting all proofs Chapter is devoted to the theory of modules over a principal ideal domain, establishing the cyclic decomposition theorem for finitely generated modules This theorem is the key to the structure theorems for finite-dimensional linear operators, discussed in Chapters and Chapter is devoted to real and complex inner product spaces The emphasis here is on the finite-dimensional case, in order to arrive as quickly as possible at the finite-dimensional spectral theorem for normal operators, in Chapter 10 However, we have endeavored to state as many results as is convenient for vector spaces of arbitrary dimension The second part of the book consists of a collection of independent topics, with the one exception that Chapter 13 requires Chapter 12 Chapter 11 is on metric vector spaces, where we describe the structure of symplectic and orthogonal geometries over various base fields Chapter 12 contains enough material on metric spaces to allow a unified treatment of topological issues for the basic xii Preface Hilbert space theory of Chapter 13 The rather lengthy proof that every metric space can be embedded in its completion may be omitted Chapter 14 contains a brief introduction to tensor products In order to motivate the universal property of tensor products, without getting too involved in categorical terminology, we first treat both free vector spaces and the familiar direct sum, in a universal way Chapter 15 (Chapter 16 in the second edition) is on affine geometry, emphasizing algebraic, rather than geometric, concepts The final chapter provides an introduction to a relatively new subject, called the umbral calculus This is an algebraic theory used to study certain types of polynomial functions that play an important role in applied mathematics We give only a brief introduction to the subject c emphasizing the algebraic aspects, rather than the applications This is the first time that this subject has appeared in a true textbook One final comment Unless otherwise mentioned, omission of a proof in the text is a tacit suggestion that the reader attempt to supply one Steven Roman Irvine, California 438 Advanced Linear Algebra planes º3 » and º3 » have the property that their intersection is a line through the origin, even if the lines are parallel We are now ready to define projective geometries Let = be a vector space of any dimension and let / be a hyperplane in = not containing the origin To each flat ? in / , we associate the subspace º?» of = generated by ? Thus, the linear span function Â 7/ Ư I ²= ³ maps affine subspaces ? of / to subspaces º?» of = The span function is not surjective: Its image is the set of all subspaces that are not contained in the base subspace of the flat / The linear span function is one-to-one and its inverse is intersection with / , c < ~ < q / for any subspace < not contained in The affine geometry 7²/³ is, as we have remarked, somewhat incomplete In the case dim²/³ ~ , every pair of points determines a line but not every pair of lines determines a point Now, since the linear span function is injective, we can identify 7²/³ with its image ²7²/³³, which is the set of all subspaces of = not contained in the base subspace This view of 7²/³ allows us to “complete” 7²/³ by including the base subspace In the three-dimensional case of Figure 16.1, the base plane, in effect, adds a projective line at infinity With this inclusion, every pair of lines intersects, parallel lines intersecting at a point on the line at infinity This two-dimensional projective geometry is called the projective plane Definition Let = be a vector space The set I ²= ³ of all subspaces of = is called the projective geometry of = The projective dimension pdim²:³ of : I ²= ³ is defined as pdim²:³ ~ dim²:³ c The projective dimension of F²= ³ is defined to be pdim²= ³ ~ dim²= ³ c A subspace of projective dimension is called a projective point and a subspace of projective dimension is called a projective line. Thus, referring to Figure 16.1, a projective point is a line through the origin and, provided that it is not contained in the base plane , it meets / in an affine point Similarly, a projective line is a plane through the origin and, provided that it is not , it will meet / in an affine line In short, span²affine point³ ~ line through the origin ~ projective point span²affine line³ ~ plane through the origin ~ projective line The linear span function has the following properties Affine Geometry 439 Theorem 16.12 The linear span function Â 7/ Ư I ²= ³ from the affine geometry 7²/³ to the projective geometry I ²= ³ defined by ? ~ º?» satisfies the following properties: 1) The linear span function is injective, with inverse given by c < ~ < q / for all subspaces < not contained in the base subspace of / 2) The image of the span function is the set of all subspaces of = that are not contained in the base subspace of / 3) ? @ if and only if º?» º@ » 4) If ? are flats in / with nonempty intersection, then span4 ? ~ span²? ³ 2 2 5) For any collection of flats in / , span8 ? ~ span²? ³ 2 2 6) The linear span function preserves dimension, in the sense that pdim²span²?³³ ~ dim²?³ 7) ? @ if and only if one of º?» q and º@ » q is contained in the other Proof To prove part 1), let % b : be a flat in / Then % / and so / ~ % b , which implies that : Note also that º% b :» ~ º%» b : and ' º% b :» q / ~ ²º%» b :³ q ²% b 2³ ¬ ' ~ % b ~ % b for some : , and - This implies that ² c ³% , which implies that either % or ~ But % / implies % Ô and so ~ , which implies that ' ~ % b % b : In other words, º% b :» q / % b : Since the reverse inclusion is clear, we have º% b :» q / ~ % b : This establishes 1) To prove 2), let < be a subspace of = that is not contained in We wish to show that < is in the image of the linear span function Note first that since < and dim²2³ ~ dim²= ³ c , we have < b ~ = and so dim²< q 2³ ~ dim²< ³ b dim²2³ c dim²< b 2³ ~ dim²< ³ c 440 Advanced Linear Algebra Now let Ê % < c Then % Ô ¬ º%» b ~ = ¬ % b / for some £ - Á ¬ % / Thus, % < q / for some £ - Hence, the flat % b ²< q 2³ lies in / and dim²% b ²< q 2³³ ~ dim²< q 2³ ~ dim²< ³ c which implies that span²% b ²< q 2³³ ~ º%» b ²< q 2³ lies in < and has the same dimension as < In other words, span²% b ²< q 2³³ ~ º%» b ²< q 2³ ~ < We leave proof of the remaining parts of the theorem as exercises. Exercises Show that if % Á Ã Á % = , then the set : ~ ¸' % ' ~ ¹ is a subspace of = Prove that if ? = is nonempty then affhull²?³ ~ % b º? c %» Prove that the set ? ~ á ạ in ²{ ³ is closed under the formation of lines, but not affine hulls Prove that a flat contains the origin if and only if it is a subspace Prove that a flat ? is a subspace if and only if for some % ? we have % ? for some £ - Show that the join of a collection ~ á% b : 2ạ of flats in = is the intersection of all flats that contain all flats in Is the collection of all flats in = a lattice under set inclusion? If not, how can you “fix” this? Suppose that ? ~ % b : and @ ~ & b ; Prove that if dim²?³ ~ dim²@ ³ and ? @ , then : ~ ; Suppose that ? ~ % b : and @ ~ & b ; are disjoint hyperplanes in = Show that : ~ ; 10 (The parallel postulate) Let ? be a flat in = and # Ô ? Show that there is exactly one flat containing #, parallel to ? and having the same dimension as ? 11 a) Find an example to show that the join ? v @ of two flats may not be the set of all lines connecting all points in the union of these flats b) Show that if ? and @ are flats with ? q @ £ J, then ? v @ is the union of all lines %& where % ? and & @ 12 Show that if ? @ and ? q @ ~ J, then dim²? v @ ³ ~ max¸dim²?³Á dim²@ ³¹ b Affine Geometry 441 13 Let dim²= ³ ~ Prove the following: a) The join of any two distinct points is a line b) The intersection of any two nonparallel lines is a point 14 Let dim²= ³ ~ Prove the following: a) The join of any two distinct points is a line b) The intersection of any two nonparallel planes is a line c) The join of any two lines whose intersection is a point is a plane d) The intersection of two coplanar nonparallel lines is a point e) The join of any two distinct parallel lines is a plane f) The join of a line and a point not on that line is a plane g) The intersection of a plane and a line not on that plane is a point 15 Prove that Â = Ư = is a surjective affine transformation if and only if ~ k ;$ for some $ = and B²= ³ 16 Verify the group-theoretic remarks about the group homomorphism Â aff= Ư B= and the subgroup trans= ³ of aff²= ³ Chapter 17 Singular Values and the Moore–Penrose Inverse Singular Values Let < and = be finite-dimensional inner product spaces over d or s and let B²< Á = ³ The spectral theorem applied to i can be of considerable help in understanding the relationship between and its adjoint i This relationship is shown in Figure 17.1 Note that < and = can be decomposed into direct sums < ~(l) and = ~ * l + in such a manner that Â ( Ư * and i ¢ * ¦ ( act symmetrically in the sense that Â " ê # and i Â # ª " Also, both and i are zero on ) and +, respectively We begin by noting that i B²< ³ is a positive Hermitian operator Hence, if ~ rk² ³ ~ rk² i ³, then < has an ordered orthonormal basis ~ ²" Á Ã Á " Á "b Á Ã Á " ³ of eigenvectors for i , where the corresponding eigenvalues can be arranged so that Ä ~ b ~ Ä ~ The set ²"b Á Ã Á " ³ is an ordered orthonormal basis for ker² i ³ ~ ker² ³ and so ²" Á Ã Á " ³ is an ordered orthonormal basis for ker² ³ ~ im² i ³ 444 Advanced Linear Algebra im(W*) u1 ONB of eigenvectors for W*W im(W) v1 W(uk)=skvk W (vk)=skuk ur ur+1 W vr+1 un W vm ker(W) ONB of eigenvectors for WW* vr ker(W*) Figure 17.1 For ~ Á Ã Á , the positive numbers of If we set ~ for , then ~ j are called the singular values i " ~ " for ~ Á Ã Á We can achieve some “symmetry” here between and i by setting # ~ ²° ³ " for each , giving # " " ~ F and i # ~ F The vectors # Á Ã Á # are orthonormal, since if Á , then º# Á # » ~ º " Á " » ~ º i " Á " » ~ º" Á " » ~ Á Hence, ²# Á Ã Á # ³ is an orthonormal basis for im² ³ ~ ker² i ³ , which can be extended to an orthonormal basis ~ ²# Á Ã Á # ³ for = , the extension ²#b Á Ã Á # ³ being an orthonormal basis for ker² i ³ Moreover, since i # ~ " ~ # the vectors # Á Ã Á # are eigenvectors for i with the same eigenvalues ~ as for i This completes the picture in Figure 17.1 Singular Values and the Moore–Penrose Inverse 445 Theorem 17.1 Let < and = be finite-dimensional inner product spaces over d or s and let B²< Á = ³ have rank Then there are ordered orthonormal bases and for < and = , respectively, for which ~ ² " Á Ã Á " Á "b Á Ã Á " ³ ONB for im² i ³ ONB for ker² ³ and ~ ² # Á Ã Á # Á #b Á Ã Á # ³ ONB for im² ³ ONB for ker² i ³ Moreover, for , " ~ i # ~ where # " are called the singular values of , defined by i " ~ " Á for The vectors " Á Ã Á " are called the right singular vectors for and the vectors # Á Ã Á # are called the left singular vectors for The matrix version of the previous discussion leads to the well-known singularvalue decomposition of a matrix Let ( CÁ ²- ³ and let ~ ²" Á Ã Á " ³ and ~ ²# Á Ã Á # ³ be the orthonormal bases from < and = , respectively, in Theorem 17.1, for the operator ( Then ´ µ8Á9 ~ ' ~ diag² Á Á Ã Á Á Á Ã Á ³ A change of orthonormal bases from the standard bases to and : gives ( ~ ´( µ; Á; ~ 49Á; ´( µ8Á9 4; Á8 ~ '8i where ~ 49Á; and ~ 48Á; are unitary/orthogonal This is the singularvalue decomposition of ( As to uniqueness, if ( ~ '8i , where and are unitary and ' is diagonal, with diagonal entries , then (i ( ~ ²7 '8i ³i '8i ~ 8'i '8i and since 'i ' ~ diag² Á Ã Á ³, it follows that the 's are eigenvalues of (i (, that is, they are the squares of the singular values along with a sufficient number of 's Hence, ' is uniquely determined by (, up to the order of the diagonal elements 446 Advanced Linear Algebra We state without proof the following uniqueness facts and refer the reader to [48] for details If and if the eigenvalues are distinct, then is uniquely determined up to multiplication on the right by a diagonal matrix of the form + ~ diag²' Á Ã Á ' ³ with (' ( ~ If , then is never uniquely determined If ~ ~ , then for any given there is a unique Thus, we see that, in general, the singular-value decomposition is not unique The Moore–Penrose Generalized Inverse Singular values lead to a generalization of the inverse of an operator that applies to all linear transformations The setup is the same as in Figure 17.1 Referring to that figure, we are prompted to define a linear transformation b Â = Ư < by b # ~ F " for for since then ² b ³Oº" ÁÃÁ" » ~ ² b ³Oº"b ÁÃÁ" » ~ and ² b ³Oº# ÁÃÁ# » ~ ² b ³Oº#b ÁÃÁ# » ~ Hence, if ~ ~ , then b ~ c The transformation b is called the Moore–Penrose generalized inverse or Moore–Penrose pseudoinverse of We abbreviate this as MP inverse Note that the composition b is the identity on the largest possible subspace of < on which any composition of the form could be the identity, namely, the orthogonal complement of the kernel of A similar statement holds for the composition b Hence, b is as “close” to an inverse for as is possible We have said that if is invertible, then b ~ c More is true: If is injective, then b ~ and so b is a left inverse for Also, if is surjective, then b is a right inverse for Hence the MP inverse b generalizes the onesided inverses as well Here is a characterization of the MP inverse Theorem 17.2 Let B²< Á = ³ The MP inverse b of is completely characterized by the following four properties: 1) b ~ 2) b b ~ b 3) b is Hermitian 4) b is Hermitian Singular Values and the Moore–Penrose Inverse 447 Proof We leave it to the reader to show that b does indeed satisfy conditions 1)–4) and prove only the uniqueness Suppose that and satisfy 1)–4) when substituted for b Then ~ ~ ² ³ i ~ i i ~ ² ³ i i ~ i i i i ~ ² ³ i i i ~ i i ~ ~ and ~ ~ ² ³ i ~ i i ~ i ² ³i ~ i i i i ~ i i ² ³i ~ i i ~ ~ which shows that ~ . The MP inverse can also be defined for matrices In particular, if ( 4Á ²- ³, then the matrix operator ( has an MP inverse (b Since this is a linear transformation from - to - , it is just multiplication by a matrix (b ~ ) This matrix ) is the MP inverse for ( and is denoted by (b Since (b ~ (b and () ~ ( ) , the matrix version of Theorem 17.2 implies that (b is completely characterized by the four conditions 1) 2) 3) 4) ((b ( ~ ( (b ((b ~ (b ((b is Hermitian (b ( is Hermitian Moreover, if ( ~