Otto bretscher linear algebra with applications 4th edition

Contents Text Features ix Preface xi Linear Equations 1.1 1.2 1.3 Linear Transformations 2.1 2.2 2.3 2.4 Introduction to Linear Systems Matrices, Vectors, and Gauss-Jordan Elimination On the Solutions of Linear Systems; Matrix Algebra Introduction to Linear Transformations and Their Inverses Linear Transformations in Geometry Matrix Products The Inverse of a Linear Transformation Subspaces of Mn and Their Dimensions 3.1 Image and Kernel of a Linear Transformation 3.2 Subspaces of R"; Bases and Linear Independence 3.3 The Dimension of a Subspace of R" 3.4 Coordinates Linear Spaces 4.1 4.2 4.3 Introduction to Linear Spaces Linear Transformations and Isomorphisms The Matrix o f a Linear Transformation Orthogonality and Least Squares 5.1 Orthogonal Projections and Orthonormal Bases 5.2 Gram-Schmidt Process and QR Factorization 5.3 Orthogonal Transformations and Orthogonal Matrices 5.4 Least Squares and Data Fitting 5.5 Inner Product Spaces 1 25 40 40 54 69 79 101 101 113 123 137 153 153 165 112 187 187 203 210 220 233 vii viii Contents Determinants 6.1 6.2 6.3 Introduction to Determinants Properties of the Determinant Geometrical Interpretations of the Determinant; Cramer’s Rule Eigenvalues and Eigenvectors 7.1 249 249 261 277 294 Dynamical Systems and Eigenvectors: An Introductory Example Finding the Eigenvalues of a Matrix Finding the Eigenvectors of a Matrix Diagonalization Complex Eigenvalues Stability 294 308 319 332 343 357 Symmetric Matrices and Quadratic Forms 367 7.2 7.3 7.4 7.5 7.6 8.1 8.2 8.3 Symmetric Matrices Quadratic Forms Singular Values Linear Differential Equations 9.1 9.2 9.3 An Introduction to Continuous Dynamical Systems The Complex Case: Euler’s Formula Linear Differential Operators and Linear Differential Equations Appendix A Vectors 437 Answers to Odd-Numbered Exercises 446 Subject Index 471 Name Index 478 367 376 385 397 397 410 423 Text Features Continuing Text Features • Linear transformations are introduced early on in the text to make the discus­ sion of matrix operations more meaningful and easier to visualize • Visualization and geometrical interpretation are emphasized extensively throughout • The reader will find an abundance of thought-provoking (and occasionally delightful) problems and exercises • Abstract concepts are introduced gradually throughout the text The major ideas are carefully developed at various levels of generality before the student is introduced to abstract vector spaces • Discrete and continuous dynamical systems are used as a motivation for eigen­ vectors, and as a unifying theme thereafter New Features in the Fourth Edition Students and instructors generally found the third edition to be accurate and well structured While preserving the overall organization and character of the text, some changes seemed in order • A large number of exercises have been added to the problem sets, from the elementary to the challenging For example, two dozen new exercises on conic and cubic curve fitting lead up to a discussion of the Cramer-Euler Paradox on fitting a cubic through nine points • The section on matrix products now precedes the discussion of the inverse of a matrix, making the presentation more sensible from an algebraic point of view • Striking a balance between the earlier editions, the determinant is defined in terms o f “patterns”, a transparent way to deal with permutations Laplace expansion and Gaussian elimination are presented as alternative approaches • There have been hundreds of small editorial improvements—offering a hint in a difficult problem for example— or choosing a more sensible notation in a theorem Preface (with David Steinsaltz) police officer on patrol at midnight, so runs an old joke, notices a man crawling about on his hands and knees under a streetlamp He walks over to investigate, whereupon the man explains in a tired and somewhat slurred voice that he has lost his housekeys The policeman offers to help, and for the next five minutes he too is searching on his hands and knees At last he exclaims, “Are you absolutely certain that this is where you dropped the keys?” A “Here? Absolutely not I dropped them a block down, in the middle of the street.” “Then why the devil have you got me hunting around this lamppost?” “Because this is where the light is.” It is mathematics, and not just (as Bismarck claimed) politics, that consists in “the art of the possible.” Rather than search in the darkness for solutions to problems of pressing interest, we contrive a realm of problems whose interest lies above all in the fact that solutions can conceivably be found Perhaps the largest patch of light surrounds the techniques of matrix arithmetic and algebra, and in particular matrix multiplication and row reduction Here we might begin with Descartes, since it was he who discovered the conceptual meetingpoint of geometry and algebra in the identification of Euclidean space with R 3; the techniques and applications proliferated since his day To organize and clarify those is the role of a modem linear algebra course Computers and Computation An essential issue that needs to be addressed in establishing a mathematical m ethod' ology is the role of computation and of computing technology Are the proper subjects of mathematics algorithms and calculations, or are they grand theories and abstractions that evade the need for computation? If the former, is it important that the students learn to carry out the computations with pencil and paper, or should the al­ gorithm “press the calculator’s x ~ button” be allowed to substitute for the traditional method of finding an inverse? If the latter, should the abstractions be taught through elaborate notational mechanisms or through computational examples and graphs? We seek to take a consistent approach to these questions: Algorithms and com­ putations are primary, and precisely for this reason computers are not Again and again we examine the nitty-gritty of row reduction or matrix multiplication in or­ der to derive new insights Most of the proofs, whether of rank-nullity theorem, the volume-change formula for determinants, or the spectral theorem for symmetric matrices, are in this way tied to hands-on procedures The aim is not just to know how to compute the solution to a problem, but to imagine the computations The student needs to perform enough row reductions by hand to be equipped to follow a line of argument of the form: “If we calculate the reduced row echelon form of such a matrix , ” and to appreciate in advance the possible outcomes of a particular computation In applications the solution to a problem is hardly more important than recog­ nizing its range o f validity and appreciating how sensitive it is to perturbations o f the input We emphasize the geometric and qualitative nature of the solutions, notions of approximation, stability, and “typical” matrices The discussion of Cram er’s rule, for instance, underscores the value of closed-form solutions for visualizing a system’s behavior and understanding its dependence from initial conditions xii Preface The availability of computers is, however, neither to be ignored nor regretted Each student and instructor will have to decide how much practice is needed to be sufficiently familiar with the inner workings of the algorithm As the explicit compu­ tations are being replaced gradually by a theoretical overview of how the algorithm works, the burden of calculation will be taken up by technology, particularly for those wishing to carry out the more numerical and applied exercises Examples, Exercises, Applications, and History The exercises and examples are the heart of this book Our objective is not just to show our readers a “patch of light” where questions may be posed and solved, but to convince them that there is indeed a great deal of useful, interesting material to be found in this area if they take the time to look around Consequently, we have included genuine applications of the ideas and methods under discussion to a broad range of sciences: physics, chemistry, biology, economics, and, of course, mathematics itself Often we have simplified them to sharpen the point, but they use the methods and models of contemporary scientists With such a large and varied set of exercises in each section, instructors should have little difficulty in designing a course that is suited to their aims and to the needs of their students Quite a few straightforward computation problems are offered, of course Simple (and, in a few cases, not so simple) proofs and derivations are required in some exercises In many cases, theoretical principles that are discussed at length in more abstract linear algebra courses are here found broken up in bite-size exercises The examples make up a significant portion of the text; we have kept abstract exposition to a minimum It is a matter of taste whether general theories should give rise to specific examples or be pasted together from them In a text such as this one, attempting to keep an eye on applications, the latter is clearly preferable: The examples always precede the theorems in this book Scattered throughout the mathematical exposition are quite a few names and dates, some historical accounts, and anecdotes Students of mathematics are too rarely shown that the seemingly strange and arbitrary concepts they study are the results of long and hard struggles It will encourage the readers to know that a mere two centuries ago some of the most brilliant mathematicians were wrestling with problems such as the meaning of dimension or the interpretation of el\ and to realize that the advance of time and understanding actually enables them, with some effort of their own, to see farther than those great minds Outline of the Text C hapter I This chapter provides a careful introduction to the solution of systems of linear equations by Gauss-Jordan elimination Once the concrete problem is solved, we restate it in terms of matrix formalism and discuss the geometric properties of the solutions C hapter Here we raise the abstraction a notch and reinterpret matrices as linear transformations The reader is introduced to the modem notion of a function, as an arbitrary association between an input and an output, which leads into a dis­ cussion of inverses The traditional method for finding the inverse of a matrix is explained: It fits in naturally as a sort of automated algorithm for Gauss-Jordan elimination Preface xiii We define linear transformations primarily in terms of matrices, since that is how they are used; the abstract concept of linearity is presented as an auxiliary notion Rotations, reflections, and orthogonal projections in R are emphasized, both as archetypal, easily visualized examples, and as preparation for future applications C hapter We introduce the central concepts of linear algebra: subspaces, image and kernel, linear independence, bases, coordinates, and dimension, still firmly fixed in R" C hapter Generalizing the ideas of the preceding chapter and using an abun­ dance of examples, we introduce abstract vector spaces (which are called linear spaces here, to prevent the confusion some students experience with the term “vector”) C hapter This chapter includes some of the most basic applications of linear algebra to geometry and statistics We introduce orthonormal bases and the G ram Schmidt process, along with the QR factorization The calculation of correlation coefficients is discussed, and the important technique of least-squares approxima­ tions is explained, in a number of different contexts C hapter Our discussion of determinants is algorithmic, based on the counting of patterns (a transparent way to deal with permutations) We derive the properties of the determinant from careful analysis of this procedure, tieing it together with G aussJordan elimination The goal is to prepare for the main application of determinants: the computation of characteristic polynomials C hapter This chapter introduces the central application of the latter half of the text: linear dynamical systems We begin with discrete systems and are nat­ urally led to seek eigenvectors, which characterize the long-term behavior of the system Qualitative behavior is emphasized, particularly stability conditions Com­ plex eigenvalues are explained, without apology, and tied into earlier discussions of two-dimensional rotation matrices xiv Preface C hapter The ideas and methods of Chapter are applied to geometry We discuss the spectral theorem for symmetric matrices and its applications to quadratic forms, conic sections, and singular values C hapter Here we apply the methods developed for discrete dynamical systems to continuous ones, that is, to systems of first-order linear differential equations Again, the cases of real and complex eigenvalues are discussed Solutions Manuals • Student's Solutions Manual, with carefully worked solutions to all oddnumbered problems in the text (ISBN 0-13-600927-1) • Instructor's Solutions Manual, with solutions to all the problems in the text (ISBN 0-13-600928-X) Acknowledgments I first thank my students and colleagues at Harvard University, Colby College, and Koq University (Istanbul) for the key role they have played in developing this text out of a series of rough lecture notes The following colleagues, who have taught the course with me, have made invaluable contributions: Attila A§kar Persi Diaconis Jordan Ellenberg Matthew Emerton Edward Frenkel Alexandru Ghitza Fernando Gouvea Jan Holly Varga Kalantarov David Kazhdan Oliver Knill Leo Livshits Barry Mazur David Mumford David Steinsaltz Shlomo Sternberg Richard Taylor George Welch I am grateful to those who have taught me algebra: Peter Gabriel, Volker Strassen, and Bartel van der Waerden at the University of Zurich; John Tate at Harvard University; Maurice Auslander at Brandeis University; and many more I owe special thanks to John Boiler, William Calder, Devon Ducharme, and Robert Kaplan for their thoughtful review of the manuscript I wish to thank Sylvie Bessette, Dennis Kletzing, and Laura Lawrie for the careful preparation of the manuscript and Paul Nguyen for his well-drawn figures Special thanks go to Kyle Burke, who has been my assistant at Colby College for three years, having taken linear algebra with me as a first-year student Paying close attention to the way his students understood or failed to understand the material, Kyle came up with new ways to explain many of the more troubling concepts (such as the kernel of a matrix or the notion of an isomorphism) Many of his ideas have found their way into this text Kyle took the time to carefully review various iterations of the manuscript, and his suggestions and contributions have made this a much better text I am grateful to those who have contributed to the book in many ways: Marilyn Baptiste, Menoo Cung, Srdjan Divac, Devon Ducharme, Robin Gottlieb, Luke Hunsberger, George Mani, Bridget Neale, Alec van Oot, Akilesh Palanisamy, Rita Pang, Esther Silberstein, Radhika de Silva, Jonathan Tannenhauser, Ken WadaShiozaki, Larry Wilson, and Jingjing Zhou 464 A N S W E R S T O O D D - N U M B E R E D E X E R C IS E S 33 If B = S~'/4S, then B - Xl„ = S ~ l (A - XIn)S 37 a Av • w — (Av)Tw = vTA Tw = vT Aw = v • Aw b Suppose Av = kvandAw = f i w ThenAvw = k(v • w) and v • Aw = fj.(v • w) By part (a), k(v • w) = fi(v ■w), so that (k —}i)(v • u>) = Since k ^ M it follows that • 0) = 0, as claimed 39 a E\ = V and E q = V1*, so that the geometric multiplicity of is m and that of is n —m The algebraic multiplicities are the same (see Exer­ cise 31) b E\ = V and £_ \ = V1 , so that the multiplicity of is m and that of is n m , - , - , with eigen­ 2 values 1.2, —0.8, —0.4; j?o = 50v\ + 5002 + 5003 41 Eigenbasis for A: j ( = 450(1.2)' + l00(-0.8)' + 50(-0.4)f m(t) = 300(1.2)' - 100(—0.8)' - 100(-0.4)' a(t) = 100(1.2)' + (-0 )' + 100(-0.4)' The populations approach the proportion 9:6:2 "0 r 1 1 43 a A = \ c S(l) = (i + | ) + r(t) pit) Then x(t + 1) = Ax(t) where w(t) 47 L et* (/) = 35 No (consider the eigenvalues) -1 -1 -1 Carl wins if he chooses cq < A= i 1 i4 Eigenbasis for A: 2 f "f 1 -1 - , with eigenvalues 1, A, 1 1 xo = e { = + -1 for t > -1 The proportion in the long run is 1:2:1 r+i 49 [rref(A) is likely to be the matrix with all l ’s di­ rectly above the main diagonal and 0’s everywhere else] 51 fA(k) = - k + ck2 + bk + a 0 a * * b * * Bx B2 53 a B = c * * B3 0 w X 0 y z b Note that A is similar to B Thus fA(k) = f B(k) = f B3{k) f Bl{k) = h(k)(—k + ck2 + bk + a), where h(k) = /fi3(X) (See Exercise 51.) c / a(A)5 = /i(A)(—A3 + cA2 -h bA + als)v = h(A) (—A30 -I- cA20 -f bAv + av) = £0 45 a A = b B = 0.1 0.2 0.4 0.3 A 7.4 Answers to more theoretical questions are omitted S = I2, D = A ,b = = b = 2" is an eigenvector of B with eigenvalue d Will approach (I —A) ~lb = value ,D= , for example If you Not diagonalizable is an eigenvector of B Furthermore, ( h - A) rb 1 found a different solution, check that AS = SD c The eigenvalues of A are 0.5 and —0.1, those of B are 0.5, —0.1, If is an eigenvector of A, then -1 , for any initial = f ■4 -1 >0 = Not diagonalizable 0‘ -3 11 Not diagonalizable _1 1" ‘l 13 S = , D = _0 0 15 S = "2 r 1 , D= "l 0 0 o' 0" A N S W E R S T O O D D - N U M B E R E D E X E R C IS E S -1 -1 1 65 A basis of Vis dim(V) = ,D = 19 Not diagonalizable; eigenvalue 1has algebraic mul­ tiplicity , but geometric multiplicity 21 Diagonalizable for all a 23 Diagonalizable for positive a 25 Diagonalizable for all a , b, and c 27 Diagonalizable only if a = b = c = 29 Never diagonalizable 5' + ( - l ) ' 31 A' = 2(5') —2(—1)r 33 A' = l l- [A ‘-1 35 Yes, since —2 values and '- ( - l ) ' 2(5') + (—D' is diagonalizable, with eigen- 37 Yes Both matrices have the characteristic polyno­ mial k —I k + 7, so that they both have the eigen7 + >J2\ values k[ = • Thus, A and B are both 67 The dimension is 32 , and therefore, A is similar to B 39 All real numbers k are eigenvalues, with corre­ sponding eigenfunctions 41 The symmetric matrices are eigenmatrices with eigenvalue , and the skew-symmetric matrices have eigenvalue Yes, L is diagonalizable, since the sum of the dimensions of the eigenspaces is 43 and i are “eigenvectors” with eigenvalues and —1, respectively Yes, T is diagonalizable; 1, i is an eigenbasis 45 No eigensequences 47 Nonzero polynomials of the form a+ cx2 are eigen­ functions with eigenvalue , and bx (with b ^ 0) has eigenvalue —1 Yes, T is diagonalizable, with eigenbasis 1, j c , x 49 , 2x — 1, and ( — l )2 are eigenfunctions with eigenvalues 1,3, and 9, respectively These func­ tions form an eigenbasis, so that T is indeed diagonalizable o' 0q ’ 00 f -1 -1 ’ and + 22 = 13 71 The eigenvalues are and 2, and (A —/ HA —2 / 3) = Thus A is diagonalizable 73 If k \ , , k m are the distincteigenvalues of A, then f A(k) = (* - * ! ) ■ ( * - km)h(k) for some polynomial h(k), so that f \ ( A ) = ( A - k i l n) - • ■ ( A - k mIn)h(A) = 0, by Exer0 cise 70 7.5 Answers to more theoretical questions are omitted c o s ( ^ ) + / si n ( ^ ) , f o r t = - If z = r (cos() + i sin(0)), then «r ( (4> + n k \ ( + n k \ \ " = ^ r * { — — ) + 1“ { — — ) ) • for kx similar to ^2 465 = , — Clockwise rotation through an angle of j followed by a scaling by a factor of y[2 Spirals outward since \z\ > 11 /(X) = ( k - l ) ( k - - i ) ( k - + /) 13 S = r2 O' , for example 15 = 17 = , for example —! , for example 19 a tr(A) = m, det(A) = b tr(fl) — 2m — n, det(B) = (—\)n~m (compare with Exercise 7.3.39) 21 + 3/ 23 25 ± l , ± i 27 —1, —1, 2 jc 51 The only eigenfunctions are the nonzero constant functions, with eigenvalue , for example - ,B= l‘ 0 0 55 A = 59 Exercise 58 implies that A and B are both similar to "o i (T , so that A is similar to B the matrix 29 tr(A) = k\ + A.2 + X3 = and det(A) = X1X2 X3 = bed > Therefore, there are one positive and two negative eigenvalues; the positive one is largest in absolute value 31 -fi A is a regular transition matrix (compare with Exercise 30), so that lim (-rA )' exists and has t —►oo identical columns (see Exercise 30) Therefore, the columns of A' are nearly identical for large t 33 c Hint: Let k \ , k 2, • , X5 be the eigenvalues, with k\ > )Ay I, for j = , , Let 1, v2, •, £5 466 A N S W E R S T O O D D - N U M B E R E D E X E R C IS E S be corresponding eigenvectors Write = c\v\ + • • ■+ C5 C5 Then /th column of A1 = Alej = c\k\ i»i H h C5A.5 U5 is nearly parallel to i»j for large t is a stable equilibrium 39 CHAPTER S Answers to more theoretical questions are omitted Y o' * 45 If a is nonzero 47 If a is nonzero 49 If a is neither nor '2 51 Q is a field - 71 53 The binary digits form a field 55 H is not a field (multiplication is noncommutative) f= 1 'V5 '- f '- f -1 -1 ’ V5 7! ’ x/3 7.6 Answers to more theoretical questions are omitted Not stable Not stable Stable Not stable Not stable 13 For all k 17 x(t) = —sin { 41 (-^t) 14 -2 11 -2 -2 O' 468 A N S W E R S T O O D D - N U M B E R E D E X E R C IS E S 43 im(T) = span(*2), rank(T) = 1, ker(r) = span(jcijt2 , x 2), nullity(T) = ' “ " 0 11 45 im(7) = P2, rank(D = 3, ker(T) = span(*| * £ , * 1*3 nullity (T) = - * 1* * 2*3 x 2), 13 l2 aU \ V = m a n - afj > , so that an > aij or a 57 q(x) = X\c\ + A2C2 A3C3 = 1, with positive A.,-, defines an ellipsoid 59 q (*) = k 1c2 = 1, with positive k \ , defines a pair of parallel planes, c\ = ± —= V A-i (ci A.1w>i ••• + c nknwn) = A! ||u>i ||2 ••• 69 Note that x T RT A Rx = (Rx)T A(Rx) > for all x in Rm Thus R T AR is positive semidefinite.^/?^ A/? is positive definite if (and only if) ker R = {0} 71 Anything can happen: The matrix RTAR may be positive definite, positive semidefinite, negative def­ inite, negative semidefinite, or indefinite 8.3 Answers to more theoretical questions are omitted o\ = 2, o2 = All singular values are (since A A = /„) o\ = o2 = \ / p + q r ‘2 O' o 0 ! '5 O' 0 Vs o 1 7! -2 ! 1" U = C\V[ 4C2?2- Note that iimi2 — _ cf J c*2 — 1• Now Au = c\Av[ c2A v 2, so that 65 Adapt the method outlined in Exercise 63 Consider an orthonormal eigenbasis v\, v2 for A with associ­ ated eigenvalues k\ > and A.2 < Now let w\ = C t / V H a n d ^ = V2 /V-A.2 , so that \[w\ \\2 = \ / k\ and \\w2\\2 = —\ / k Then q(c\W\ c2w2) = (,c\xb\ +C2W2) •(Aici ui 1+X2C2 W2 ) = A-icf ||it>i ||2+ X2cl\\m\\2 =C2\ - Cr 67 Adapt the method outlined in Exercises 63 and 65 Consider an orthonormal eigenbasis V[ , , vp, , vr , , for A such that the asso­ ciated eigenvalues kj are positive for j = , , p, negative for j = p 1, , r, and zero for j = r 1, • • • ♦n Let wj = Vj / yj \ kj | for j = 1, , r and \bj = Vj for j = r 1, , n -1 23 A A1J ^ , = / ^ f o r i - ' ' \0 for i = r , • , n The nonzero eigenvalues of A A and A A are the same since ||w) /||2 = —, by construction V 15 Singular values of A are the reciprocals of those of A 0.8 0.6] [ - 21 -0 0.8J [—2 63 q(cxw H -4 cnwn) = (c\w\ H + cnwn) • C2 25 Choose vectors and v2 as in Theorem 8.3.3 Write L q(x) = k\c A.2C2 A.3C3 = 1, with k\ > 0, A.2 > 0, A.3 < defines a hyperboloid of one sheet c2kn\\wn\\2 = cf + • • • + l 47 The determinant of the wth principal submatrix is positive if m is even, and negative if m is odd 55 Note that det r -2 2' \\AS\\2 = c^WAviW2 + }\Av2\\2 97 = cfcrf + oo < (C| + cl)(T2 = °l We conclude that || Au\\ < o \ The proof of