The linear algebra a beginning graduate student ought to know (2nd ed)

THE LINEAR ALGEBRA A BEGINNING GRADUATE STUDENT OUGHT TO KNOW The Linear Algebra a Beginning Graduate Student Ought to Know Second Edition by JONATHAN S GOLAN University of Haifa, Israel A C.I.P Catalogue record for this book is available from the Library of Congress ISBN-10 ISBN-13 ISBN-10 ISBN-13 1-4020-5494-7 (PB) 978-1-4020-5494-5 (PB) 1-4020-5495-5 (e-book) 978-1-4020-5495-2 (e-book) Published by Springer, P.O Box 17, 3300 AA Dordrecht, The Netherlands www.springer.com Printed on acid-free paper All Rights Reserved © 2007 Springer No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work To my grandsons: Shachar, Eitan, and Sarel ĐƠĂÊ ĐƠĂƯ â ('Â đ ,ÊƯă) Contents Notation and terminology Fields Vector spaces over a field 17 Algebras over a field 33 Linear independence and dimension 49 Linear transformations 79 The endomorphism algebra of a vector space 99 Representation of linear transformations by matrices 117 The algebra of square matrices 131 10 Systems of linear equations 169 11 Determinants 199 12 Eigenvalues and eigenvectors 229 13 Krylov subspaces 267 vii viii Contents 14 The dual space 285 15 Inner product spaces 299 16 Orthogonality 325 17 Selfadjoint Endomorphisms 349 18 Unitary and Normal endomorphisms 369 19 Moore-Penrose pseudoinverses 389 20 Bilinear transformations and forms 399 A Summary of Notation 423 Index 427 For whom is this book written? Crow’s Law: Do not think what you want to think until you know what you ought to know.1 Linear algebra is a living, active branch of mathematical research which is central to almost all other areas of mathematics and which has important applications in all branches of the physical and social sciences and in engineering However, in recent years the content of linear algebra courses required to complete an undergraduate degree in mathematics – and even more so in other areas – at all but the most dedicated universities, has been depleted to the extent that it falls far short of what is in fact needed for graduate study and research or for real-world application This is true not only in the areas of theoretical work but also in the areas of computational matrix theory, which are becoming more and more important to the working researcher as personal computers become a common and powerful tool Students are not only less able to formulate or even follow mathematical proofs, they are also less able to understand the underlying mathematics of the numerical algorithms they must use The resulting knowledge gap has led to frustration and recrimination on the part of both students and faculty alike, with each silently – and sometimes not so silently – blaming the other for the resulting state of affairs This book is written with the intention of bridging that gap It was designed be used in one or more of several possible ways: (1) As a self-study guide; (2) As a textbook for a course in advanced linear algebra, either at the upper-class undergraduate level or at the first-year graduate level; or (3) As a reference book It is also designed to be used to prepare for the linear algebra portion of prelim exams or PhD qualifying exams This volume is self-contained to the extent that it does not assume any previous knowledge of formal linear algebra, though the reader is assumed to have been exposed, at least informally, to some basic ideas or techniques, such as matrix manipulation and the solution of a small system of linear equations It does, however, assume a seriousness of purpose, considerable This law, attributed to John Crow of King’s College, London, is quoted by R V Jones in his book Most Secret War ix x For whom is this book written? motivation, and modicum of mathematical sophistication on the part of the reader The book also contains a large number of exercises, many of which are quite challenging, which I have come across or thought up in over thirty years of teaching Many of these exercises have appeared in print before, in such journals as American Mathematical Monthly, College Mathematics Journal, Mathematical Gazette, or Mathematics Magazine, in various mathematics competitions or circulated problem collections, or even on the internet Some were donated to me by colleagues and even students, and some originated in files of old exams at various universities which I have visited in the course of my career Since, over the years, I did not keep track of their sources, all I can is offer a collective acknowledgement to all those to whom it is due Good problem formulators, like the God of the abbot of Citeaux, know their own Deliberately, difficult exercises are not marked with an asterisk or other symbol Solving exercises is an integral part of learning mathematics and the reader is definitely expected to so, especially when the book is used for self-study Solving a problem using theoretical mathematics is often very different from solving it computationally, and so strong emphasis is placed on the interplay of theoretical and computational results Real-life implementation of theoretical results is perpetually plagued by errors: errors in modelling, errors in data acquisition and recording, and errors in the computational process itself due to roundoff and truncation There are further constraints imposed by limitations in time and memory available for computation Thus the most elegant theoretical solution to a problem may not lead to the most efficient or useful method of solution in practice While no reference is made to particular computer software, the concurrent use of a personal computer equipped symbolic-manipulation software such as Maple, Mathematica, Matlab or MuPad is definitely advised In order to show the “human face” of mathematics, the book also includes a large number of thumbnail photographs of researchers who have contributed to the development of the material presented in this volume Acknowledgements Most of the first edition this book was written while the I was a visitor at the University of Iowa in Iowa City and at the University of California in Berkeley I would like to thank both institutions for providing the facilities and, more importantly, the mathematical atmosphere which allowed me to concentrate on writing This edition was extensively revised after I retired from teaching at the University of Haifa in April, 2004 I have talked to many students and faculty members about my plans for this book and have obtained valuable insights from them In particular, I would like to acknowledge the aid of the following colleagues and students who were kind enough to read the preliminary versions of this book and For whom is this book written? xi offer their comments and corrections: Prof Daniel Anderson (University of Iowa), Prof Adi Ben-Israel (Rutgers University), Prof Robert Cacioppo (Truman State University), Prof Joseph Felsenstein (University of Washington), Prof Ryan Skip Garibaldi (Emory University), Mr George Kirkup (University of California, Berkeley), Prof Earl Taft (Rutgers University), Mr Gil Varnik (University of Haifa) Photo credits The photograph of Dr Shmuel Winograd is used with the kind permission of the Department of Computer Science of the City University of Hong Kong The photographs of Prof Ben-Israel, Prof Blass, Prof Kublanovskaya, and Prof Strassen are used with their respective kind permissions The photograph of Prof Greville is used with the kind permission of Mrs Greville The photograph of Prof Rutishauser is used with the kind permission of Prof Walter Gander The photograph of Prof V N Faddeeva is used with the kind permission of Dr Vera Simonova The photograph of Prof Zorn is used with the kind permission of his son, Jens Zorn The photograph of J W Givens was taken from a group photograph of the participants at the 1964 Gatlinburg Conference on Numerical Algebra All other photographs are taken from the MacTutor History of Mathematics Archive website (http://www-history.mcs.standrews.ac.uk/history/index.html), the portrait gallery of mathematicians at the Trucsmatheux website (http://trucsmaths.free.fr/), or similar websites To the best knowledge of the managers of those sites, and to the best of my knowledge, they are in the public domain 420 20 Bilinear transformations and forms Exercise 1071 Let n be a positive integer, let F be a field, and let A ∈ Mn×n (F ) Show that there exists a symmetric matrix B ∈ Mn×n (F ) satisfying v · Av = v · Bv for all v ∈ F n Exercise 1072 Find a bilinear form f ∈ Bil(R3 , R3 ) which defines the   a quadratic form  b  → a2 − 2ab + 4ac − 2bc + 2c2 c Exercise 1073 Let f ∈ Bil(R3 , R3 ) be the symmetric bilinear form de  −3 fined by the matrix  −6  Find the quadratic form defined by f Exercise 1074 Let f ∈ Bil(R3 , R3 ) be the symmetric bilinear form de  −1  Find the quadratic form defined fined by the matrix  −1 −3 by f Exercise 1075 Find a symmetric bilinear form f ∈ Bil(R3 , R3 ) which   a defines the quadratic form  b  → 2ab + 4ac + 6bc c Exercise 1076 Let F be a field of characteristic other than 2, and let V be a vector space over F Let q : V → F be a function satisfying the condition that q(v + w) + q(v − w) = 2q(v) + 2q(w) for all v, w ∈ V Show that the function f : V × V → F defined by f : (v, w) → [q(v + w) − q(v − w)] is a symmetric bilinear form Exercise 1077 Let V be a vector space over a field F of characteristic other than 2, and let f ∈ Bil(V, V ) be a symmetric bilinear form which defines a quadratic form q : V → F Show that q(u + v + w) = q(u + v) + q(u + w) + q(v + w) − q(u) − q(v) − q(w) for all u, v, w ∈ V Exercise 1078 Find the numerical range of the quadratic form q : R2 → R defined by q : v −→ v T 0 v 20 Bilinear transformations and forms Exercise 1079 Let V ∼ = F ⊗ V V be a vector space over a field F 421 Show that Exercise 1080 Let V and W be vector spaces over a field F Let n x ∈ V ⊗ W be written in the form x = i=1 vi ⊗ wi , where n is minimal k in the sense that there is no way to express x in the form i=1 vi ⊗ wi for any k < n Show that {v1 , } is a linearly-independent subset of V and that {w1 , wn } is a linearly-independent subset of W Exercise 1081 Let K be a field containing F as a subfield If V is a vector space over F, show that K ⊗ V is a vector space over K Exercise 1082 Let V be a vector space of finite dimension n over a field F and let Y be the subspace of V ⊗ V generated by all elements of the form v ⊗ v − v ⊗ v, where v, v ∈ V Find the dimension of Y Exercise 1083 Let V and W be finite dimensional vector spaces over a field F Let v, v ∈ V and w, w ∈ W be vectors satisfying the condition v ⊗ w = v ⊗ w and this is not the identity element of V ⊗ W with respect to addition Show that there exists a scalar c ∈ F such that v = cv and w = cw Exercise 1084 Let F be a field and, for all A, B ∈ M2×2 (F ), denote the Kronecker product of A and B by A ⊗ B If {H1 , , H4 } is the canonical basis for M2×2 (F ), is {Hi ⊗ Hj | ≤ i, j ≤ 4} a basis for M4×4 (F ) Exercise 1085 Let n be a positive integer and let F be a field If A ∈ Mn×n (F ) is a magic matrix, is the same true for A ⊗ A ∈ M2n×2n (F )? Exercise 1086 Let F be a field and let k and n be positive integers If matrices A ∈ Mk×k (F ) and B ∈ Mn×n (F ) have eigenvalues a and b respectively, show that ab is an eigenvalue of A ⊗ B Exercise 1087 Let F be a field and let k and n be positive integers If matrices A ∈ Mk×k (F ) and B ∈ Mn×n (F ) have eigenvalues a and b respectively, find a matrix C ∈ Mkn×kn (F ) with eigenvalue a + b Exercise 1088 Let F be a field of characteristic other than and let V be a vector space over F Find the minimal polynomial of the endomorphism α of V ⊗ V defined by n n (vi ⊗ wi ) → α: i=1 (wi ⊗ vi ) i=1 Exercise 1089 Let F be a field, let k, n, s, and t be positive integers, and consider matrices A ∈ Mk×n (F ) and B ∈ Ms×t (F ) Is the rank of A ⊗ B necessarily equal to the product of the ranks of A and B? 422 20 Bilinear transformations and forms Exercise 1090 Let V, V , W, W be vector spaces over a field F and let α : V → V and β : W → W be monic linear transformations Let α ⊗ β be the linear transformation from V ⊗ V to W ⊗ W n n α⊗β : (vi ⊗ vi ) → i=1 [α(vi ) ⊗ β(vi )] i=1 Is α ⊗ β necessarily monic? Exercise 1091 Let F be a field and let (K, •) and (L, ∗) be F -algebras Define an operation on V ⊗ W by setting (v ⊗ w) (v ⊗ w ) = (v • v ) ⊗ (w ∗ w ) for all v.v ∈ K and w, w ∈ L Is (K ⊗ L, ) an F -algebra? Exercise 1092 Let V = R2 and let W = V ⊗ V If w ∈ W is normal, there necessarily exist normal vectors v, v ∈ V such that w = v ⊗ v ? Exercise 1093 Let V be an inner product space over R having a basis {vi | i ∈ Ω} and let W be an inner product space over R having a basis {wj | j ∈ Λ} Define a function µ : (V ⊗ W ) × (V ⊗ W ) → R by setting   µ :  bij (vi ⊗ wj ) → aij (vi ⊗ wj ), i∈Ω j∈Λ aij bij i∈Ω j∈Λ vi , vi + wj , wj i∈Ω j∈Λ Is µ an inner product on V ⊗ W ? Exercise 1094 Let V be a vector space over a field F End(V ) Is the function V ∧ V → V ∧ V defined by n n ci (vi ∧ wi ) → i=1 a linear transformation? ci (α(vi ) ∧ α(wi )) i=1 and let α ∈ Appendix A Summary of Notation Re(z), im(z), z, √ Q( p), Z/(p), GF (p), i∈Ω Vi , 19 Mk×n (V ), 20 V Ω , 19 χB , 21 i∈Ω Vi , 22 (Ω) V , 22 F v, 23 F D, 24 i∈Ω Wi , 27 K − , 36 v × w, 36 K + , 37 deg(f ), 38 F [X], 38 F [g(X)], 38 Φq (X), 42 423 424 Appendix A Summary of Notation µ(d), 42 F [X1 , , Xn ], 44 dim(V ), 64 U ⊕ W, 66 i∈Ω Wi , 67 gr(f ), 81 Hom(V, W ), 82 ker(α), 84 im(α), 85 AT , 85 Af f (V, W ), 87 ∼ =, 87 rk(α), 89 null(α), 89 V /W, 98 End(V ), 99 σ c , 99 Aut(V ), 101 εhk , 102 εh;c , 102 εhk;c , 102 ΦBD , 118 v w, 121 v ∧ w, 121 Ehk , 137 Eh;c , 137 Ehk;c , 137 Sn , 202 sgn(π), 203 |A|, 203 adj(A), 213 spec(α), 229 ρ(A), 233 comp(p), 240 A ∼ B, 241 Ann(v), 247 mA (X), 248 F [α]v0 , 267 LR(V ), 268 D(V ), 285 tr(A), 286 v, w , 300 v · w, 300 DH , 301 v , 305 Appendix A Summary of Notation eA , 316 cos(A), 15 sin(A), 15 v ⊥ w, 326 W ⊥ , 330 α∗ , 339 Rα , 354 SO(n), 375 √ α, 381 α+ , 390 Bil(V × W, Y ), 399 f op , 399 Bil(V × W ), 401 TBD (f ), 402 v ⊥f w, 403 A⊥f , 403 q(v), 407 V ⊗ W, 410 v ⊗ w, 410 A ⊗ B, 413 V ∧ V, 416 V ⊗k , 415 ∧k V, 416 ∧(V ), 416 425 Index alphabet, 20 angle, 325 annihilate, 247 anti-Hermitian matrix, 366 anticommutative algebra, 34 antitrace, 296 Apollonius’ identity, 323 Arnoldi process, 331 associative algebra, 33 automorphism, 101 elementary, 102 of algebras, 101 unitary, 369 Axiom of Choice, addition in a field, vector, 17 adjacency matrix, 353 adjoint matrix, 213 adjoint transformation, 339 affine subset, 86 affine transformation, 86 algebra, 33 anticommutative, 34 associative, 33 Cayley, 302 commutative, 34 division, 45 entire, 38 exterior, 416 Jordan, 37 Lie, 35 optimization, 10 quaternion, 58 tensor, 415 unital, 33 algebraic, 65 algebraic multiplicity, 244 algebraically closed, 43 band matrix, 133 basis, 55 canonical, 57, 271 dual, 290 Hamel, 61 Bernstein function, 94 Bessel’s identity, 336 best approximation, 395 bijective, bilinear form, 401 427 428 Index nondegenerate, 401 bilinear transformation, 399 symmetric, 399 Binet-Cauchy identity, 322 binomial formula, 16 block form, 123 bounded, 60, 307 bounded support, 347 bra-ket product, 121 Brauer’s Theorem, 314 canonical basis, 57, 271 Cantor set, 61 cartesian product, Cassini oval, 314 Cauchy-Schwarz-Bunyakovsky Theorem, 304 Cayley algebra, 302 Cayley-Hamilton Theorem, 250 chain, 53 chain subset, 53 change-of-basis matrix, 146 characteristic, 10 characteristic function, 21 characteristic polynomial, 239, 268 characteristic value, 229 characteristic vector, 229 Chebyshev polynomial, 327 Cholesky decomposition, 360 circulant matrix, 156 co-independent hyperplanes, 298 coefficient, 38 Fourier, 333 leading, 38 Taylor, 104 coefficient matrix, 171 column equivalent, 145 column space, 179 combination linear, 24 commutative algebra, 34 commuting pair, 36 companion matrix, 240 complement, 68 orthogonal, 330 completely reducible, 43 complex conjugate, complex number, condition number, 190 spectral, 383 congruent matrices, 403 conjugate transpose, 301 conjugate, complex, continuant, 221 convex subset, 365 coproduct direct, 22 Courant-Fischer Minimax Theorem, 354 Cramer’s Theorem, 215 cross product, 36 Crout’s algorithm, 153 cyclic, 104 cyclotomic polynomial, 42 decomposition Cholesky, 360 direct sum, 67 LU, 153 polar, 382 QR, 335 singular value, 382 spectral, 379 defective eigenvalue, 244 degree of a generalized eigenvector, 275 of a polynomial, 38 of a polynomial function, 41 derivation, 100 derogatory endomorphism, 244 determinant, 205 determinant function, 199 diagonal matrix, 132 diagonalizable endomorphism, 236 difference set, differential, 80 differential operator, 100 dimension, 63 Index Dirac functional, 289 direct coproduct, 22 direct product, 19 direct sum, 66 direct sum decomposition, 67 discrete cosine transform, 137 discrete Fourier transform, 136, 308 disjoint subspaces, 23 distance, 315 distribution, 286 division algebra, 45 Division Algorithm, 40 domain, integral, 10 dot product, 300 weighted, 301 dual basis, 290 dual space, 285 weak, 290 dyadic product, 130 eigenspace, 231, 233 generalized, 276 eigenvalue, 229, 232 defective, 244 semisimple, 244 simple, 244 eigenvector, 229, 232 generalized, 275 Eisenstein’s criterion, 42 elementary automorphism, 102 elementary matrix, 137 elementary operation, 143 endomorphism, 99 bounded, 112 derogatory, 244 diagonalizable, 236 nilpotent, 272 normal, 375 orthogonally diagonalizable, 352 positive definite, 356 selfadjoint, 349 entangled tensors, 410 entire, 38 429 epic, epimorphism, 85 equal functions, equivalence relation, 106 equivalent matrices, 145 euclidean norm, 305 subfield, 299 evaluation functional, 291 even function, 69 even permutation, 203 Exchange Property, 26 extended coefficient matrix, 171 exterior algebra, 416 exterior power, 416 exterior product, 121 exterior square, 416 factor space, 98 fan, 168 fast Fourier transform, 136 Fibonacci sequence, 269 field, algebraically closed, 43 formally-real, 300 Galois, orderable, 16 field of algebraic numbers, 65 finite dimensional, 63 finitely generated, 26 fixed point, 231 fixed space, 231 form bilinear, 401 Lorentz, 408 quadratic, 407 formal differentiation, 104 formally real field, 300 Fourier coefficient, 333 Fredholm alternative, 293 Frobenius norm, 311 full pivoting, 152 function, characteristic, 21 determinant, 199 430 Index even, 69 inverse, odd, 69 periodic, 62 piecewise constant, 30 spline, 48 functional Dirac, 289 evaluation, 291 linear, 285 zero, 285 Fundamental Theorem of Algebra, 43 Galois field, Gauss-Jordan method, 175 Gauss-Seidel iteration method, 186 Gaussian elimination, 175 general Lie algebra, 131 general quadratic equation, 410 generalized eigenspace, 276 generalized eigenvector, 275 generating set, 25 geometric multiplicity, 244 Gershgorin bound, 312 Gershgorin’s Theorem, 312 Givens rotation matrix, 370 GMRES algorithm, 331 golden ratio, 269 Google matrix, 254 Gram matrix, 303 Gram-Schmidt process, 328 Gram-Schmidt Theorem, 328 graph, 81 Grassmann’s Theorem, 69 Greville’s method, 392 group of automorphisms, 107 Guttman’s Theorem, 138 Haar wavelet, 332 Hadamard inequality, 329 Hadamard matrix, 207 Hadamard product, 157 Hamel basis, 61 Hamming norm, 315 Hankel matrix, 219 Hausdorff Maximum Principle, 60 Hermitian matrix, 351 Hermitian transpose, 301 Hibert subset, 332 Hilbert matrix, 144 Hilbert-Schmidt norm, 311 homogeneous system of linear equations, 170 homomorphism, 79 of algebras, 79 of unital algebras, 79 Householder matrix, 373 hyperplane, 292 ill-conditioned, 189 image, 85 imaginary part, improper subspace, 22 independent subspaces, 67 indeterminate, 38 index of nilpotence, 272 induced norm, 310 infinite dimensional, 63 inner product, 299 inner product space, 300 integral domain, 10 interior product, 121 interpolation problem, 169 intersection, invariant subspace, 103 inverse function, inversion, 202 involution, 341 irreducible, 41 isometry, 362 isomorphic vector spaces, 87 isomorphism, 85 of algebras, 85 of unital algebras, 85 iteration method Gauss-Seidel, 186 Jacobi, 186 Jacobi identity, 36 Index Jacobi iteration method, 186 Jacobi overrelaxation method, 188 Jacobi polynomial, 327 JOR, 188 Jordan algebra, 37 Jordan canonical form, 275, 278 Jordan identity, 37 Jordan product, 37 Karatsuba’s algorithm, 39 kernel, 84 ket-bra product, 121 Kovarik algorithm, 393 Kronecker product, 413 Krylov algorithm, 271 Krylov subspace, 267 Lagrange identity, 306 Lagrange interpolation polynomial, 146 Lanczos algorithm, 271 leading coefficient, 38 leading entry, 174 least squares method, 395 Legendre polynomial, 326 Lie algebra, 35 general, 131 special, 286 Lie product, 36 lightcone inequality, 409 linear combination, 24 linear functional, 285 linear transformation, 79 adjoint of, 339 linearly dependent, 49 locally, 82 linearly independent, 49 linearly recurrent sequence, 268 list, locally linearly dependent, 82 Loewner partial order, 357 Lorentz form, 408 lower-triangular matrix, 134 LU-decomposition, 153 Mă obius function, 42 431 magic matrix, 259 Markov matrix, 135 matrices column-equivalent, 145 congruent, 403 equivalent, 145 row-equivalent, 145 similar, 241 unitarily similar, 370 matrix, 20 adjacency, 353 adjoint, 213 anti-Hermitian, 366 band, 133 change-of-basis, 146 circulant, 156 coefficient, 171 companion, 240 determinant of, 205 diagonal, 132 elementary, 137 extended coefficient, 171 Givens rotation, 370 Google, 254 Gram, 303 Hadamard, 207 Hankel, 219 Hermitian, 351 Hilbert, 144 Householder, 373 in block form, 123 in reduced row echelon form, 174 in row echelon form, 173 lower-triangular, 134 magic, 259 Markov, 135 Nievergelt’s, 144 nonsingular, 135 normal, 384 orthogonal, 372 permutation, 142 quasidefinite, 366 scalar, 132 singular, 135 432 Index skew-symmetric, 134 sparse, 187 special orthogonal, 375 stochastic, 135 symmetric, 134 symmetric Toeplitz, 183 transpose, 85 tridiagonal, 133 unitary, 370 upper Hessenberg, 419 upper-triangular, 133 Vandermonde, 147 zero, 21 maximal, 53 maximal subspace, 292 minimal, 53 minimal polynomial, 248, 268 Minkowski’s inequality, 307 minor, 209 Modular Law, 28 monic function, polynomial, 38 monomorphism, 85 Moore-Penrose pseudoinverse, 389 multiplication in a field, scalar, 17 multiplication table, 58 multiplicity algebraic, 244 geometric, 244 mutually orthogonal, 326 Nievergelt’s matrix, 144 nilpotent, 272 nondegenerate bilinear form, 401 nonhomogeneous system of linear equations, 170 nonsingular matrix, 135 nontrivial subspace, 22 norm, 305, 309 euclidean, 305 Frobenius, 311 Hamming, 315 Hilbert-Schmidt, 311 induced, 310 spectral, 311 normal endomorphism, 375 normal matrix, 384 normal vector, 305 normed space, 309 nullity, 89 number complex, rational, real, numerical range, 409 odd function, 69 odd permutation, 203 operation elementary, 143 opposite transformation, 399 optimization algebra, 10 order of recurrence, 268 orderable field, 16 orthogonal, 326 orthogonal complement, 330 right, 403 orthogonal matrix, 372 orthogonal projection, 330 orthogonality with respect to a bilinear form, 403 orthogonally diagonalizable, 352 orthonormal, 331 Pad´e approximant, 216 pairwise disjoint, 24 Parallelogram law, 307 parity, 401 Parseval’s identity, 336 partial order, 53 Loewner, 357 partial pivoting, 152 partially-ordered set, 53 periodic function, 62 permutation, even, 203 odd, 203 Index permutation matrix, 142 Pfaffian, 206 piecewise constant, 30 pivot, 152 pivoting full, 152 partial, 152 Poincar´e-Birkhoff-Witt Theorem, 36 polar decomposition, 382 polynomial, 38 characteristic, 239, 268 Chebyshev, 327 completely reducible, 43 cyclotomic, 42 in several indeterminates, 44 irreducible, 41 Jacobi, 327 Lagrange interpolation, 146 Legendre, 326 minimal, 248, 268 monic, 38 reducible, 41 zero, 38 polynomial function, 40 positive definite, 356 power exterior, 416 pre-Banach space, 309 pre-Hilbert space, 300 primitive root of unity, 136 process Arnoldi, 331 Gram-Schmidt, 328 product bra-ket, 121 cartesian, cross, 36 direct, 19 dot, 300 dyadic, 130 exterior, 121 Hadamard, 157 inner, 299 interior, 121 433 Jordan, 37 ket-bra, 121 Kronecker, 413 Lie, 36 scalar triple, 306 Schur, 157 tensor, 411 vector triple, 306 projection, 104 onto an affine set, 342 orthogonal, 330 proper subspace, 22 pseudoinverse Moore-Penrose, 389 QR-decomposition, 335 quadratic form, 407 quadratic surface, 410 quasidefinite matrix, 366 quaternion algebra, 58 real, 58 range, numerical, 409 rank, 89, 179, 403 Rational Decomposition Theorem, 274 rational number, Rayleigh quotient function, 354 iteration scheme, 355 real euclidean, 299 real number, real part, real quaternion, 58 reduced row echelon form, 174 reducible, 41 relation equivalence, 106 partial order, 53 relaxation method, 188 restriction, Riesz Representation Theorem, 338 right orthogonal complement, 403 434 Index row echelon form, 173 row equivalent, 145 row space, 179 scalar, 18 scalar matrix, 132 scalar multiplication, 17 scalar triple product, 306 Schur product, 157 Schur’s Theorem, 371 selfadjoint, 349 semifield, 10 semisimple eigenvalue, 244 sequence, Fibonacci, 269 linearly recurrent, 268 set difference, generating, 25 partially-ordered, 53 spanning, 25 Sherman-Morrison-Woodbury Theorem, 138 signum, 203 similar matrices, 241 simple eigenvalue, 244 simple tensors, 410 singular matrix, 135 singular value, 383 Singular Value Decomposition Theorem, 382 skew symmetric matrix, 134 solution set, 171 solution space, 171 SOR, 188 space dual, 285 inner product, 300 normed, 309 pre-Banach, 309 pre-Hilbert, 300 solution, 171 spanning set, 25 sparse matrix, 187 special Lie algebra, 286 special orthogonal matrix, 375 spectral condition number, 383 Spectral Decomposition Theorem, 379 spectral norm, 311 spectral radius, 233 spectrum, 229 spline function, 48 Steinitz Replacement Property, 52 stochastic matrix, 135 Strassen-Winograd algorithm, 150 subalgebra, 35 unital, 35 subfield, euclidean, 299 real euclidean, 299 subset affine, 86 bounded, 60 chain, 53 convex, 365 Hilbert, 332 orthonormal, 331 underlying, subspace, 22 cyclc, 104 generated by, 25 improper, 22 invariant, 103 Krylov, 267 maximal, 292 nontrivial, 22 proper, 22 spanned by, 25 trivial, 22 subspaces disjoint, 23 independent, 67 pairwise disjoint, 24 successive overrelaxation method, 188 Sylvester’s Theorem, 89 symmetric bilinear transformation, 399 symmetric difference, 21 Index symmetric matrix, 134 symmetric Toeplitz matrix, 183 system of linear equations, 169 homogeneous, 170 nonhomogeneous, 170 Taylor coefficient, 104 tensor algebra, 415 tensor product, 411 tensors entangled, 410 simple, 410 trace, 286 transcendental, 65 transform discrete cosine, 137 discrete Fourier, 136, 308 fast Fourier, 136 transformation affine, 86 bilinear, 399 linear, 79 opposite, 399 transpose, 85 conjugate, 301 Hermitian, 301 Triangle difference inequality, 307 Triangle inequality, 315 tridiagonal matrix, 133 trivial subspace, 22 underlying subset, union, unit, 34 unital algebra, 33 unital subalgebra, 35 unitarily similar matrices, 370 unitary automorphism, 369 unitary matrix, 370 upper Hessenberg matrix, 419 upper-triangular matrix, 133 Vandermonde matrix, 147 vector, 18 normal, 305 435 vector addition, 17 vector space, 18 finite dimensional, 63 finitely generated, 26 infinite dimensional, 63 vector triple product, 306 vectors orthogonal, 326 orthogonal with respect to a bilinear form, 403 wavelet Haar, 332 weak dual space, 290 weighted dot product, 301 Well Ordering Principle, 53 width of a band matrix, 133 word, 20 Wronskian, 207 Zlobec’s formula, 393 Zorn’s Lemma, 60 .. .THE LINEAR ALGEBRA A BEGINNING GRADUATE STUDENT OUGHT TO KNOW The Linear Algebra a Beginning Graduate Student Ought to Know Second Edition by JONATHAN S GOLAN University of Haifa, Israel A. .. quantum mechanics and quantum electrodynamics The algebraic structure of Lie algebras and Jordan algebras was studied in detail by 20th-century American mathematicians Nathan Jacobson and A Adrian... algebra is a living, active branch of mathematical research which is central to almost all other areas of mathematics and which has important applications in all branches of the physical and social