May 27, 2005 12:24 l57-fm-student Sheet number Page number i Linear Algebra and Its Applications THIRD EDITION UPDATE David C Lay University of Maryland – College Park Boston San Francisco New York London Toronto Sydney Tokyo Singapore Madrid Mexico City Munich Paris Cape Town Hong Kong Montreal cyan magenta yellow black May 27, 2005 12:24 l57-fm-student Sheet number Page number ii cyan magenta yellow black Publisher: Greg Tobin Acquisitions Editor: William Hoffman Project Editor: Joanne Ha Editorial Assistant: Emily Portwood Managing Editor: Karen Wernholm Production Supervisor: Sheila Spinney Senior Designer/Cover Designer: Barbara T Atkinson Photo Researcher: Beth Anderson Digital Assets Manager: Jason Miranda Media Producer: Sara Anderson Software Development: David Malone and Mary Durnwald Marketing Manager: Phyllis Hubbard Marketing Coordinator: Celena Carr Senior Author Support/Technology Specialist: Joe Vetere Rights and Permissions Advisor: Dana Weightman Senior Manufacturing Buyer: Evelyn Beaton Composition: Techsetters, Inc Illustrations: Techsetters, Inc Photo Credits: Bettmann/Corbis; Hulton Archive 58, 63, 98, 156, 185, 252, 426, 469 PhotoDisc 105 The Boeing Company 106 Boeing Phantom Works 140 Jet Propulsion Lab/NASA 161 Bo Strain; Reprinted by permission of University of North Carolina at Chapel Hill 215 Kennedy Space Center 289, 469 Eyewire 301 Stone 373 Corbis 374 From North American Datum of 1983, Charles Schwartz editor, National Geodetic Information Center 426 Anglo-Australian Observatory/Royal Observatory, Edinburgh 447 NASA 448 GEOPIC images courtesy of Earth Satellite Corporation, Rockville, MD Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks Where those designations appear in this book, and Addison-Wesley was aware of a trademark claim, the designations have been printed in initial caps or all caps MATLAB is a registered trademark of The MathWorks, Inc Library of Congress Cataloging-in-Publication Data Lay, David C Linear algebra and its applications / David C Lay – 3rd ed update p cm Includes index ISBN 0-321-28713-4 (alk paper) Algebra, Linear–Textbooks I Title QA184.2.L39 2006 512 5–dc22 2005042186 Copyright © 2006 Pearson Education, Inc All rights reserved No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher Printed in the United States of America For information on obtaining permission for use of material in this work, please submit written request to Pearson Education, Inc., Rights and Contracts Department, 75 Arlington Street, Suite 300, Boston, MA 02116, fax your request to 617-848-7047, or e-mail at http://www.pearsoned.com/legal/permissions.htm 10—QWT—09 08 07 06 05 May 27, 2005 12:24 l57-fm-student Sheet number Page number iii cyan magenta yellow black To my wife, Lillian, and our children, Christina, Deborah, and Melissa, whose support, encouragement, and faithful prayers made this book possible May 27, 2005 12:24 l57-fm-student Sheet number Page number iv cyan magenta yellow black About the Author David C Lay holds a B.A from Aurora University (Illinois), and an M.A and Ph.D from the University of California at Los Angeles Lay has been an educator and research mathematician since 1966, mostly at the University of Maryland, College Park He has also served as a visiting professor at the University of Amsterdam, the Free University in Amsterdam, and the University of Kaiserslautern, Germany He has over 30 research articles published in functional analysis and linear algebra As a founding member of the NSF-sponsored Linear Algebra Curriculum Study Group, Lay has been a leader in the current movement to modernize the linear algebra curriculum Lay is also a co-author of several mathematics texts, including Introduction to Functional Analysis with Angus E Taylor, Calculus and Its Applications, with L J Goldstein and D I Schneider, and Linear Algebra Gems—Assets for Undergraduate Mathematics, with D Carlson, C R Johnson, and A D Porter A top-notch educator, Professor Lay has received four university awards for teaching excellence, including, in 1996, the title of Distinguished Scholar–Teacher of the University of Maryland In 1994, he was given one of the Mathematical Association of America’s Awards for Distinguished College or University Teaching of Mathematics He has been elected by the university students to membership in Alpha Lambda Delta National Scholastic Honor Society and Golden Key National Honor Society In 1989, Aurora University conferred on him the Outstanding Alumnus award Lay is a member of the American Mathematical Society, the Canadian Mathematical Society, the International Linear Algebra Society, the Mathematical Association of America, Sigma Xi, and the Society for Industrial and Applied Mathematics Since 1992, he has served several terms on the national board of the Association of Christians in the Mathematical Sciences May 27, 2005 12:24 l57-fm-student Sheet number Page number v cyan magenta yellow black Contents Preface ix A Note to Students CHAPTER xv Linear Equations in Linear Algebra INTRODUCTORY EXAMPLE: and Engineering 1.1 Systems of Linear Equations 1.2 Row Reduction and Echelon Forms 1.3 Vector Equations 1.4 The Matrix Equation Ax = b 1.5 Solution Sets of Linear Systems 50 1.6 Applications of Linear Systems 57 1.7 Linear Independence 1.8 Introduction to Linear Transformations 73 1.9 The Matrix of a Linear Transformation 82 1.10 Linear Models in Business, Science, and Engineering Matrix Algebra 14 40 65 92 102 105 INTRODUCTORY EXAMPLE: Design 28 Supplementary Exercises CHAPTER Linear Models in Economics Computer Models in Aircraft 105 2.1 Matrix Operations 107 2.2 The Inverse of a Matrix 2.3 Characterizations of Invertible Matrices 2.4 Partitioned Matrices 2.5 Matrix Factorizations 2.6 The Leontief Input–Output Model 118 128 134 142 152 v June 1, 2005 11:33 vi l57-fm-student Sheet number Page number vi cyan magenta yellow black Contents 2.7 Applications to Computer Graphics 2.8 Subspaces of Rn 2.9 Dimension and Rank 176 Supplementary Exercises CHAPTER Determinants 183 185 Determinants in Analytic Geometry INTRODUCTORY EXAMPLE: 3.1 Introduction to Determinants 3.2 Properties of Determinants 3.3 Cramer’s Rule, Volume, and Linear Transformations Supplementary Exercises CHAPTER Vector Spaces 186 192 201 211 Space Flight and Control Systems 4.1 Vector Spaces and Subspaces 4.2 Null Spaces, Column Spaces, and Linear Transformations 4.3 Linearly Independent Sets; Bases 4.4 Coordinate Systems 216 4.5 The Dimension of a Vector Space 4.6 Rank 4.7 Change of Basis 4.8 Applications to Difference Equations 4.9 Applications to Markov Chains 237 246 256 262 271 Supplementary Exercises 277 288 298 Eigenvalues and Eigenvectors INTRODUCTORY EXAMPLE: and Spotted Owls 185 215 INTRODUCTORY EXAMPLE: CHAPTER 158 167 301 Dynamical Systems 301 5.1 Eigenvectors and Eigenvalues 302 5.2 The Characteristic Equation 5.3 Diagonalization 5.4 Eigenvectors and Linear Transformations 5.5 Complex Eigenvalues 5.6 Discrete Dynamical Systems 310 319 335 342 327 226 215 May 27, 2005 12:24 l57-fm-student Sheet number Page number vii cyan magenta yellow black Contents 5.7 Applications to Differential Equations 5.8 Iterative Estimates for Eigenvalues Supplementary Exercises CHAPTER INTRODUCTORY EXAMPLE: Readjusting the North American 6.1 Inner Product, Length, and Orthogonality 6.2 Orthogonal Sets 6.3 Orthogonal Projections 6.4 The Gram–Schmidt Process 6.5 Least-Squares Problems 6.6 Applications to Linear Models 6.7 Inner Product Spaces 6.8 Applications of Inner Product Spaces 375 384 394 402 409 419 427 436 444 Symmetric Matrices and Quadratic Forms INTRODUCTORY EXAMPLE: 7.1 Diagonalization of Symmetric Matrices 7.2 Quadratic Forms 449 7.3 Constrained Optimization 7.4 The Singular Value Decomposition 7.5 Applications to Image Processing and Statistics 455 463 471 491 The Geometry of Vector Spaces INTRODUCTORY EXAMPLE: 8.1 Affine Combinations 8.2 Affine Independence 8.3 Convex Combinations 8.4 Hyperplanes 8.5 Polytopes 8.6 Curves and Surfaces 447 Multichannel Image Processing Supplementary Exercises CHAPTER ONLINE ONLY 373 373 Supplementary Exercises 370 Orthogonality and Least Squares Datum CHAPTER 353 363 The Platonic Solids 482 447 vii May 31, 2005 12:21 viii l57-fm-student Sheet number Page number viii Contents CHAPTER ONLINE ONLY Optimization INTRODUCTORY EXAMPLE: The Berlin Airlift 9.1 Matrix Games 9.2 Linear Programming—Geometric Method 9.3 Linear Programming—Simplex Method 9.4 Duality Appendixes A Uniqueness of the Reduced Echelon Form B Complex Numbers Glossary A3 A9 Answers to Odd-Numbered Exercises Index I1 A19 A1 cyan magenta yellow black May 27, 2005 12:24 l57-fm-student Sheet number Page number ix cyan magenta yellow black Preface The response of students and teachers to the first three editions of Linear Algebra and Its Applications has been most gratifying This Third Edition Update provides substantial new support both for teaching and for using technology in the course As before, the text provides a modern elementary introduction to linear algebra and a broad selection of interesting applications The material is accessible to students with the maturity that should come from successful completion of two semesters of college-level mathematics, usually calculus The main goal of the text is to help students master the basic concepts and skills they will use later in their careers The topics here follow the recommendations of the Linear Algebra Curriculum Study Group, which were based on a careful investigation of the real needs of the students and a consensus among professionals in many disciplines that use linear algebra Hopefully, this course will be one of the most useful and interesting mathematics classes taken as an undergraduate DISTINCTIVE FEATURES Early Introduction of Key Concepts Many fundamental ideas of linear algebra are introduced within the first seven lectures, in the concrete setting of Rn , and then gradually examined from different points of view Later generalizations of these concepts appear as natural extensions of familiar ideas, visualized through the geometric intuition developed in Chapter A major achievement of the text, I believe, is that the level of difficulty is fairly even throughout the course A Modern View of Matrix Multiplication Good notation is crucial, and the text reflects the way scientists and engineers actually use linear algebra in practice The definitions and proofs focus on the columns of a matrix rather than on the matrix entries A central theme is to view a matrix–vector product Ax as a linear combination of the columns of A This modern approach simplifies many arguments, and it ties vector space ideas into the study of linear systems ix May 27, 2005 12:24 x l57-fm-student Sheet number 10 Page number x cyan magenta yellow black Preface Linear Transformations Linear transformations form a “thread” that is woven into the fabric of the text Their use enhances the geometric flavor of the text In Chapter 1, for instance, linear transformations provide a dynamic and graphical view of matrix–vector multiplication Eigenvalues and Dynamical Systems Eigenvalues appear fairly early in the text, in Chapters and Because this material is spread over several weeks, students have more time than usual to absorb and review these critical concepts Eigenvalues are motivated by and applied to discrete and continuous dynamical systems, which appear in Sections 1.10, 4.8, 4.9, and in five sections of Chapter Some courses reach Chapter after about five weeks by covering Sections 2.8 and 2.9 instead of Chapter These two optional sections present all the vector space concepts from Chapter needed for Chapter Orthogonality and Least-Squares Problems These topics receive a more comprehensive treatment than is commonly found in beginning texts The Linear Algebra Curriculum Study Group has emphasized the need for a substantial unit on orthogonality and least-squares problems, because orthogonality plays such an important role in computer calculations and numerical linear algebra and because inconsistent linear systems arise so often in practical work PEDAGOGICAL FEATURES Applications A broad selection of applications illustrates the power of linear algebra to explain fundamental principles and simplify calculations in engineering, computer science, mathematics, physics, biology, economics, and statistics Some applications appear in separate sections; others are treated in examples and exercises In addition, each chapter opens with an introductory vignette that sets the stage for some application of linear algebra and provides a motivation for developing the mathematics that follows Later, the text returns to that application in a section near the end of the chapter A Strong Geometric Emphasis Every major concept in the course is given a geometric interpretation, because many students learn better when they can visualize an idea There are substantially more drawings here than usual, and some of the figures have never appeared before in a linear algebra text Examples This text devotes a larger proportion of its expository material to examples than most linear algebra texts There are more examples than an instructor would ordinarily present in class But because the examples are written carefully, with lots of detail, students can read them on their own May 24, 2005 12:52 A16 L57-glossary Sheet number Page number A16 cyan magenta yellow black Glossary principal components (of the data in a matrix B of observations): The unit eigenvectors of a sample covariance matrix S for B, with the eigenvectors arranged so that the corresponding eigenvalues of S decrease in magnitude If B is in mean-deviation form, then the principal components are the right singular vectors in a singular value decomposition of B T probability vector: A vector in Rn whose entries are nonnegative and sum to one product Ax: The linear combination of the columns of A using the corresponding entries in x as weights production vector: The vector in the Leontief input–output model that lists the amounts that are to be produced by the various sectors of an economy projection matrix (or orthogonal projection matrix): A symmetric matrix B such that B = B A simple example is B = vvT , where v is a unit vector proper subspace: Any subspace of a vector space V other than V itself pseudoinverse (of A): The matrix VD −1 U T , when UDV T is a reduced singular value decomposition of A Q quadratic form: A function Q defined for x in Rn by Q(x) = xTAx, where A is an n×n symmetric matrix (called the matrix of the quadratic form) QR factorization: A factorization of an m×n matrix A with linearly independent columns, A = QR, where Q is an m×n matrix whose columns form an orthonormal basis for Col A, and R is an n×n upper triangular invertible matrix with positive entries on its diagonal R range (of a linear transformation T ): The set of all vectors of the form T (x) for some x in the domain of T rank (of a matrix A): The dimension of the column space of A, denoted by rank A Rayleigh quotient: R(x) = (xTAx)/(xT x) An estimate of an eigenvalue of A (usually a symmetric matrix) recurrence relation: See difference equation reduced echelon form (or reduced row echelon form): A reduced echelon matrix that is row equivalent to a given matrix reduced echelon matrix: A rectangular matrix in echelon form that has these additional properties: The leading entry in each nonzero row is 1, and each leading is the only nonzero entry in its column reduced singular value decomposition: A factorization A = UDV T , for an m×n matrix A of rank r, where U is m×r with orthonormal columns, D is an r ×r diagonal matrix with the r nonzero singular values of A on its diagonal, and V is n×r with orthonormal columns regression coefficients: The coefficients β0 and β1 in the leastsquares line y = β0 + β1 x regular stochastic matrix: A stochastic matrix P such that some matrix power P k contains only strictly positive entries relative change or relative error (in b): The quantity b / b when b is changed to b + b repellor (of a dynamical system in R2 ): The origin when all trajectories except the constant zero sequence or function tend away from residual vector: The quantity that appears in the general linear model: y = Xβ + ; that is, = y − Xβ, the difference between the observed values and the predicted values (of y) Re x: The vector in Rn formed from the real parts of the entries of a vector x in Cn right inverse (of A): Any rectangular matrix C such that AC = I right-multiplication (by A): Multiplication of a matrix on the right by A right singular vectors (of A): The columns of V in the singular value decomposition A = U V T roundoff error: Error in floating point arithmetic caused when the result of a calculation is rounded (or truncated) to the number of floating point digits stored Also, the error that results when the decimal representation of a number such as 1/3 is approximated by a floating point number with a finite number of digits row–column rule: The rule for computing a product AB in which the (i, j )-entry of AB is the sum of the products of corresponding entries from row i of A and column j of B row equivalent (matrices): Two matrices for which there exists a (finite) sequence of row operations that transforms one matrix into the other row reduction algorithm: A systematic method using elementary row operations that reduces a matrix to echelon form or reduced echelon form row replacement: An elementary row operation that replaces one row of a matrix by the sum of the row and a multiple of another row row space (of a matrix A): The set Row A of all linear combinations of the vectors formed from the rows of A; also denoted by Col AT row sum: The sum of the entries in a row of a matrix May 24, 2005 12:52 L57-glossary Sheet number Page number A17 cyan magenta yellow black Glossary row vector: A matrix with only one row, or a single row of a matrix that has several rows row–vector rule for computing Ax: The rule for computing a product Ax in which the ith entry of Ax is the sum of the products of corresponding entries from row i of A and from the vector x S saddle point (of a dynamical system in R2 ): The origin when some trajectories are attracted to and other trajectories are repelled from same direction (as a vector v): Avector that is a positive multiple of v sample mean: The average M of a set of vectors, X1 , , XN , given by M = (1/N)(X1 + · · · + XN ) scalar: A (real) number used to multiply either a vector or a matrix scalar multiple of u by c: The vector cu obtained by multiplying each entry in u by c scale (a vector): Multiply a vector (or a row or column of a matrix) by a nonzero scalar Schur complement: A certain matrix formed from the blocks of a 2×2 partitioned matrix A = [Aij ] If A11 is invertible, its Schur complement is given by A22 − A21 A−1 11 A12 If A22 is invertible, its Schur complement is given by A11 − A12 A−1 22 A21 Schur factorization (of A, for real scalars): A factorization A = URU T of an n×n matrix A having n real eigenvalues, where U is an n×n orthogonal matrix and R is an upper triangular matrix set spanned by {v1 , , vp }: The set Span {v1 , , vp } signal (or discrete-time signal): A doubly infinite sequence of numbers, {yk }; a function defined on the integers; belongs to the vector space S similar (matrices): Matrices A and B such that P −1 AP = B, or equivalently, A = PBP −1 , for some invertible matrix P similarity transformation: A transformation that changes A into P −1 AP singular (matrix): A square matrix that has no inverse singular value decomposition (of an m×n matrix A): A = U V T , where U is an m×m orthogonal matrix, V is an n×n orthogonal matrix, and is an m×n matrix with nonnegative entries on the main diagonal (arranged in decreasing order of magnitude) and zeros elsewhere If rank A = r, then has exactly r positive entries (the nonzero singular values of A) on the diagonal A17 singular values (of A): The (positive) square roots of the eigenvalues of ATA, arranged in decreasing order of magnitude size (of a matrix): Two numbers, written in the form m×n, that specify the number of rows (m) and columns (n) in the matrix solution (of a linear system involving variables x1 , , xn ): A list (s1 , s2 , , sn ) of numbers that makes each equation in the system a true statement when the values s1 , , sn are substituted for x1 , , xn , respectively solution set: The set of all possible solutions of a linear system The solution set is empty when the linear system is inconsistent Span {v1 , , vp }: The set of all linear combinations of v1 , , vp Also, the subspace spanned (or generated) by v1 , , vp spanning set (for a subspace H ): Any set {v1 , , vp } in H such that H = Span {v1 , , vp } spectral decomposition (of A): A representation A = λ1 u1 uT1 + · · · + λn un uTn where {u1 , , un } is an orthonormal basis of eigenvectors of A, and λ1 , , λn are the corresponding eigenvalues of A spiral point (of a dynamical system in R2 ): The origin when the trajectories spiral about stage-matrix model: A difference equation xk+1 = Axk where xk lists the number of females in a population at time k, with the females classified by various stages of development (such as juvenile, subadult, and adult) standard basis: The basis E = {e1 , , en } for Rn consisting of the columns of the n×n identity matrix, or the basis {1, t, , t n } for Pn standard matrix (for a linear transformation T ): The matrix A such that T (x) = Ax for all x in the domain of T standard position: The position of the graph of an equation xTAx = c when A is a diagonal matrix state vector: A probability vector In general, a vector that describes the “state” of a physical system, often in connection with a difference equation xk+1 = Axk steady-state vector (for a stochastic matrix P ): A probability vector q such that P q = q stiffness matrix: The inverse of a flexibility matrix The j th column of a stiffness matrix gives the loads that must be applied at specified points on an elastic beam in order to produce a unit deflection at the j th point on the beam stochastic matrix: A square matrix whose columns are probability vectors strictly dominant eigenvalue: An eigenvalue λ1 of a matrix A with the property that |λ1 | > |λk | for all other eigenvalues λk of A May 24, 2005 12:52 A18 L57-glossary Sheet number 10 Page number A18 cyan magenta yellow black Glossary submatrix (of A): Any matrix obtained by deleting some rows and/or columns of A; also, A itself subspace: A subset H of some vector space V such that H has these properties: (1) the zero vector of V is in H ; (2) H is closed under vector addition; and (3) H is closed under multiplication by scalars symmetric matrix: A matrix A such that AT = A system of linear equations (or a linear system): A collection of one or more linear equations involving the same set of variables, say, x1 , , xn T total variance: The trace of the covariance matrix S of a matrix of observations trace (of a square matrix A): The sum of the diagonal entries in A, denoted by tr A trajectory: The graph of a solution {x0 , x1 , x2 , } of a dynamical system xk+1 = Axk , often connected by a thin curve to make the trajectory easier to see Also, the graph of x(t) for t ≥ 0, when x(t) is a solution of a differential equation x (t) = Ax(t) transfer matrix: A matrix A associated with an electrical circuit having input and output terminals, such that the output vector is A times the input vector transformation (or function, or mapping) T from Rn to Rm : A rule that assigns to each vector x in Rn a unique vector T (x) in Rm Notation: T : Rn → Rm Also, T : V → W denotes a rule that assigns to each x in V a unique vector T (x) in W translation (by a vector p): The operation of adding p to a vector or to each vector in a given set transpose (of A): An n×m matrix AT whose columns are the corresponding rows of the m×n matrix A trend analysis: The use of orthogonal polynomials to fit data, with the inner product given by evaluation at a finite set of points triangle inequality: u + v ≤ u + v for all u, v triangular matrix: A matrix A with either zeros above or zeros below the diagonal entries trigonometric polynomial: Alinear combination of the constant function and sine and cosine functions such as cos nt and sin nt trivial solution: The solution x = of a homogeneous equation Ax = U uncorrelated variables: Any two variables xi and xj (with i = j ) that range over the ith and j th coordinates of the observation vectors in an observation matrix, such that the covariance sij is zero underdetermined system: A system of equations with fewer equations than unknowns uniqueness question: Asks, “If a solution of a system exists, is it unique; that is, is it the only one?” unit consumption vector: A column vector in the Leontief input–output model that lists the inputs a sector needs for each unit of its output; a column of the consumption matrix unit lower triangular matrix: A square lower triangular matrix with ones on the main diagonal unit vector: A vector v such that v = upper triangular matrix: A matrix U (not necessarily square) with zeros below the diagonal entries u11 , u22 , V Vandermonde matrix: An n×n matrix V or its transpose, when V has the form x x 21 · · · x1n−1 n−1 1 x2 x2 · · · x2 V = n−1 xn xn · · · xn variance (of a variable xj ): The diagonal entry sjj in the covariance matrix S for a matrix of observations, where xj varies over the j th coordinates of the observation vectors vector: A list of numbers; a matrix with only one column In general, any element of a vector space vector addition: Adding vectors by adding corresponding entries vector equation: An equation involving a linear combination of vectors with undetermined weights vector space: A set of objects, called vectors, on which two operations are defined, called addition and multiplication by scalars Ten axioms must be satisfied See the first definition in Section 4.1 vector subtraction: Computing u + (−1)v and writing the result as u − v W weighted least squares: Least-squares problems with a weighted inner product such as x, y = w12 x1 y1 + · · · + wn2 xn yn weights: The scalars used in a linear combination Z zero subspace: The subspace {0} consisting of only the zero vector zero vector: The unique vector, denoted by 0, such that u + = u for all u In Rn , is the vector whose entries are all zero May 24, 2005 12:54 L57-index Sheet number Page number cyan magenta yellow black Index Accelerator-multiplier model, 286n Adjoint, classical, 203 Adjugate, 203 Affine transformation, 81 Aircraft design, 105, 134 Algebraic multiplicity of an eigenvalue, 314 Algebraic properties of Rn , 32, 40 Algorithms bases for Col A, Row A, Nul A, 262–265 compute a B-matrix, 332 decouple a system, 347–348, 358 diagonalization, 321–322 finding A−1 , 124–125 finding change-of-coordinates matrix, 274 Gram–Schmidt process, 402–405 inverse power method, 366 Jacobi’s method, 317 LU factorization, 142–146 QR algorithm, 317, 318, 368 reduction to first-order system, 284 row–column rule for computing AB, 111 row reduction, 17–20 row–vector rule for computing Ax, 45 singular value decomposition, 476 solving a linear system, 24 steady-state vector, 293 writing solution set in parametric vector form, 54 Amps, 95 Analysis of data, 142 See also Matrix factorization Angles in R2 and R3 , 381 Anticommutativity, 183 Approximation, 314 Area approximating, 208–209 ellipse, 209 parallelogram, 205–207 triangle, 210 Argument of complex number, A6 Associative law (multiplication), 113 Associative property (addition), 108 Attractor, 345, 356 Augmented matrix, Auxiliary equation, 282 Average value, 434 Axioms inner product space, 428 vector space, 215 B-coordinate vector, 176, 247 B-coordinates, 246 B-matrix, 329 Back-substitution, 22–23 Backward phase, 20, 23, 144 Balancing chemical equations, 59–60, 63 Band matrix, 150 Basic variable, 20–21 Basis, 170–173, 238, 256–257 change of, 271–275 change of, in Rn , 274 column space, 171–172, 240–242, 264–265 coordinate systems, 246–253 eigenspace, 304 eigenvectors, 321, 324 fundamental set of solutions, 354 fundamental subspaces, 478–479 null space, 240, 264–265 orthogonal, 385–386, 402, 430–431 orthonormal, 389, 405–406, 451, 473 row space, 263, 265n solution space, 283 spanning set, 239 standard, 170, 238, 247–248, 389 subspace, 170 two views, 242 Basis Theorem, 179, 259 Beam model, 120–121 Bessel’s inequality, 444 Best approximation C[a, b], 440 Fourier, 441 P4 , 431 to y by elements of W , 398 Best Approximation Theorem, 398 Bidiagonal matrix, 151 Bill of final demands, 152 Block matrix, 134 diagonal, 138 multiplication, 136 upper triangular, 137 Boundary condition, 286 I1 May 24, 2005 12:54 I2 L57-index Sheet number Page number cyan magenta yellow black Index Branch current, 97 Branches in network, 60, 95 Budget constraint, 468 C (language), 46, 115 C[a, b], 224, 433, 440 Cn , 335 Cambridge Diet, 93, 100 Casorati matrix, 279 Cauchy, Augustin-Louis, 185 Cauchy–Schwarz inequality, 432 Cayley–Hamilton Theorem, 371 Center of gravity (mass), 39 Center of projection, 163 CFD See Computational fluid dynamics Change of basis, 271–273 in Rn , 274 Change-of-coordinates matrix, 249, 273–275 Change of variable for complex eigenvalue, 340 in differential equation, 358 in dynamical system, 347–348 in principal component analysis, 486 in a quadratic form, 457–458 Characteristic equation of matrix, 310, 313, 335 Characteristic polynomial, 314, 317 Characterization of Linearly Dependent Sets Theorem, 68 Chemical equations, 59–60, 63 Cholesky factorization, 462, 492 Classical adjoint, 203 Codomain, 74 Coefficient correlation, 382 filter, 280 Fourier, 441 of linear equation, matrix, regression, 419 trend, 439 Cofactor expansion, 188, 196 Column(s) augmented, 125 determinants, 196 operations, 196 orthogonal, 414 orthonormal, 390–391 pivot, 15, 241, 266, A1 span Rm , 43 sum, 154 vector, 28 Column–row expansion, 137 Column space, 229 basis for, 171–172, 240–241, 264–265 dimension of, 259, 265 least-squares problem, 409–411 and null space, 230–232 subspace, 169, 229 See also Fundamental subspaces Comet, orbit of, 426 Commutativity, 114, 183 Companion matrix, 372 Complement, orthogonal, 380 Complex number, A3 absolute value of, A4 argument of, A6 conjugate, A4 imaginary axis, A5 polar coordinates, A6–A7 powers of, A7 real and imaginary parts, A3 and R2 , A8 Complex root, 282, 314, 335 See also Auxiliary equation; Eigenvalue, complex Complex vector, 28n real and imaginary parts, 337 Complex vector space, 217n, 335 Component of y orthogonal to u, 386 Composition of linear transformations, 110, 148 Composition of mappings, 109, 160 Computational fluid dynamics (CFD), 105 Computer graphics, 158 center of projection, 163 composite transformations, 160 homogeneous coordinates, 159, 162–163 perspective projections, 163–165 shear transformations, 159 3D, 161–163 Condition number, 131, 133, 200, 445 singular value decomposition, 478 Conformable partition, 136 Conjugate pair, 338, A4 Consistent system, 4, 8–9, 24 matrix equation, 42–43 Constant of adjustment, positive, 286 Constrained optimization, 463–470 eigenvalues, 465, 468 feasible set, 468 indifference curve, 469–470 See also Quadratic form Consumption matrix, 154 Continuous dynamical systems, 302, 356–360 Continuous functions, 224, 233, 262, 433–436, 440–442 Contraction transformation, 77, 86 Contrast between Nul A and Col A, 230–232 Control system, 140, 215–216, 300, 342 control sequence, 300 controllable pair, 300 Schur complement, 139 space shuttle, 215–216 state-space model, 300 state vector, 140, 289, 300 steady-state response, 342 system matrix, 140, 147–148 transfer function, 140 Controllability matrix, 300 Convergence, 155, 294, 316, 317, 342 See also Iterative methods Coordinate mapping, 247, 250–251, 272 Coordinate system(s), 176–177, 246–248 change of basis, 271–273 graphical, 247–248 isomorphism, 251–253 polar, A6 Rn , 248–249 Coordinate vector, 176, 247 Correlation coefficient, 382 Cost vector, 36 Counterexample, 72 Covariance, 485 matrix, 484, 488 Cramer’s rule, 201 Cross-product term, 456, 458 Crystallography, 248, 255 May 24, 2005 12:54 L57-index Sheet number Page number cyan magenta yellow black Index Crystals, 185 Current flow, 95 Current law, 97 Curve-fitting, 26, 422–423, 431–432 De Moivre’s Theorem, A7 Decomposition eigenvector, 342, 363 force, 388 orthogonal, 386, 395 polar, 492 singular value, 474–481 See also Factorization Decoupled system, 348, 354, 358 Degenerate line, 81 Design matrix, 419 Determinant, 185–187 adjugate, 203 area and volume, 204–205 Casoratian, 279 characteristic equation, 313 cofactor expansion, 188, 196 column operations, 196 Cramer’s rule, 201 echelon form, 194 eigenvalues, 313, 318 elementary matrix, 197 geometric interpretation, 204, 312 and inverse, 118, 194, 203–204 linearity property, 197, 212 multiplicative property, 196, 314 n×n matrix, 187 product of pivots, 194, 311 recursive definition, 187 row operations, 192–194, 197 3×3 matrix, 186 transformations, 207–209 triangular matrix, 189, 313 volume, 204, 312 See also Matrix Diagonal entries, 107 Diagonal matrix, 107, 138, 319, 474 Diagonal Matrix Representation Theorem, 331 Diagonalizable matrix, 320 distinct eigenvalues, 323 nondistinct eigenvalues, 324 orthogonally, 450 Diagonalization Theorem, 320 Difference equation, 97, 277, 280–286 dimension of solution space, 283 eigenvectors, 307, 315–316, 343 first-order, 284–285 homogeneous, 280, 282 nonhomogeneous, 280, 283 population model, 97–99 recurrence relation, 97, 280, 282 reduction to first order, 284 signal processing, 280 solution sets of, 282, 284 (fig.) stage-matrix model, 302 state-space model, 300 See also Dynamical system; Markov chain Differential equation, 233, 353–354 circuit problem, 355, 360, 362 decoupled system, 354, 358 eigenfunctions, 355 initial value problem, 354 solutions of, 354 See also Laplace transform Differentiation, 233 Digital signal processing See Signal processing Dilation transformation, 77–78, 83 Dimension (vector space), 256 classification of subspaces, 258 column space, 178–179, 260 null space, 178, 260 row space, 265–267 subspace, 177 Directed line segment, 29 Direction of greatest attraction, 345, 356 of greatest repulsion, 346, 357 Discrete linear dynamical system See Dynamical system Discrete-time signal See Signals Distance between vector and subspace, 387, 399 between vectors, 378 Distributive laws, 113 Domain, 73 Dot product, 375 Dynamical system, 302, 342 attractor, 345, 356 change of variable, 347 decoupling, 354, 358 eigenvalues and eigenvectors, 307, 315, 343 graphical solutions, 344–347 owl population model, 301, 349 predator–prey model, 343 repellor, 345, 357 saddle point, 346, 347, 357 spiral point, 360–361 stage-matrix model, 302, 349 See also Difference equation; Mathematical model Eccentricity of orbit, 426 Echelon form, 14 basis for row space, 264 consistent system, 24 determinant, 194, 311 flops, 23 LU factorization, 142–144 pivot positions, 15 Effective rank, 180, 268, 474 Eigenfunctions, 355, 359 Eigenspace, 304–305 dimension of, 324, 452 orthogonal basis for, 451 Eigenvalue, 303 characteristic equation, 313, 335 complex, 314, 335, 338, 348, 359 constrained optimization, 464–468 determinants, 311–313, 318 diagonalization, 319–323, 450–452 differential equations, 355–359 distinct, 323, 324 dynamical systems, 315–316, 342, 348 invariant plane, 340 Invertible Matrix Theorem, 312 iterative estimates, 317, 318, 363, 366–368 multiplicity of, 314 and quadratic forms, 461 and rotation, 335, 338 (fig.), 340, 350 (fig.), 360 (fig.) row operations, 304, 315 I3 May 24, 2005 12:54 I4 L57-index Sheet number Page number cyan magenta yellow black Index Eigenvalue (continued) similarity, 314–315 strictly dominant, 363 triangular matrix, 306 See also Dynamical system Eigenvector, 303 basis, 321, 324 complex, 335, 340 decomposition, 343, 363 diagonalization, 320–323, 450–452 difference equations, 307 dynamical system, 315–316, 343, 345, 346, 355–359 linearly independent, 307, 320 Markov chain, 316 principal components, 486 row operations, 304 Electrical network model, 2, 95–97 circuit problem, 355, 360, 362 matrix factorization, 147 minimal realization, 148 Elementary matrix, 122–124 determinant, 197 interchange, 197 reflector, 444 row replacement, 197 scale, 197 Elementary reflector, 444 Elementary row operation, 7, 122 Ellipse, 459 area, 209 singular values, 471–473 Equal matrices, 107 Equation auxiliary, 282 characteristic, 313 difference, 92, 97, 280 differential, 233, 353–355 of a line, 53, 81 linear, 2–3, 53, 419 normal, 376, 411 parametric, 52–54 price, 157 production, 153 three-moment, 286 vector, 28, 32–34, 41–42, 48, 56 Equilibrium, unstable, 352 Equilibrium prices, 57–59, 63 Equilibrium vector, 292–294 Equivalence relation, 333 Equivalent linear systems, Existence and Uniqueness Theorem, 24 Existence of solution, 75, 85 Existence questions, 8, 23, 43, 75, 84, 130 Explicit description, 52, 170, 228, 232 Factorization analysis of a dynamical system, 319 of block matrices, 138 complex eigenvalue, 340 diagonal, 319, 331 for a dynamical system, 319 in electrical engineering, 147 See also Matrix factorization; Singular value decomposition Feasible set, 468 Filter, linear, 280 low-pass, 281, 419 moving average, 286 Final demand vector, 152 Finite-dimensional vector space, 257 subspace, 259 Finite set, 257 First-order difference equation See Difference equation First principal component, 486 Flexibility matrix, 120 Flight control system, 215–216 Floating point arithmetic, 10 Floating point operation (flop), 10, 23 Flow in network, 60–62, 64, 95 Force, decomposition, 388 Fortran, 46 Forward phase, 20 Fourier approximation, 441 Fourier coefficients, 441 Fourier series, 440–442 Free variable, 20, 24, 50, 260 Full rank, 270 Function, 73 continuous, 433, 440 eigenfunction, 355 transfer, 140 trend, 439 utility, 469 Fundamental solution set, 283, 354 Fundamental subspaces, 267 (fig.), 270, 380 (fig.), 478 Gauss, Carl Friedrich, 14n, 426n Gaussian elimination, 14n General least-squares problem, 409 General linear model, 421 General solution, 21, 51, 283 Geometric descriptions of R2 , 29 of Span{v}, 35 of Span{u, v}, 35 Geometric point, 29 Givens rotation, 104 Gram matrix, 492 Gram–Schmidt process, 402–405, 430 in inner product spaces, 430 Legendre polynomials, 436 in P4 , 430, 439 in Rn , 404 Gram–Schmidt Process Theorem, 404 Heat conduction, 151 Helmert blocking, 374 Hermite polyomials, 261 Hilbert matrix, 134 Homogeneous coordinates, 159, 162 Homogenous system, 50–52 difference equations, 280–281 in economics, 57–59 subspace, 170, 227 Hooke’s law, 120 Householder matrix, 444 reflection, 184 Howard, Alan H., 93 Hyperbola, 459 Hyperspectral image processing, 488 Identity matrix, 45, 113, 122–124 Ill-conditioned matrix, 131, 416 Image, vector, 74 Image processing, multichannel, 447, 482, 486–488 Imaginary axis, A5 Imaginary numbers, pure, A5 Imaginary part complex number, A3 complex vector, 337 Implicit definition of Nul A, 170, 228, 232 Implicit description, 52, 299 Inconsistent system, 4, See also Linear system May 24, 2005 12:54 L57-index Sheet number Page number cyan magenta yellow black Index Indexed set, 65, 237 Indifference curve, 469 Inequality Bessel’s, 444 Cauchy–Schwarz, 432 triangle, 433 Infinite dimensional space, 257 Infinite set, 257n Initial value problem, 354 Inner product, 117, 375, 428 angles, 381 axioms, 427 on C[a, b], 433–434 Cauchy–Schwarz inequality, 432 evaluation, 433 length/norm, 378, 429 on Pn , 429 properties, 376 space, 428 triangle inequality, 433 Input–output model, 148, 152 Input sequence, 300 See also Control system Interchange matrix, 123, 197 Intermediate demand, 152 Interpolating polynomial, 26, 184 Invariant plane, 340 Inverse, 119 algorithm for, 124 augmented columns, 125 condition number, 131, 133 determinant, 119 elementary matrix, 122–124 flexibility matrix, 120 formula, 119, 203 identity matrix, 123 ill-conditioned matrix, 131 linear transformation, 130 Moore–Penrose, 480 partitioned matrix, 137, 140 product, 122 stiffness matrix, 120–121 transpose, 121 Inverse power method, 366–368 Invertible linear transformation, 130 matrix, 119, 123, 194 Invertible Matrix Theorem, 129–130, 179, 194, 267, 312, 479 Isomorphic vector spaces, 177, 262 Isomorphism, 177, 251, 283, 430n Iterative methods eigenspace, 364–365 eigenvalues, 317, 363, 366–368 formula for (I –C)–1 , 154, 157 inverse power method, 366 Jacobi’s method, 317 power method, 363 QR algorithm, 317, 318, 368 Jacobian matrix, 345n Jacobi’s method, 317 Jordan, Wilhelm, 14n Jordan form, 332 Junctions, 60 Kernel, 232 Kirchhoff’s laws, 95, 97 Ladder network, 147–148, 150 Laguerre polynomial, 261 Lamberson, R., 302 Landsat image, 447–448, 488, 489 LAPACK, 115, 138 Laplace transform, 140, 202 Law of cosines, 381 Leading entry, 14 Leading variable, 20n Least-squares fit cubic trend, 423 linear trend, 438–439 quadratic trend, 422, 439, 440 (fig.) scatter plot, 422 seasonal trend, 425 trend surface, 423 Least-squares problem, 373, 409 column space, 410–411 curve-fitting, 422–423 error, 413 lines, 419–421 mean-deviation form, 421 multiple regression, 423–424 normal equations, 374, 411, 420 orthogonal columns, 414 plane, 424 QR factorization, 414–415 residuals, 419 singular value decomposition, 480 sum of the squares for error, 427, 437 weighted, 436–438 I5 See also Inner product space Least-squares solution, 375, 409, 480 alternative calculation, 414 minimum length, 480, 492 QR factorization, 414–415 Left distributive law, 113 Left-multiplication, 113, 124, 200, 407 Left singular vector, 475 Legendre polynomial, 436 Leibniz, Gottfried, 185 Length of vector, 376–377, 429 singular values, 473 Leontief, Wasily, 1, 152, 157n exchange model, 57–59 input–output model, 152–157 production equation, 153 Line degenerate, 81 equation of, 3, 53 least-squares, 419–421 parametric vector equation, 52 Span{v}, 35 translation of, 53 Line segment, directed, 29 Linear combination, 32, 41, 221 in applications, 36 weights, 32, 41, 228 Linear dependence in R3 , 68 (fig.) Linear dependence relation, 65, 237 column space, 240 row-equivalent matrices, A1 row operations, 265 Linear difference equation See Difference equation Linear equation, 2–3 See also Linear system Linear filter, 280 Linear independence, 65, 237 eigenvectors, 307 matrix columns, 66, 89 in P3 , 251 in Rn , 69 sets, 65, 237, 259 signals, 279 zero vector, 69 Linear model See Mathematical model Linear programming, partitioned matrix, 138 Linear recurrence relation See Difference equation May 24, 2005 12:54 I6 L57-index Sheet number Page number cyan magenta yellow black Index Linear system, 3, 34, 42 basic strategy for solving, coefficient matrix, consistent/inconsistent, 4, 8–9 equivalent, existence of solutions, 8, 23–24 general solution, 21 homogeneous, 50–52, 57–59 linear independence, 65–70 and matrix equation, 40–42 matrix notation, 4–5 nonhomogeneous, 52–53, 267 over-/underdetermined, 26 parametric solution, 22, 52 solution sets, 3–8, 20–24, 50–54 and vector equations, 34 See also Linear transformation; Row operation Linear transformation, 80, 83, 99, 232, 282, 327 B-matrix, 329, 331 composite, 109, 160 contraction/dilation, 77–78, 83 of data, 79 determinants, 207–209 diagonal matrix representation, 331 differentiation, 233 domain/codomain, 73–74 geometric, 84–87 Givens rotation, 104 Householder reflection, 184 invertible, 130–131 isomorphism, 251 kernel, 232 matrix for, 83, 328–329, 332 null space, 232 one-to-one/onto, 87–89 projection, 87 properties, 76–77 on Rn , 330 range, 74, 232 reflection, 85, 184, 393 rotation, 78, 84 shear, 76, 86, 159 similarity, 314–315, 331 standard matrix, 83 vector space, 232–233, 329–330 See also Isomorphism; Superposition principle Linear trend, 440 Linearly dependent set, 65, 68–70, 237 Linearly independent eigenvectors, 307, 320 Linearly independent set, 65, 66, 237 See also Basis Long-term behavior of a dynamical system, 342 of a Markov chain, 291, 294 Loop current, 95 Lower triangular matrix, 132, 142, 144, 146 Low-pass filter, 281, 417 LU factorization, 106, 142–146, 149, 367 Mm×n , 224 Main diagonal, 107 Maple, 317 Mapping, 73 composition of, 109 coordinate, 247, 250–253, 272 eigenvectors, 329–330 matrix factorizations, 327–332 one-to-one, 87–89 onto Rm , 87, 89 signal processing, 282 See also Linear transformation Marginal propensity to consume, 286 Mark II computer, Markov chain, 288–294 convergence, 294 eigenvectors, 316 predictions, 291 probability vector, 288 state vector, 289 steady-state vector, 292, 316 stochastic matrix, 288 Mass–spring system, 223, 233, 244 Mathematica, 317 Mathematical ecologists, 301 Mathematical model, 1, 92 aircraft, 105, 158 beam, 120 electrical network, 95 linear, 92–99, 152, 288, 301, 342, 421 nutrition, 93 population, 97, 289, 293 predator–prey, 343 spotted owl, 301–302 stage-matrix, 302, 349 See also Markov chain MATLAB, 27, 134, 149, 211, 298, 317, 350, 367, 368, 408 Matrix, 107–115 adjoint/adjugate, 203 anticommuting, 183 augmented, band, 151 bidiagonal, 151 block, 134–141 Casorati, 279–280 change-of-coordinates, 249, 273–275 characteristic equation, 310–317 coefficient, 5, 44 of cofactors, 203 column space, 229 column sum, 154 column vector, 28 commutativity, 113, 183 companion, 372 consumption, 153, 157 controllability, 300 covariance, 484–485 design, 419 diagonal, 107, 138 diagonalizable, 320 echelon, 14–15 elementary, 122–124, 197–198, 444 equal, 107 flexibility, 120 Gram, 492 Hilbert, 134 Householder, 184, 444 identity, 45, 107, 113, 122–124 ill-conditioned, 131, 133, 414 interchange, 197 inverse, 119 invertible, 119, 121, 129 Jacobian, 345n leading entry, 14 of a linear transformation, 83, 328–329 m×n, migration, 98, 289, 316 multiplication, 109–110, 136 nonzero row/column, 14 notation, 4–5 null space, 169–170, 226 of observations, 483 orthogonal, 391, 450 orthonormal, 391n May 24, 2005 12:54 L57-index Sheet number Page number cyan magenta yellow black Index Matrix (continued) orthonormal columns, 390–391 partitioned, 134–138 Pauli spin, 183 positive definite/semidefinite, 461 powers of, 114 products, 110, 196 projection, 453, 455 pseudoinverse, 480 of quadratic form, 455 rank of, 178–265 reduced echelon, 14 regular stochastic, 294 row–column rule, 111 row equivalent, 7, 34n, A1 row space, 263 scalar multiple, 108 scale, 197 Schur complement, 139 singular/nonsingular, 119, 130, 131 size of, square, 128, 131 standard, 83, 110 stiffness, 120 stochastic, 288, 297 submatrix of, 135, 300 sum, 107–108 symmetric, 449–453 system, 140 trace of, 334, 485 transfer, 147 transpose of, 114–115, 121 tridiagonal, 151 unit cost, 79 unit lower triangular, 142 Vandermonde, 184, 212, 372 zero, 107 See also Determinant; Diagonalizable matrix; Inverse; Matrix factorization; Row operations; Triangular matrix Matrix of coefficients, 5, 44 Matrix equation, 42 Matrix factorization (decomposition), 142 Cholesky, 462, 492 complex eigenvalue, 340 diagonal, 319–320, 331 in electrical engineering, 147–148 full QR, 408 linear transformations, 327–332 LU, 142–146 permuted LU, 142–146 polar, 492 QR, 150, 405–407, 414–415 rank, 150 rank-revealing, 492 reduced LU, 150 reduced SVD, 480 Schur, 445 similarity, 314, 331 singular value decomposition, 150, 471–480 spectral, 150, 453 Matrix inversion, 118–121 Matrix multiplication, 109–110 block, 136 column–row expansion, 137 and determinants, 196 properties, 112, 114 row–column rule, 111 See also Composition of linear transformations Matrix notation See Back-substitution Matrix of observations, 483 Matrix program, 27 Matrix transformation, 74–76, 83 See also Linear transformation Matrix–vector product, 40 properties, 45 rule for computing, 45 Maximum of quadratic form, 464–468 Mean, sample, 484 Mean-deviation form, 421, 484 Mean square error, 442 Microchip, 135 Migration matrix, 98, 289, 316 Minimal realization, 148 Minimum length solution, 492 Minimum of quadratic form, 464–468 Model, mathematical See Mathematical model Modulus, A4 Molecular modeling, 161 Moore-Penrose inverse, 480 Moving average, 286 Muir, Thomas, 185 Multichannel image See Image processing, multichannel Multiple regression, 423–424 I7 Multiplicative property of det, 196, 313 Multiplicity of eigenvalue, 314 Multivariate data, 482, 487–488 National Geodetic Survey, 373 Negative definite quadratic form, 461 Negative flow, in a network branch, 95 Negative semidefinite form, 461 Negative of a vector, 217 Network, 60 branch, 95 branch current, 97 electrical, 95–97, 100, 147–148 flow, 60–62, 64, 95 loop currents, 95, 100 Nodes, 60 Noise, random, 286 Nonhomogeneous system, 52, 267 difference equations, 280, 283 Nonlinear dynamical system, 345n Nonsingular matrix, 119, 130 Nontrivial solution, 50 Nonzero column, 14 Nonzero row, 14 Nonzero singular values, 473 Norm of vector, 376–377, 429 Normal equation, 374, 411 North American Datum (NAD), 373–374 Null space, 169, 226 basis, 171, 240, 264 and column space, 230–232 dimension of, 260, 265–267 eigenspace, 304 explicit description of, 228–229 linear transformation, 233 See also Fundamental subspaces; Kernel Nullity, 265 Nutrition model, 93–94 Observation vector, 419, 483 Ohm’s law, 95 Oil exploration, One-to-one linear transformation, 88, 245 See also Isomorphism One-to-one mapping, 87–89 Onto mapping, 87, 89 Optimization, constrained See Constrained optimization Orbit of a comet, 426 May 24, 2005 12:54 I8 L57-index Sheet number Page number cyan magenta yellow black Index Ordered n-tuple, 31 Ordered pair, 28 Orthogonal eigenvectors, 450 matrix, 391, 450 polynomials, 431, 439 regression, 491 set, 384, 440 vectors, 379, 429 Orthogonal basis, 385, 430, 451, 473 for fundamental subspaces, 478–479 Gram–Schmidt process, 402, 430 Orthogonal complement, 380 Orthogonal Decomposition Theorem, 395 Orthogonal diagonalization, 450 principal component analysis, 485 quadratic form, 457 spectral decomposition, 453 Orthogonal projection, 386, 394 geometric interpretation, 388, 397 matrix, 399, 453, 455 properties of, 397 onto a subspace, 386, 395 sum of, 388, 397 (fig.) Orthogonality, 379, 390 Orthogonally diagonalizable, 450 Orthonormal basis, 389, 399, 405 columns, 390–391 matrix, 391n rows, 391 set, 389 Outer product, 117, 136, 184, 270, 453 Overdetermined system, 26 Owl population model, 301, 351 P, 220 Pn , 220 dimension, 257 inner product, 429 standard basis, 238 trend analysis, 439 Parabola, 422 Parallel line, 53 processing, 2, 115 solution sets, 53 (fig.), 54 (fig.), 284 (fig.) Parallelepiped, 185, 205–207, 312 Parallelogram area of, 205–207 law, for vectors, 383, 436 rule for addition, 30 region inside, 81, 208 Parameter vector, 419 Parametric description, 22 equation of a line, 52, 81 equation of a plane, 52 vector equation, 52 vector form, 52, 54 Partial pivoting, 20, 146 Partitioned matrix, 106, 134–141 addition and multiplication, 135–137 algorithms, 138 block diagonal, 138 block upper triangular, 137 column–row expansion, 137 conformable, 136 inverse of, 137–138, 140 outer product, 136 Schur complement, 139 submatrices, 135 Partitions, 134 Pauli spin matrix, 183 Permuted LU factorization, 146 Perspective projection, 163–165 Phase backward, 20, 144 forward, 20 Pivot, 17 column, 16, 172, 242, 265, A1 positions, 15 product, 194, 311 Pixel, 447 Point masses, 39 Polar coordinates, A6 Polar decomposition, 492 Polynomial characteristic, 314, 315 degree of, 219 Hermite, 261 interpolating, 26, 184 Laguerre, 261 Legendre, 436 orthogonal, 431, 439 in Pn , 218, 220, 239, 251–252 set, 218–220 trignometric, 440 zero, 219 Population model, 97–99, 288, 293, 343, 349, 353 Positive definite matrix, 461 Positive definite quadratic form, 461 Positive semidefinite matrix, 461 Power method, 363–366 Powers of a matrix, 114 Predator–prey model, 343–344 Predicted y-value, 419 Preprocessing, 142 Price equation, 157 Price vector, 157 Prices, equilibrium, 57–59, 63 Principal Axes Theorem, 458 Principal component analysis, 447, 483, 485 covariance matrix, 484 first principal component, 486 matrix of observations, 483 multivariate data, 482, 487–488 singular value decomposition, 488 Probability vector, 288 Process control data, 483 Product of complex numbers, A7 dot, 375 of elementary matrices, 122, 198 inner, 117, 375, 428 of matrices, 110, 196 of matrix inverses, 122 of matrix transposes, 114 matrix–vector, 41 outer, 117, 136 scalar, 117 See also Column–row expansion; Inner product Production equation, 153 Production vector, 152 Projection matrix, 453, 455 perspective, 163–165 transformations, 76, 87, 184 See also Orthogonal projection Properties determinants, 192 inner product, 376, 427, 433 linear transformation, 77, 88 May 24, 2005 12:54 L57-index Sheet number Page number cyan magenta yellow black Index Properties (continued) matrix addition, 108 matrix inversion, 121 matrix multiplication, 112 matrix–vector product, Ax, 45 orthogonal projections, 397, 399 of Rn , 32 rank, 300 transpose, 115 See also Invertible Matrix Theorem Properties of Determinants Theorem, 313 Pseudoinverse, 480, 492 Public work schedules, 468–469 feasible set, 468 indifference curve, 469 utility, 469 Pure imaginary number, A5 Pythagorean Theorem, 380, 398 QR algorithm, 317, 318, 368 QR factorization, 150, 405–407, 445 Cholesky factorization, 492 full QR factorization, 408 least squares, 414–415 QR Factorization Theorem, 405 Quadratic form, 455 change of variable, 457 classifying, 460–461 cross-product term, 456 indefinite, 461 maximum and minimum, 463 orthogonal diagonalization, 457–458 positive definite, 461 principal axes, 459 See also Constrained optimization; Symmetric matrix Quadratic Forms and Eigenvalues Theorem, 461 Rn , 31 algebraic properties of, 32, 40 change of basis, 274 dimension, 257 inner product, 375 length (norm), 376 quadratic form, 456 standard basis, 238, 389 subspace, 167, 395 R2 and R3 , 28, 29, 31, 220 Range of transformation, 74, 299, 232 Rank, 178, 179, 262, 265 in control systems, 300 effective, 180, 474 estimation, 268, 474n factorization, 150, 300 full, 270 Invertible Matrix Theorem, 179, 267 properties of, 300 See also Outer product Rank-revealing factorization, 492 Rank Theorem, 178, 265 Rayleigh quotient, 367, 445 Real part complex number, A3 complex vector, 337 Real vector space, 217 Rectangular coordinate system, 29 Recurrence relation See Difference equation Reduced echelon form, 14, 15 basis for null space, 228, 264–265 solution of system, 20, 23, 24 uniqueness of, A1 Reduced LU factorization, 150 Reduced singular value decomposition, 480, 492 Reduction to first-orderequation, 284 Reflection, 85, 393 Householder, 184 Reflector matrix, 184, 444–445 Regression coefficients, 419 line, 419 multiple, 423 orthogonal, 491 Relative change, 445 Relative error, 445 See also Condition number Repellor, 345, 357 Residual, 419, 421 Resistance, 95 Riemann sum, 434 Right-distributive law, 113 Right-multiplication, 113, 200 Right singular vector, 475 RLC circuit, 244 Rotation due to a complex eigenvalue, 338 (fig.), 340 I9 Rotation transformation, 78, 84, 104, 160, 162, 165 Roundabout, 64 Roundoff error, 10, 131, 366, 407, 474, 478 Row–column rule, 111 Row equivalent matrices, 7, 15, 123, 315, A1 notation, 21, 34n Row operation, 7, 192 back-substitution, 22 basic/free variable, 20 determinants, 192, 197–198, 313 echelon form, 15 eigenvalues, 304, 315 elementary, 7, 123 existence/uniqueness, 23–24 inverse, 121, 123 linear dependence relations, 172, 265 pivot positions, 15–17 rank, 268, 474 See also Linear system Row reduction algorithm, 17–20 backward phase, 20, 23, 144 forward phase, 20, 23 See also Row operation Row replacement matrix, 123, 197 Row space, 263 basis, 263, 265n dimension of, 265 Invertible Matrix Theorem, 267 See also Fundamental subspaces Row vector, 263 Row–vector rule, 45 S, 218, 278, 279 Saddle point, 346, 347 (fig.), 349 (fig.), 357 Sample covariance matrix, 484 Sample mean, 484 Sample variance, 490 Samuelson, P.A., 286n Scalar, 29, 217 Scalar multiple, 28, 31 (fig.), 107, 217 Scalar product See Inner product Scale a nonzero vector, 377 Scale matrix, 197 Scatter plot, 483 Scene variance, 448 May 24, 2005 12:54 I10 L57-index Sheet number 10 Page number 10 cyan magenta yellow black Index Schur complement, 139 Schur factorization, 445 Series circuit, 147 Set, vector See Vector set Shear-and-scale transformation, 166 Shear transformation, 76, 86, 159 Shunt circuit, 147 Signal processing, 280 auxiliary equation, 281 filter coefficients, 280 fundamental solution set, 283 linear difference equation, 280 linear filter, 280 low-pass filter, 281, 417 moving average, 286 reduction to first-order, 284 See also Dynamical system Signals control systems, 215, 216 discrete-time, 218 function, 215 noise, 286 sampled, 218, 278 vector space, S, 218, 278 Similar matrices, 314, 317, 318, 320, 331 See also Diagonalizable matrix Similarity transformation, 314 Singular matrix, 119, 130, 131 Singular value decomposition (SVD), 150, 471, 474 condition number, 478 estimating matrix rank, 180, 474 fundamental subspaces, 478 least-squares solution, 480 m×n matrix, 473 principal component analysis, 488 pseudoinverse, 480 rank of matrix, 474 reduced, 480 singular vectors, 475 Singular Value Decomposition Theorem, 475 Sink of dynamical system, 356 Size of a matrix, Solution (set), 3, 20, 54, 282, 354 difference equations, 282–284, 307 differential equations, 354–355 explicit description of, 21, 52, 307 fundamental, 283, 354 general, 21, 50–52, 283–284, 343, 358 geometric visualization, 53 (fig.), 54 (fig.), 284 (fig.) homogeneous system, 50, 170, 282 minimum length, 492 nonhomogeneous system, 52–53, 283 null space, 226 parametric, 22, 52, 54 row equivalent matrices, subspace, 170, 227, 282, 283, 304, 354 superposition, 96, 354 trivial/nontrivial, 50 unique, 8, 24, 87 See also Least-squares solution Source of dynamical system, 357 Space shuttle, 215 Span, 35, 43 linear independence, 68 orthogonal projection, 386 subspace, 179 Span{u, v} as a plane, 35 (fig.) Span{v} as a line, 35 (fig.) Span{v1 , , vp }, 35, 221 Spanning set, 221, 242 Spanning Set Theorem, 239 Sparse matrix, 106, 155, 195 Spatial dimension, 484 Spectral components, 483 Spectral decomposition, 452–453 Spectral dimension, 484 Spectral factorization, 150 Spectral Theorem, 452 Spiral point, 360–361 Spotted owl, 301, 342, 349 Square matrix, 128, 131 Stage-matrix model, 302, 349 Standard basis, 170, 238, 274, 389 Standard matrix, 83, 110, 327 Standard position, 459 State-space model, 300, 342 State vector, 140, 289, 300 Steady-state heat flow, 150 response, 342 temperature, 12, 101, 150 vector, 292, 294, 303, 316 Stiffness matrix, 120 Stochastic matrix, 288, 297, 303 regular, 294 Strictly dominant eigenvalue, 363 Submatrix, 135, 300 Subspace, 167, 220 basis for, 170, 238 column space, 169, 229 dimension of, 177, 258 eigenspace, 304 fundamental, 270, 380 (fig.), 478 homogeneous system, 228 intersection of, 225 linear transformation, 233 (fig.) null space, 169, 227 spanned by a set, 169, 221 sum, 225 zero, 169, 220 See also Vector space Sum of squares for error, 427, 437 Surface rendering, 165 Superposition principle, 77, 96, 354 SVD See Singular value decomposition Symmetric matrix, 341, 369, 449 positive definite/semidefinite, 461 See also Quadratic form System, linear See Linear system System matrix, 140 Takakazu, Seki, 185 Tetrahedron, 185, 210 Theorem Basis, 179, 259 Best Approximation, 398–399 Cauchy–Schwarz Inequality, 432 Cayley–Hamilton, 371 Characterization of Linearly Dependent Sets, 68, 237 Column–Row Expansion of AB, 137 Cramer’s Rule, 201–202 De Moivre’s, A7 Diagonal Matrix Representation, 331 Diagonalization, 320 Existence and Uniqueness, 24 Gram–Schmidt Process, 404 Inverse Formula, 203 Invertible Matrix, 129–130, 179, 194, 267, 312, 479 Multiplicative Property (of det), 196 Orthogonal Decomposition, 395–396 Principal Axes, 458 Pythagorean, 380 May 24, 2005 12:54 L57-index Sheet number 11 Page number 11 cyan magenta yellow black Index Theorem (continued) QR Factorization, 405–406 Quadratic Forms and Eigenvalues, 461 Rank, 178, 265–267 Row Operations, 192 Singular Value Decomposition, 475 Spanning Set, 239–240, 242 Spectral, 452 Triangle Inequality, 433 Unique Representation, 246 Uniqueness of the Reduced Echelon Form, 15, A1 Three-moment equation, 286 Total variance, 485 fraction explained, 487 Trace of a matrix, 334, 485 Trajectory, 344 Transfer function, 140 Transfer matrix, 147 Transformation affine, 81 codomain, 74 definition of, 73 domain of, 73 identity, 329 image of a vector x under, 74 range of, 74 See also Linear transformation Translation, vector, 53 in homogeneous coordinates, 160 Transpose, 114–115 conjugate, 445n determinant of, 196 of inverse, 121 of product, 115 properties of, 115 Trend analysis, 438–440 Trend surface, 423 Triangle, area of, 210 Triangle inequality, 433 Triangular matrix, determinants, 189 eigenvalues, 306 lower, 132, 142, 144, 146 upper, 132, 137 Tridiagonal matrix, 151 Trignometric polynomial, 440 Trivial solution, 50 Uncorrelated variable, 485 Underdetermined system, 26 Unique Representation Theorem, 246 Unique vector, 224 Uniqueness question, 8, 23, 50, 75, 84 Unit cell, 248 Unit consumption vector, 152 Unit cost matrix, 79 Unit lower triangular matrix, 142 Unit square, 84 Unit vector, 377, 429, 464 Upper triangular matrix, 132, 137 Utility function, 469 Value added vector, 157 Vandermonde matrix, 184, 212, 372 Variable, 20 basic/free, 20 leading, 20n uncorrelated, 485 See also Change of variable Variance, 412, 437n, 485 sample, 490 scene, 448 total, 485 Vector(s), 28 addition/subtraction, 28, 29, 30 angles between, 381–382 as arrows, 29 (fig.) column, 28 complex, 28n coordinate, 176, 247 cost, 36 decomposing, 388 distance between, 378 equal, 28 equilibrium, 292 final demand, 152 image, 74 left singular, 475 length/norm, 376–377, 429, 473 linear combinations, 32–37, 70 linearly dependent/independent, 65–70 negative, 217 normalizing, 377 observation, 419, 483 orthogonal, 379 parameter, 419 I11 as a point, 29 price, 157 probability, 288 production, 152 in Rn , 31 in R3 , 31 in R2 , 28–31 reflection, 395 residual, 421 singular, 475 state, 140, 289, 300 steady-state, 292, 294, 303, 316 sum, 28 translations, 53 unique, 224 unit, 152, 377, 429 value added, 157 weights, 32 zero, 31, 69, 168, 217, 379 See also Eigenvector Vector addition, 29 as translation, 53 Vector equation, 33, 35 linear dependence relation, 65 parametric, 52, 54 Vector set, 65–70, 384–391 indexed, 65 linear independence, 237–242, 256–260 orthogonal, 384–386, 449 orthonormal, 389–391, 399, 405 polynomial, 218, 220 Vector space, 215, 217 of arrows, 217 axioms, 217 complex, 217n and difference equations, 282–284 and differential equations, 233, 354 of discrete-time signals, 218 finite-dimensional, 257, 259 of functions, 219, 433, 440 infinite-dimensional, 257 of polynomials, 218, 429 real, 217n See also Inner product space; Subspace Vector subtraction, 28–32 Vector sum, 28 May 24, 2005 12:54 I12 L57-index Sheet number 12 Page number 12 cyan magenta yellow black Index Vertex, 158 Vibration of a weighted spring, 223, 233, 244 Viewing plane, 163 Virtual reality, 161 Volt, 95 Volume ellipsoid, 210 parallelepiped, 185, 205–207, 312 tetrahedron, 210 Weighted least squares, 428, 436 Weights, 32, 41 as free variables, 229 Wire-frame models, 105, 158 Zero matrix, 107 Zero polynomial, 219 Zero solution, 50 Zero subspace, 169, 220 Zero vector, 31, 69 orthogonal, 379 subspace, 169 unique, 217, 224 ... analysis and linear algebra As a founding member of the NSF-sponsored Linear Algebra Curriculum Study Group, Lay has been a leader in the current movement to modernize the linear algebra curriculum... Sets of Linear Systems 50 1.6 Applications of Linear Systems 57 1.7 Linear Independence 1.8 Introduction to Linear Transformations 73 1.9 The Matrix of a Linear Transformation 82 1.10 Linear Models... numerical issues in linear algebra Written by Rick Smith, they were developed to accompany a computational linear algebra course at the University of Florida, which has used Linear Algebra and Its