LECTURES ON APPLIED MATHEMATICS pptx

LECTURES ON APPLIED MATHEMATICS Part 1: Linear Algebra Ray M. Bowen Former Professor of Mechanical Engineering President Emeritus Texas A&M University College Station, Texas Copyright Ray M. Bowen Updated February, 2013 ii ____________________________________________________________________________ PREFACE To Part 1 It is common for Departments of Mathematics to offer a junior-senior level course on Linear Algebra. This book represents one possible course. It evolved from my teaching a junior level course at Texas A&M University during the several years I taught after I served as President. I am deeply grateful to the A&M Department of Mathematics for allowing this Mechanical Engineer to teach their students. This book is influenced by my earlier textbook with C C Wang, Introductions to Vectors and Tensors, Linear and Multilinear Algebra. This book is more elementary and is more applied than the earlier book. However, my impression is that this book presents linear algebra in a form that is somewhat more advanced than one finds in contemporary undergraduate linear algebra courses. In any case, my classroom experience with this book is that it was well received by most students. As usual with the development of a textbook, the students that endured its evolution are due a statement of gratitude for their help. As has been my practice with earlier books, this book is available for free download at the site http://www1.mengr.tamu.edu/rbowen/ or, equivalently, from the Texas A&M University Digital Library’s faculty repository, http://repository.tamu.edu/handle/1969.1/2500. It is inevitable that the book will contain a variety of errors, typographical and otherwise. Emails to rbowen@tamu.edu that identify errors will always be welcome. For as long as mind and body will allow, this information will allow me to make corrections and post updated versions of the book. College Station, Texas R.M.B. Posted January, 2013 iii ______________________________________________________________________________ CONTENTS Part 1 Linear Algebra Selected Readings for Part I………………………………………………………… 2 CHAPTER 1 Elementary Matrix Theory……………………………………… 3 Section 1.1 Basic Matrix Operations……………………………………… 3 Section 1.2 Systems of Linear Equations…………………………………… 13 Section 1.3 Systems of Linear Equations: Gaussian Elimination………… 21 Section 1.4 Elementary Row Operations, Elementary Matrices…………… 39 Section 1.5 Gauss-Jordan Elimination, Reduced Row Echelon Form……… 45 Section 1.6 Elementary Matrices-More Properties…………………………. 53 Section 1.7 LU Decomposition……………………………………………. 69 Section 1.8 Consistency Theorem for Linear Systems……………………… 91 Section 1.9 The Transpose of a Matrix……………………………………… 95 Section 1.10 The Determinant of a Square Matrix…………………………….101 Section 1.11 Systems of Linear Equations: Cramer’s Rule ……………… …125 CHAPTER 2 Vector Spaces…………………………………………… 131 Section 2.1 The Axioms for a Vector Space…………………………… 131 Section 2.2 Some Properties of a Vector Space……………………… 139 Section 2.3 Subspace of a Vector Space…………………………… 143 Section 2.4 Linear Independence……………………………………. 147 Section 2.5 Basis and Dimension…………………………………… 163 Section 2.6 Change of Basis………………………………………… 169 Section 2.7 Image Space, Rank and Kernel of a Matrix………………… 181 CHAPTER 3 Linear Transformations…………………………………… 207 Section 3.1 Definition of a Linear Transformation…………… 207 Section 3.2 Matrix Representation of a Linear Transformation 211 Section 3.3 Properties of a Linear Transformation……………. 217 Section 3.4 Sums and Products of Linear Transformations… 225 Section 3.5 One to One Onto Linear Transformations………. 231 Section 3.6 Change of Basis for Linear Transformations 235 CHAPTER 4 Vector Spaces with Inner Product……………………… 247 Section 4.1 Definition of an Inner Product Space…………… 247 iv Section 4.2 Schwarz Inequality and Triangle Inequality……… 255 Section 4.3 Orthogonal Vectors and Orthonormal Bases…… 263 Section 4.4 Orthonormal Bases in Three Dimensions……… 277 Section 4.5 Euler Angles…………………………………… 289 Section 4.6 Cross Products on Three Dimensional Inner Product Spaces 295 Section 4.7 Reciprocal Bases………………………………… 301 Section 4.8 Reciprocal Bases and Linear Transformations…. 311 Section 4.9 The Adjoint Linear Transformation……………. 317 Section 4.10 Norm of a Linear Transformation……………… 329 Section 4.11 More About Linear Transformations on Inner Product Spaces 333 Section 4.12 Fundamental Subspaces Theorem…………… 343 Section 4.13 Least Squares Problem………………………… 351 Section 4.14 Least Squares Problems and Overdetermined Systems 357 Section 4.14 A Curve Fit Example…………………………… 373 CHAPTER 5 Eigenvalue Problems…………………………………… 387 Section 5.1 Eigenvalue Problem Definition and Examples… 387 Section 5.2 The Characteristic Polynomial…………………. 395 Section 5.3 Numerical Examples…………………………… 403 Section 5.4 Some General Theorems for the Eigenvalue Problem 421 Section 5.5 Constant Coefficient Linear Ordinary Differential Equations 431 Section 5.6 General Solution……………………………… 435 Section 5.7 Particular Solution……………………………… 453 CHAPTER 6 Additional Topics Relating to Eigenvalue Problems…… 467 Section 6.1 Characteristic Polynomial and Fundamental Invariants 467 Section 6.2 The Cayley-Hamilton Theorem………………… 471 Section 6.3 The Exponential Linear Transformation……… 479 Section 6.4 More About the Exponential Linear Transformation 493 Section 6.5 Application of the Exponential Linear Transformation 499 Section 6.6 Projections and Spectral Decompositions………. 511 Section 6.7 Tensor Product of Vectors……………………… 525 Section 6.8 Singular Value Decompositions………………… 531 Section 6.9 The Polar Decomposition Theorem…………… 555 INDEX………………………………………………………………… vii v PART I1 NUMERICAL ANALYSIS Selected Readings for Part II………………………………………… PART III. ORDINARY DIFFERENTIAL EQUATIONS Selected Readings for Part III………………………………………… PART IV. PARTIAL DIFFERENTIAL EQUATIONS Selected Readings for Part IV………………………………………… vi _______________________________________________________________________________ PART I LINEAR ALGEBRA Selected Reading for Part I BOWEN, RAY M., and C C. WANG, Introduction to Vectors and Tensors, Linear and Multilinear Algebra, Volume 1, Plenum Press, New York, 1976. BOWEN, RAY M., and C C. WANG, Introduction to Vectors and Tensors: Second Edition—Two Volumes Bound as One, Dover Press, New York, 2009. FRAZER, R. A., W. J. DUNCAN, and A. R. COLLAR, Elementary Matrices, Cambridge University Press, Cambridge, 1938. GREUB, W. H., Linear Algebra, 3 rd ed., Springer-Verlag, New York, 1967. HALMOS, P. R., Finite Dimensional Vector Spaces, Van Nostrand, Princeton, New Jersey, 1958. L EON, S. J., Linear Algebra with Applications, 7 th Edition, Pearson Prentice Hall, New Jersey, 2006. M OSTOW, G. D., J. H. SAMPSON, and J. P. MEYER, Fundamental Structures of Algebra, McGraw- Hill, New York, 1963. SHEPHARD, G. C., Vector Spaces of Finite Dimensions, Interscience, New York, 1966. LEON, STEVEN J., Linear Algebra with Applications 7 th Edition, Pearson Prentice Hall, New Jersey, 2006. 3 __________________________________________________________ Chapter 1 ELEMENTARY MATRIX THEORY When we introduce the various types of structures essential to the study of linear algebra, it is convenient in many cases to illustrate these structures by examples involving matrices. Also, many of the most important practical applications of linear algebra are applications focused on matrix algebra. It is for this reason we are including a brief introduction to matrix theory here. We shall not make any effort toward rigor in this chapter. In later chapters, we shall return to the subject of matrices and augment, in a more careful fashion, the material presented here. Section 1.1. Basic Matrix Operations We first need some notations that are convenient as we discuss our subject. We shall use the symbol R to denote the set of real numbers, and the symbol C to denote the set of complex numbers. The sets R and C are examples of what is known in mathematics as a field. Each set is endowed with two operations, addition and multiplication such that For Addition: 1. The numbers 1 x and 2 x obey (commutative) 12 21 x xxx   2. The numbers 1 x , 2 x , and 3 x obey (associative) 12 31 23 () () x xxxxx    3. The real (or complex) number 0 is unique (identity) and obeys 00 x x   4. The number x has a unique “inverse” x  such that. ()0xx   For Multiplication 5. The numbers 1 x and 2 x obey (commutative) 12 21 x xxx  4 Chap. 1 • ELEMENTARY MATRIX THEORY 6. The numbers 1 x , 2 x , and 3 x obey (associative) 12 3 1 23 () () x xx xxx  7. The real (complex) number 1 is unique (identity) and obeys (1) (1) x xx   8. For every 0x  , there exists a number 1 x (inverse under multiplication) such that 11 1 x x xx      9. For every 123 ,, x xx, (distribution axioms) 12 3 12 13 1231213 () () x xx xxxx x xx xx xx    While it is not especially important to this work, it is appropriate to note that the concept of a field is not limited to the set of real numbers or complex numbers. Given the notation R for the set of real numbers and a positive integer N , we shall use the notation N R to denote the set whose elements are N-tuples of the form   1 , , N x x where each element is a real number. A convenient way to write this definition is     1 , , N Nj xxxRR (1.1.1) The notation in (1.1.1) should be read as saying “ N R equals the set of all N-tuples of real numbers.” In a similar way, we define the N-tuple of complex numbers, N C , by the formula     1 , , N Nj zzzCC (1.1.2) An M by N matrix A is a rectangular array of real or complex numbers ij A arranged in M rows and N columns. A matrix is usually written [...]... symbol  on the right side of (1.1.8) refers to addition and subtraction of the complex or real numbers Aij and Bij , while the symbol  on the left side is an operation defined by (1.1.8) It is an operation defined on the set M M N Two matrices of the same order are said to be conformable for addition and subtraction Addition and subtraction are not defined for matrices which are not conformable... equations in N unknowns obtained from (1.2.1) by a) b) c) d) switching two rows, multiplying one of the rows by a nonzero constant multiply one row by a nonzero constant and adding it to another row, or combinations of a),b) and c) Equivalent systems have the same solution as the original system The point that is embedded in this concept is that given the problem of solving (1.2.1), one can convert... addition and subtraction of matrices, equation (1.1.9) defines multiplication of a matrix by a real or complex number It is a consequence of the definitions (1.1.8) and (1.1.9) that  A  ( 1) A    Aij    (1.1.10) These definitions of addition and subtraction and, multiplication by a number imply that A B  B  A (1.1.11) A  ( B  C )  ( A  B)  C (1.1.12) Sec 1.1 • Basic Matrix Operations... M  1 equations to obtain M  1 equations in N  1 unknowns, x2 , x3 , , x N Repeat the process with these M  1 equations to obtain an equation for one of the unknowns This solution is then back substituted into the previous equations to obtain the answers for the other two variables If the original system of equations does not have a solution, the elimination process will yield an inconsistency which... that overdetermined systems do not have a solution Likewise, undetermined solutions usually do not have a unique solutions If there are an equal number of unknowns as equations, i.e., M  N , he system may or may not have a solution If it has a solution, it may not be unique In the special case where A is a square matrix that is also nonsingular, the solution of (1.2.3)is formally x  A1b (1.2.6) Unfortunately,... ELEMENTARY MATRIX THEORY Sec 1.3 • Systems of Linear Equations: Gaussian Elimination 21 Section 1.3 Systems of Linear Equations: Gaussian Elimination Elimination methods, which represent methods learned in high school algebra, form the basis for the most powerful methods of solving systems of linear algebraic equations We begin this discussion by introducing the idea of an equivalent system to the given... illustrate, can perform the operations on the rows of the augmented matrix, rather than on the equations themselves Note: In matrix algebra, we are using what is known as row operations when we manipulate the augmented matrix Step 1: Forward Elimination of Unknowns: If A11  0 , we first multiply the row of the augmented matrix equation by the result from the second row The result is the augmented... that planes associated with three linear algebraic equations can intersect in a point, as with (1.2.11), or as a line or, perhaps, they will not intersect This geometric observation reveals the fact that systems of linear equations can have unique solutions, solutions that are not unique and no solution An example where there is not a unique solution is provided by the following: Example 1.2.3: 2 x1 ... substitution into (1.2.16) one can establish that  x1   2  x   x2    x3       x3   x3      (1.2.17) is a solution for all values x3 Thus, there are an infinite number of solutions of (1.2.16) Example 1.2.6: Consider the overdetermined system x1  x2  2 x1  x2  1 x1  4 (1.2.18) Sec 1.2 • Systems of Linear Equations 19 If (1.2.18)3 is substituted into (1.2.18)1 and (1.2.18)2 the inconsistent... graphical representations to illustrate the range of solutions We need solution procedures that will yield numerical values for the solution developed within a theoretical framework that allows one to characterize the solution properties in advance of the attempted solution Our goal, in this introductory phase of this linear algebra course is to develop components of this theoretical framework and to illustrate . Transformation……………. 217 Section 3.4 Sums and Products of Linear Transformations… 225 Section 3.5 One to One Onto Linear Transformations………. 231 Section 3.6. 493 Section 6.5 Application of the Exponential Linear Transformation 499 Section 6.6 Projections and Spectral Decompositions………. 511 Section 6.7 Tensor