Lyche t numerical linear algebra and matrix factorizations 2020

22 Tom Lyche Numerical Linear Algebra and Matrix Factorizations Editorial Board T J.Barth M.Griebel D.E.Keyes R.M.Nieminen D.Roose T.Schlick Texts in Computational Science and Engineering Editors Timothy J Barth Michael Griebel David E Keyes Risto M Nieminen Dirk Roose Tamar Schlick 22 More information about this series at http://www.springer.com/series/5151 Tom Lyche Numerical Linear Algebra and Matrix Factorizations Tom Lyche Blindern University of Oslo Oslo, Norway ISSN 1611-0994 ISSN 2197-179X (electronic) Texts in Computational Science and Engineering ISBN 978-3-030-36467-0 ISBN 978-3-030-36468-7 (eBook) https://doi.org/10.1007/978-3-030-36468-7 Mathematics Subject Classification (2010): 15-XX, 65-XX © Springer Nature Switzerland AG 2020 This work is subject to copyright All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed The use of general descriptive names, registered names, trademarks, service marks, etc in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Foreword It is a pleasure to write this foreword to the book “Numerical Linear Algebra and Matrix Factorizations” by Tom Lyche I see this book project from three perspectives, corresponding to my three different roles: first, as a friend and close colleague of Tom for a number of years, secondly as the present department head, and, finally, as a researcher within the international linear algebra and matrix theory community The book actually has a long history and started out as lecture notes that Tom wrote for a course in numerical linear algebra For almost forty years this course has been an important and popular course for our students in mathematics, both in theoretical and more applied directions, as well as students in statistics, physics, mechanics and computer science These notes have been revised multiple times during the years, and new topics have been added I have had the pleasure to lecture the course myself, using Tom’s lecture notes, and I believe that both the selection of topics and the combined approach of theory and algorithms is very appealing This is also what our students point out when they have taken this course As we know, the area presented in this book play a highly central role in many applications of mathematics and in scientific computing in general Sometimes, in the international linear algebra and matrix theory community, one divides the area into numerical linear algebra, applied linear algebra and core (theoretical) linear algebra This may serve some purpose, but often it is fruitful to have a more unified view on this, in order to see the interplay between theory, applications and algorithms I think this view dominates this book, and that this makes the book interesting to a wide range of readers Finally, I would like to thank Tom for his work with this book and the mentioned course, and for being a good colleague from whom I have learned a lot I know that his international research community in spline theory also share this view Most importantly, I hope that you, the reader, will enjoy the book! Oslo, Norway June 2019 Geir Dahl v Preface This book, which has grown out of a one semester course at the University of Oslo, targets upper undergraduate and beginning graduate students in mathematics, statistics, computational physics and engineering who need a mathematical background in numerical linear algebra and related matrix factorizations Mastering the material in this book should enable a student to analyze computational problems and develop his or her own algorithms for solving problems of the following kind, • System of linear equations Given a (square) matrix A and a vector b Find a vector x such that Ax = b • Least squares Given a (rectangular) matrix A and a vector b Find a vector x such that the sum of squares of the components of b − Ax is as small as possible • Eigenvalues and eigenvectors Given a (square) matrix A Find a number λ and/or a nonzero vector x such that Ax = λx Such problems can be large and difficult to handle, so much can be gained by understanding and taking advantage of special structures For this we need a good understanding of basic numerical linear algebra and matrix factorizations Factoring a matrix into a product of simpler matrices is a crucial tool in numerical linear algebra for it allows one to tackle large problems through solving a sequence of easier ones The main characteristics of this book are as follows: It is self-contained, only assuming first year calculus, an introductory course in linear algebra, and some experience in solving mathematical problems on a computer A special feature of this book is the detailed proofs of practically all results Parts of the book can be studied independently making it suitable for self study There are numerous exercises which can be found at the end of each chapter In a separate book we offer solutions to all problems Solutions of many exam problems given for this course at the University of Oslo are included in this separate volume vii viii Preface The book, consisting of an introductory first chapter and 15 more chapters, naturally disaggregating into six thematically related parts The chapters are designed to be suitable for a one week per chapter one semester course Toward the goal of being self-contained, the first chapter contains a review of linear algebra, and is provided to the reader for convenient occasional reference Many of the chapters contain material beyond what might normally be covered in one week of lectures A typical 15 week semester’s curriculum could consist of the following curated material LU and QR factorizations 2.4, 2.5, 3.2, 3.3, 3.5, 4.1, 4.2, 5.1 − 5.4, 5.6 SVD, norms and LSQ 6.1, 6.3, 7.1 − 7.4, 8.1 − 8.3, 9.1 − 9.3, 9.4.1 Kronecker products 10.1, 10.2, 10.3, 11.1, 11.2, 11.3 Iterative methods 12.1 − 12.4, 13.1 − 13.3, 13.5 Eigenpairs 14.1 − 14.5, 15.1 − 15.3 Chapters 2–4 give a rather complete treatment of various LU factorizations Chapters 5–9 cover QR and singular value factorizations, matrix norms, least squares methods and perturbation theory for linear equations and least squares problems Chapter 10 gives an introduction to Kronecker products We illustrate their use by giving simple proofs of properties of the matrix arising from a discretization of the dimensional Poison Equation Also, we study fast methods based on eigenvector expansions and the Fast Fourier Transform in Chap 11 Some background from Chaps 2, and may be needed for Chaps 10 and 11 Iterative methods are studied in Chaps 12 and 13 This includes the classical methods of Jacobi, Gauss Seidel Richardson and Successive Over Relaxation (SOR), as well as a derivation and convergence analysis of the methods of steepest descent and conjugate gradients The preconditioned conjugate gradient method is introduced and applied to the Poisson problem with variable coefficients In Chap 14 we consider perturbation theory for eigenvalues, the power method and its variants, and use the Inertia Theorem to find a single eigenvalue of a symmetric matrix Chapter 15 gives a brief informal introduction to one of the most celebrated algorithms of the twentieth century, the QR method for finding all eigenvalues and eigenvectors of a matrix In this book we give many detailed numerical algorithms for solving linear algebra problems We have written these algorithms as functions in MATLAB A list of these functions and the page number where they can be found is included after the table of contents Moreover, their listings can be found online at http:// folk.uio.no/tom/numlinalg/code Complexity is discussed briefly in Sect 3.3.2 As for programming issues, we often vectorize the algorithms leading to shorter and more efficient programs Stability is important both for the mathematical problems and for the numerical algorithms Stability can be studied in terms of perturbation theory that leads to condition numbers, see Chaps 8, and 14 We Preface ix will often use phrases like “the algorithm is numerically stable” or “the algorithm is not numerically stable” without saying precisely what we mean by this Loosely speaking, an algorithm is numerically stable if the solution, computed in floating point arithmetic, is the exact solution of a slightly perturbed problem To determine upper bounds for these perturbations is the topic of backward error analysis We refer to [7] and [17, 18] for an in-depths treatment A list of freely available software tools for solving linear algebra problems can be found at www.netlib.org/utk/people/JackDongarra/la-sw.html To supplement this volume the reader might consult Björck [2], Meyer [15] and Stewart [17, 18] For matrix analysis the two volumes by Horn and Johnson [9, 10] contain considerable additional material Acknowledgments I would like to thank my colleagues Elaine Cohen, Geir Dahl, Michael Floater, Knut Mørken, Richard Riesenfeld, Nils Henrik Risebro, Øyvind Ryan and Ragnar Winther for all the inspiring discussions we have had over the years Earlier versions of this book were converted to LaTeX by Are Magnus Bruaset and Njål Foldnes with help for the final version from Øyvind Ryan I thank Christian Schulz, Georg Muntingh and Øyvind Ryan who helped me with the exercise sessions and we have, in a separate volume, provided solutions to practically all problems in this book I also thank an anonymous referee for useful suggestions Finally, I would like to give a special thanks to Larry Schumaker for his enduring friendship and encouragement over the years Oslo, Norway June 2019 Tom Lyche Contents A Short Review of Linear Algebra 1.1 Notation 1.2 Vector Spaces and Subspaces 1.2.1 Linear Independence and Bases 1.2.2 Subspaces 1.2.3 The Vector Spaces Rn and Cn 1.3 Linear Systems 1.3.1 Basic Properties 1.3.2 The Inverse Matrix 1.4 Determinants 1.5 Eigenvalues, Eigenvectors and Eigenpairs 1.6 Exercises Chap 1.6.1 Exercises Sect 1.1 1.6.2 Exercises Sect 1.3 1.6.3 Exercises Sect 1.4 Part I 1 10 11 12 13 15 18 20 20 21 22 LU and QR Factorizations Diagonally Dominant Tridiagonal Matrices; Three Examples 2.1 Cubic Spline Interpolation 2.1.1 Polynomial Interpolation 2.1.2 Piecewise Linear and Cubic Spline Interpolation 2.1.3 Give Me a Moment 2.1.4 LU Factorization of a Tridiagonal System 2.2 A Two Point Boundary Value Problem 2.2.1 Diagonal Dominance 2.3 An Eigenvalue Problem 2.3.1 The Buckling of a Beam 2.4 The Eigenpairs of the 1D Test Matrix 27 27 28 28 31 34 37 38 40 40 41 xi Index A Absolute error, 181 A-inner product, 279 Algebraic multiplicity, 132 Algorithms assemble Householder transformations, 326 backsolve, 63 backsolve column oriented, 75 bandcholesky, 89 cg, 286 fastpoisson, 241 findsubintervals, 50 forwardsolve, 63 forwardsolve column oriented, 64 housegen, 110 Householder reduction to Hessenberg form, 325 Householder triangulation, 112 Jacobi, 256 L1U factorization, 73 LDL* factorization, 85 the power method, 338 preconditioned cg, 301 Rayleigh quotient iteration, 341 SOR, 257 spline evaluation, 50 splineint, 50 testing conjugate gradient, 287 trifactor, 36 trisolve, 36 upper Hessenberg linear system, 123 A-norm, 279, 306 B Back substitution, 58 Bandsemi-cholesky, 94 Biharmonic equation, 235 fast solution method, 249 nine point rule, 250 Block LU theorem, 74 C Cauchy-Binet formula, 17 Cauchy determinant, 23 Cauchy-Schwarz inequality, 101 Cayley Hamilton Theorem, 148 Central difference, 52 Central difference approximation second derivative, 52 Change of basis matrix, Characteristic equation, 19, 129 Characteristic polynomial, 19, 129 Chebyshev polynomial, 293 Cholesky factorization, 83 Column operations, 16 Column space (span), 10 Companion matrix, 146 Complementary subspaces, 104 Complete pivoting, 70 Complexity of an algorithm, 65 Condition number ill-conditioned, 181 Congruent matrices, 328 Conjugate gradient method, 279 convergence, 288 derivation, 283 © Springer Nature Switzerland AG 2020 T Lyche, Numerical Linear Algebra and Matrix Factorizations, Texts in Computational Science and Engineering 22, https://doi.org/10.1007/978-3-030-36468-7 357 358 Krylov spaces, 289 least squares problem, 308 preconditioning, 299 preconditioning algorithm, 301 preconditioning convergence, 301 Convex combinations, 139, 185, 296 Convex function, 185 Courant-Fischer theorem, 142 Cubic spline minimal norm, 49 D Defective matrix, 129 Deflation, 137 Determinant area of a triangle, 23 Cauchy, 23 Cauchy-Binet, 17 cofactor expansion, 16 definition, 15 elementary operations, 16 plane equation, 22 principal minor, 60 straight line equation, 16 Vandermonde, 23 Dirac delta, Direct sum, 104 Discrete cosine transform, 242 Discrete Fourier transform (DFT), 242, 243 Fourier matrix, 243 Discrete sine transform (DST), 242 E Eigenpair, 18, 40, 129 eigenvalue, 18 eigenvector, 18 left eigenpair, 143 1D test matrix, 42 right eigenpair, 143 spectrum, 18 Eigenvalue, 40, 129 algebraic multiplicity, 132 characteristic equation, 19, 129 characteristic polynomial, 19, 129 Courant-Fischer theorem, 142 geometric multiplicity, 132 Hoffman-Wielandt theorem, 143 left eigenvalue, 143 location, 318 Rayleigh quotient, 139 right eigenvalue, 143 spectral theorem, 140 Index spectrum, 129 triangular matrix, 20 Eigenvector, 40, 129 left eigenvector, 143 right eigenvector, 143 Eigenvector expansion, 130 Elementary reflector, 107 Elsner’s theorem, 321 Equivalent norms, 173 Euclidian norm, 11 Exams 1977-1, inverse update, 21 1977-2, weighted least squares, 217 1977-3, symmetrize matrix, 53 1978-1, computing the inverse, 79 1979-3, x T Ay inequality, 119 1980-2, singular values perturbation, 221 1981-2, perturbed linear equation, 194 1981-3, solving T H x = b, 79 1982-1, L1U factorization, 96 1982-2, QL-factorization, 121 1982-3, QL-factorization, 121 1982-4, an A-norm inequality, 190 1983-1, L1U factorization update, 80 1983-2, a least squares problem, 217 1983-3, antisymmetric system, 310 1983-4, cg antisymmetric system, 311 1990-1, U1L factorization, 80 1991-3, Iterative method, 275 1993-2, periodic spline interpolation, 195 1994-2, upper Hessenberg system, 76 1995-4, A orthogonal bases, 190 1996-3, Cayley Hamilton Theorem, 148 2005-1, right or wrong?, 147 2005-2, singular values, 164 2006-1, yes or no, 330 2006-2, QR fact of band matrices, 122 2008-1, Gauss-Seidel method, 276 2008-2, find QR factorization, 122 2008-3, eigenpairs of Kronecker prod, 234 2009-1, matrix products, 21 2009-2, Gershgorin disks, 331 2009-3, eigenvalues of tridiag matrix, 147 2010-1, Householder transformation, 120 2010-2, eigenvalue perturbations, 331 2011-1, steepest descent, 307 2011-2, polar decomposition, 165 2013-2, a Givens transformation, 123 2013-4, LSQ MATLAB program, 197 2015-1, underdetermined system, 166 2015-2, Cholesky update, 97 2015-3, Rayleigh quotient, 149 2016-1, Norms, Cholesky and SVD, 168 2016-2 Cholesky update, 97 Index 2016-3, operation counts, 124 2017-1, Strassen multiplication, 20 2017-2, SVD , 220 2017-3, an iterative method, 192 2018-1, least square fit, 217 2018-2, Cholesky and Givens, 125 2018-3, AT A inner product, 308 Extension of basis, 104 F Fast Fourier transform (FFT), 242, 244 recursive FFT, 246 Fill-inn, 238 Finite difference method, 37 Fixed-point, 258 Fixed point form of discrete Poisson equation, 255 Fixed-point iteration, 258 Fourier matrix, 243 Fredholm’s alternative, 220 Frobenius norm, 161 G Gaussian elimination, 59 complete pivoting, 70 interchange matrix, 67 multipliers, 60 pivot, 66 pivoting, 66 pivot row, 66 pivots, 60 pivot vector, 67 scaled partial pivoting, 70 Generalized eigenvectors, 134 Generalized inverse, 209 Geometric multiplicity, 132 Gerschgorin’s theorem, 318 Given’s rotation, 117 Gradient, 86, 351 Gradient method, 282 Gram-Schmidt, 103 H Hadamard’s inequality, 121 Hessian, 86, 351 Hilbert matrix, 24, 203 Hoffman-Wielandt theorem, 143 Hölder’s inequality, 172, 187 Householder transformation, 107 359 I Ill-conditioned problem, 181 Inequality geometric/arithmetic mean, 187 Hölder, 187 Kantorovich, 296 Minkowski, 188 Inner product, 99, 100 inner product norm, 100 Pythagoras’ theorem, 102 standard, 11 standard inner product in Cn , 100 Inner product space orthogonal basis, 102 orthonormal basis, 102 Interchange matrix, 67 Inverse power method, 339 Inverse triangle inequality, 173 Iterative method convergence, 260 Gauss-Seidel, 254 Jacobi, 254 SOR, 254 SOR, convergence, 263 SSOR, 255 Iterative methods, 251 J Jacobian, 352 Jordan factorization, 134 generalized eigenvectors, 134 Jordan block, 133 Jordan canonical form, 133 principal vectors, 134 Jordan factors, 134 K Kronecker product, 229 eigenvectors, 231 inverse, 231 left product, 229 mixed product rule, 230 nonsingular, 231 positive definite, 231 propertis, 231 right product, 229 symmetry, 231 transpose, 231 Kronecker sum, 230 nonsingular, 231 positive definite, 231 symmetry, 231 Krylov space, 289 360 L Laplacian, 351 LDL theorem, 84 Leading principal block submatrices, 74 Leading principal minor, 60 Leading principal submatrices, 60 Least squares error analysis, 210 Least squares problem, 199 Least squares solution, 199 Left eigenpair, 143 Left eigenvalue, 143 Left eigenvector, 143 Left triangular, 70 Linear combination nontrivial, span, Linear system Cramer’s rule, 17 existence and uniqueness, 12, 13 homogeneous, 12 overdetermined, 12 residual vector, 182 square, 12 underdetermined, 12 Linearly dependent, Linearly independent, LSQ, 199 LU factorization, 70 LU theorem, 72 M Matrix A∗ A, 86 addition, adjoint, 17 adjoint formula for the inverse, 17 banded, block matrix, 44 block triangular, 46 cofactor, 16 column space (span), 10 companion matrix, 146 computing inverse, 75 conjugate transpose, deflation, 137 diagonal, element-by-element operations, Hadamard product, Hermitian, 3, 43 Hilbert, 24 Hilbert matrix, inverse, 24 idempotent, 146 Index identity matrix, ill-conditioned, 182 inverse, 13 invertible, 13, 14 Kronecker product, 229 LDL* and LL*, 88 leading principal minor, 60 leading principal submatrices, 60 left inverse, 13 left triangular, lower Hessenberg, lower triangular, LU theorem, 72 multiplication, negative (semi)definite, 86 Neumann series, 270 nilpotent, 146 nonsingular, 12, 13 nonsingular products, 13 nullity, 10 null space (N), 10 operator norm, 176 outer product, 54 outer product expansion, 54 permutation, 66 positive definite, 86 positive semidefinite, 86 principal minor, 60 principal submatrix, 60 product of triangular matrices, 47 quasi-triangular, 139 right inverse, 13 right triangular, row space, 10 scalar multiplication, Schur product, second derivative, 38 Sherman-Morrison formula, 21 similar matrices, 131 similarity transformation, 131 singular, 12, 13 spectral radius, 261, 268 Strassen multiplication, 20 strictly diagonally dominant, 33 symmetric, test matrix,1D , 41 test matrix, 2D, 229 trace, 19 transpose, triangular, 47 tridiagonal, unit triangular, 48 upper Hessenberg, upper trapezoidal, 111 Index upper triangular, vec Operation, 226 weakly diagonally dominant, 38 well-conditioned, 182 Matrix norm consistent norm, 175 Frobenius norm, 161, 174 max norm, 174 operator norm, 175 spectral norm, 178 subordinate norm, 175 sum norm, 174 two-norm, 178 Minimal polynomial, 148 Minkowski’s inequality, 172, 188 Mixed product rule, 230 N Natural ordering, 227 Negative (semi)definite, 86 Neumann series, 270 Nilpotent matrix, 146 Nondefective matrix, 129 Nonsingular matrix, 12 Nontrivial subspaces, Norm, 171 l1 -norm, 172 l2 -norm, 172 l∞ -norm, 172 absolute norm, 180 continuity, 173 Euclidian norm, 172 infinity-norm, 172 max norm, 172 monotone norm, 180 one-norm, 172 triangle inequality, 171 two-norm, 172 Normal matrix, 137 Normal equations, 200 Null space (N), 10 O Oblique projection, 104 1D test matrix, 41 Operation count, 64 Operator norm, 176 Optimal relaxation parameter, 265 Optimal step length, 282 Orthogonal matrix, 106 Orthogonal projection, 105 361 Orthogonal sum, 104 Outer product, 54 P Paraboloid, 306 Parallelogram identity, 188 Partial pivoting, 69 Permutation matrix, 66 Perpendicular vectors, 102 Pivot vector, 67 Plane rotation, 117 p-norms, 172 Poisson matrix, 227 Poisson problem, 225 five point stencil, 227 nine point scheme, 234 Poisson matrix, 227 variable coefficients, 302 Poisson problem (1D), 37 Polarization identity, 197 Polynomial degree, linear interpolation, 28 nontrivial, Runge phenomenon, 28 zero, Positive definite, 86, 95 Positive semidefinite, 86, 95 Power method, 335 inverse, 339 Rayleigh quotient iteration, 340 shifted, 339 Preconditioned conjugate gradient method, 279 Preconditioning, 299 Preconditioning matrix, 258 Principal submatrix, 60 Principal vectors, 134 Q QR algorithm implicit shift, 346 Rayleigh quotient shift, 346 shifted, 345 Wilkinson shift, 346 QR decomposition, 114 QR factorization, 114 Quadratic form, 86 R Rank of a matrix, 10 Rate of convergence, 266 362 Rayleigh quotient, 139 generalized, 150 Rayleigh quotient iteration, 340 Relative error, 181 Residual vector, 182 Richardson’s method, 261 Right eigenpair, 143 Right eigenvalue, 143 Right eigenvector, 143 Rotation in the i, j -plane, 118 Row operations, 16 Row space, 10 S Scalar product, 99 Schur factorization, 136 Schur factors, 136 Second derivative matrix, 38 positive definite, 87 Semi-Cholesky factorization, 91 Sherman-Morrison formula, 21 Shifted power method, 339 Similar matrices, 131 Similarity transformation, 131 Singular value Courant-Fischer theorem, 213 Hoffman-Wielandt theorem, 215 Singular value factorization (SVF), 157 Singular values, 153 error analysis, 213 well conditioned, 216 Singular vectors, 153 Spectral radius, 261, 268 Spectral theorem, 140 Spectrum, 18, 129 Splitting matrices for J, GS and SOR, 259 Splitting matrix, 258 Standard inner product in Cn , 106 Steepest descent, 282 Stencil, 227 Sum of subspaces, 104 Sums of integers, 76 Sylvester’s inertia theorem, 328 T Trace, 19 Triangle inequality, 171 Triangular matrix lower triangular, 70 upper triangular, 70 2D test matrix, 229 Two point boundary value problem, 37 Index U Unitary matrix, 106 Unit vectors, Upper trapezoidal matrix, 111 Upper triangular, 70 V Variable coefficient, 313 Vector addition, angle, 102 linearly dependent, linearly independent, negative, orthogonal, 102 orthonormal, 102 scalar multiplication, zero, Vector norm, 101, 171 Vector space basis, change of basis, 10 change of basis matrix, complex, complex inner product space, 100 dimension, dimension formula, sums of subspaces, direct sum, enlarging vectors to a basis, examples of subspaces, existence of basis, finite dimensional, intersection, nontrivial subspaces, normed, 171 orthogonal vectors, 102 real, span, subspace, sum, trivial subspace, union, vectors, Vectorization, 226 W Weights, 203 Z Zero function, Editorial Policy Textbooks on topics in the field of computational science and engineering will be considered They should be written for courses in CSE education Both graduate and undergraduate textbooks will be published in TCSE Multidisciplinary topics and multidisciplinary teams of authors are especially welcome Format: Only works in English will be considered For evaluation purposes, manuscripts may be submitted in print or electronic form, in the latter case, preferably as pdfor zipped ps-files Authors are requested to use the LaTeX style files available from Springer at: http://www.springer.com/gp/authors-editors/book-authors-editors/resourcesguidelines/rights-permissions-licensing/manuscript-preparation/5636#c3324 (Layout & templates – LaTeX template – contributed books) Electronic material can be included if appropriate Please contact the publisher Those considering a book which might be suitable for the series are strongly advised to contact the publisher or the series editors at an early stage General Remarks Careful preparation of manuscripts will help keep production time short and ensure a satisfactory appearance of the finished book The following terms and conditions hold: Regarding free copies and royalties, the standard terms for Springer mathematics textbooks hold Please write to martin.peters@springer.com for details Authors are entitled to purchase further copies of their book and other Springer books for their personal use, at a discount of 33.3% directly from Springer-Verlag Series Editors Timothy J Barth NASA Ames Research Center NAS Division Moffett Field, CA 94035, USA barth@nas.nasa.gov Michael Griebel Institut für Numerische Simulation der Universität Bonn Wegelerstr 53115 Bonn, Germany griebel@ins.uni-bonn.de Risto M Nieminen Department of Applied Physics Aalto University School of Science and Technology 00076 Aalto, Finland risto.nieminen@tkk.fi Dirk Roose Department of Computer Science Katholieke Universiteit Leuven Celestijnenlaan 200A 3001 Leuven-Heverlee, Belgium dirk.roose@cs.kuleuven.be David E Keyes Mathematical and Computer Sciences and Engineering King Abdullah University of Science and Technology P.O Box 55455 Jeddah 21534, Saudi Arabia david.keyes@kaust.edu.sa Tamar Schlick Department of Chemistry and Courant Institute of Mathematical Sciences New York University 251 Mercer Street New York, NY 10012, USA schlick@nyu.edu and Editor for Computational Science and Engineering at Springer: Martin Peters Springer-Verlag Mathematics Editorial Tiergartenstrasse 17 69121 Heidelberg, Germany martin.peters@springer.com Department of Applied Physics and Applied Mathematics Columbia University 500 W 120 th Street New York, NY 10027, USA kd2112@columbia.edu Texts in Computational Science and Engineering H P Langtangen, Computational Partial Differential Equations Numerical Methods and Diffpack Programming 2nd Edition A Quarteroni, F Saleri, P Gervasio, Scientific Computing with MATLAB and Octave 4th Edition H P Langtangen, Python Scripting for Computational Science 3rd Edition H Gardner, G Manduchi, Design Patterns for e-Science M Griebel, S Knapek, G Zumbusch, Numerical Simulation in Molecular Dynamics H P Langtangen, A Primer on Scientific Programming with Python 5th Edition A Tveito, H P Langtangen, B F Nielsen, X Cai, Elements of Scientific Computing B Gustafsson, Fundamentals of Scientific Computing M Bader, Space-Filling Curves 10 M Larson, F Bengzon, The Finite Element Method: Theory, Implementation and Applications 11 W Gander, M Gander, F Kwok, Scientific Computing: An Introduction using Maple and MATLAB 12 P Deuflhard, S Röblitz, A Guide to Numerical Modelling in Systems Biology 13 M H Holmes, Introduction to Scientific Computing and Data Analysis 14 S Linge, H P Langtangen, Programming for Computations - A Gentle Introduction to Numerical Simulations with MATLAB/Octave 15 S Linge, H P Langtangen, Programming for Computations - A Gentle Introduction to Numerical Simulations with Python 16 H.P Langtangen, S Linge, Finite Difference Computing with PDEs - A Modern Software Approach 17 B Gustafsson, Scientific Computing from a Historical Perspective 18 J.A Trangenstein, Scientific Computing - Vol I – Linear and Nonlinear Equations 19 J.A Trangenstein, Scientific Computing - Vol II – Eigenvalues and Optimization 20 J.A Trangenstein, Scientific Computing - Vol III – Approximation and Integration 21 H.P Langtangen, K.-A Mardal, Introduction to Numerical Methods for Variational Problems 22 T Lyche, Numerical Linear Algebra and Matrix Factorizations For further information on these books please have a look at our mathematics catalogue at the following URL: www.springer.com/series/5151 Monographs in Computational Science and Engineering J Sundnes, G.T Lines, X Cai, B.F Nielsen, K.-A Mardal, A Tveito, Computing the Electrical Activity in the Heart For further information on this book, please have a look at our mathematics catalogue at the following URL: www.springer.com/series/7417 Lecture Notes in Computational Science and Engineering D Funaro, Spectral Elements for Transport-Dominated Equations H.P Langtangen, Computational Partial Differential Equations Numerical Methods and Diffpack Programming W Hackbusch, G Wittum (eds.), Multigrid Methods V P Deuflhard, J Hermans, B Leimkuhler, A.E Mark, S Reich, R.D Skeel (eds.), Computational Molecular Dynamics: Challenges, Methods, Ideas D Kröner, M Ohlberger, C Rohde (eds.), An Introduction to Recent Developments in Theory and Numerics for Conservation Laws S Turek, Efficient Solvers for Incompressible Flow Problems An Algorithmic and Computational Approach R von Schwerin, Multi Body System SIMulation Numerical Methods, Algorithms, and Software H.-J Bungartz, F Durst, C Zenger (eds.), High Performance Scientific and Engineering Computing T.J Barth, H Deconinck (eds.), High-Order Methods for Computational Physics 10 H.P Langtangen, A.M Bruaset, E Quak (eds.), Advances in Software Tools for Scientific Computing 11 B Cockburn, G.E Karniadakis, C.-W Shu (eds.), Discontinuous Galerkin Methods Theory, Computation and Applications 12 U van Rienen, Numerical Methods in Computational Electrodynamics Linear Systems in Practical Applications 13 B Engquist, L Johnsson, M Hammill, F Short (eds.), Simulation and Visualization on the Grid 14 E Dick, K Riemslagh, J Vierendeels (eds.), Multigrid Methods VI 15 A Frommer, T Lippert, B Medeke, K Schilling (eds.), Numerical Challenges in Lattice Quantum Chromodynamics 16 J Lang, Adaptive Multilevel Solution of Nonlinear Parabolic PDE Systems Theory, Algorithm, and Applications 17 B.I Wohlmuth, Discretization Methods and Iterative Solvers Based on Domain Decomposition 18 U van Rienen, M Günther, D Hecht (eds.), Scientific Computing in Electrical Engineering 19 I Babuška, P.G Ciarlet, T Miyoshi (eds.), Mathematical Modeling and Numerical Simulation in Continuum Mechanics 20 T.J Barth, T Chan, R Haimes (eds.), Multiscale and Multiresolution Methods Theory and Applications 21 M Breuer, F Durst, C Zenger (eds.), High Performance Scientific and Engineering Computing 22 K Urban, Wavelets in Numerical Simulation Problem Adapted Construction and Applications 23 L.F Pavarino, A Toselli (eds.), Recent Developments in Domain Decomposition Methods 24 T Schlick, H.H Gan (eds.), Computational Methods for Macromolecules: Challenges and Applications 25 T.J Barth, H Deconinck (eds.), Error Estimation and Adaptive Discretization Methods in Computational Fluid Dynamics 26 M Griebel, M.A Schweitzer (eds.), Meshfree Methods for Partial Differential Equations 27 S Müller, Adaptive Multiscale Schemes for Conservation Laws 28 C Carstensen, S Funken, W Hackbusch, R.H.W Hoppe, P Monk (eds.), Computational Electromagnetics 29 M.A Schweitzer, A Parallel Multilevel Partition of Unity Method for Elliptic Partial Differential Equations 30 T Biegler, O Ghattas, M Heinkenschloss, B van Bloemen Waanders (eds.), Large-Scale PDEConstrained Optimization 31 M Ainsworth, P Davies, D Duncan, P Martin, B Rynne (eds.), Topics in Computational Wave Propagation Direct and Inverse Problems 32 H Emmerich, B Nestler, M Schreckenberg (eds.), Interface and Transport Dynamics Computational Modelling 33 H.P Langtangen, A Tveito (eds.), Advanced Topics in Computational Partial Differential Equations Numerical Methods and Diffpack Programming 34 V John, Large Eddy Simulation of Turbulent Incompressible Flows Analytical and Numerical Results for a Class of LES Models 35 E Bänsch (ed.), Challenges in Scientific Computing - CISC 2002 36 B.N Khoromskij, G Wittum, Numerical Solution of Elliptic Differential Equations by Reduction to the Interface 37 A Iske, Multiresolution Methods in Scattered Data Modelling 38 S.-I Niculescu, K Gu (eds.), Advances in Time-Delay Systems 39 S Attinger, P Koumoutsakos (eds.), Multiscale Modelling and Simulation 40 R Kornhuber, R Hoppe, J Périaux, O Pironneau, O Wildlund, J Xu (eds.), Domain Decomposition Methods in Science and Engineering 41 T Plewa, T Linde, V.G Weirs (eds.), Adaptive Mesh Refinement – Theory and Applications 42 A Schmidt, K.G Siebert, Design of Adaptive Finite Element Software The Finite Element Toolbox ALBERTA 43 M Griebel, M.A Schweitzer (eds.), Meshfree Methods for Partial Differential Equations II 44 B Engquist, P Lötstedt, O Runborg (eds.), Multiscale Methods in Science and Engineering 45 P Benner, V Mehrmann, D.C Sorensen (eds.), Dimension Reduction of Large-Scale Systems 46 D Kressner, Numerical Methods for General and Structured Eigenvalue Problems 47 A Boriỗi, A Frommer, B Joú, A Kennedy, B Pendleton (eds.), QCD and Numerical Analysis III 48 F Graziani (ed.), Computational Methods in Transport 49 B Leimkuhler, C Chipot, R Elber, A Laaksonen, A Mark, T Schlick, C Schütte, R Skeel (eds.), New Algorithms for Macromolecular Simulation 50 M Bücker, G Corliss, P Hovland, U Naumann, B Norris (eds.), Automatic Differentiation: Applications, Theory, and Implementations 51 A.M Bruaset, A Tveito (eds.), Numerical Solution of Partial Differential Equations on Parallel Computers 52 K.H Hoffmann, A Meyer (eds.), Parallel Algorithms and Cluster Computing 53 H.-J Bungartz, M Schäfer (eds.), Fluid-Structure Interaction 54 J Behrens, Adaptive Atmospheric Modeling 55 O Widlund, D Keyes (eds.), Domain Decomposition Methods in Science and Engineering XVI 56 S Kassinos, C Langer, G Iaccarino, P Moin (eds.), Complex Effects in Large Eddy Simulations 57 M Griebel, M.A Schweitzer (eds.), Meshfree Methods for Partial Differential Equations III 58 A.N Gorban, B Kégl, D.C Wunsch, A Zinovyev (eds.), Principal Manifolds for Data Visualization and Dimension Reduction 59 H Ammari (ed.), Modeling and Computations in Electromagnetics: A Volume Dedicated to JeanClaude Nédélec 60 U Langer, M Discacciati, D Keyes, O Widlund, W Zulehner (eds.), Domain Decomposition Methods in Science and Engineering XVII 61 T Mathew, Domain Decomposition Methods for the Numerical Solution of Partial Differential Equations 62 F Graziani (ed.), Computational Methods in Transport: Verification and Validation 63 M Bebendorf, Hierarchical Matrices A Means to Efficiently Solve Elliptic Boundary Value Problems 64 C.H Bischof, H.M Bücker, P Hovland, U Naumann, J Utke (eds.), Advances in Automatic Differentiation 65 M Griebel, M.A Schweitzer (eds.), Meshfree Methods for Partial Differential Equations IV 66 B Engquist, P Lötstedt, O Runborg (eds.), Multiscale Modeling and Simulation in Science 67 I.H Tuncer, Ü Gülcat, D.R Emerson, K Matsuno (eds.), Parallel Computational Fluid Dynamics 2007 68 S Yip, T Diaz de la Rubia (eds.), Scientific Modeling and Simulations 69 A Hegarty, N Kopteva, E O’Riordan, M Stynes (eds.), BAIL 2008 – Boundary and Interior Layers 70 M Bercovier, M.J Gander, R Kornhuber, O Widlund (eds.), Domain Decomposition Methods in Science and Engineering XVIII 71 B Koren, C Vuik (eds.), Advanced Computational Methods in Science and Engineering 72 M Peters (ed.), Computational Fluid Dynamics for Sport Simulation 73 H.-J Bungartz, M Mehl, M Schäfer (eds.), Fluid Structure Interaction II - Modelling, Simulation, Optimization 74 D Tromeur-Dervout, G Brenner, D.R Emerson, J Erhel (eds.), Parallel Computational Fluid Dynamics 2008 75 A.N Gorban, D Roose (eds.), Coping with Complexity: Model Reduction and Data Analysis 76 J.S Hesthaven, E.M Rønquist (eds.), Spectral and High Order Methods for Partial Differential Equations 77 M Holtz, Sparse Grid Quadrature in High Dimensions with Applications in Finance and Insurance 78 Y Huang, R Kornhuber, O.Widlund, J Xu (eds.), Domain Decomposition Methods in Science and Engineering XIX 79 M Griebel, M.A Schweitzer (eds.), Meshfree Methods for Partial Differential Equations V 80 P.H Lauritzen, C Jablonowski, M.A Taylor, R.D Nair (eds.), Numerical Techniques for Global Atmospheric Models 81 C Clavero, J.L Gracia, F.J Lisbona (eds.), BAIL 2010 – Boundary and Interior Layers, Computational and Asymptotic Methods 82 B Engquist, O Runborg, Y.R Tsai (eds.), Numerical Analysis and Multiscale Computations 83 I.G Graham, T.Y Hou, O Lakkis, R Scheichl (eds.), Numerical Analysis of Multiscale Problems 84 A Logg, K.-A Mardal, G Wells (eds.), Automated Solution of Differential Equations by the Finite Element Method 85 J Blowey, M Jensen (eds.), Frontiers in Numerical Analysis - Durham 2010 86 O Kolditz, U.-J Gorke, H Shao, W Wang (eds.), Thermo-Hydro-Mechanical-Chemical Processes in Fractured Porous Media - Benchmarks and Examples 87 S Forth, P Hovland, E Phipps, J Utke, A Walther (eds.), Recent Advances in Algorithmic Differentiation 88 J Garcke, M Griebel (eds.), Sparse Grids and Applications 89 M Griebel, M.A Schweitzer (eds.), Meshfree Methods for Partial Differential Equations VI 90 C Pechstein, Finite and Boundary Element Tearing and Interconnecting Solvers for Multiscale Problems 91 R Bank, M Holst, O Widlund, J Xu (eds.), Domain Decomposition Methods in Science and Engineering XX 92 H Bijl, D Lucor, S Mishra, C Schwab (eds.), Uncertainty Quantification in Computational Fluid Dynamics 93 M Bader, H.-J Bungartz, T Weinzierl (eds.), Advanced Computing 94 M Ehrhardt, T Koprucki (eds.), Advanced Mathematical Models and Numerical Techniques for Multi-Band Effective Mass Approximations 95 M Azaïez, H El Fekih, J.S Hesthaven (eds.), Spectral and High Order Methods for Partial Differential Equations ICOSAHOM 2012 96 F Graziani, M.P Desjarlais, R Redmer, S.B Trickey (eds.), Frontiers and Challenges in Warm Dense Matter 97 J Garcke, D Pflüger (eds.), Sparse Grids and Applications – Munich 2012 98 J Erhel, M Gander, L Halpern, G Pichot, T Sassi, O Widlund (eds.), Domain Decomposition Methods in Science and Engineering XXI 99 R Abgrall, H Beaugendre, P.M Congedo, C Dobrzynski, V Perrier, M Ricchiuto (eds.), High Order Nonlinear Numerical Methods for Evolutionary PDEs - HONOM 2013 100 M Griebel, M.A Schweitzer (eds.), Meshfree Methods for Partial Differential Equations VII 101 R Hoppe (ed.), Optimization with PDE Constraints - OPTPDE 2014 102 S Dahlke, W Dahmen, M Griebel, W Hackbusch, K Ritter, R Schneider, C Schwab, H Yserentant (eds.), Extraction of Quantifiable Information from Complex Systems 103 A Abdulle, S Deparis, D Kressner, F Nobile, M Picasso (eds.), Numerical Mathematics and Advanced Applications - ENUMATH 2013 104 T Dickopf, M.J Gander, L Halpern, R Krause, L.F Pavarino (eds.), Domain Decomposition Methods in Science and Engineering XXII 105 M Mehl, M Bischoff, M Schäfer (eds.), Recent Trends in Computational Engineering - CE2014 Optimization, Uncertainty, Parallel Algorithms, Coupled and Complex Problems 106 R.M Kirby, M Berzins, J.S Hesthaven (eds.), Spectral and High Order Methods for Partial Differential Equations - ICOSAHOM’14 107 B Jüttler, B Simeon (eds.), Isogeometric Analysis and Applications 2014 108 P Knobloch (ed.), Boundary and Interior Layers, Computational and Asymptotic Methods – BAIL 2014 109 J Garcke, D Pflüger (eds.), Sparse Grids and Applications – Stuttgart 2014 110 H P Langtangen, Finite Difference Computing with Exponential Decay Models 111 A Tveito, G.T Lines, Computing Characterizations of Drugs for Ion Channels and Receptors Using Markov Models 112 B Karazösen, M Manguo˘glu, M Tezer-Sezgin, S Gưktepe, Ư U˘gur (eds.), Numerical Mathematics and Advanced Applications - ENUMATH 2015 113 H.-J Bungartz, P Neumann, W.E Nagel (eds.), Software for Exascale Computing - SPPEXA 20132015 114 G.R Barrenechea, F Brezzi, A Cangiani, E.H Georgoulis (eds.), Building Bridges: Connections and Challenges in Modern Approaches to Numerical Partial Differential Equations 115 M Griebel, M.A Schweitzer (eds.), Meshfree Methods for Partial Differential Equations VIII 116 C.-O Lee, X.-C Cai, D.E Keyes, H.H Kim, A Klawonn, E.-J Park, O.B Widlund (eds.), Domain Decomposition Methods in Science and Engineering XXIII 117 T Sakurai, S Zhang, T Imamura, Y Yusaku, K Yoshinobu, H Takeo (eds.), Eigenvalue Problems: Algorithms, Software and Applications, in Petascale Computing EPASA 2015, Tsukuba, Japan, September 2015 118 T Richter (ed.), Fluid-structure Interactions Models, Analysis and Finite Elements 119 M.L Bittencourt, N.A Dumont, J.S Hesthaven (eds.), Spectral and High Order Methods for Partial Differential Equations ICOSAHOM 2016 120 Z Huang, M Stynes, Z Zhang (eds.), Boundary and Interior Layers, Computational and Asymptotic Methods BAIL 2016 121 S.P.A Bordas, E.N Burman, M.G Larson, M.A Olshanskii (eds.), Geometrically Unfitted Finite Element Methods and Applications Proceedings of the UCL Workshop 2016 122 A Gerisch, R Penta, J Lang (eds.), Multiscale Models in Mechano and Tumor Biology Modeling, Homogenization, and Applications 123 J Garcke, D Pflüger, C.G Webster, G Zhang (eds.), Sparse Grids and Applications - Miami 2016 124 M Schăafer, M Behr, M Mehl, B Wohlmuth (eds.), Recent Advances in Computational Engineering Proceedings of the 4th International Conference on Computational Engineering (ICCE 2017) in Darmstadt 125 P.E Bjørstad, S.C Brenner, L Halpern, R Kornhuber, H.H Kim, T Rahman, O.B Widlund (eds.), Domain Decomposition Methods in Science and Engineering XXIV 24th International Conference on Domain Decomposition Methods, Svalbard, Norway, February 6–10, 2017 126 F.A Radu, K Kumar, I Berre, J.M Nordbotten, I.S Pop (eds.), Numerical Mathematics and Advanced Applications – ENUMATH 2017 127 X Roca, A Loseille (eds.), 27th International Meshing Roundtable 128 Th Apel, U Langer, A Meyer, O Steinbach (eds.), Advanced Finite Element Methods with Applications Selected Papers from the 30th Chemnitz Finite Element Symposium 2017 129 M Griebel, M A Schweitzer (eds.), Meshfree Methods for Partial Differencial Equations IX 130 S Weißer, BEM-based Finite Element Approaches on Polytopal Meshes 131 V A Garanzha, L Kamenski, H Si (eds.), Numerical Geometry, Grid Generation and Scientific Computing Proceedings of the 9th International Conference, NUMGRID 2018/Voronoi 150, Celebrating the 150th Anniversary of G F Voronoi, Moscow, Russia, December 2018 132 E H van Brummelen, A Corsini, S Perotto, G Rozza (eds.), Numerical Methods for Flows For further information on these books please have a look at our mathematics catalogue at the following URL: www.springer.com/series/3527 ... column vector and x T is a row vector © Springer Nature Switzerland AG 2020 T Lyche, Numerical Linear Algebra and Matrix Factorizations, Texts in Computational Science and Engineering 22, https://doi.org/10.1007/978-3-030-36468-7_1... to shorter and more efficient programs Stability is important both for the mathematical problems and for the numerical algorithms Stability can be studied in terms of perturbation theory that... as the present department head, and, finally, as a researcher within the international linear algebra and matrix theory community The book actually has a long history and started out as lecture