Computer Methods for Ordinary Differential Equations and Differential Algebraic Equations

Computer Methods for Ordinary Di erential Equations and Di erential-Algebraic Equations Uri M Ascher and Linda R Petzold December 2, 1997 Preface This book has been developed from course notes that we wrote, having repeatedly taught courses on the numerical solution of ordinary di erential equations (ODEs) and related problems We have taught such courses at a senior undergraduate level as well as at the level of a rst graduate course on numerical methods for di erential equations The audience typically consists of students from Mathematics, Computer Science and a variety of disciplines in engineering and sciences such as Mechanical, Electrical and Chemical Engineering, Physics, Earth Sciences, etc The material that this book covers can be viewed as a rst course on the numerical solution of di erential equations It is designed for people who want to gain a practical knowledge of the techniques used today The course aims to achieve a thorough understanding of the issues and methods involved and of the reasons for the successes and failures of existing software On one hand, we avoid an extensive, thorough, theorem-proof type exposition: we try to get to current methods, issues and software as quickly as possible On the other hand, this is not a quick recipe book, as we feel that a deeper understanding than can usually be gained by a recipe course is required to enable the student or the researcher to use their knowledge to design their own solution approach for any nonstandard problems they may encounter in future work The book covers initial-value and boundary-value problems, as well as di erential-algebraic equations (DAEs) In a one-semester course we have been typically covering over 75% of the material it contains We wrote this book partially as a result of frustration at not being able to assign a textbook adequate for the material that we have found ourselves covering There is certainly excellent, in-depth literature around In fact, we are making repeated references to exhaustive texts which, combined, cover almost all the material in this book Those books contain the proofs and references which we omit They span thousands of pages, though, and the time commitment required to study them in adequate depth may be more than many students and researchers can a ord to invest We have tried to stay below a 350-page limit and to address all three ODE-related areas menii iii tioned above A signi cant amount of additional material is covered in the Exercises Other additional important topics are referred to in brief sections of Notes and References Software is an important and well-developed part of this subject We have attempted to cover the most fundamental software issues in the text Much of the excellent and publicly-available software is described in the Software sections at the end of the relevant chapters, and available codes are cross-referenced in the index Review material is highlighted and presented in the text when needed, and it is also cross-referenced in the index Traditionally, numerical ODE texts have spent a great deal of time developing families of higher order methods, e.g Runge-Kutta and linear multistep methods, applied rst to nonsti problems and then to sti problems Initial value problems and boundary value problems have been treated in separate texts, although there is much in common There have been fundamental di erences in approach, notation, and even in basic de nitions, between ODE initial value problems, ODE boundary value problems, and partial di erential equations (PDEs) We have chosen instead to focus on the classes of problems to be solved, mentioning wherever possible applications which can lend insight into the requirements and the potential sources of di culty for numerical solution We begin by outlining the relevant mathematical properties of each problem class, then carefully develop the lower-order numerical methods and fundamental concepts for the numerical analysis Next we introduce the appropriate families of higher-order methods, and nally we describe in some detail how these methods are implemented in modern adaptive software An important feature of this book is that it gives an integrated treatment of ODE initial value problems, ODE boundary value problems, and DAEs, emphasizing not only the di erences between these types of problems but also the fundamental concepts, numerical methods and analysis which they have in common This approach is also closer to the typical presentation for PDEs, leading, we hope, to a more natural introduction to that important subject Knowledge of signi cant portions of the material in this book is essential for the rapidly emerging eld of numerical dynamical systems These are numerical methods employed in the study of the long term, qualitative behavior of various nonlinear ODE systems We have emphasized and developed in this work relevant problems, approaches and solutions But we avoided developing further methods which require deeper, or more speci c, knowledge of dynamical systems, which we did not want to assume as a prerequisite The plan of the book is as follows Chapter is an introduction to the di erent types of mathematical models which are addressed in the book We use simple examples to introduce and illustrate initial- and boundaryvalue problems for ODEs and DAEs We then introduce some important applications where such problems arise in practice iv Each of the three parts of the book which follow starts with a chapter which summarizes essential theoretical, or analytical issues (i.e before applying any numerical method) This is followed by chapters which develop and analyze numerical techniques For initial value ODEs, which comprise roughly half this book, Chapter summarizes the theory most relevant for computer methods, Chapter introduces all the basic concepts and simple methods (relevant also for boundary value problems and for DAEs), Chapter is devoted to one-step (Runge-Kutta) methods and Chapter discusses multistep methods Chapters 6-8 are devoted to boundary value problems for ODEs Chapter discusses the theory which is essential to understand and to make e ective use of the numerical methods for these problems Chapter brie y considers shooting-type methods and Chapter is devoted to nite di erence approximations and related techniques The remaining two chapters consider DAEs This subject has been researched and solidi ed only very recently (in the past 15 years) Chapter is concerned with background material and theory It is much longer than Chapters and because understanding the relationship between ODEs and DAEs, and the questions regarding reformulation of DAEs, is essential and already suggests a lot regarding computer approaches Chapter 10 discusses numerical methods for DAEs Various courses can be taught using this book A 10-week course can be based on the rst chapters, with an addition from either one of the remaining two parts In a 13-week course (or shorter in a more advanced graduate class) it is possible to cover comfortably Chapters 1-5 and either Chapters 6-8 or Chapters 9-10, with a more super cial coverage of the remaining material The exercises vary in scope and level of di culty We have provided some hints, or at least warnings, for those exercises that we (or our students) have found more demanding Many people helped us with the tasks of shaping up, correcting, ltering and re ning the material in this book First and foremost there are our students in the various classes we taught on this subject They made us acutely aware of the di erence between writing with the desire to explain and writing with the desire to impress We note, in particular, G Lakatos, D Aruliah, P Ziegler, H Chin, R Spiteri, P Lin, P Castillo, E Johnson, D Clancey and D Rasmussen We have bene ted particularly from our earlier collaborations on other, related books with K Brenan, S Campbell, R Mattheij and R Russell Colleagues who have o ered much insight, advice and criticism include E Biscaia, G Bock, C W Gear, W Hayes, C Lubich, V Murata, D Pai, J B Rosen, L Shampine and A Stuart Larry Shampine, in particular, did an incredibly extensive refereeing job and o ered many comments which have helped us to signi cantly improve this text We have v PREFACE also bene ted from comments of numerous anonymous referees December 2, 1997 U M Ascher L R Petzold vi PREFACE Contents Ordinary Di erential Equations 1.1 1.2 1.3 1.4 1.5 1.6 Initial Value Problems Boundary Value Problems Di erential-Algebraic Equations Families of Application Problems Dynamical Systems Notation On Problem Stability 2.1 2.2 2.3 2.4 2.5 2.6 2.7 Test Equation and General De nitions Linear, Constant Coe cient Systems Linear, Variable Coe cient Systems Nonlinear Problems Hamiltonian Systems Notes and References Exercises Basic Methods, Basic Concepts 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 A Simple Method: Forward Euler Convergence, Accuracy, Consistency and 0-Stability Absolute Stability Sti ness: Backward Euler A-Stability, Sti Decay Symmetry: Trapezoidal Method Rough Problems Software, Notes and References 3.8.1 Notes 3.8.2 Software 3.9 Exercises One Step Methods 11 15 16 19 21 22 26 28 29 31 31 35 35 38 42 47 56 58 61 64 64 65 67 73 4.1 The First Runge-Kutta Methods 75 4.2 General Formulation of Runge-Kutta Methods 81 vii viii CONTENTS 4.3 4.4 4.5 4.6 4.7 Convergence, 0-Stability and Order for Runge-Kutta Methods 83 Regions of Absolute Stability for Explicit Runge-Kutta Methods 89 Error Estimation and Control 91 Sensitivity to Data Perturbations 96 Implicit Runge-Kutta and Collocation Methods 101 4.7.1 Implicit Runge-Kutta Methods Based on Collocation 102 4.7.2 Implementation and Diagonally Implicit Methods 105 4.7.3 Order Reduction 108 4.7.4 More on Implementation and SIRK Methods 109 4.8 Software, Notes and References 110 4.8.1 Notes 110 4.8.2 Software 112 4.9 Exercises 113 Linear Multistep Methods 125 More BVP Theory and Applications 163 5.1 The Most Popular Methods 127 5.1.1 Adams Methods 128 5.1.2 Backward Di erentiation Formulae 131 5.1.3 Initial Values for Multistep Methods 132 5.2 Order, 0-Stability and Convergence 134 5.2.1 Order 134 5.2.2 Stability: Di erence Equations and the Root Condition 137 5.2.3 0-Stability and Convergence 139 5.3 Absolute Stability 143 5.4 Implementation of Implicit Linear Multistep Methods 146 5.4.1 Functional Iteration 146 5.4.2 Predictor-Corrector Methods 146 5.4.3 Modi ed Newton Iteration 148 5.5 Designing Multistep General-Purpose Software 149 5.5.1 Variable Step-Size Formulae 150 5.5.2 Estimating and Controlling the Local Error 152 5.5.3 Approximating the Solution at O -Step Points 155 5.6 Software, Notes and References 155 5.6.1 Notes 155 5.6.2 Software 156 5.7 Exercises 157 6.1 6.2 6.3 6.4 6.5 Linear Boundary Value Problems and Green's Function Stability of Boundary Value Problems BVP Sti ness Some Reformulation Tricks Notes and References 166 168 171 172 174 CONTENTS ix 6.6 Exercises 175 Shooting 7.1 Shooting: a Simple Method and its Limitations 7.1.1 Di culties 7.2 Multiple Shooting 7.3 Software, Notes and References 7.3.1 Notes 7.3.2 Software 7.4 Exercises Finite Di erence Methods for BVPs 8.1 Midpoint and Trapezoidal Methods 8.1.1 Solving Nonlinear Problems: Quasilinearization 8.1.2 Consistency, 0-stability and Convergence 8.2 Solving the Linear Equations 8.3 Higher Order Methods 8.3.1 Collocation 8.3.2 Acceleration Techniques 8.4 More on Solving Nonlinear Problems 8.4.1 Damped Newton 8.4.2 Shooting for Initial Guesses 8.4.3 Continuation 8.5 Error Estimation and Mesh Selection 8.6 Very Sti Problems 8.7 Decoupling 8.8 Software, Notes and References 8.8.1 Notes 8.8.2 Software 8.9 Exercises More on Di erential-Algebraic Equations 9.1 Index and Mathematical Structure 9.1.1 Special DAE Forms 9.1.2 DAE Stability 9.2 Index Reduction and Stabilization: ODE with Invariant 9.2.1 Reformulation of Higher-Index DAEs 9.2.2 ODEs with Invariants 9.2.3 State Space Formulation 9.3 Modeling with DAEs 9.4 Notes and References 9.5 Exercises 177 177 180 183 186 186 187 187 193 194 197 201 204 206 206 208 210 210 211 211 213 215 220 222 222 223 223 231 232 238 245 247 248 250 253 254 256 257 x CONTENTS 10 Numerical Methods for Di erential-Algebraic Equations 263 10.1 Direct Discretization Methods 264 10.1.1 A Simple Method: Backward Euler 265 10.1.2 BDF and General Multistep Methods 268 10.1.3 Radau Collocation and Implicit Runge-Kutta Methods 270 10.1.4 Practical Di culties 276 10.1.5 Specialized Runge-Kutta Methods for Hessenberg Index2 DAEs 280 10.2 Methods for ODEs on Manifolds 282 10.2.1 Stabilization of the Discrete Dynamical System 283 10.2.2 Choosing the Stabilization Matrix F 287 10.3 Software, Notes and References 290 10.3.1 Notes 290 10.3.2 Software 292 10.4 Exercises 293 Bibliography Index 300 307 Bibliography 1] V.I Arnold Mathematical Methods of Classical Mechanics SpringerVerlag, 1978 2] U Ascher Stabilization of invariants of discretized di erential systems Numerical Algorithms, 14:1{24, 1997 3] U Ascher, H Chin, L Petzold, and S Reich Stabilization of constrained mechanical systems with DAEs and invariant manifolds J Mech Struct Machines, 23:135{158, 1995 4] U Ascher, H Chin, and S Reich Stabilization of DAEs and invariant manifolds Numer Math., 67:131{149, 1994 5] U Ascher, J Christiansen, and R Russell Collocation software for boundary value ODE's ACM Trans Math Software, 7:209{222, 1981 6] U Ascher and P Lin Sequential regularization methods for nonlinear higher index DAEs SIAM J Scient Comput., 18:160{181, 1997 7] U Ascher, P Markowich, P Pietra, and C Schmeiser A phase plane analysis of transonic solutions for the hydrodynamic semiconductor model Mathematical Models and Methods in Applied Sciences, 1:347{ 376, 1991 8] U Ascher, R Mattheij, and R Russell Numerical Solution of Boundary Value Problems for Ordinary Di erential Equations SIAM, second edition, 1995 9] U Ascher and L Petzold Projected implicit Runge-Kutta methods for di erential-algebraic equations SIAM J Numer Anal., 28:1097{1120, 1991 10] U Ascher, S Ruuth, and B Wetton Implicit-explicit methods for timedependent PDE's SIAM J Numer Anal., 32:797{823, 1995 11] U Ascher and R Spiteri Collocation software for boundary value di erential-algebraic equations SIAM J Scient Comp., 15:938{952, 1994 301 302 12] U.M Ascher and S Reich The midpoint scheme and variants for Hamiltonian systems: advantages and pitfalls 1997 Manuscript 13] G Bader and U Ascher A new basis implementation for a mixed order boundary value ODE solver SIAM J Scient Comput., 8:483{500, 1987 14] R Barrett, M Beary, T Chan, J Demmel, J Donald, J Dongarra, V Eijkhout, R Pozo, C Romaine, and H Van der Vorst Templates for the Solution of Linear Systems SIAM, 1996 15] J Baumgarte Stabilization of constraints and integrals of motion in dynamical systems Comp Methods Appl Mech., 1:1{16, 1972 16] C Bischof, A Carle, G Corliss, A Griewank, and P Hovland adifor - generating derivative codes from fortran programs Scienti c Programming, 1:11{29, 1992 17] G Bock Recent advances in parameter identi cation techniques for ODE In P Deu hard and E Hairer, editors, Numerical treatment of inverse problems, Boston, 1983 Birkhauser 18] R.W Brankin, I Gladwell, and L.F Shampine Rksuite: A suite of Runge-Kutta codes for the initial value problem for ODEs Report 92-s1, Dept Mathematics, SMU, Dallas, Texas, 1992 19] K Brenan, S Campbell, and L Petzold Numerical Solution of InitialValue Problems in Di erential-Algebraic Equations SIAM, second edition, 1996 20] F Brezzi and M Fortin Mixed and Hybrid Finite Element Methods Springer-Verlag, New York, 1991 21] P N Brown, G D Byrne, and A C Hindmarsh vode, a variablecoe cient ODE solver SIAM J Sci Stat Comput., 10:1038{1051, 1989 22] A Bryson and Y.C Ho Applied Optimal Control Ginn and Co., Waltham, MA, 1969 23] K Burrage Parallel and Sequential Methods for Ordinary Di erential Equations Oxford University Press, 1995 24] K Burrage and J.C Butcher Stability criteria for implicit Runge-Kutta methods SIAM J Numer Anal., 16:46{57, 1979 25] K Burrage, J.C Butcher, and F Chipman An implementation of singly-implicit Runge-Kutta methods BIT, 20:452{465, 1980 Bibliography Bibliography 26] J C Butcher The Numerical Analysis of Ordinary Di erential Equations Wiley, 1987 27] M.P Calvo, A Iserles, and A Zanna Numerical solution of isospectral ows Technical report, DAMTP, Cambridge, 1995 28] S L Campbell Least squares completions of nonlinear di erentialalgebraic equations Numer Math., 65:77{94, 1993 29] S L Campbell Numerical methods for unstructured higher-index DAEs Annals of Numer Math., 1:265{278, 1994 30] J.R Cash and M.H Wright User's guide for twpbvp: a code for solving two-point boundary value problems Technical report, on line in netlib, 1996 31] H Chin Stabilization methods for simulations of constrained multibody dynamics PhD thesis, Institute of Applied Mathematics, University of British Columbia, 1995 32] W.A Coppel Dichotomies in stability theory Springer-Verlag, 1978 Lecture Notes in Math Vol 629 33] G Dahlquist A special stability problem for linear multistep methods BIT, 3:27{43, 1963 34] G Dahlquist Error analysis for a class of methods for sti nonlinear initial value problems In Numerical Analysis, Dundee, pages 50{74 Springer, 1975 35] C de Boor Good approximation by splines with variable knots ii In Springer Lecture Notes in Mathematics, 353, 1973 36] C de Boor and B Swartz Collocation at Gaussian points SIAM J Numer Anal., 10:582{606, 1973 37] L Dieci, R.D Russell, and E.S Van Vleck Unitary integrators and applications to continuous orthonormalization techniques SIAM J, Numer Anal., 31:261{281, 1994 38] E Doedel and J Kernevez Software for continuation problems in ordinary di erential equations SIAM J Numer Anal., 25:91{111, 1988 39] J.R Dormand and P.J Prince A family of embedded Runge-Kutta formulae J Comp Appl Math., 6:19{26, 1980 303 304 40] E Fehlberg Low order classical Runge-Kutta formulas with step size control and their application to some heat transfer problems Computing, 6:61{71, 1970 41] L Fox The Numerical Solution of Two-Point Boundary Value Problems in Ordinary Di erential Equations Oxford University Press, 1957 42] C W Gear The simultaneous numerical solution of di erentialalgebraic equations IEEE Trans Circuit Theory, CT-18:89{95, 1971 43] C W Gear Numerical Initial Value Problems in Ordinary Di erential Equations Prentice-Hall, 1973 44] C W Gear and J B Keiper The analysis of generalized BDF methods applied to Hessenberg form DAEs SIAM J Numer Anal., 28:833{858, 1991 45] C.W Gear, G Gupta, and B Leimkuhler Automatic integration of the Euler-Lagrange equations with constraints J Comput Appl Math., pages 77{90, 1985 46] E Griepentrog and R Marz Di erential-Algebraic Equations and Their Numerical Treatment Teubner, 1986 47] J Guckenheimer and P Holmes Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields Springer-Verlag, New York, 1983 48] W Hackbusch Iterative Solution of Large Sparse Systems of Equations Springer-Verlag, 1994 49] E Hairer, Ch Lubich, and M Roche The Numerical Solution of Di erential-Algebraic Systems by Runge-Kutta Methods, volume 1409 Springer-Verlag, 1989 50] E Hairer, S.P Norsett, and G Wanner Solving Ordinary Di erential Equations I: Nonsti Problems Springer-Verlag, second edition, 1993 51] E Hairer and D Sto er Reversible long term integration with variable step sizes SIAM J Scient Comput., 18:257{269, 1997 52] E Hairer and G Wanner Solving Ordinary Di erential Equations II: Sti and Di erential-Algebraic Problems Springer-Verlag, 1991 53] N.A Haskell The dispersion of surface waves in multilayered media Bull Seis Soc Am., 43:17{34, 1953 Bibliography Bibliography 54] P Henrici Discrete Variable Methods in Ordinary Di erential Equations John Wiley, 1962 55] M Hirsch, C Pugh, and M Shub Invariant manifolds, volume 583 Springer-Verlag, 1976 56] T.E Hull, W.H Enright, and K.R Jackson User's guide for dverk { a subroutine for solving non-sti ODEs Report 100, Dept Computer Science, U Toronto, 1975 57] A Jameson Computational transonics Comm Pure Appl Math., XLI:507{549, 1988 58] W Kampowsky, P Rentrop, and W Schmidt Classi cation and numerical simulation of electric circuits Surv Math Ind., 2:23{65, 1992 59] H B Keller Numerical Solution of Two Point Boundary Value Problems SIAM, 1976 60] B.L.N Kennett Seismic wave propagation in strati ed media Cambridge University Press, 1983 61] W Kutta Beitrag zur naherungsweisen integration totaler di erentialgleichungen Zeitschr fur Math u Phys., 46:435{453, 1901 62] J D Lambert Numerical Methods for Ordinary Di erential Systems Wiley, 1991 63] M Lentini and V Pereyra An adaptive nite di erence solver for nonlinear two-point boundary value problems with mild boundary layers SIAM J Numer Anal., 14:91{111, 1977 64] Ch Lubich, U Nowak, U Pohle, and Ch Engstler mexx { numerical software for the integration of constrained mechanical multibody systems Preprint sc 92-12, ZIB Berlin, 1992 65] Jerry B Marion and Stephen T Thornton Classical Dynamics of Particles and Systems Harcourt Brace Jovanovich, third edition, 1988 66] R Marz Numerical methods for di erential-algebraic equations Acta Numerica, 1:141{198, 1992 67] R.M.M Mattheij and J Molnaar Ordinary Di erential Equations in Theory and Practice Wiley, Chichester, 1996 68] R.M.M Mattheij and G.W.M Staarink Implementing multiple shooting for nonlinear BVPs Rana 87-14, EUT, 1987 305 306 69] C.C Pantelides The consistent initialization of di erential-algebraic systems SIAM J Scient Comput., 9:213{231, 1988 70] V Pereyra and G Sewell Mesh selection for discrete solution of boundary value problems in ordinary di erential equations Numer Math., 23:261{268, 1975 71] L.R Petzold, L.O Jay, and J Yen Numerical solution of highly oscillatory ordinary di erential equations Acta Numerica, pages 437{484, 1997 72] F Potra and W Rheinboldt On the numerical solution of the EulerLagrange equations Mech Structures Mach., 1, 1991 73] A Prothero and A Robinson On the stability and accuracy of onestep methods for solving sti systems of ordinary di erential equations Math Comp., 28:145{162, 1974 74] P Rabier and W Rheinboldt On the computation of impasse points of quasilinear di erential algebraic equations Math Comp., 62:133{154, 1994 75] P J Rabier and W C Rheinboldt A general existence and uniqueness theorem for implicit di erential algebraic equations Di Int Eqns., 4:563{582, 1991 76] P J Rabier and W C Rheinboldt A geometric treatment of implicit di erential-algebraic equations J Di Eqns., 109:110{146, 1994 77] M Rao Ordinary Di erential Equations Theory and Applications Edward Arnold, 1980 78] S Reddy and N Trefethen Stability of the method of lines Numer Math., 62:235{267, 1992 79] W.C Rheinboldt Di erential-algebraic systems as di erential equations on manifolds Math Comp., 43:473{482, 1984 80] H Rubin and P Ungar Motion under a strong constraining force Comm Pure Appl Math., 10:65{87, 1957 81] C Runge Ueber die numerische au osung von di erentialgleichungen Math Ann., 46:167{178, 1895 82] J.M Sanz-Serna and M.P Calvo Numerical Hamiltonian Problems Chapman and Hall, 1994 Bibliography Index 83] T Schlick, M Mandziuk, R.D Skeel, and K Srinivas Nonlinear resonance artifacts in molecular dynamics simulations 1997 Manuscript 84] M.R Scott and H.A Watts Computational solution of linear two-point boundary value problems SIAM J Numer Anal., 14:40{70, 1977 85] L F Shampine Numerical Solution of Ordinary Di erential Equations Chapman & Hall, 1994 86] L F Shampine and M K Gordon Computer Solution of Ordinary Di erential Equations W H Freeman and Co., 1975 87] L.F Shampine and H.A Watts The art of writing a Runge-Kutta code, part i In J.R Rice, editor, Mathematical Software III, pages 257{275 Academic Press, 1977 88] I Stakgold Green's functions and boundary value problems Wiley, 1979 89] H Stetter Analysis of Discretization Methods for Ordinary Di erential Equations Springer, 1973 90] G Strang and G Fix An Analysis of the Finite Element Method Prentice-Hall, Englewood Cli s, NJ, 1973 91] J.C Strikwerda Finite Di erence Schemes and Partial Di erential Equations Wadsworth & Brooks/Cole, 1989 92] S.H Strogatz Nonlinear dynamics and chaos Addison-Wesley, Reading, MA, 1994 93] A.M Stuart and A.R Humphries Dynamical systems and numerical analysis Cambridge University Press, Cambridge, England, 1996 94] J.H Verner Explicit Runge-Kutta methods with estimates of the local truncation error SIAM J Numer Anal., 15:772{790, 1978 95] R.A Wehage and E.J Haug Generalized coordinate partitioning for dimension reduction in analysis of constrained dynamic systems J of Mechanical Design, 104:247{255, 1982 96] R Weiss The convergence of shooting methods BIT, 13:470{475, 1973 97] S.J Wright Stable parallel algorithms for two-point boundary value problems SIAM J Scient Comput., 13:742{764, 1992 307 Index Imsl, 66 Mathematica, 65 Matlab, 65, 99, 113 Nag, 66, 187 Netlib, 66, 112, 187 Boundary conditions Dirichlet, 227 non-separate, 174 periodic, 163, 174, 192, 223 separated, 163, 204 two-point, 163 Boundary layer, see Layer Boundary value problems (BVP), continuation, 211 damped Newton method, 210 decoupling, 186, 220 deferred correction, 208 error estimation, 213 extrapolation, 208 nite di erence methods, 193{230 0-stability, 201 collocation, 206 consistency, 201 convergence, 201 solving the linear equations, 204 sti problems, 215 for PDEs, 206 in nite interval, 188 mesh selection, 213 midpoint method, 194 multiple shooting method, 183{186 Newton's method, 197 reduced superposition, 186 Riccati method, 186 simple shooting method, 177{182 software, 223 stabilized march method, 186 superposition, 186 trapezoid method, 223 BVP codes Absolute stability, 42{47 implicit Runge-Kutta methods, 104 plotting the region of, 89, 144 region of, 43 explicit Runge-Kutta methods, 89{91 multistep methods, 143{145 Accuracy, order of, 38 Adams methods, 128{131 0-stability, 143 absolute stability, 144 Adams-Bashforth (explicit) method, 129 Adams-Moulton (implicit) method, 129 Algebraic variables (DAE), 234 Almost block diagonal, 208 Arti cial di usion, 228 Asymptotic stability, see Stability, asymptotic Automatic di erentiation, 65, 292 Autonomous, 3, 33, 83 B-convergence, 110 Backward di erentiation formulae, see BDF Backward Euler method, 47{56, 130 DAE, 265{268 region of absolute stability, 50 solution of nonlinear system, 50 BDF methods, 131{132 0-stability, 143 DAE, 268{270 Bifurcation diagram, 212 308 Index auto, 223 colnew, 223 colsys, 223 mus, 187 pasvar, 223 suport, 187 twpbvp, 223 Chaos, 158 Characteristic polynomial, 26 Chemical reaction BVP example, 178, 213, 224 Collocation methods basic idea, 102 for BVPs, 206 Gauss formulae, 101 Lobatto formulae, 102 order for DAEs, 272 order of, 103 projected, for DAE, 281 Radau formulae, 101 relation to implicit Runge-Kutta, 102 Compact nite di erence methods, 226 Compacti cation multiple shooting method, 191 Condition number eigenvalue matrix, 46 iteration matrix (DAE), 278 orthogonal matrix, 56 Conservative system, 29 Consistency, 38 BVPs, nite di erence methods, 201 multistep methods, 137 Constraint manifold, 240 Constraints (DAE), hidden, 234 Continuation methods, 90, 211{213 arclength, 213 Continuous extension, 110 Contraction mapping, 51 Convection-di usion equation (PDE), 161 Convergence, 38 BDF methods for DAEs, 268 BVPs, nite di erence methods, 201 309 calculated rate, 79 multistep methods, 134 of order p, 39 Runge-Kutta methods, 83 Coordinate partitioning (DAE), 254 Corrector formula, 147 Crank-Nicolson method for PDEs, 69 DAE codes coldae, 293 daspk, 293 dassl, 293 mexx, 293 radau5, 293 Damped Newton method, 210 Decoupling, 172, 186, 222, 267 BVP, 220{221 Decoupling methods (BVP), 172 Deferred correction method, 208 Degrees of freedom (DAE), 233 Delay di erential equation, 111, 189 Dense output, 110 Diagonally implicit Runge-Kutta methods (DIRK), 106 Dichotomy, 170, 220 exponential, 170 Di erence equations, 137 Di erence operator, 38 Di erential variables (DAE), 234 Di erential-algebraic equations (DAE), 10, 231 index reduction and stabilization, 247 algebraic variables, 234 BDF methods, 268 consistent initial conditions, 233, 276 constraint stabilization, 253 convergence of BDF methods, 268 coordinate partitioning, 254 di erential geometric approach, 256 di erential variables, 234 direct discretization methods, 264 existence and uniqueness, 256 fully-implicit index-1, 263 310 Hessenberg form, 238, 257 Hessenberg index-2, 239 Hessenberg index-3, 240 hidden constraints, 234 higher-index, 234 index reduction, unstabilized, 249 index, de nition, 235 least squares methods, 291 multistep methods, 269 numerical methods, 263 ODE with constraints, 10 reformulation of higher-index DAEs, 248 regularization, 264 semi-explicit, 10, 234 semi-explicit index-1, 238 simple subsystems, 232 singular, 257, 264 stabilization of the constraint, 251 stabilized index-2 formulation, 260 state space formulation, 253 underlying ODE, 246, 254 Di erentiation automatic, 75 symbolic, 75 Discontinuity discretization across, 61 location of, 63 Dissipativity, 111 Divergence, 17, 30 Divided di erences, 126 Drift o the constraint (DAE), 251 Dry friction, 63 Dynamical system, 15 discrete, 112 Eigenvalue, 20 Eigenvector, 20 Error constant (multistep methods), 136 equidistribution, 215, 222 global, 39 local, 41 local truncation, 38 Index tolerance absolute and relative, 91 Error estimation BVPs, 213 embedded Runge-Kutta methods, 92 global error, 95 index-2 DAE, 279 multistep methods, 152 Runge-Kutta methods, 91 step doubling, 94 Euler method backward (implicit), 35 forward (explicit), 35 symplectic, 116 written as Runge-Kutta, 81 Event location, 63, 111 Explicit method, 37 ODE, Extraneous roots, 139 Extrapolation, 110, 208 Finite element method, 222 Fully-implicit index-1 DAEs, 263 Functional iteration, 50 multistep methods, 146 Fundamental solution, 26, 166 in shooting method, 178 Fundamental theorem, di erence methods, 39 Gauss collocation, 101, 104, 115, 119, 121, 207, 210, 213, 220, 223, 293 Gaussian points, 76, 104, 120 quadrature, 76 Global error, 39 estimates of, 95, 213 Gradient, 17, 29 Green's function, 168, 185 Half-explicit Runge-Kutta methods (DAE), 281 Hamiltonian, 29 Index Hamiltonian systems, 29, 111, 116, 123 invariants, 250 preservation of the invariant, 255 Hermite interpolation, 111 Hessenberg form (DAE), 238, 257 Higher index DAEs, 234 Homotopy path, 211 Hopf bifurcation, 67 Implicit method, 49 ODE, 10 Runge-Kutta methods, 101{109 implementation, 105, 109 Implicit Euler method, see Backward Euler method Implicit-explicit (IMEX) methods, 161 Incompressible Navier-Stokes equations, 239 Index, 232{247 de nition, 235 di erential, 257 perturbation, 257 reduction stabilized index-2 formulation, 260 unstabilized, 249 Initial conditions, consistent (DAE), 233 Initial layer, see Layer Initial value problem (IVP), Instability DAE, drift o the constraint, 251 Interpolating polynomial and divided differences review, 126 Invariant integral, 120, 251 ODE with, 120 Invariant set, 15, 32, 247, 249, 253, 282 Isolated solution (BVP), 165 Isolated solution (IVP), 159 Isospectral ow, 121 Iteration matrix, 53 Jacobian matrix, 7, 17 di erence approximation, 54 311 Kepler problem, modi ed, 123 Kronecker product review, 105 Krylov space methods, 156 Lagrange multiplier, 10 DAEs and constrained optimization, 240 Layer boundary, 195, 207, 215, 218{220, 224 initial, 47, 58, 60, 229 Leapfrog (Verlet) method, 116, 256 Limit cycle, 5, 67 Limit set, 15 Linearization, local, 28 Lipschitz constant, continuity, 6, 40 Lobatto collocation, 102, 104, 121, 207 Local elimination, 208 Local error, 41 control of, in Runge-Kutta methods, 91 estimation by step doubling, 94 relationship to local truncation error, 41 Local extrapolation, 94 Local truncation error, 38, 64 BVPs, nite di erence methods, 201 estimation of (multistep methods), 153 multistep methods, 134 principal term (multistep methods), 154 relation to local error, 64 Long time integration, 111 Lyapunov function, 32 Matrix banded, 56 sparse, 56 Matrix decompositions LU , 55 QR, 56 review, 54 Matrix eigenvalues review, 19 Matrix exponential 312 review, 24 Mechanical systems, 11, 240 generalized coordinate partitioning method, 257 reformulation of higher-index DAEs, 248 Mesh, 35 locally almost uniform, 229 Mesh function, 38 Mesh Reynolds number, 227 Mesh selection (BVP), 213 Method of lines, 6, 12, 64, 161, 212, 280 heat equation stability restriction, 69 transverse, 13 Midpoint method, 68, 194 dynamic equivalence to trapezoid method, 68 explicit, 78 explicit, written as Runge-Kutta, 82 staggered, 225 Milne's estimate local truncation error (predictor-corrector methods), 153 Milne's method (multistep method), 141 Mode solution, 27, 167 Model reduction, 99 Molecular dynamics, 116 Moving mesh method (PDEs), 255 Multiple shooting method, 183 compacti cation, 191 matrix, 185, 202 on parallel processors, 185 patching conditions, 183 Multiple time scales, 47 Multirate method, 112 Multistep codes daspk, 156 dassl, 156 difsub, 156 ode, 156 vode, 156 vodpk, 156 Multistep methods, 125 Index absolute stability, 143 Adams methods, 128 BDF, 131 characteristic polynomials, 137 consistency, 137 DAE, 269 error constant, 136 implementation, 146 initial values, 132 local truncation error, 134 order of accuracy, 134 predictor-corrector, 146 software design, 149 variable step-size formulae, 150 Newton iteration backward Euler method, 51 DAE, 268 di erence approximation, 54 implicit Runge-Kutta methods, 105 in shooting method, 178 Newton's method damped, 210 modi ed, 148 quasi-Newton, 180 review, 53 Newton-Kantorovich Theorem, 190 Nonautonomous ODE transformation to autonomous form, 83 ODE explicit, implicit, 10, 72 linear constant-coe cient system, 22 on a manifold, 248 with constraints, 234 with invariant, 247, 250 O -step points (multistep methods), 155 One-step methods, 73 Optimal control, 13 adjoint variables, 14 Hamiltonian function, 14 Order notation review, 36 Index Order of accuracy multistep methods, 134 Runge-Kutta methods, 83, 86, 88, 103 Runge-Kutta methods for DAEs, 291 Order reduction, 108{109 DIRK, for DAEs, 273 in BVPs, 220 Runge-Kutta methods (DAE), 271 Order selection (multistep methods), 154 Order stars, 110 Oscillator, harmonic, 30, 63 Oscillatory system, 65, 242, 255 Parallel method Runge-Kutta, 112 Parallel shooting method, 185 Parameter condensation, 208 Parameter estimation, 15 Parasitic roots, 139 Partial di erential equation (PDE), 12, 206, 223 Path following, 212 Pendulum, sti spring, 242 Perturbations initial data, 21 inhomogeneity, 27 Preconditioning, 156 Predator-prey model, Predictor polynomial, 152 Predictor-corrector methods, 146 Principal error function, 95 Principal root, 139 Projected collocation methods, 281 Projected Runge-Kutta methods, 281 Projection matrix, orthogonal, 170 Quadrature rules review, 75 Quasilinearization, 197{200 with midpoint method for BVPs, 200 Radau collocation, 101, 104, 119, 274, 292 Reduced solution, 57, 243 Reduced superposition, 186 313 Reformulation, boundary value problems, 172 Regularization (DAE), 264 Review Basic quadrature rules, 75 Kronecker product, 105 Matrix decompositions, 54 Matrix eigenvalues, 19 Matrix exponential, 24 Newton's method, 52 Order notation, 36 Taylor's theorem for a function of several variables, 73 The interpolating polynomial and divided di erences, 126 Riccati method, 172, 186 Root condition, 140 Rough problems, 61 Runge-Kutta codes dopri5, 112 dverk, 112 ode45 (Matlab), 112 radau5, 113 rkf45, 112 rksuite, 112 stride, 113 Runge-Kutta methods absolute stability, 89, 104 Butcher tree theory, 110 DAE, 270{282 diagonally implicit (DIRK), 106 Dormand & Prince 4(5) embedded pair, 94 embedded methods, 92 explicit, 81 Fehlberg 4(5) embedded pair, 93 fourth order classical, 79, 82, 86 general formulation, 81 half-explicit, for DAEs, 281 historical development, 110 implicit, 101 low order, 76 mono-implicit, 223 314 Index order barriers, 110 order of accuracy by Butcher trees, 84 order results for DAEs, 291 projected, for DAEs, 281 singly diagonally implicit (SDIRK), 107 singly implicit (SIRK), 109 Semi-explicit index-1 DAE, 238 Sensitivity analysis, 96, 292 boundary value problems, 176 parameters, 96 Shooting method, 177 algorithm description, 179 di culties, 180 di culties for nonlinear problems, 182 multiple shooting method, 183 simple shooting, 177 single shooting, 177 stability considerations, 180 Similarity transformation, 20, 24 Simple pendulum, 3, 10, 120, 241 Singly diagonally implicit (SDIRK) RungeKutta methods, 107 Singly implicit Runge-Kutta methods (SIRK), 109 Singular perturbation problems relation to DAEs, 243 Smoothing, 231 Sparse linear system, 199, 204 Spectral methods (PDEs), 161 Spurious solution, 112 Stability 0-stability, 39, 42, 83, 139{143, 201{203 A-stability, 56, 104, 144 absolute stability, 42 algebraic stability, 119 AN-stability, 68 asymptotic di erence equations, 139 asymptotic, of the constraint manifold, 251 boundary value ODE, 168{171 di erence equations, 138 initial value DAE, 245{247 initial value ODE, 19{33 asymptotic, 21, 25, 27 nonlinear, 28 relative stability, 143 resonance instability, 117 root condition (multistep methods), 140 scaled stability region, 114 strong stability (multistep methods), 141 weak stability (multistep methods), 141 Stability constant boundary value problem, 169, 185, 203 initial value problem, 27 Stabilization Baumgarte, 253, 260 coordinate projection (DAE), 283, 286 of the constraint (DAE), 251 post-stabilization (DAE), 283 Stabilized index-2 formulation (DAE), 260 Stabilized march method, 186 Stage order, 273 State space formulation (DAE), 253 Steady state, 28, 212 Step size, 35 Step size selection multistep methods, 154 Runge-Kutta methods, 91 Sti boundary value problems nite di erence methods, 215 Sti decay, 57, 65, 102, 104, 114, 131 DAE, 272 ODE methods for DAEs, 264 Sti y accurate, 102, 114 Sti ness, 47 boundary value problems, 171 de nition, 48 system eigenvalues, 48 transient, 47 Strange attractor, 158 Superposition method, 186 Switching function, 63 Symmetric methods, 58{61 Index Symmetric Runge-Kutta methods, 119 Symplectic map, 30 Symplectic methods, 111 Taylor series method, 74 Taylor's theorem, several variables review, 73 Test equation, 21, 42 Theta method, 115 Transformation decoupling (BVP), 217 decoupling (DAE), 267 Transformation, stretching, 70 Trapezoid method, 35, 130 derivation, 58 dynamic equivalence to midpoint method, 68 explicit, 78 explicit, written as Runge-Kutta, 82 Upstream di erence, 216, 227 Upwind di erence, 216, 227 Variable step size multistep methods xed leading-coe cient strategy, 152 variable-coe cient strategy, 150 Variational boundary value problem, 165 equation, 28, 178 Vibrating spring, 9, 25 Waveform relaxation, 112 Well-posed problem, continuous dependence on the data, existence, uniqueness, 315 ... relevant for computer methods, Chapter introduces all the basic concepts and simple methods (relevant also for boundary value problems and for DAEs), Chapter is devoted to one-step (Runge-Kutta) methods. .. between ODEs and DAEs, and the questions regarding reformulation of DAEs, is essential and already suggests a lot regarding computer approaches Chapter 10 discusses numerical methods for DAEs Various... denote domains and operators Also, Re and I m denote the real and imaginary parts of a complex scalar, and R is the set of real numbers Chapter 1: Ordinary Di erential Equations For a vector function