scientific computing

SCIENTIFIC COMPUTING An Introductory Survey Michael T. Heath University of Illinois at Urbana-Champaign ii Copyright c 1997 by The McGraw-Hill Companies. All rights reserved. About the Author Michael T. Heath holds four positions at the University of Illinois at Urbana-Champaign: Professor in the Department of Computer Science, Director of the Computational Science and Engineering Program, Director of the Center for Simulation of Advanced Rockets, and Senior Research Scientist at the National Center for Supercomputing Applications (NCSA). He received a B.A. in Mathematics from the University of Kentucky, an M.S. in Mathematics from the University of Tennessee, and a Ph.D. in Computer Science from Stanford University. Before joining the University of Illinois in 1991, he spent a number of years at Oak Ridge National Laboratory, first as Eugene P. Wigner Postdoctoral Fellow and later as Computer Science Group Leader in the Mathematical Sciences Research Section. His research interests are in numerical analysis—particularly numerical linear algebra and optimization—and in parallel computing. He has has been an editor of the SIAM Journal on Scientific Computing, SIAM Review, and the International Journal of High Performance Computing Applications, as well as several conference proceedings. In 2000, he was named an ACM Fellow. iii iv To Mona Contents Preface xiii Notation xvii 1 Scientific Computing 1 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 General Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Approximations in Scientific Computation . . . . . . . . . . . . . . . . . . . 2 1.2.1 Sources of Approximation . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.2 Data Error and Computational Error . . . . . . . . . . . . . . . . . 3 1.2.3 Truncation Error and Rounding Error . . . . . . . . . . . . . . . . . 4 1.2.4 Absolute Error and Relative Error . . . . . . . . . . . . . . . . . . . 5 1.2.5 Sensitivity and Conditioning . . . . . . . . . . . . . . . . . . . . . . 5 1.2.6 Backward Error Analysis . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2.7 Stability and Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.3 Computer Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.3.1 Floating-Point Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.3.2 Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.3.3 Properties of Floating-Point Systems . . . . . . . . . . . . . . . . . . 10 1.3.4 Rounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.3.5 Machine Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.3.6 Subnormals and Gradual Underflow . . . . . . . . . . . . . . . . . . 13 1.3.7 Exceptional Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.3.8 Floating-Point Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . 14 1.3.9 Cancellation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.4 Mathematical Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 1.4.1 Mathematical Software Libraries . . . . . . . . . . . . . . . . . . . . 21 1.4.2 Scientific Computing Environments . . . . . . . . . . . . . . . . . . . 22 1.4.3 Practical Advice on Software . . . . . . . . . . . . . . . . . . . . . . 23 v vi CONTENTS 1.5 Historical Notes and Further Reading . . . . . . . . . . . . . . . . . . . . . 25 2 Systems of Linear Equations 37 2.1 Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 2.1.1 Singularity and Nonsingularity . . . . . . . . . . . . . . . . . . . . . 37 2.2 Solving Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 2.2.1 Triangular Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . 40 2.2.2 Elementary Elimination Matrices . . . . . . . . . . . . . . . . . . . . 41 2.2.3 Gaussian Elimination and LU Factorization . . . . . . . . . . . . . . 42 2.2.4 Pivoting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 2.2.5 Implementation of Gaussian Elimination . . . . . . . . . . . . . . . . 49 2.2.6 Complexity of Solving Linear Systems . . . . . . . . . . . . . . . . . 50 2.2.7 Gauss-Jordan Elimination . . . . . . . . . . . . . . . . . . . . . . . . 51 2.2.8 Solving Modified Problems . . . . . . . . . . . . . . . . . . . . . . . 52 2.3 Norms and Condition Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 54 2.3.1 Vector Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 2.3.2 Matrix Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.3.3 Condition Number of a Matrix . . . . . . . . . . . . . . . . . . . . . 57 2.4 Accuracy of Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 2.4.1 Residual of a Solution . . . . . . . . . . . . . . . . . . . . . . . . . . 58 2.4.2 Estimating Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 2.4.3 Improving Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 2.5 Special Types of Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . 63 2.5.1 Symmetric Positive Definite Systems . . . . . . . . . . . . . . . . . . 63 2.5.2 Symmetric Indefinite Systems . . . . . . . . . . . . . . . . . . . . . . 65 2.5.3 Band Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 2.6 Iterative Methods for Linear Systems . . . . . . . . . . . . . . . . . . . . . . 67 2.7 Software for Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 2.7.1 LINPACK and LAPACK . . . . . . . . . . . . . . . . . . . . . . . . 69 2.7.2 Basic Linear Algebra Subprograms . . . . . . . . . . . . . . . . . . . 69 2.8 Historical Notes and Further Reading . . . . . . . . . . . . . . . . . . . . . 70 3 Linear Least Squares 83 3.1 Data Fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 3.2 Linear Least Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 3.3 Normal Equations Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 3.3.1 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 3.3.2 Normal Equations Method . . . . . . . . . . . . . . . . . . . . . . . 87 3.3.3 Augmented System Method . . . . . . . . . . . . . . . . . . . . . . . 89 3.4 Orthogonalization Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 3.4.1 Triangular Least Squares Problems . . . . . . . . . . . . . . . . . . . 90 3.4.2 Orthogonal Transformations . . . . . . . . . . . . . . . . . . . . . . . 90 3.4.3 QR Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 3.4.4 Householder Transformations . . . . . . . . . . . . . . . . . . . . . . 91 3.4.5 Givens Rotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 CONTENTS vii 3.4.6 Gram-Schmidt Orthogonalization . . . . . . . . . . . . . . . . . . . . 98 3.4.7 Rank Deficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 3.4.8 Column Pivoting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 3.5 Comparison of Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 3.6 Software for Linear Least Squares . . . . . . . . . . . . . . . . . . . . . . . . 103 3.7 Historical Notes and Further Reading . . . . . . . . . . . . . . . . . . . . . 105 4 Eigenvalues and Singular Values 115 4.1 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . . . . . . . 115 4.1.1 Nonuniqueness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 4.1.2 Characteristic Polynomial . . . . . . . . . . . . . . . . . . . . . . . . 116 4.1.3 Properties of Eigenvalue Problems . . . . . . . . . . . . . . . . . . . 117 4.1.4 Similarity Transformations . . . . . . . . . . . . . . . . . . . . . . . 118 4.1.5 Conditioning of Eigenvalue Problems . . . . . . . . . . . . . . . . . . 120 4.2 Methods for Computing All Eigenvalues . . . . . . . . . . . . . . . . . . . . 121 4.2.1 Characteristic Polynomial . . . . . . . . . . . . . . . . . . . . . . . . 121 4.2.2 Jacobi Method for Symmetric Matrices . . . . . . . . . . . . . . . . 122 4.2.3 QR Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 4.2.4 Preliminary Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . 125 4.3 Methods for Computing Selected Eigenvalues . . . . . . . . . . . . . . . . . 126 4.3.1 Power Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 4.3.2 Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 4.3.3 Geometric Interpretation . . . . . . . . . . . . . . . . . . . . . . . . 128 4.3.4 Shifts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 4.3.5 Deflation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 4.3.6 Inverse Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 4.3.7 Rayleigh Quotient . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 4.3.8 Rayleigh Quotient Iteration . . . . . . . . . . . . . . . . . . . . . . . 131 4.3.9 Lanczos Method for Symmetric Matrices . . . . . . . . . . . . . . . . 132 4.3.10 Spectrum-Slicing Methods for Symmetric Matrices . . . . . . . . . . 133 4.4 Generalized Eigenvalue Problems . . . . . . . . . . . . . . . . . . . . . . . . 135 4.5 Singular Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 4.5.1 Singular Value Decomposition . . . . . . . . . . . . . . . . . . . . . . 136 4.5.2 Applications of SVD . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 4.6 Software for Eigenvalues and Singular Values . . . . . . . . . . . . . . . . . 138 4.7 Historical Notes and Further Reading . . . . . . . . . . . . . . . . . . . . . 140 5 Nonlinear Equations 151 5.1 Nonlinear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 5.1.1 Solutions of Nonlinear Equations . . . . . . . . . . . . . . . . . . . . 152 5.1.2 Convergence Rates of Iterative Methods . . . . . . . . . . . . . . . . 153 5.2 Nonlinear Equations in One Dimension . . . . . . . . . . . . . . . . . . . . . 154 5.2.1 Bisection Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 5.2.2 Fixed-Point Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 5.2.3 Newton’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 viii CONTENTS 5.2.4 Secant Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 5.2.5 Inverse Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 5.2.6 Linear Fractional Interpolation . . . . . . . . . . . . . . . . . . . . . 163 5.2.7 Safeguarded Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 164 5.2.8 Zeros of Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 5.3 Systems of Nonlinear Equations . . . . . . . . . . . . . . . . . . . . . . . . . 165 5.3.1 Fixed-Point Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 5.3.2 Newton’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 5.3.3 Secant Updating Methods . . . . . . . . . . . . . . . . . . . . . . . . 169 5.3.4 Broyden’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 5.3.5 Robust Newton-Like Methods . . . . . . . . . . . . . . . . . . . . . . 171 5.4 Software for Nonlinear Equations . . . . . . . . . . . . . . . . . . . . . . . . 171 5.5 Historical Notes and Further Reading . . . . . . . . . . . . . . . . . . . . . 173 6 Optimization 183 6.1 Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 6.1.1 Local versus Global Optimization . . . . . . . . . . . . . . . . . . . . 184 6.1.2 Relationship to Nonlinear Equations . . . . . . . . . . . . . . . . . . 185 6.1.3 Accuracy of Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . 186 6.2 One-Dimensional Optimization . . . . . . . . . . . . . . . . . . . . . . . . . 186 6.2.1 Golden Section Search . . . . . . . . . . . . . . . . . . . . . . . . . . 186 6.2.2 Successive Parabolic Interpolation . . . . . . . . . . . . . . . . . . . 188 6.2.3 Newton’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 6.2.4 Safeguarded Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 191 6.3 Multidimensional Unconstrained Optimization . . . . . . . . . . . . . . . . 191 6.3.1 Direct Search Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 191 6.3.2 Steepest Descent Method . . . . . . . . . . . . . . . . . . . . . . . . 191 6.3.3 Newton’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 6.3.4 Quasi-Newton Methods . . . . . . . . . . . . . . . . . . . . . . . . . 195 6.3.5 Secant Updating Methods . . . . . . . . . . . . . . . . . . . . . . . . 196 6.3.6 Conjugate Gradient Method . . . . . . . . . . . . . . . . . . . . . . . 197 6.3.7 Truncated Newton Methods . . . . . . . . . . . . . . . . . . . . . . . 199 6.4 Nonlinear Least Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 6.4.1 Gauss-Newton Method . . . . . . . . . . . . . . . . . . . . . . . . . . 200 6.4.2 Levenberg-Marquardt Method . . . . . . . . . . . . . . . . . . . . . 201 6.5 Constrained Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 6.5.1 Linear Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 6.6 Software for Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 6.7 Historical Notes and Further Reading . . . . . . . . . . . . . . . . . . . . . 208 7 Interpolation 219 7.1 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 7.1.1 Purposes for Interpolation . . . . . . . . . . . . . . . . . . . . . . . . 219 7.1.2 Interpolation versus Approximation . . . . . . . . . . . . . . . . . . 220 7.1.3 Choice of Interpolating Function . . . . . . . . . . . . . . . . . . . . 220 CONTENTS ix 7.1.4 Basis Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 7.2 Polynomial Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 7.2.1 Evaluating Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . 224 7.2.2 Lagrange Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . 224 7.2.3 Newton Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . 225 7.2.4 Orthogonal Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . 229 7.2.5 Interpolating a Function . . . . . . . . . . . . . . . . . . . . . . . . . 230 7.2.6 High-Degree Polynomial Interpolation . . . . . . . . . . . . . . . . . 231 7.2.7 Placement of Interpolation Points . . . . . . . . . . . . . . . . . . . 231 7.3 Piecewise Polynomial Interpolation . . . . . . . . . . . . . . . . . . . . . . . 232 7.3.1 Hermite Cubic Interpolation . . . . . . . . . . . . . . . . . . . . . . 233 7.3.2 Cubic Spline Interpolation . . . . . . . . . . . . . . . . . . . . . . . . 233 7.3.3 Hermite Cubic versus Cubic Spline Interpolation . . . . . . . . . . . 234 7.3.4 B-splines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 7.4 Software for Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 7.4.1 Software for Special Functions . . . . . . . . . . . . . . . . . . . . . 239 7.5 Historical Notes and Further Reading . . . . . . . . . . . . . . . . . . . . . 239 8 Numerical Integration and Differentiation 245 8.1 Numerical Quadrature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 8.1.1 Quadrature Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 8.2 Newton-Cotes Quadrature . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 8.2.1 Newton-Cotes Quadrature Rules . . . . . . . . . . . . . . . . . . . . 246 8.2.2 Method of Undetermined Coefficients . . . . . . . . . . . . . . . . . 247 8.2.3 Error Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 8.2.4 Polynomial Degree . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 8.3 Gaussian Quadrature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 8.3.1 Gaussian Quadrature Rules . . . . . . . . . . . . . . . . . . . . . . . 251 8.3.2 Change of Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 8.3.3 Gauss-Kronrod Quadrature Rules . . . . . . . . . . . . . . . . . . . 254 8.4 Composite and Adaptive Quadrature . . . . . . . . . . . . . . . . . . . . . . 255 8.4.1 Composite Quadrature Rules . . . . . . . . . . . . . . . . . . . . . . 255 8.4.2 Automatic and Adaptive Quadrature . . . . . . . . . . . . . . . . . . 256 8.5 Other Integration Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 8.5.1 Integrating Tabular Data . . . . . . . . . . . . . . . . . . . . . . . . 257 8.5.2 Infinite Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 8.5.3 Double Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 8.5.4 Multiple Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258 8.6 Integral Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 8.7 Numerical Differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 8.7.1 Finite Difference Approximations . . . . . . . . . . . . . . . . . . . . 262 8.7.2 Automatic Differentiation . . . . . . . . . . . . . . . . . . . . . . . . 263 8.8 Richardson Extrapolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 8.9 Software for Numerical Integration and Differentiation . . . . . . . . . . . . 266 8.10 Historical Notes and Further Reading . . . . . . . . . . . . . . . . . . . . . 267 x CONTENTS 9 Initial Value Problems for ODEs 275 9.1 Ordinary Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . 275 9.1.1 Initial Value Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 276 9.1.2 Higher-Order ODEs . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 9.1.3 Stable and Unstable ODEs . . . . . . . . . . . . . . . . . . . . . . . 277 9.2 Numerical Solution of ODEs . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 9.2.1 Euler’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 9.3 Accuracy and Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282 9.3.1 Order of Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282 9.3.2 Stability of a Numerical Method . . . . . . . . . . . . . . . . . . . . 284 9.3.3 Stepsize Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 9.4 Implicit Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286 9.5 Stiff Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 9.6 Survey of Numerical Methods for ODEs . . . . . . . . . . . . . . . . . . . . 290 9.6.1 Taylor Series Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 290 9.6.2 Runge-Kutta Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 291 9.6.3 Extrapolation Methods . . . . . . . . . . . . . . . . . . . . . . . . . 293 9.6.4 Multistep Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 9.6.5 Multivalue Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 9.7 Software for ODE Initial Value Problems . . . . . . . . . . . . . . . . . . . 299 9.8 Historical Notes and Further Reading . . . . . . . . . . . . . . . . . . . . . 300 10 Boundary Value Problems for ODEs 309 10.1 Boundary Value Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 10.2 Shooting Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310 10.3 Superposition Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312 10.4 Finite Difference Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312 10.5 Finite Element Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314 10.6 Eigenvalue Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 10.7 Software for ODE Boundary Value Problems . . . . . . . . . . . . . . . . . 319 10.8 Historical Notes and Further Reading . . . . . . . . . . . . . . . . . . . . . 319 11 Partial Differential Equations 325 11.1 Partial Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 325 11.1.1 Classification of Partial Differential Equations . . . . . . . . . . . . . 325 11.2 Time-Dependent Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326 11.2.1 Semidiscrete Methods Using Finite Differences . . . . . . . . . . . . 327 11.2.2 Semidiscrete Methods Using Finite Elements . . . . . . . . . . . . . 328 11.2.3 Fully Discrete Methods . . . . . . . . . . . . . . . . . . . . . . . . . 329 11.2.4 Implicit Finite Difference Methods . . . . . . . . . . . . . . . . . . . 332 11.2.5 Hyperbolic versus Parabolic Problems . . . . . . . . . . . . . . . . . 333 11.3 Time-Independent Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 11.3.1 Finite Difference Methods . . . . . . . . . . . . . . . . . . . . . . . . 335 11.3.2 Finite Element Methods . . . . . . . . . . . . . . . . . . . . . . . . . 337 11.4 Direct Methods for Sparse Linear Systems . . . . . . . . . . . . . . . . . . . 337 [...]... Consider the problem of computing values of the cosine function for arguments near π/2 Let x ≈ π/2 and let h be a small perturbation to x Then the error in computing cos(x + h) is given by Absolute error = cos(x + h) − cos(x) ≈ −h sin(x) ≈ −h, and hence Relative error ≈ −h tan(x) ≈ ∞ Thus, small changes in x near π/2 cause large relative changes in cos(x) regardless of the method for computing it For example,... values of the exponent field Whether Inf and NaN are supported at the user level in a given computing environment depends on the language, compiler, and run-time system If available, these quantities can be helpful in designing software that deals gracefully with exceptional situations rather than 14 CHAPTER 1 SCIENTIFIC COMPUTING abruptly aborting the program In MATLAB (see Section 1.4.2), for example, if... properties of good algorithms, such as efficiency, numerical algorithms should also be as reliable and accurate as possible despite the various approximations made along the way 1 2 1.1.1 CHAPTER 1 SCIENTIFIC COMPUTING General Strategy In seeking a solution to a given computational problem, a basic general strategy, which occurs throughout this book, is to replace a difficult problem with an easier one... others are due to subsequent computational processes Although this distinction is not always clearcut (rounding, for example, may affect both the input data and subsequent computational 4 CHAPTER 1 SCIENTIFIC COMPUTING results), it is nevertheless helpful in understanding the overall effects of approximations in numerical computations A typical problem can be viewed as the computation of the value of a... where x is a point near x A problem is sensitive, or ill-conditioned, if its condition number ˆ is much larger than 1 Anyone who has felt a shower go from freezing to scalding, or vice 6 CHAPTER 1 SCIENTIFIC COMPUTING versa, at the slightest touch of the temperature control has had first-hand experience with a sensitive system Example 1.2 Evaluating a Function Consider the propagated data error when a... problems are generic, but others are directly related to specific applications in various scientific and engineering disciplines This book provides a fairly comprehensive introduction to scientific computing, but scientific computing is only part of what has become known as computational science Computational science is a relatively new mode of scientific investigation that includes several phases: 1 Development... the particular input value x = 1, we have, to seven decimal places, f (x) = 2.718282, ˆ f (x) = 2.666667, x = log(2.666667) = 0.980829, ˆ ˆ Forward error = f (x) − f (x) = −0.051615, 8 CHAPTER 1 SCIENTIFIC COMPUTING Backward error = x − x = −0.019171 ˆ The point here is not to compare the numerical values of the forward and backward errors quantitatively, but merely to illustrate the concepts involved... mainframes and supercomputers as well The IEEE standard was carefully crafted to eliminate the many anomalies and ambiguities in earlier vendor-specific floating-point implementations and has 10 CHAPTER 1 SCIENTIFIC COMPUTING greatly facilitated the development of portable and reliable numerical software It also allows for sensible and consistent handling of exceptional situations, such as division by zero 1.3.2... following results Number 1.649 1.650 1.651 1.699 Chop 1.6 1.6 1.6 1.6 Round to nearest 1.6 1.6 1.7 1.7 Number 1.749 1.750 1.751 1.799 Chop 1.7 1.7 1.7 1.7 Round to nearest 1.7 1.8 1.8 1.8 12 CHAPTER 1 SCIENTIFIC COMPUTING A potential source of additional error that is often overlooked is in the decimal-to-binary and binary-to-decimal conversions that usually take place upon input and output of floatingpoint... instruction Chapter 1 Scientific Computing 1.1 Introduction The subject of this book is traditionally called numerical analysis Numerical analysis is concerned with the design and analysis of algorithms for solving mathematical problems that arise in computational science and engineering For this reason, numerical analysis has more recently become known as scientific computing Numerical analysis is distinguished . and optimization—and in parallel computing. He has has been an editor of the SIAM Journal on Scientific Computing, SIAM Review, and the International Journal of High Performance Computing Applications,. SCIENTIFIC COMPUTING An Introductory Survey Michael T. Heath University of Illinois at Urbana-Champaign ii Copyright c 1997. engineering disciplines. This book provides a fairly comprehensive introduction to scientific computing, but scientific computing is only part of what has become known as computational science. Computational