NUmerical methods for chemical engineering application in matlab

This book has been developed with these needs in mind.This text first presents a fundamental discussion of linear algebra, to provide the necessaryfoundation to read the applied mathemati

Numerical Methods for Chemical Engineering

Suitable for a first-year graduate course, this textbook unites the applications of numericalmathematics and scientific computing to the practice of chemical engineering Written in

a pedagogic style, the book describes basic linear and nonlinear algebraic systems all theway through to stochastic methods, Bayesian statistics, and parameter estimation Thesesubjects are developed at a nominal level of theoretical mathematics suitable for graduateengineers The implementation of numerical methods in M® is integrated withineach chapter and numerous examples in chemical engineering are provided, together with alibrary of corresponding M programs Although the applications focus on chemicalengineering, the treatment of the topics should also be of interest to non-chemical engineersand other applied scientists that work with scientific computing This book will provide thegraduate student with the essential tools required by industry and research alike

Supplementary material includes solutions to homework problems set in the text,

M programs and tutorial, lecture slides, and complicated derivations for the moreadvanced reader These are available online at www.cambridge.org/9780521859714

K J B has been Assistant Professor at MIT since 2000 He has taught sively across the engineering discipline at both the undergraduate and graduate level Thisbook is a result of the successful course the author devised at MIT for numerical methodsapplied to chemical engineering

exten-Numerical Methods for Chemical Engineering

K E N N E T H J B E E R S

Massachusetts Institute of Technology

Cambridge University Press

The Edinburgh Building, Cambridge cb2 2ru, UK

First published in print format

isbn-13 978-0-521-85971-4

isbn-13 978-0-511-25650-9

© K J Beers 2007

2006

Information on this title: www.cambridge.org/9780521859714

This publication is in copyright Subject to statutory exception and to the provision of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press.

isbn-10 0-511-25650-7

isbn-10 0-521-85971-9

Cambridge University Press has no responsibility for the persistence or accuracy of urls for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate.

Published in the United States of America by Cambridge University Press, New York www.cambridge.org

hardback

eBook (EBL) eBook (EBL) hardback

Existence and uniqueness of solutions to a nonlinear algebraic equation 61

Example Steady-state modeling of a condensation

v

Eigenvalues and the existence/uniqueness properties of linear

Applying Gershgorin’s theorem to study the convergence of iterative

Numerical calculation of eigenvalues and eigenvectors in MATLAB 123

Initial value problems of ordinary differential equations

Contents vii

Chemical reaction and diffusion in a spherical catalyst pellet 265

Modeling a tubular chemical reactor with dispersion; treating

Numerical issues for discretized PDEs with more than two

Brownian dynamics and stochastic differential equations

MATLAB summary 364

This text focuses on the application of quantitative analysis to the field of chemical neering Modern engineering practice is becoming increasingly more quantitative, as theuse of scientific computing becomes ever more closely integrated into the daily activities

engi-of all engineers It is no longer the domain engi-of a small community engi-of specialist practitioners.Whereas in the past, one had to hand-craft a program to solve a particular problem, carefullyhusbanding the limited memory and CPU cycles available, now we can very quickly solve farmore complex problems using powerful, widely-available software This has introduced theneed for research engineers and scientists to become computationally literate – to know thepossibilities that exist for applying computation to their problems, to understand the basicideas behind the most important algorithms so as to make wise choices when selecting andtuning them, and to have the foundational knowledge necessary to navigate independentlythrough the literature

This text meets this need, and is written at the level of a first-year graduate student

in chemical engineering, a consequence of its development for use at MIT for the course10.34, “Numerical methods applied to chemical engineering.” This course was added in

2001 to the graduate core curriculum to provide all first-year Masters and Ph.D studentswith an overview of quantitative methods to augment the existing core courses in transportphenomena, thermodynamics, and chemical reaction engineering Care has been taken todevelop any necessary material specific to chemical engineering, so this text will proveuseful to other engineering and scientific fields as well The reader is assumed to have takenthe traditional undergraduate classes in calculus and differential equations, and to havesome experience in computer programming, although not necessarily in A AB ®.Even a cursory search of the holdings of most university libraries shows there to be agreat number of texts with titles that are variations of “Advanced Engineering Mathematics”

or “Numerical Methods.” So why add yet another?

I find that there are two broad classes of texts in this area The first focuses on ducing numerical methods, applied to science and engineering, at the level of a junior

intro-or seniintro-or undergraduate elective course The scope is necessarily limited to rather simpletechniques and applications The second class is targeted to research-level workers, eitherhigher graduate-level applied mathematicians or computationally-focused researchers inscience and engineering These may be either advanced treatments of numerical methodsfor mathematicians, or detailed discussions of scientific computing as applied to a specificsubject such as fluid mechanics

ix

Neither of these classes of text is appropriate for teaching the fundamentals of scientificcomputing to beginning chemical engineering graduate students Examples should be typ-ical of those encountered in graduate-level chemical engineering research, and while thestudents should gain an understanding of the basis of each method and an appreciation ofits limitations, they do not need exhaustive theory-proof treatments of convergence, erroranalysis, etc It is a challenge for beginning students to identify how their own problemsmay be mapped into ones amenable to quantitative analysis; therefore, any appropriate textshould have an extensive library of worked examples, with code available to serve later astemplates Finally, the text should address the important topics of model development andparameter estimation This book has been developed with these needs in mind.

This text first presents a fundamental discussion of linear algebra, to provide the necessaryfoundation to read the applied mathematical literature and progress further on one’s own.Next, a broad array of simulation techniques is presented to solve problems involvingsystems of nonlinear algebraic equations, initial value problems of ordinary differentialand differential-algebraic (DAE) systems, optimizations, and boundary value problems ofordinary and partial differential equations A treatment of matrix eigenvalue analysis isincluded, as it is fundamental to analyzing these simulation techniques

Next follows a detailed discussion of probability theory, stochastic simulation, statistics,and parameter estimation As engineering becomes more focused upon the molecular level,stochastic simulation techniques gain in importance Particular attention is paid to Browniandynamics, stochastic calculus, and Monte Carlo simulation Statistics and parameter esti-mation are addressed from a Bayesian viewpoint, in which Monte Carlo simulation proves apowerful and general tool for making inferences and testing hypotheses from experimentaldata

In each of these areas, topically relevant examples are given, along with A ABwww atw r s c programs that serve the students as templates when later writingtheir own code An accompanying website includes a A AB tutorial, code listings ofall examples, and a supplemental material section containing further detailed proofs andoptional topics Of course, while significant effort has gone into testing and validating theseprograms, no guarantee is provided and the reader should use them with caution

The problems are graded by difficulty and length in each chapter Those of grade A aresimple and can be done easily by hand or with minimal programming Those of grade Brequire more programming but are still rather straightforward extensions or implementations

of the ideas discussed in the text Those of grade C either involve significant thinking beyondthe content presented in the text or programming effort at a level beyond that typical of theexamples and grade B problems

The subjects covered are broad in scope, leading to the considerable (though hopefullynot excessive) length of this text The focus is upon providing a fundamental understanding

of the underlying numerical algorithms without necessarily exhaustively treating all of theirdetails, variations, and complexities of use Mastery of the material in this text should enablefirst-year graduate students to perform original work in applying scientific computation totheir research, and to read the literature to progress independently to the use of moresophisticated techniques

Preface xi

Writing a book is a lengthy task, and one for which I have enjoyed much help andsupport Professor William Green of MIT, with whom I taught this course for one semester,generously shared his opinions of an early draft The teaching assistants who have worked

on the course have also been a great source of feedback and help in problem-development,

as have, of course, the students who have wrestled with intermediate drafts and my evolvingapproach to teaching the subject My Ph.D students Jungmee Kang, Kirill Titievskiy, ErikAllen, and Brian Stephenson have shown amazing forbearance and patience as the textbecame an additional, and sometimes demanding, member of the group Above all, I mustthank my family, and especially my supportive wife Jen, who have been tracking my progressand eagerly awaiting the completion of the book

1 Linear algebra

This chapter discusses the solution of sets of linear algebraic equations and defines basicvector/matrix operations The focus is upon elimination methods such as Gaussian elim-ination, and the related LU and Cholesky factorizations Following a discussion of thesemethods, the existence and uniqueness of solutions are considered Example applicationsinclude the modeling of a separation system and the solution of a fluid mechanics boundaryvalue problem The latter example introduces the need for sparse-matrix methods and thecomputational advantages of banded matrices Because linear algebraic systems have, underwell-defined conditions, a unique solution, they serve as fundamental building blocks inmore-complex algorithms Thus, linear systems are treated here at a high level of detail, asthey will be used often throughout the remainder of the text

Linear systems of algebraic equations

We wish to solve a system of N simultaneous linear algebraic equations for the N unknowns

x1, x2, , x N, that are expressed in the general form

a i j is the constant coefficient (assumed real) that multiplies the unknown x j in equation

i b i is the constant “right-hand-side” coefficient for equation i, also assumed real As a

particular example, consider the system

x1+ x2+ x3= 4

3x1+ x2+ 6x3= 2for which

1

It is common to write linear systems in matrix/vector form as

Row i of A contains the values a i 1 , a i 2 , , a i N that are the coefficients multiplying each

unknown x1, x2, , x N in equation i Column j contains the coefficients a 1 j , a 2 j , , a N j

that multiply x j in each equation i = 1, 2, , N Thus, we have the following associations,

coefficients multiplyingrows⇔ equations columns⇔ a specific unknown



InMATLABwe solve Ax = b with the single command,x = A\b For the example (1.2),

we compute the solution with the code

fundamen-Review of scalar, vector, and matrix operations 3

differential equations, etc Also, as we shall see, this solver fails for certain commonclasses of very large systems of equations, and we need to know enough about linearalgebra to diagnose such situations and to propose other methods that do work in suchinstances This chapter therefore contains not only an explanation of how the MATLAB

solver is implemented, but also a detailed, fundamental discussion of the properties of linearsystems

Our discussion is intended only to provide a foundation in linear algebra for the practice ofnumerical computing, and is continued in Chapter 3 with a discussion of matrix eigenvalueanalysis For a broader, more detailed, study of linear algebra, consult Strang (2003) orGolub & van Loan (1996)

Review of scalar, vector, and matrix operations

As we use vector notation in our discussion of linear systems, a basic review of the concepts

of vectors and matrices is necessary

Scalars, real and complex

Most often in basic mathematics, we work with scalars, i.e., single-valued numbers These

may be real, such as 3, 1.4, 5/7, 3.14159 , or they may be complex, 1+ 2i, 1/2 i, where

i =√−1 The set of all real scalars is denoted  The set of all complex scalars we call

a = Re{z} = real part of z

(1.8)

b = Im{z} = imaginary part of z The complex conjugate, ¯z = z∗, of z = a + ib is

Note that the product ¯zz is always real and nonnegative,

¯zz = (a − ib)(a + ib) = a2− iab + iab − i2

b2= a2+ b2≥ 0 (1.10)

so that we may define the real-valued, nonnegative modulus of z, |z|, as

Often, we write complex numbers in polar notation,

z = a + ib = |z|(cos θ + i sin θ) θ = tan−1(b /a) (1.12)

Using the important Euler formula, a proof of which is found in the supplemental material

found at the website that accompanies this book,

Vector notation and operations

We write a three-dimensional (3-D) vectorv (Figure 1.1) as

v is real if v1, v2, v3∈ ; we then say v ∈ 3 We can easily visualize this vector in

3-D space, defining the three coordinate basis vectors in the 1(x), 2(y), and 3(z) directions

as

e[1]=



100





010





001



to write v ∈ 3as

v = v1e[1]+ v2e[2]+ v3e[3] (1.17)

We extend this notation to defineN

, the set of N-dimensional real vectors,

where v j ∈  for j = 1, 2, , N By writing v in this manner, we define a column vector;

however,v can also be written as a row vector,

The difference between column and row vectors only becomes significant when we startcombining them in equations with matrices

Review of scalar, vector, and matrix operations 5

We write v ∈  Nas an expansion in coordinate basis vectors as

.0







We further add to the list of operations associated with the vectors v, w ∈  N the dot

(inner, scalar) product,

For example, for the two vectors

v =



123





456

of the angle between the two vectors,

k=1

v2

If v · w = 0, v and w are said to be orthogonal, the extension of the adjective

“perpendic-ular” from3toN If v · w = 0 and |v| = |w| = 1, i.e., both vectors are normalized to

unit length,v and w are said to be orthonormal.

The formula for the length|v| of a vector v ∈  Nsatisfies the more general properties

of a norm v of v ∈  N A norm v is a rule that assigns a real scalar, v ∈ , to each

vector v ∈  N

such that for everyv, w ∈  N

, and for every c∈ , we have

Review of scalar, vector, and matrix operations 7

Table 1.1 p-norm values for the 3-D

The length of a vector is thus also the 2-norm For v = [1 −2 3], the values of the p-norm,

computed from (1.35), are presented in Table 1.1

v p = [|1|p+ | −2|p+ |3|p

]1/p = [(1)p+ (2)p+ (3)p

We define the infinity norm as the limit of v p as p → ∞, which merely extracts from v

the largest magnitude of any component,

v ∞≡ lim

p→∞ v p= maxj ∈[1,N] {|v j|} (1.36)For v = [1 −2 3], v ∞= 3

Like scalars, vectors can be complex We define the set of complex N-dimensional vectors

as C N, and write each component of v ∈ C N

For complex vectors v, w ∈ C N

, to form the dot product v · w, we take the complex

conjugates of the first vector’s components,

to have a unique solution, there must be as many equations as unknowns, and so typically

A will have an equal number N of columns and rows and thus be a square matrix A matrix

is said to be of dimension M × N if it has M rows and N columns We now consider some

simple matrix operations

Review of scalar, vector, and matrix operations 9

A v is also an N-dimensional vector, whose j th component is

From the rule for forming Av, we see that the number of columns of A must equal the

dimension ofv; however, we also can define Av when M = N,

The transpose operation is essentially a mirror reflection across the principal diagonal

a11, a22, a33, Consider the following examples:

to be an N × 1 matrix if in column form and to be a 1 × N matrix if in

row form Thus, for v, w ∈  N, expressing vectors by default as column vectors, we writethe dot product as

Elimination methods for solving linear systems

With these basic definitions in hand, we now begin to consider the solution of the linear

system Ax = b, in which x, b ∈  N

and A is an N × N real matrix We consider here

elimination methods in which we convert the linear system into an equivalent one that is

easier to solve These methods are straightforward to implement and work generally forany linear system that has a unique solution; however, they can be quite costly (perhapsprohibitively so) for large systems Later, we consider iterative methods that are moreeffective for certain classes of large systems

Elimination methods for solving linear systems 11

Elementary row operations

We first note that we can select any two equations, say j and k, and add them to obtain

another one that is equally valid,

(a j 1 + a k1 )x1+ (a j 2 + a k2 )x2+ · · · + (a j N + a k N )x N = (b j + b k) (1.57)

with no effect upon the solution x Similarly, we can take any equation, say j, multiply it by

a nonzero scalar c, to obtain

we can choose any j ∈ [1, N], k ∈ [1, N] and any scalar c = 0 to form the following

equivalent system that has exactly the same solution(s):

As we must change both A and b, it is common to perform row operations on the augmented

Elimination methods for solving linear systems 13

We now build a systematic approach to solving Ax = b based upon a sequence of

ele-mentary row operations, known as Gaussian elimination First, we start with the original

We use the notation ( A , b)(2,1)for the augmented matrix obtained after placing a zero at

the (2,1) position through the operation 2← 2 − λ21× 1,

Again, it is important to note that the linear system A(2,1) x = b(2,1) has the same solution(s)

x as the original system Ax = b.

As we develop this method, let us consider the example (1.2),

We continue this process and place zeros in all elements in the first column below the

diagonal, considering next the element in row 3 If a11= 0,

λ31 =a31

a11

(1.75)and the row operation 3← 3 − λ31× 1 yields the new (3, 1) element

Elimination methods for solving linear systems 15

The augmented matrix after this operation is

In general, for N > 3, we continue this procedure until all elements of the first column are

zero except a11 The augmented system matrix after this sequence of row operations is

We now move to the second column and perform row operations to place zeros everywhere

below the diagonal (2, 2) position If a(222,1)= 0, we define

For N > 3, we perform another N − 3 row operations to place all zeros below the principal

diagonal in the second column, yielding

For our example, the final augmented matrix is

Elimination methods for solving linear systems 17

Solving triangular systems by substitution

The equations defined by A (N ,N−1) x = b (N ,N−1)(1.88) are



For the example, x3= −8/1 = −8.

We then work backwards, next solving for x N−1,

until we obtain the values of all unknowns through this procedure of backward substitution.

For our example, we have

While we have found a solution, it remains to be established that this is the only solution.

The following algorithm transforms a linear system to upper triangular form by Gaussian

elimination:

allocate N2memory locations to store A , and N for b

for i = 1, 2, , N − 1; iterate over columns from left to right

if a ii = 0, STOP to avoid division by zero

for j = i + 1, i + 2, , N; iterate over rows below diagonal

that comprises two floating point operations (FLOPs), one a scalar multiplication and

the other a scalar addition The term “floating point operation” originates from the binarysystem for representing real numbers in digital memory that is described in the supplementalmaterial found at the accompanying website It is common to describe the computationaleffort required by an algorithm in terms of the number of FLOPs that it performs duringexecution

The algorithm for Gaussian elimination contains three nested loops that each run overall or part of the set of indices{1, 2, , N} Therefore, the total number of FLOPs is

proportional to N3(the exact number, 2N3/3 is computed in the supplemental material).

If we increase N by a factor of 10, the amount of work required increases by a factor of

103= 1000 We see here the major problem with Gaussian elimination; it can become verycostly for large systems!

Backward substitution follows the algorithm:

allocate N memory locations for the components of x

for i = N, N − 1, , 1; iterate over rows from bottom to top

As there are only two nested loops, the number of FLOPs scales only as N2 Therefore, for

large systems, it takes far (N times) longer to transform the system into triangular form by elimination (2N3/3 FLOPs) than it does to solve the triangular system by substitution (N2FLOPs)

Elimination methods for solving linear systems 19

The solution is now computed very easily, x j = f j /d j j This approach, Gauss–Jordan

elimination, is rarely used in computational practice.

We avoid such divisions by zero through the technique of partial pivoting Before ning any row operations, let us examine the first column of A,

rows 1 and j to form the equivalent system

By contrast, if all of the elements of the first column are zero, as they must be if ¯a11=

a j 1= 0, there can be no unique solution We thus stop the elimination procedure at thispoint

In Gaussian elimination with partial pivoting, when moving to column i, we first examine all elements in this column at or below the diagonal, and select the row j ≥ i with the largest

magnitude element,

|a j i | ≥ |a ki| ∀ k = i, i + 1, , N (1.106)

If j = i, we swap rows j and i, and thus, unless all elements at or below the diagonal

in column i are zero, |a j i | = 0 We then can compute the scalars λ i +1,i, , λ N ,i withouthaving to divide by zero If |a j i | is zero, then all of the elements a i ,i , a i +1,i, , a N ,imust be

zero

To see what happens in this case, consider the following system, obtained after placingzeros successfully below the diagonal in the first three columns When preparing to eliminatethe values in the fourth column, we find that all values at or below the diagonal are already

Elimination methods for solving linear systems 21

without changing the value of Ax Obviously, there can be no unique solution of Ax = b,

and we thus stop the elimination procedure at this point

If a unique solution exists, Gaussian elimination with partial pivoting will find it, even

if there are zeros along the principal diagonal of the original augmented matrix It is calledpartial pivoting because we only swap rows; if we swap columns as well (which necessi-tates more complex book-keeping, but results in better propagation of round-off error), we

perform Gaussian elimination with full pivoting.

The algorithm for Gaussian elimination with partial pivoting is

for i = 1, 2, , N − 1; iterate over columns

select row j ≥ i such that |a j i | = max j ≥i {|a ii |, |a i +1, i |, , |a N 1|}

if a j i = 0, no unique solution exists, STOP

if j = i, interchange rows i and j of augmented matrix

for j = i + 1, i + 2, , N; iterate over rows j > i

Backward substitution proceeds in the same manner as before Even if rows must beswapped for each column, the computational overhead of partial pivoting is relativelysmall.

To demonstrate Gaussian elimination with partial pivoting, consider the system of tions (1.70) with the augmented matrix

First, we examine the first column to see that the element of largest magnitude is in row 3

We thus swap rows 1 and 3,



1 

3−2 3



6 

7−2 3



3 

1−1 3



1 

1−1 3



6 

4−1 3

Existence and uniqueness of solutions 23

in the third row We thus swap rows 2 and 3,

 

23

23



Finally, from the first equation,

The solution to (1.70) is thus (x1, x2, x3)= (19, −7, −8).

Existence and uniqueness of solutions

With Gaussian elimination and partial pivoting, we have a method for solving linear systemsthat either finds a solution or fails under conditions in which no unique solution exists Inthis section, we consider at more depth the question of when a linear system possesses a real

solution (existence) and if so, whether there is exactly one (uniqueness) These questions are

vitally important, for linear algebra is the basis upon which we build algorithms for solvingnonlinear equations, ordinary and partial differential equations, and many other tasks

As a first step, we consider the equation Ax = b from a somewhat more abstract viewpoint.

We note that A is an N × N real matrix and x and b are both N-dimensional real vectors.

Figure 1.2 Interpretation of a real N × N matrix A as a linear transformation from the domain  N

into the codomainN

We have introduced the notation N

for the set of all real N-dimensional vectors The term

Also, for any u , v, w ∈  N

, any c1, c2∈ , a null vector 0 ∈ N

, and for every v ∈  N

defining an additive inverse, −v ∈  N, we have the identities

Using the concept of the vector spaceN , we now interpret the N × N real matrix A

in a new fashion We note that for any v ∈  N

, the matrix-vector product with A is also in

N , Av ∈  N This product is formed by the rule

Also, since for any v, w ∈  N

, c∈ , we have the linearity properties

we say that A is a linear transformation mappingN into itself, A :N → N The action

of A upon vectors v, w ∈  N is sketched in Figure 1.2 From this interpretation of A as

Existence and uniqueness of solutions 25

Figure 1.3 Interpreting matrix multiplication C = AB as sequence of linear transformations.

a linear transformation, we can understand better the existence and uniqueness properties

of Ax = b We see that a solution exists if there is some vector x ∈  N that is mapped

into b∈ N by A, and that this solution is unique if there exists no other y = x that is also mapped into b by A.

Multiplication of matrices

Before continuing with our discussion of existence and uniqueness, let us see how the pretation of a matrix as a linear transformation immediately dictates a rule for multiplying

inter-two matrices Let us say that we have an M × P real matrix A that maps every v ∈  P

into some Av ∈  M In addition, we also have some P × N real matrix B that maps every

w ∈ N into some Bw ∈ P We therefore make the association v = Bw and define a

composite mapping fromNintoM , C, such that for w ∈ N , we obtain Cw ∈ M by

first multiplying w by B and then by A (Figure 1.3) That is,

Comparing this to the product of an M × N matrix C and w ∈  N yields

To obtain c i j , we sum the products of the elements of A in row i with those of B in column

j The matrix product AB is defined only if the number of columns of A equals the number

of rows of B.

We also note that, in general, matrix multiplication is not commutative,

From this rule for matrix multiplication, we obtain the following relation for the transpose

of a product of two equal-sized square matrices,

( A B)T= BT

Vector spaces and basis sets

We must discuss one more topic before considering the existence and uniqueness of solutions

to linear systems: the use of basis sets The set of vectors{b[1], b[2], , b [P] } in  Nis said

to be linearly independent if there exists no set of real scalars { c1, c2, , c P } except that

of all zeros, c j = 0, j = 1, 2, , P, such that

c1b[1]+ c2b[2]+ · · · + c P b [P]= 0 (1.134)

In other words, if a set is linearly independent, no member of the set can be expressed as

a linear combination of the others For vectors in N, a set{b[1], b[2], , b [P] } can be

linearly independent only if P ≤ N.

If P = N, we say that the set of N linearly independent vectors {b[1], b[2], , b [N ] }

forms a basis set forN

Then any vectorv ∈  N

may be expressed as a linear combination

v = c1b[1]+ c2b[2]+ · · · + c N b [N ] c j ∈  (1.135)

A common question is