Springer d t nguyen finite element methods parallel sparse statics and eigen solutions(ocr)

FINITE ELEMENT METHODS: Parallel-Sparse Statics and Eigen-Solutions FINITE ELEMENT METHODS: Parallel-Sparse Statics and Eigen-Solutions Duc Thai Nguyen Old Dominion University N o f i l k , Virginia Q - Springer Prof Duc Thai Nguyen 135 Kaufman Old Dominion University Department of Civil & Environmental Engineering Multidisc Parallel-Vector Comp Ctr Norfolk VA 23529 Finite Element Methods: Parallel-Sparse Statics and Eigen-Solutions Library of Congress Control Number: 2005937075 ISBN 0-387-29330-2 e-ISBN 0-387-30851-2 ISBN 978-0-387-29330-1 Printed on acid-free paper O 2006 Springer Science+Business Media, Inc All rights reserved This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, Inc., 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now know or hereafter developed is forbidden The use in this publication of trade names, trademarks, service marks and similar terms, even if the are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights Printed in the United States of America To Dac K Nguyen Thinh T Thai Hang N Nguyen Eric N D Nguyen and Don N Nguyen Contents A Review of Basic Finite Element Procedures I Introduction Numerical Techniques for Solving Ordinary Differential Equations (ODE) 1.3 Identifying the "Geometric" versus "Natural" Boundary Conditions 1.4 The Weak Formulations 1.5 Flowcharts for Statics Finite Element Analysis 1.6 Flowcharts for Dynamics Finite Element Analysis 13 1.7 Uncoupling the Dynamical Equilibrium Equations 14 17 1.8 One-Dimensional Rod Finite Element Procedures 1.8.1 One-Dimensional Rod Element Stiffness Matrix 18 1.8.2 Distributed Loads and Equivalent Joint Loads -21 1.8.3 Finite Element Assembly Procedures 22 1.8.4 Imposing the Boundary Conditions 24 1.8.5 Alternative Derivations of System of Equations from Finite Element Equations 25 Truss Finite Element Equations 27 1.9 1.10 Beam (or Frame) Finite Element Equations 29 1.11 Tetrahedral Finite Element Shape Functions .31 1.12 Finite Element Weak Formulations for General 2-D Field Equations 35 44 1.13 The Isoparametric Formulation 51 1.14 Gauss Quadrature 59 1.15 Summary 1.16 Exercises 59 1.1 1.2 Simple MPIfFORTRAN Applications 63 2.1 2.2 2.3 2.4 2.5 2.6 63 Introduction 63 Computing Value of "IT" by Integration Matrix-Matrix Multiplication 68 MPI Parallel 110 72 Unrolling Techniques 75 Parallel Dense Equation Solvers 77 77 2.6.1 Basic Symmetrical Equation Solver 2.6.2 Parallel Data Storage Scheme 78 2.6.3 Data Generating Subroutine 80 2.6.4 Parallel Choleski Factorization 80 2.6.5 A Blocked and Cache-Based Optimized Matrix-Matrix Multiplication 81 81 2.6.5.1 Loop Indexes and Temporary Array Usage 2.6.5.2 Blocking and Strip Mining 82 2.6.5.3 Unrolling of Loops 82 2.6.6 Parallel "Block" Factorization 83 2.6.7 "Block" Forward Elimination Subroutine 85 2.6.8 "Block" Backward Elimination Subroutine 86 2.6.9 "Block" Error Checking Subroutine + -88 2.6.10 Numerical Evaluation , ,, 91 2.6.11 ConcIusions #95 2.7 Devcloping/Debugging Parallcl MPI Application Code on Your Own Laptop -95 2.8 Summary 103 2.9 Excrciscs 103 ., Direct Sparse Equation Solvers 105 Introduction 105 Sparse Storage Schemes lU5 Three Basic Steps and Re-Ordering Algorithms .,110 Sym'bolic Factorization with Re-Ordering Column Numbers ,.118 Sparse Numerical Factorization -132 Super (Master) Nodes (Depms-of-Freedom) 134 Numerical Factorization with Unrolling Strategies .w a 137 ForwardlBackward Solutions with Unrolling Strategies 137 Alternative Approach for Handling an Indefinite Matrix 154 165 Unsymmetrical Matrix Equation Solver Summary 180 Exercises ,181 , Sparse Assembly Process ,183 Introduction 183 A Simple Finitc Element Model (Symmetrical Matrices) 183 Finite Element Sparse Assembly Algorithms for Symmetrical Matrices 188 Symbolic Sparse Assembly of Symmetrical Matrices , 189 Numerical Sparse Assembly of Symmetrical Matrices 192 Step-by-step Algorithms for Symmetrical Sparse Assembly 2W) A Simple Finitc Element Model (Unsymmetrical Matrices) 219 224 Re-Ordering Algorithms Imposing Dirichlet Boundary Conditions , ,,, , * a 229 Unsy rnmetrical Sparse Equations Data Formats , ,, 254 Symbolic Sparse Assembly of Unsymmetrical Matrices , 259 Numerical Sparse Assembly of Unsymmetrical Matrices , .260 Step-by-step Algorithms for Unsymmetrical Sparse Assembly and Unsymmetrical Sparse Equation Solver 260 A Numerical Example 265 Summary , * , * * +.265 Exercises 266 ix Generalized Eigen-Solvers 269 Introduction 269 269 A Simple Generalized Eigen-Example 271 Inverse and Forward Iteration Procedures 274 Shifted Eigen-Problems 276 Transformation Methods 286 Sub-space Iteration Method 290 Lanczns Eigen-Solution Algorithms 290 5.7.1 Derivation of Lanczos Algorithms 295 5.7.2 Lanczos Eigen-Solution Error Analysis 302 5.7.3 Sturm Sequence Check .306 5.7.4 Proving the Lanczos Vectors Are M-Orthogonal 308 5.7.5 "Classical" Gram-Schmidt Re-Orthogonalization 314 5.7.6 Detailed Step-by-step Lanczos Algorithms 316 5.7.7 Educational Software for Lanczos Algorithms 5.7.8 Efficient Software for Lanczos Eigen-Solver 336 339 Unsymmetrical Eigen-Solvers 5.8 5.9 Balanced Matrix 339 5.10 Reduction to Hessenberg Form 340 341 5.1 QR Factoruat~on 341 5.12 Householder QR Transformation 348 5.13 "Modified" Gram-Schmidt Re-Orthogonalization 350 5.14 QR Iteration for Unsymmetrical Eigen-Solutions 5.15 QR Iteration with Shifts for Unsymmetrical Eigen-Solutions 353 5.16 Panel Flutter Analysis 355 5.17 Block Lanczos Algorithms 365 366 5.17.1 Details of "Block Lanczos" Algorithms 5.17.2 A Numerical Example for "Block Lanczos" Algorithms .371 377 5.18 Summary 5.19 Exercises 378 5.1 5.2 5.3 5.4 5.5 5.6 5.7 Finite Element Domain Decomposition Procedures 379 Introduction 379 A Simple Numerical Example Using Domain Decomposition (DD) Procedures 382 Imposing Boundary Conditions on "Rectangular" Matrices K$! 390 How to Construct Sparse Assembly of "Rectangular" Matrix K$; .392 Mixed Direct-Iterative Solvers for Domain Decomposition 393 Preconditioned Matrix for PCG Algorithm with DD Formulation 397 Generalized Inverse 404 FETI Domain Decomposition F o r m n l a t i ~ n ' ~ ~~~." 409 Preconditioned Conjugate Projected Gradient (PCPG) of the Dual Interface h b l e m 16"' A Automated Procedures for Computing Generalized Inverse and Rigid Body Motions 422 Numerical Examples of a 2-D Truss by FETI Formuldion .433 A Preconditioning Technique for Indefinite Linear S stem I6.l2' 459 FETI-DP Domain Decomposition ~orrnulation 463 Multi-Level Sub-Domains and Multi-Frontal Solver Id.13 488 Iterative Solution with Successive Right-Hand Sides '6.23'6.24' 490 Summary .b a 510 Exercises '.'! Appcndix A Singular Value Decomposition (SVD) 515 References 521 Index , .527 Finite element methods (FEM) and associated computer software have been widely accepted as one of the most effective, general tools for solving large-scale, practical engineering and science applications It is no wonder there is a vast number of excellent textbooks in FEM (not including hundreds of journal articles related to FEM) written in the past decades! While existing FEM textbooks have thoroughly discussed different topics, such as lineartnonlinear, statictdynamic analysis, with varieties of 1-Dl2-Dt3-D finite element libraries, for thermal, electrical, contact, and electromagnetic applications, most (if not all) current FEM textbooks have mainly focused on the developments of "finite element libraries," how to incorporate boundary conditions, and some general discussions about the assembly process, solving systems of "banded" (or "skyline") linear equations For implicit finite element codes, it is a well-known fact that efficient equation and eigen-solvers play critical roles in solving large-scale, practical engineeringlscience problems Sparse matrix technologies have evolved and become mature enough that all popular, commercialized FEM codes have inserted sparse solvers into their software Furthermore, modern computer hardware usually has multiple processors; clusters of inexpensive personal computers (under WINDOWS, or LINUX environments) are available for parallel computing purposes to dramatically reduce the computational time required for solving large-scale problems Most (if not all) existing FEM textbooks discuss the assembly process and the equation solver based on the "variable banded" (or "skyline") strategies Furthermore, only limited numbers of FEM books have detailed discussions about Lanczos eigen-solvers or explanation about domain decomposition (DD) finite element formulation for parallel computing purposes This book has been written to address the concerns mentioned above and is intended to serve as a textbook for graduate engineering, computer science, and mathematics students A number of state-of-the-art FORTRAN software, however, have been developed and explained with great detail Special efforts have been made by the author to present the material in such a way to minimize the mathematical background requirements for typical graduate engineering students Thus, compromises between rigorous mathematics and simplicities are sometimes necessary The materials from this book have evolved over the past several years through the author's research work and graduate courses (CEE71.51815 = Finite Element I, CEE695 = Finite Element Parallel Computing, CEE7111811 = Finite Element 11) at Old Dominion University (ODU) In Chapter 1, a brief review of basic finite element xii procedures for LinearlStatics/Dynamics analysis is given One, two, and threedimensional finite element types are discussed The weak formulation is emphasized, Finite element general field equations are derived, isoparametric formulation is explained, and Gauss Quadrature formulas for efficient integration are discussed In this chapter, only simple (non-efficient) finite element assembly procedures are explained, Chapter illustrates some salient features offered by Message Passing Interface {MPT) FORTRAN environments Unrolling techniques, efficient usage of computer cache memory, and some basic hdHlFORTRAN applications in matrix linear algebra operations are also discussed in this chapter Different versions of direct, "SPARSE" equation solvers' slrategies are thoroughly discussed in Chapter The "truly sparse" finite element ''assembly process" is explained in Chapter Different versions of the Lanczos algorithms for the solution of generalized eigenequntions (in a sparse matrix envjrnnment) are derived in Chapter Finally, the ovcrall finite element domain decomposi~oncomputer implementation, which can exploit "diroct" sparse matrix equation, eigen-solvers, sparse assembly, "iterative" solvers (for both "symmetrical" and 4bnsymmetrical"systems of linear equations}, nnd parallel processing computation, me thoroughly explained and demonstrated in Chapter Attempts have been made by the author to explain some difficult conccpts/algorithms in simple language and through simple (hand-calculated) numerical examples Many FORTRAN codes (in the forms of main program, and sub-routines) are given in Chapters - Several large-scale, practical engineering problems involved with severd hundred thousand to over million degree-offreedoms (dof) have been used to demonstrate the efficiency of the algorithms discussed in this textbook This textbook should be useful for graduate students, practicing engineers, and researchers who wish to thoroughly understand the detailed step-by-step algorithms used during the finite element (truely sparse) assembly, the 'Uirect" and "iterative" sparse equation and eigen-solvers, and incorporating the DD formulation for efficient parallel computation The book can be used in any of the following "stand-alone" courses: (a) Chapter can be expanded (with more numerical examples) and portions of Chapter (wly cover thc sparse formats, and some "key components" of the sparse solver) can be used as a first (introductive type) course in finite element analysis at the senior undergraduate (or 1'' year graduate) level @) Chapters 1,3,4, and can be used as n "stand-alone" graduate course such as "Special Topics in FEM:Sparse Linear Statics and Eigen-Solutions." jc) Chapters 1, 2, 3, 4, and can be used as a "stand-alone" graduate course, such as "Special Topics in FEm: Parnllel Sparse Linear Statics Solutions." (d) Chapters 2, 3, and 5, and portions of Chapter 6, can be used as a "standalone" graduate course such as "High Performance Parallel Matrix Computation." Duc T Nguyen Example 4: (Relationship between SVD and generalized inverse) "Let the m x n matrix A of rank k have the SVD A = ~ Z ~ ~ ; w i ttoh2o2 >csk)0 ~ Then the generalized inverse A+ of A is the nxm matrix A+ = v z + u H ; w h e r e Z = Hence : [Ei and E is the kxk diagonal matrix, with References J N Reddy, An Introduction to the Finite Element Method, 2ndedition, McGraw-Hill (1993) K J Bathe, Finite Element Procedures, Prentice Hall (1996) K H Huebner, The Finite Element Method for Engineers, John Wiley & Sons (1975) T R Chandrupatla and A.D Belegundu, Introduction to Finite Elements in Engineering, Prentice-Hall (1991) D S Burnett, Finite Element Analysis: From Concepts to Applications, Addison-Wesley Publishing Company (1987) M A Crisfield, Nonlinear Finite Element Analysis of Solids and Structures, volume 2, John Wiley & Sons (2001) 0.C Zienkiewicz, The Finite Element Method, 31d edition, McGraw-Hill(1977) D R Owen and E Hinton, Finite Elements in Plasticity: Theory and Practice, Pineridge Press Limited, Swansea, UK (1980) D T Nguyen, Parallel-Vector Equation Solvers for Finite Element Engineering Applications, Kluwer/Plenum Publishers (2002) J Jin, The Finite Element Method in Electromagnetics, John Wiley & Sons (1 993) P P Sivester and R L Ferrari, Finite Elements for Electrical Engineers, 31d edition, Cambridge University Press (1996) R D Cook, Concepts and Applications of Finite Element Analysis, 2"d edition, John Wiley & Sons (1981) S Pissanetzky, Sparse Matrix Technology, Academic Press, Inc (1984) J A Adam, "The effect of surface curvature on wound healing in bone: 11 The critical size defect." Mathematical and Computer Modeling, 35 (2002), p 1085 - 1094 W Gropp, "Tutorial on MPI: The Message Passing Interface," Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL 60439 522 Finite Element Methods: Parallel-Sparse Statics and Eigen-Solutions SGI sparse solver library sub-routine, Scientific Computing Software Library (SCSL) User's Guide, document number 007-4325-001, published Dec 30,2003 I S Duff and J K Reid, "MA47, a FORTRAN Code for Direct Solution of Indefinite Sparse Symmetric Linear Systems," RAL (Report) #95-001, Rutherford Appleton Laboratory, Oxon, OX1 OQX (Jan 1995) G Karypis and V Kumar, "ParMETiS: Parallel Graph Partitioning and Sparse Matrix Ordering Library," University of Minnesota, CS Dept., Version 2.0 (1998) J W H Liu, "Reordering Sparse Matrices For Parallel Elimination," Technical Report #87-01, Computer Science, York University, North York, Ontario, Canada (1987) D T Nguyen, G Hou, B Han, and H Runesha, "Alternative Approach for Solving Indefinite Symmetrical System of Equation," Advances in Engineering Softwure, Vol 31 (2000), pp 581 - 584, Elsevier Science Ltd I S Duff, and G W Stewart (editors), Sparse Matrix Proceedings 1979, SlAM (1979) I S Duff, R G Grimes, and J G Lewis, "Sparse Matrix Test Problems," ACM Trans Math Software, 15, pp - 14 (1989) G H Golub and C F VanLoan, "Matrix Computations," Johns Hopkins University Press, Baltimore, MD, 2ndedition (1989) A George and J W Liu, Computer Solution of Large Sparse Positive Definite Systems, Prentice-Hall (1981) E Ng and B W Peyton, "Block Sparse Choleski Algorithm on Advanced Uniprocessor Computer," SIAM J of Sci Comput., volume 14, pp 1034 - 1056 (1993) H B Runesha and D T Nguyen, "Vectorized Sparse Unsymmetrical Equation Solver for Computational Mechanics," Advances in Engr Software, volume 31, nos - 9, pp 563 - 570 (Aug - Sept 2000), Elsevier J A George, "Nested Disection of a Regular Finite Element Mesh," SlAM J Numer Anal., volume 15, pp 1053 - 1069 (1978) I S Duff and J K Reid, "The Design of MA48: A Code for the Direct Solution of Sparse Unsymmetric Linear Systems of Equations," ACM Trans Math Software., 22 (2): 187 - 226 (June 1996) Duc T Nguyen I S Duff and J Reid, "MA27: A Set of FORTRAN Subroutines for Solving Sparse Symmetric Sets of Linear Equations," AERE Technical Report, R-10533, IIarwell, England (1982) Nguyen, D T., Bunting, C., Moeller, K J., Runesha H B., and Qin, J., "Subspace and Lanczos Sparse Eigen-Solvers for Finite Element Structural and Electromagnetic Applications," Advances in Engineering Software, volume 31, nos - 9, pages 599 - 606 (August - Sept 2000) Nguyen, D T and Arora, J S., "An Algorithm for Solution of Large Eigenvalue Problems," Computers & Structures, vol 24, no 4, pp 645 - 650, August 1986 Arora, J S and Nguyen, D T., "Eigen-solution for Large Structural Systems with Substructures," International Journal for Numerical Methods in Engineering, vol 15, 1980, pp 333 - 341 Qin, J and Nguyen, D T., "A Vector Out-of-Core Lanczons Eigensolver for Structural Vibration Problems," presented at the 35th Structures, Structural Dynamics, and Material Conference, Hilton Head, SC, (April 18 - 20,1994) K J Bathe, Finite Element Procedures, Prentice Hall (1996) G Golub, R Underwood, and J H Wilkinson, "The Lanczos Algorithm for Symmetric Ax=Lamda*Bx Problem," Tech Rep STAN-CS-72-720, Computer Science Dept., Stanford University (1972) B Nour-Omid, B N Parlett, and R L Taylor, "Lanczos versus Subspace Iteration for Solution of Eigenvalue Problems," IJNM in Engr., volume 19, pp 859 - 871 (1983) B N Parlett and D Scott, "The Lanczos Algorithm with Selective Orthogonalization,"Mathematics of Computation, volume 33, no 145, pp 217 - 238 (1979) H.D Simon, "The Lanczos Algorithm with Partial Reorthogonalization", Mathematics of Computation, 42, no 165, pp 115-142 (1984) J J Dongarra, C B Moler, J R Bunch, and G W Stewart, LINPACK Users' Guide, SIAM, Philadelphia (1979) S Rahmatalla and C C Swan, "Continuum Topology Optimization of Buckling-Sensitive Structures," AIAA Journal, volume 41, no 6, pp 1180 - 1189 (June 2003) 524 Finite Element Methods: Parallel-Sparse Statics and Eigen-Solutions W H Press, B P Flannery, S A Teukolsky, and W T Vetterling, Numerical Recipes (FORTRAN Version), Cambridge University Press (1989) M T Heath, Scientijc Computing: An Introductory Survey, McGraw-Hill(1997) Tuna Baklan, "CEE7111811: Topics in Finite Element Analysis," Homework #5, Old Dominion University, Civil & Env Engr Dept., Norfolk, VA (private communication) W R Watson, "Three-Dimensional Rectangular Duct Code with Application to Impedance Eduction," AIAA Journal, 40, pp 217-226 (2002) D T Nguyen, S Tungkahotara, W R Watson, and S D Rajan "Parallel Finite Element Domain Decomposition for Structural/ Acoustic Analysis," Journal of Computational and Applied Mechanics, volume 4, no 2, pp 189 - 201 (2003) C Farhat and F X Roux, "Implicit Parallel Processing in Structural Mechanics," Computational Mechanics Advances, volume 2, pp - 124 (1994) D T Nguyen and P Chen, "Automated Procedures for Obtaining Generalized Inverse for FETl Formulations," Structures Research Technical Note No 03-22-2004, Civil & Env Engr Dept., Old Dominion University, Norfolk, VA 23529 (2004) C Farhat, M Lesoinne, P LeTallec, K Pierson, and D Rixen, "FETI-DP: A Dual-Primal Unified FETI Method- Part I: A Faster Alternative to the Level FETI Method," IJNME, volume 50, pp 1523 - 1544 (2001) R Kanapady and K K Tamma, "A Scalability and SpacePTime Domain Decomposition for Structural Dynamics - Part I: Theoretical Developments and Parallel Formulations," Research Report UMSI 20021 188 (November 2002) X S Li and J W Dernrnel, "SuperLU-DIST: A Scalable DistributedMemory Sparse Direct Solver for Unsymmetric Linear Systems," ACM Trans Mathematical Software, volume 29, no 2, pp 110 - 140 (June 2003) A D Belegundu and T R Chandrupatla, Optimization Concepts and Applications in Engineering, Prentice-Hall (1999) Duc T Nguyen [6.10] D T Nguyen and P Chen, "Automated Procedures For Obtaining Generalized Inverse for FETI Formulation," Structures Technical Note # 03-22-2004, Civil & Env Engr Dept ODU, Norfolk, VA 23529 [6.11] M Papadrakakis, S Bitzarakis, and A Kotsopulos, "Parallel Solution Techniques in Computational Structural Mechanics," B H V Topping (Editor), Parallel and Distributed Processing for Computational Mechanics: Systems and Tools, pp 180 - 206, Saxe-Coburg Publication, Edinburgh, Scotland (1999) [6.12] L Kornzsik, P Poschmann, and I Sharapov, "A Preconditioning Technique for Indefinite Linear Systems," Finite Element in Analysis and Design, volume 26, pp 253-258 (1997) [6.13] P Chen, H Runesha, D T Nguyen, P Tong, and T Y P Chang, "Sparse Algorithms for Indefinite System of Linear Equations," pp 712 - 717, Advances in Computational Engineering Science, edited (1997) by S N Atluri and G Yagawa, Tech Science Press, Forsyth, Georgia [6.14] D T Nguyen, G Hou, H Runesha, and B Han, "Alternative Approach for Solving Sparse Indefinite Symmetrical System of Equations," Advances in Engineering S o f ~ r evolume , 31 (8 - 9), pp 581 - 584 (2000) [6.15] J Qin, D T Nguyen, T Y P Chang, and P Tong, "Efficient Sparse Equation Solver With Unrolling Strategies for Computational Mechanics", pp 676 - 681, Advances in Computational Engineering Science, edited (1997) by S N Atluri and G Yagawa, Tech Science Press, Forsyth, Georgia [6.16] A George and J W Liu, Computer Solution of Large Sparse Positive Definite Systems, Prentice-Hall (1981) [6.17] C Farhat, M Lesoinne, and K Pierson, "A Scalable Dual-Primal Domain Decomposition Method," Numerical Linear Algebra with Applications, volume 7, pp 687 - 714 (2000) [6.18] Nguyen, D T., "Multilevel Structural Sensitivity Analysis," Computers & Structures Journal, volume 25, no 2, pp 191 - 202, April 1987 526 Finite Element Methods: Parallel-Sparse Statics and Eigen-Solutions [6.19] S J Kim, C S Lee, J H Kim, M Joh, and S Lee, "ISAP: A High Performance Parallel Finite Element Code for Large-Scale Structural Analysis Based on Domain-wise Multifrontal Technique," proceedings of Super Computing, Phoenix, AZ (November 15 - 21,2003) [6.20] J H Kim, and S J Kim, "Multifrontal Solver Combined with Graph Patitioners," AlAA Journal, volume 37, no 8, pp 964 - 970 (Aug 1999) [6.21] I Duff and J Reid, "The Multifrontal Solution of Indefinite Sparse Symmetric Linear Systems," Association for Computing Machinery Transactions Mathematical Sofmare, volume 9, pp 302 - 325 (1983) [6.22] B M Iron, "A Frontal Solution Program for Finite Element Analysis," IJNME, volume 2, pp - 32 (1970) [6.23] F J Lingen, "A Generalized Conjugate Residual Method for the Solution of Non-Symmetric Systems of Equations with Multiple Right-Hand Sides," IJNM in Engr., volume 44, pp 641 - 656 (1999) [6.24] P F Fischer, "Projection Techniques for Iterative Solution of Ax = b with Successive Right-Hand Sides," ICASE Report # 93-90, NASA LaRC, Hampton, VA [6.25] S Tungkahotara, D T Nguyen, W R Watson, and H B Runesha, "Simple and Efficient Parallel Dense Equation Solvers," 9' International Conference on Numerical Methods and Computational Mechanics, University of Miskolc, Miskoic, Hungary (July 15 - 19, 2002) Index ABAQUS, 105 Absolute displacement, 163 Acoustic finite element model, 400 Adam, J.A., 521 Aerodynamic equations, 356 Aerodynamic influence, 356 Adjacency array, 115 Algebraic equations, Arora, J.S., 523 Assembled, 12 Assembly procedures, 22 Axial displacement, 18 Axial reaction, 24 Axially distributed load, 17 Axially loaded rod, 17 Balanced matrix, 339 Banded sparse matrix, 401 Basic matrices, 40 Bathe, K.J., 521 Beam deflection, Belegundu, A.D., 521 Bending moment, Berry, M.W., 515 Block column storage, 79 Block forward elimination, 85 Block Lanczos algorithms, 365, 366,368,371 Body force, 35 Boeing's sparse indefinite equation solver, 163 Boolean transformation, 414, 466 Boundary conditions, 1, Boundary displacement, 380 Boundary dof, 488 Boundary force, 35 Brick element, 33 Buckling analysis, 294 Burnett, D.S., 521 Bworne, M., 515 Cache, 80 Chandrupatla, T.R., 521 Chen, P., 524,525 Choleski factorization, 295, 369 Classical Gram-Schmidt, 308 Colloquation, Compact column storage, 107 Compact row storage, 105 Compatibility requirements, 465 Compilation, 101 Complete polynomial, 18 Conjugate direction, 497 Conjugate gradient method, 394 Conjugate Projected Gradient (CPG), 417 Conjugate vectors, 496 Connectivity information, 186 Continuity condition, Convergence, Cook, R.D., 521 Coordinate transformation, 11 Corner dof, 466,471 Corner nodes, 465 Corner point, 465 Cray-C90, 77 Crisfield, M.A., 521 Critical oscillation, 355 Cross-sectional area, 17 Curvature, Curve boundary, 44 Damping matrix, 356 Decompose (a matrix), 77 Deflection, Degree-of-freedom, 10 Demmel, J.W., 515 Dependent variable field, 37 Dependent variable, Derivatives, Determinant, 270 Determinant, 50 Diagonal matrix, 16 Diagonal terms, 162 Differential equation, DlPSS (MPI software), 401 Direct sparse equation solvers, 105 Dirichlet boundary conditions, 187,229 Discretized locations, Displacement compatibility, 27 528 Finite Element Methods: Parallel-Sparse Statics and Eigen-Solutions Distributed loads, 21 Domain decomposition (DD) 379, 382 Dongara, J.J., 523 DOT product operations, 77 Dual DD formulation, 464 Dual interface problem, 417 Duff, I., 489 Dynamic pressure, 355,356 Dynamical equilibrium equations, 13,14 Dynamics, 13 Effective boundary stiffness (load), 380, 381 Eigen-matrix, 270 Eigen-solution error analysis, 295 Eigen-values matrix, 277 Eigen-values, 14, 15 Eigen-vectors, 14, 15 Element connectivity matrix, 187 Element local coordinate, 11 Element mass matrix, 13 Element shape function, 31,33 Element stiffness matrix, 21, 26 Energy approach, 20 Equivalent joint loads, 21,26, 31 Error norms computation, 89 Essential boundary conditions, 79, 17 Euler-Bernoulli beam, 30 Extended GCR algorithm, 509 External virtual work, 10 Factorization, 110 Farhat, C., 524, 525 FETl domain decomposition (DD), 409 FETI-1 algorithms, 414 FETI-DP formulation, 463 FETI-DP step-by-step procedures, 472 Field equations, 35 Fill-in terms, 114 Finite element analysis, Finite element connectivity, 115 Finite element model (symmetrical matrices), 183 Finite element model (unsymmetrical matrices), 219 Finite element stiffness equations, 12 Finite elements, First sub-diagonal, 340 Fischer, P.F., 526 Floating sub-domains, 411, 456 Floating substructure, 427 Forcing function, Forth order differential equation, 30 FORTRAN-90,63 Forward substitution of blocks, 86 Forwardtbackward elimination, 78 Frame finite element, 29 Free vibration, 14 Galerkin, 1, 7, Gauss quadrature formulas, 56 Gauss quadrature, 51 Gaussian elimination, 340 Generalized Conjugate Residual (GCR) algorithms, 503 Generalized coordinates, 45 Generalized eigen-equation, 365 Generalized eigen-solvers, 269 Generalized eigen-value problem, 14 Generalized inverse, 404,427, 456 Generalized Jacobi method, 284 Geometric boundary conditions, Geometric stiffness matrix, 294 George, A., 525 Gippspool Stock Meiger, 115 Global coordinate reference, 12 Global coordinate references, 387 Global dof, 186 Gropp, W., 521 Heat, Heath, M.T., 524 Hessenberg (form) matrix, 340 Hessenberg reduction, 377 Hinton, E., 521 Homogeneous equation, 501 Homogeneous form, Duc T Nguyen Hooke's law, 10 Householder transformation, 341, 342,344 Huebner, K.H., 521 Identity matrix, 16 Ill-posed (matrix), 411 lncomplete factorized, 133 lncomplete Choleski factorization, 394 lncore memory requirements, 162 lndefinite (matrix), 410 lndefinite linear system, 456 lndefinite matrix, 154 lndefinite matrix, 294 lndefinite matrix, 456 Independent variables, 10 Initial conditions, 1, 14, 16 Integral form, Integrating by parts, 25 Integration by parts, 36 Integration, lnterface constraints, 467 Interior displacement, 380 lnterior dof, 488 lnterior load vectors, 386 lnternal nodal load vector, 27 Internal virtual work, 10 Interpolant, 18 Interpolation function, 19, 26 Inverse (and forward) iteration procedures, 271 Irons, B.M., 489 lsoparametric bar element, 45 lsoparametric formulation, 44 Iterative solver, 416 Jacobi method, 277,305 Jacobian matrix, 44 Jacobian, 46 Kernel (of a matrix), 411 Kim, J.H., 526 Kinetic energy, 13 Komzsik, L., 525 Lagrange multiplier method, 460 Lagrange multipliers, 163 Lagrangian function, 410 Lanczos eigen-algorithms, 305 Lanczos eigen-solver, 336 Lanczos vectors, 290,294,296 Lanczos vectors, 306 Large amplitude vibration, 357 Lanczos eigen-solution, 290 LDL Transpose, 110,132 LDU, 110,114,168,172 Least square problems, 515 Li, X.S., 524 Linearly independent vectors, 502 Linearly independent, 15 Lingen, F.J., 526 Liu, W., 525 Lowest eigen-pairs, 359 Lumped mass matrix, 294 MA28 unsymmetrical sparse solver, 108 MA28,415 MA47,415 Mass matrices, 272 Mass, 13 Material matrix, 10,20,21 Mathematical operator, MATLAB (software), 4,425 Matrix notations, 4, 19 Matrix times matrix, 77 Matrix-matrix multiplication, 81 Message Passing lnterface (MPI), 63 METiS, 115,224,305 Minimize residual, Mixed direct-iterative solvers, 393 Mixed finite element types, 207 ModifGCR algorithm, 509 Modified Gram-Schmidt, 348 Modified minimum degree (MMD), 163 Moment of inertia, Moment of inertia, 379 M-orthogonal, 306 M-orthonormality, 294 MPI-BCAST, 67 MPI-COMM-RANK, 67 MPI-COMM-SIZE, 65,67 MPI-DOUBLE-PRECISION, 66 530 Finite Element Methods: Parallel-Sparse Statics and Eigen-Solutions MPl-FILE-CLOSE, 73 MPI-FILE-OPEN, 73 MPI-FILE-READ, 73 MPI-FILE-SET-VIEW, 73 MPI-FILE-WRITE, 73 MPI-FINALIZE, 65, 67 MPI-INIT, 65, 67 MPI-RECV, 71 MPI-REDUCE, 66 MPI-SSEND, 71 MPI,WTIME, 70 MSC-NASTRAN, 105 Multi-level substructures, 488 Multiple Minimum Degree (MMD), 115 Multipliers, 143 Off-diagonal terms, 163 Omid, B.M., 523 Optimization problems, 490 Ordinary differential equations, Orthogonal condition, 310 Orthonormality conditions, 272 Othogonalize (Lanczos vector), 315 Out-of-core memory, 160 Outward normal, Overall boundary node numbering system, 383 Overhead computational costs, 160 Owen, D.R., 521 Natural boundary conditions, 6-9, 17 Natural coordinate system, 44, 48 Natural frequency, 14 Necessary condition, 491 Nested Dissection (ND), 115 Nested dissection (ND), 163 Nguyen, D.T., 339, 521,523-526 Nguyen-Runesha's unsymmetrical sparse matrix storage scheme, 256 Noble, B., 515 Nodal displacement, 10 Nodal loads, 10 Non-homogeneous, Non-linear flutter analysis, 357 Nonlinear, 39 Non-singular, 13,24 Non-trivial solution, 14 Normalized, 15 Normalized eigen-matrix, 16 Normalized eigen-vector, 15,271, 299 Numerical integration, 44 Numerical recipe, 339 Numerical sparse assembly of unsymmetrical matrices, 260 Numerical sparse assembly, 192, 201 Panel flutter, 355 Papadrakakis, M., 525 Parallel (MPI) Gram-Schmidt QR, 361 Parallel block factorization, 83 Parallel Choleski factorization, 80 Parallel computer, 64 Parallel dense equation solvers, 77 Parallel 110, 72 Parlett, B.N., 523 Partial derivatives, 10 Partial differential equations, PCPG iterative solver, 457 Pissanetzsky, S., 521 Pivoting (2x2), 154 Plane cross-section, 30 Plane element, 47 Plane isoperimetric element, 47 Polak-Rebiere algorithm, 498 Polynomial function, 52 Polynomial, Positive definite matrix, 13 Positive definite, 110 Positive definite, 155 Potential energy, 469 Preconditioned conjugate gradient (D.D.), 396,397 Preconditioning matrix, 393 Prescribed boundary conditions, 386 Off-diagonal term, 107 Duc T Nguyen Press, Flannery, Teukolsky and Vetterling, 339 Primal DD formulation, 464 Primary dependent function, 32 Primary variable, 9, 18 Processor, 64 Projected residual, 421 Proportional damping matrix, 13 Pseudo force, 41 Pseudo rigid body motion, 412 Qin, J., 523 QR algorithm, 340 QR factorization, 341 QR iteration with shifts, 353 QR iteration, 350 QR, 361 Quadratic solid element, 44 Quadrilateral element, 47 Range (of a matrix), 411 Rank (of a matrix), 415 Rayleigh Ritz, 59 Rectangular element, 42 Reddy, J.N., 42-44, 521 Reduced eigen-equation, 361 Reduced eigen-problem, 287 Reduced stiffness, mass matrices, 287 Reduced tri-diagonal system, 316 Reid, J., 489 Relative error norm, 163 Remainder displacement, 487 Remainder dof, 465,467,470 Re-ordering algorithms, 110, 117, 224 Re-orthogonalize, 361 Residual, Reversed Cuthill-Mckee, 115 Right-hand-side columns, 160 Rigid body displacement, 405 Rod finite element, 17 Runesha, H.B., 523 Saxpy operations, 76 Saxpy unrolling strategies, 141 Scalar field problem, 47 Scalar product operations, 80 Schur complement 380,462 Scott, D., 523 Search direction, 398, 491, 494 Secondary variable, SGI (parallel) sparse solver, 401 SGlIOrigin 2000, 91 SGl's unsymmetrical sparse matrix storage scheme, 258 Shape functions, 9-10 Shear force, Shifted eigen-problem, 304 Shifted eigen-problems, 274 Simon, H.D., 523 Simply supported beam, Simply supports, Simpson's integration rule, 53 Singular matrix, 13 Singular value decomposition (SVD), 515 Skyline column storage, 78 Slope, 5, Solid elements, 33 Sparse assembly of rectangular matrix, 392 Sparse assembly of unsymmetrical matrices, 259 Sparse assembly, 183 Sparse eigen-solution, 17 Sparse matrix time vector, 398 Sparse storage scheme, 105 Sparse, 13 Standard eigen-problem, 296, 299,316 Standard eigen-value problems, 269 Static condensation, 295 Statics, Steepest descent direction, 494 Step-by-step optimization procedures, 491 Step-size, 491 Stiffness matrix, 11 Strain energy density, 20 Strain-displacement relationships, 10,20,46, 50 Stress, 10 Stress-strain relationship, 10 Stride, 81 532 Finite Element Methods: Parallel-SparseStatics and Eigen-Solutions Strip mining, 81 Structural banded finite element model, 402 Structural Engineering, 1, Structural problem, 10 Sturm sequence, 302,304 Sub-domains, 9, 381 Subspace iteration method, 286 Substructures, 381 Sub-structuring numbering system, 383 Successive right-hand-sides, 490 Sun-10000 processors, 401 Super (k-th) row, 143 Super (master) nodes, 134 Super linear speed-up, 401 Super row, 180 Support boundary condition, 386 Supported node, 24 Symbolic factorization, 118 Symbolic sparse assembly, 189 Symmetrical equation solver, 77 Symmetrical positive definite, 369 Symmetrical sparse assembly, 200 Symmetrical, System effective boundary load vector, 389 System global coordinate, 11 System mass matrix, 13 System stiffness equations, 12 Tamma, K.K., 524 Tangent stiffness matrix, 294 Taylor, R.L., 523 Tetrahedral, 31 Thickness of plate (or shell), 379 Three-node element, 45 Transformation methods, 276, 277 Transposing (a sparse matrix), 130 Transverse deflection, 30 Transverse distributed loads, 30 Trapezoid integration rule, 52 Triangular area, 39 Triangular element, 39,41, 205 Tri-diagonal matrix, 291, 315, 365 Truss 2-D by FETl formulation, 433 Truss finite element, 27, 184 Tungkahotara, S., 526 Twice differiable Unconstrained finite element model, 219 Uncoupling, 14 Uniform load, Unitary matrices, 515 Unknown displacement vector, 12 Unrolling numerical factorization, 137 Unrolling of loops, 82 Unrolling techniques, 76 Unstable, 23 Unsymmetrical eigen-solver, 339, 354 Unsymmetrical equation solver, 168 Unsymmetrical matrix, 166, 167 Unsymmetrical sparse assembly, 230 Upper Hessenberg matrix, 340, 359 Upper triangular matrix, 77 Upper triangular, 114 Variational, Vector/cache computer, 360 Velocity, 13 Virtual displacement, Virtual nodal displacement, 10 Virtual strain, 10 Virtual work equation, 11 Virtual work, Watson, W.R., 524 Weak form, Weak formulations, 6, 7, 32 Weighted integral statement, Weighted residual, 32, 35 Weighting function, 3, Weighting function, Weighting residual, 3, 25 Young modulus, 2,10,117 Duc T Nguyen Zienkewicz, O.C., 521 ZPSLDLT (SGI subroutine), 401 FINITE ELEMENT METHODS: PARALLEL-SPARSE STATICS AND EIGEN-SOLUTIONS Duc T Nguyen Dr Duc T Nguyen is the founding Director of the Institute for Multidisciplinary Parallel-Vector Computation and Professor of Civil and Environmental Engineering at Old Dominion University His research work in parallel procedures for computational mechanics has been supported by NASA Centers, AFOSR, CIT, Virginia Power, NSF, Lawrence Livermore National Laboratory, Jonathan Corp., NorthropGrumman Corp., and Hong Kong University of Science and Technology He is the recipient of numerous awards, including the 1989 Gigaflop Award presented by Cray Research Incorporated, the 1993 Tech Brief Award presented by NASA Langley Research Center for his fast Parallel-Vector Equation Solvers, and Old Dominion University, 2001 A Rufus Tonelson distinguished faculty award Dr Nguyen has been listed among the Most Highly Cited Researchers in Engineering in the world ... is divided into three rod finite elements as indicated in Figure 1.5 Finite Element Methods: Parallel- Sparse Statics and Eigen- Solutions Figure 1.5 an Axially Loaded Rod with three Finite Elements... the detailed step-by-step algorithms used during the finite element (truely sparse) assembly, the ''Uirect" and "iterative" sparse equation and eigen- solvers, and incorporating the DD formulation... 10 Finite Element Methods: Parallel- Sparse Statics and Eigen- Solutions The unknown primary function (say, deflection function) f(xi) at any location within a finite element can be computed in terms