High performance computations in NMR 2006

High Performance Computations in NMR by Wyndham Bolling Blanton B.S Chemistry (Carnegie Mellon University) 1998 B.S Physics (Carnegie Mellon University) 1998 A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Chemistry in the GRADUATE DIVISION of the UNIVERSITY OF CALIFORNIA, BERKELEY Committee in charge: Professor Alexander Pines, Chair Professor Jeffrey A Reimer Professor Raymond Y Chiao David E Wemmer Fall 2002 The dissertation of Wyndham Bolling Blanton is approved: Chair Date Date Date Date University of California, Berkeley Fall 2002 High Performance Computations in NMR Copyright c 2002 by Wyndham Bolling Blanton Abstract High Performance Computations in NMR by Wyndham Bolling Blanton Doctor of Philosophy in Chemistry University of California, Berkeley Professor Alexander Pines, Chair As an analytic noninvasive technique to study molecules in their natural environment, NMR has little equal The advancement of the technique is beginning to enter a new phase, where many body dynamics, complex control, and precise measurements of many body spin properties preclude any exact theoretical treatment Approximation methods and other reductions in the set of parameter spaces are currently used to obtain some form of intuition about a simplified NMR system; however, to exactly profile a real system, numerical simulation is required The scope of most NMR simulations is chiefly regulated to small spin systems, where the dynamics are simplified enough to simulate efficiently The cause is typically based on a poor understanding of how to simulate an NMR situation effectively and efficiently This seems consistent with the fact that most NMR spectroscopists are not computer scientists as well The introduction of novel programming paradigms and numerical techniques seems to have eluded the field A complete simulation environment for NMR is presented here marrying three fundamental aspects of simulations 1) numerical speed and efficiency, 2) simplicity in implementation, and 3) NMR specific algorithmic developments The majority of numerical NMR is reduced to a simple simulation framework The framework allows for more complex simulations for explorations of both many body spin dynamics and control A specific large scale simulation is applied to recoupling sequences in solid–state NMR Using simple permutations on base pulse sequences can result in control enhancements on both the simple system and the many body system beyond a theoretical approach The sheer number of permutations required to solve the problem would have certainly been impossible without the aid of this framework This new framework now opens other unexplored possibilities of using simulation as a development tool for the larger problems of many body dynamics and control Professor Alexander Pines Dissertation Committee Chair i To my Grandmother and Grandfather, Lucy and Wyndham Jr ii Contents List of Figures v List of Tables viii Introduction Computer Mechanics 2.1 Data Types 2.2 The Object 2.2.1 Syntax 2.3 Expression Templates 2.3.1 Motivations 2.3.2 Stacks 2.3.3 An Array Object and Stacks 2.3.4 Expression Template Implementation 2.4 Optimizing For Hardware 2.4.1 Basic Computer Architecture 2.4.2 A Faster Matrix Multiplication 13 13 14 15 19 27 30 37 NMR Forms 3.1 Classical Mechanics 3.2 Bloch Equation Magnetic Fields 3.3 Quantum Mechanics 3.3.1 Rotations 3.3.2 Rotational Frames 3.3.3 The Hamiltonians 3.4 NMR Initial Conditions 3.4.1 Quantum 3.4.2 Classical 42 42 43 59 60 64 67 73 73 74 NMR Algorithms 4.1 Classical Algorithms 4.1.1 Eigenvalue Problem 4.1.2 ODE solvers 4.2 Quantum Algorithms 76 76 76 78 82 iii 4.3 4.2.1 The Direct Method 4.2.2 Periodicity and Propagator Reduction 4.2.3 Eigenspace 4.2.4 Periodicity and Eigen–Space methods 4.2.5 Non-periodic Hamiltonians 4.2.6 Powder Average Integration Conclusions and Comments BlochLib 5.1 Introduction 5.2 The Abstract NMR Simulation 5.2.1 Experimental Evolutions (EE) 5.2.2 Theoretical Evolutions (TE) 5.2.3 Existing NMR Tool Kits 5.2.4 Why Create a new Tool Kit? 5.3 BlochLib Design 5.3.1 Existing Numerical Tool Kits 5.3.2 Experimental and Theoretical Evolutions for NMR 5.3.3 BlochLib Layout 5.3.4 Drawbacks 5.4 Various Implementations 5.4.1 Solid 5.4.2 Classical Program: Magnetic Field Calculators 5.4.3 Classical Programs: Bloch Simulations 5.5 Conclusions 82 83 89 95 100 100 103 simulations 105 105 106 106 106 108 109 109 110 111 112 121 123 124 129 131 140 141 141 142 143 143 145 150 151 155 155 156 158 161 161 185 196 Massive Permutations of Rotor Synchronized Pulse Sequences 6.1 Introduction 6.1.1 Rotor Synchronization 6.2 Background Theory 6.2.1 Average Hamiltonian 6.2.2 Recoupling RSS 6.2.3 C7 6.2.4 Removable of Higher Order Terms 6.3 Permutations 6.3.1 The Sub–Units 6.3.2 The Measure 6.3.3 Algorithmic Flow 6.4 Data and Results 6.4.1 Sequence Measures 6.4.2 Transfer Efficiencies 6.5 Conclusions Future Expansions 201 7.1 Evolutionary Algorithms (EA) 202 7.2 Neural Networks 209 7.3 Final Remarks 211 iv Bibliography 213 A Auxillary code A.1 General C++ code and examples A.1.1 C++ Template code used to generate prime number at compilation A.1.2 C++ Template meta-program to unroll a fixed length vector at compilation time A.1.3 C++ code for performing a matrix multiplication with L2 cache blocking and partial loop unrolling A.1.4 An MPI master/slave implimentation framework A.1.5 C++ class for a hidden layer Fully connected back–propagation Neural Network A.2 NMR algorithms A.2.1 Mathematica Package to generate Wigner Rotation matrices and Spin operators A.2.2 Rational Reduction C++ Class A.2.3 Optimized static Hamiltonian FID propogation A.2.4 γ − COM P U T E C++ Class A.3 BlochLib Configurations and Sources A.3.1 Solid configuration files A.3.2 Magnetic Field Calculator input file A.3.3 Quantum Mechanical Single Pulse Simulations A.3.4 Example Classical Simulation of the Bulk Susceptibility A.3.5 Example Classical Simulation of the Modulated Demagnetizing Field 225 225 225 226 228 230 232 239 239 244 252 253 263 263 266 267 267 274 v List of Figures 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 2.10 2.11 2.12 A two state Turing machine A simple stack tree How the compiler unrolls an expression template set of operations DAXPY speed tests A pictorial representation for the matrix–matrix tensor multiplication Speed in MFLOPS of a matrix–matrix multiplication A generic computer data path Pipe lines and loop unrolling A 128 bit SIMD registers made of 4–32 bit data values Cache levels in modern Processors Speed comparison in MFLOPS of loop unrolling Speed comparison in MFLOPS of L2 cache blocking and loop unrolling 15 25 26 28 29 30 34 35 36 39 40 3.1 3.2 3.3 The magnitude of the dipole field The magnetization of a sample inside a magneti field Magnetization in iso–surfaces versus the applied magnetic field, Bo , the temperature T , and number of moles 52 55 84 89 4.4 Various propagators needed for an arbitrary rational reduction Effectiveness of the rational propagator reduction method Diagram of one Hamiltonian period and the propagator labels used for the COMPUTE algorithm Octants of equal volume of a sphere 96 102 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 Experimental Evolutions and Theoretical Evolutions The basic design layout of the BlochLib NMR tool kit C=A*B*adjoint(A) speed of BlochLib Solid vs Simpson The design of the EE program Solid derived from the input syntax 1D static and spinning spin simulation 1D and 2D post-C7 simulation The basic design for the Field Calculator program Magnetic field of a D–circle A rough design for a classical Bloch simulation over various interactions 107 113 115 125 127 128 128 130 132 133 4.1 4.2 4.3 75 A.3 BLOCHLIB CONFIGURATIONS AND SOURCES 266 #number o f 2D p o i n t s f i d p t =128 #c o l l e c t i o n a m a t r i x o f d a t a 2D( ) #s e t t h e s p i n n i n g wr=5000 #t h e b a s i c r o t o r a n g l e r o t o r=rad2deg ∗ a c o s ( / sqrt ( ) ) ) #s e t t h e d e t e c t i o n m a t r i x d e t e c t ( Ip ) #r e s e t t h e ro b a c k t o t h e eq ro ( Iz ) #9 time a m p l i t u d e s amp=150000 t =1/amp/4 #l o o p o v e r t h e r o t o r s t e p s l o o p (m=0: f i d p t −1) #may use ’ r e u s e ’ a l l v a r i a b l e s a r e s t a t i c i n sub1 # must be r e p e a t m t i m e s t o advance t h e d e n s i t y m a t r i x # f o r each f i d ( t h e f i r s t f i d g e t s no c7 ) r e u s e ( sub1 , m) #p u l s e t h e IZ down t o t h e xy p l a n e f o r d e t e c t i o n 1H: p u l s e ( t90 , , amp) #c o l l e c t t h e f i d a t t h e ’ mth ’ p o s i t i o n f i d (m) #r e s e t t h e ro b a c k t o t h e eq ro ( Iz ) end s a v e f i d m a t l a b ( dc7 ) #s a v e t h e m a t l a b f i l e } } A.3.2 Magnetic Field Calculator input file The input coil type ‘Dcircle’ is a user registered function, and not part of the normal please view the source code in the distribution for details par MyCoil { subcoil1 { type h e l m h o l t z loops 25 amps −4 numpts 0 R length axis z } subcoil2 { A.3 BLOCHLIB CONFIGURATIONS AND SOURCES 267 type D c i r c l e loops amps numpts 0 R theta1 theta2 180 axis z center , −.6 ,5 } subcoil3 { type D c i r c l e loops amps numpts 0 R theta1 theta2 180 axis z c e n t e r , −.6 , −5 } } grid { −1,−1,−1 max , , dim , , } params { #which magnetic f i e l d s e c t i o n t o use s e c t i o n MyCoil #o u t p u t textout #o u t p u t matout t e x t f i l e name shape b i o t m a t l a b f i l e name f i e l d mat } A.3.3 Quantum Mechanical Single Pulse Simulations A.3.4 Example Classical Simulation of the Bulk Susceptibility This simulation is a replication of the simulation performed by M Augustine in Figure of Ref [100] It demonstrates the slight offset effect imposed by the magnetization of one spin on another Both the C++ source using the BlochLib framework and the A.3 BLOCHLIB CONFIGURATIONS AND SOURCES configuration file is given Results from this simulation can be found in Figure 5.11 C++ source #include ” b l o c h l i b h” // t h e r e q u i r e d namespaces using namespace BlochLib ; using namespace s t d ; /∗ THis s i m u l a t e s t h e e f f e c t o f t h e Bulk S u s e p t i b i l i t y on a HETCOR e x p e r i e m e n t h o p e f u l l y we s h a l l s e e s e v e r a l e c h o s i n t h e i n d i r e c t dimension a HETOCR i s a 2D e x p e r i e m e n t s p i n :: 90 − − t −−90−−−−− s p i n 2:: −−−−−−−90−FID ∗/ t i m e r stopwatch ; void printTime ( int nrounds =1){ s t d : : c o u t