sethna p. statistical mechanics.. entropy, order parameters and complexity

Statistical Mechanics: Entropy, Order Parameters and Complexity James P. Sethna, Physics, Cornell University, Ithaca, NY c April 19, 2005 Q B b SYSTEM x F E δE+ E Water Alcohol Oil Two-Phase Mix Electronic version of text available at http://www.physics.cornell.edu/sethna/StatMech/book.pdf Contents 1 Why Study Statistical Mechanics? 3 Exercises 7 1.1 QuantumDice 7 1.2 Probability Distributions. . . . . . . . . . . . . . . 8 1.3 Waitingtimes. 8 1.4 Stirling’s Approximation and Asymptotic Series. . 9 1.5 RandomMatrixTheory 10 2 Random Walks and Emergent Properties 13 2.1 Random Walk Examples: Universality and Scale Invariance 13 2.2 TheDiffusionEquation 17 2.3 CurrentsandExternalForces 19 2.4 SolvingtheDiffusionEquation 21 2.4.1 Fourier 21 2.4.2 Green 22 Exercises 23 2.1 Random walks in Grade Space. . . . . . . . . . . . 24 2.2 Photon diffusion in the Sun. . . . . . . . . . . . . . 24 2.3 Ratchet and Molecular Motors. . . . . . . . . . . . 24 2.4 Solving Diffusion: Fourier and Green. . . . . . . . 26 2.5 Solving the Diffusion Equation. . . . . . . . . . . . 26 2.6 FryingPan 26 2.7 ThermalDiffusion 27 2.8 PolymersandRandomWalks. 27 3 Temperature and Equilibrium 29 3.1 The Microcanonical Ensemble . . . . . . . . . . . . . . . . 29 3.2 The Microcanonical Ideal Gas . . . . . . . . . . . . . . . . 31 3.2.1 Configuration Space . . . . . . . . . . . . . . . . . 32 3.2.2 MomentumSpace 33 3.3 WhatisTemperature? 37 3.4 Pressure and Chemical Potential . . . . . . . . . . . . . . 40 3.5 Entropy, the Ideal Gas, and Phase Space Refinements . . 44 Exercises 46 3.1 EscapeVelocity. 47 3.2 TemperatureandEnergy. 47 i ii CONTENTS 3.3 Hard Sphere Gas . . . . . . . . . . . . . . . . . . . 47 3.4 Connecting Two MacroscopicSystems 47 3.5 GaussandPoisson 48 3.6 MicrocanonicalThermodynamics 49 3.7 Microcanonical Energy Fluctuations. . . . . . . . . 50 4 Phase Space Dynamics and Ergodicity 51 4.1 Liouville’s Theorem . . . . . . . . . . . . . . . . . . . . . 51 4.2 Ergodicity 54 Exercises 58 4.1 The Damped Pendulum vs. Liouville’s Theorem. . 58 4.2 Jupiter! and the KAM Theorem . . . . . . . . . . 58 4.3 InvariantMeasures. 60 5 Entropy 63 5.1 Entropy as Irreversibility: Engines and Heat Death . . . . 63 5.2 EntropyasDisorder 67 5.2.1 Mixing: Maxwell’s Demon and Osmotic Pressure . 67 5.2.2 Residual Entropy of Glasses: The Roads Not Taken 69 5.3 Entropy as Ignorance: Information and Memory . . . . . 71 5.3.1 Nonequilibrium Entropy . . . . . . . . . . . . . . . 72 5.3.2 InformationEntropy 73 Exercises 76 5.1 Life and the Heat Death of the Universe. . . . . . 77 5.2 P-VDiagram 77 5.3 CarnotRefrigerator 78 5.4 DoesEntropyIncrease? 78 5.5 EntropyIncreases:Diffusion. 80 5.6 Informationentropy 80 5.7 Shannonentropy 80 5.8 EntropyofGlasses 81 5.9 RubberBand 82 5.10 DerivingEntropy. 83 5.11 Chaos, Lyapunov, and Entropy Increase. . . . . . . 84 5.12 BlackHoleThermodynamics 84 5.13 FractalDimensions. 85 6 Free Energies 87 6.1 The Canonical Ensemble . . . . . . . . . . . . . . . . . . . 88 6.2 Uncoupled Systems and Canonical Ensembles . . . . . . . 92 6.3 GrandCanonicalEnsemble 95 6.4 WhatisThermodynamics? 96 6.5 Mechanics:FrictionandFluctuations 100 6.6 Chemical Equilibrium and Reaction Rates . . . . . . . . . 101 6.7 Free Energy Density for the Ideal Gas . . . . . . . . . . . 104 Exercises 106 6.1 Two–statesystem. 107 6.2 BarrierCrossing 107 To be pub. Oxford UP, ∼Fall’05 www.physics.cornell.edu/sethna/StatMech/ CONTENTS iii 6.3 Statistical Mechanics and Statistics. . . . . . . . . 108 6.4 Euler, Gibbs-Duhem, and Clausius-Clapeyron. . . 109 6.5 NegativeTemperature 110 6.6 Laplace 110 6.7 Lagrange 111 6.8 Legendre. 111 6.9 Molecular Motors: Which Free Energy? . . . . . . 111 6.10 Michaelis-Menten and Hill . . . . . . . . . . . . . . 112 6.11 PollenandHardSquares. 113 7 Quantum Statistical Mechanics 115 7.1 Mixed States and Density Matrices 115 7.2 Quantum Harmonic Oscillator . . . . . . . . . . . . . . . . 120 7.3 BoseandFermiStatistics 120 7.4 Non-Interacting Bosons and Fermions . . . . . . . . . . . 121 7.5 Maxwell-Boltzmann “Quantum” Statistics . . . . . . . . . 125 7.6 Black Body Radiation and Bose Condensation . . . . . . 127 7.6.1 Free Particles in a Periodic Box . . . . . . . . . . . 127 7.6.2 BlackBodyRadiation 128 7.6.3 Bose Condensation . . . . . . . . . . . . . . . . . . 129 7.7 MetalsandtheFermiGas 131 Exercises 132 7.1 Phase Space Units and the Zero of Entropy. . . . . 133 7.2 Does Entropy Increase in Quantum Systems? . . . 133 7.3 Phonons on a String. . . . . . . . . . . . . . . . . . 134 7.4 CrystalDefects. 134 7.5 DensityMatrices 134 7.6 Ensembles and Statistics: 3 Particles, 2 Levels. . . 135 7.7 Bosons are Gregarious: Superfluids and Lasers . . 135 7.8 Einstein’sAandB 136 7.9 Phonons and Photons are Bosons. . . . . . . . . . 137 7.10 Bose Condensation in a Band. . . . . . . . . . . . 138 7.11 Bose Condensation in a Parabolic Potential. . . . . 138 7.12 Light Emission and Absorption. . . . . . . . . . . . 139 7.13 Fermions in Semiconductors. . . . . . . . . . . . . 140 7.14 White Dwarves, Neutron Stars, and Black Holes. . 141 8 Calculation and Computation 143 8.1 What is a Phase? Perturbation theory. . . . . . . . . . . . 143 8.2 TheIsingModel 146 8.2.1 Magnetism 146 8.2.2 BinaryAlloys 147 8.2.3 Lattice Gas and the Critical Point . . . . . . . . . 148 8.2.4 HowtoSolvetheIsingModel. 149 8.3 MarkovChains 150 Exercises 154 8.1 TheIsingModel 154 8.2 Coin Flips and Markov Chains. . . . . . . . . . . . 155 c James P. Sethna, April 19, 2005 Entropy, Order Parameters, and Complexity iv CONTENTS 8.3 RedandGreenBacteria 155 8.4 DetailedBalance 156 8.5 Heat Bath, Metropolis, and Wolff. . . . . . . . . . 156 8.6 StochasticCells. 157 8.7 TheRepressilator. 159 8.8 Entropy Increases! Markov chains. . . . . . . . . . 161 8.9 Solving ODE’s: The Pendulum . . . . . . . . . . . 162 8.10 SmallWorldNetworks. 165 8.11 Building a Percolation Network. . . . . . . . . . . 167 8.12 Hysteresis Model: Computational Methods. . . . . 169 9 Order Parameters, Broken Symmetry, and Topology 171 9.1 IdentifytheBrokenSymmetry 172 9.2 DefinetheOrderParameter 172 9.3 ExaminetheElementaryExcitations 176 9.4 ClassifytheTopologicalDefects 178 Exercises 183 9.1 Topological Defects in the XY Model. . . . . . . . 183 9.2 Topological Defects in Nematic Liquid Crystals. . 184 9.3 Defect Energetics and Total Divergence Terms. . . 184 9.4 Superfluid Order and Vortices. . . . . . . . . . . . 184 9.5 Landau Theory for the Ising model. . . . . . . . . 186 9.6 BlochwallsinMagnets. 190 9.7 Superfluids: Density Matrices and ODLRO. . . . . 190 10 Correlations, Response, and Dissipation 195 10.1 Correlation Functions: Motivation . . . . . . . . . . . . . 195 10.2ExperimentalProbesofCorrelations 197 10.3 Equal–Time Correlations in the Ideal Gas . . . . . . . . . 198 10.4 Onsager’s Regression Hypothesis and Time Correlations . 200 10.5 Susceptibility and the Fluctuation–Dissipation Theorem . 203 10.5.1 Dissipation and the imaginary part χ  (ω) 204 10.5.2 Static susceptibility χ 0 (k) 205 10.5.3 χ(r,t)andFluctuation–Dissipation 207 10.6 Causality and Kramers Kr¨onig 210 Exercises 212 10.1 Fluctuations in Damped Oscillators. . . . . . . . . 212 10.2 Telegraph Noise and RNA Unfolding. . . . . . . . 213 10.3 Telegraph Noise in Nanojunctions. . . . . . . . . . 214 10.4 Coarse-Grained Magnetic Dynamics. . . . . . . . . 214 10.5 Noise and Langevin equations. . . . . . . . . . . . 216 10.6 Fluctuations, Correlations, and Response: Ising . . 216 10.7 Spin Correlation Functions and Susceptibilities. . . 217 11 Abrupt Phase Transitions 219 11.1MaxwellConstruction 220 11.2 Nucleation: Critical Droplet Theory. . . . . . . . . . . . . 221 11.3 Morphology of abrupt transitions. . . . . . . . . . . . . . 223 To be pub. Oxford UP, ∼Fall’05 www.physics.cornell.edu/sethna/StatMech/ CONTENTS 1 11.3.1 Coarsening. . . . . . . . . . . . . . . . . . . . . . . 223 11.3.2Martensites 227 11.3.3DendriticGrowth. 227 Exercises 228 11.1 van der Waals Water. . . . . . . . . . . . . . . . . 228 11.2 Nucleation in the Ising Model. . . . . . . . . . . . 229 11.3 Coarsening and Criticality in the Ising Model. . . . 230 11.4 Nucleation of Dislocation Pairs. . . . . . . . . . . . 231 11.5 Oragami Microstructure. . . . . . . . . . . . . . . . 232 11.6 Minimizing Sequences and Microstructure. . . . . . 234 12 Continuous Transitions 237 12.1Universality 239 12.2ScaleInvariance 246 12.3 Examples of Critical Points. . . . . . . . . . . . . . . . . . 253 12.3.1 Traditional Equilibrium Criticality: Energy versus Entropy.253 12.3.2 Quantum Criticality: Zero-point fluctuations versus energy.253 12.3.3 Glassy Systems: Random but Frozen. . . . . . . . 254 12.3.4 Dynamical Systems and the Onset of Chaos. . . . 256 Exercises 256 12.1 Scaling: Critical Points and Coarsening. . . . . . . 257 12.2 RG Trajectories and Scaling. . . . . . . . . . . . . 257 12.3 Bifurcation Theory and Phase Transitions. . . . . 257 12.4 Onset of Lasing as a Critical Point. . . . . . . . . . 259 12.5 Superconductivity and the Renormalization Group. 260 12.6 RG and the Central Limit Theorem: Short. . . . . 262 12.7 RG and the Central Limit Theorem: Long. . . . . 262 12.8 PeriodDoubling 264 12.9 Percolation and Universality. . . . . . . . . . . . . 267 12.10 Hysteresis Model: Scaling and Exponent Equalities.269 A Appendix: Fourier Methods 273 A.1 FourierConventions 274 A.2 Derivatives, Convolutions, and Correlations . . . . . . . . 276 A.3 Fourier Methods and Function Space . . . . . . . . . . . . 277 A.4 Fourier and Translational Symmetry . . . . . . . . . . . . 279 Exercises 281 A.1 FourierforaWaveform 281 A.2 Relations between the Fouriers. . . . . . . . . . . . 281 A.3 Fourier Series: Computation. . . . . . . . . . . . . 281 A.4 Fourier Series of a Sinusoid. . . . . . . . . . . . . . 282 A.5 Fourier Transforms and Gaussians: Computation. 282 A.6 Uncertainty 284 A.7 WhiteNoise. 284 A.8 FourierMatching. 284 A.9 Fourier Series and Gibbs Phenomenon. . . . . . . . 284 c James P. Sethna, April 19, 2005 Entropy, Order Parameters, and Complexity 2 CONTENTS To be pub. Oxford UP, ∼Fall’05 www.physics.cornell.edu/sethna/StatMech/ Why Study Statistical Mechanics? 1 Many systems in nature are far too complex to analyze directly. Solving for the motion of all the atoms in a block of ice – or the boulders in an earthquake fault, or the nodes on the Internet – is simply infeasible. Despite this, such systems often show simple, striking behavior. We use statistical mechanics to explain the simple behavior of complex systems. Statistical mechanics brings together concepts and methods that infil- trate many fields of science, engineering, and mathematics. Ensembles, entropy, phases, Monte Carlo, emergent laws, and criticality – all are concepts and methods rooted in the physics and chemistry of gases and liquids, but have become important in mathematics, biology, and com- puter science. In turn, these broader applications bring perspective and insight to our fields. Let’s start by briefly introducing these pervasive concepts and meth- ods. Ensembles: The trick of statistical mechanics is not to study a single system, but a large collection or ensemble of systems. Where under- standing a single system is often impossible, calculating the behavior of an enormous collection of similarly prepared systems often allows one to answer most questions that science can be expected to address. For example, consider the random walk (figure 1.1). (You might imag- ine it as the trajectory of a particle in a gas, or the configuration of a polymer in solution.) While the motion of any given walk is irregular (left) and hard to predict, simple laws describe the distribution of mo- tions of an infinite ensemble of random walks starting from the same initial point (right). Introducing and deriving these ensembles are the themes of chapters 3, 4, and 6. Entropy: Entropy is the most influential concept arising from statis- tical mechanics (chapter 5). Entropy, originally understood as a thermo- dynamic property of heat engines that could only increase, has become science’s fundamental measure of disorder and information. Although it controls the behavior of particular systems, entropy can only be defined within a statistical ensemble: it is the child of statistical mechanics, with no correspondence in the underlying microscopic dynamics. En- tropy now underlies our understanding of everything from compression algorithms for pictures on the Web to the heat death expected at the end of the universe. Phases. Statistical mechanics explains the existence and properties of 3 4 Why Study Statistical Mechanics? Fig. 1.1 Random Walks. The motion of molecules in a gas, or bacteria in a liquid, or photons in the Sun, is described by an irregular trajectory whose velocity rapidly changes in direction at random. Describing the specific trajectory of any given random walk (left) is not feasible or even interesting. Describing the statistical average properties of a large number of random walks is straightforward; at right is shown the endpoints of random walks all starting at the center. The deep principle underlying statistical mechanics is that it is often easier to understand the behavior of ensembles of systems. phases. The three common phases of matter (solids, liquids, and gases) have multiplied into hundreds: from superfluids and liquid crystals, to vacuum states of the universe just after the Big Bang, to the pinned and sliding ‘phases’ of earthquake faults. Phases have an integrity or stability to small changes in external conditions or composition 1 –with deep connections to perturbation theory, section 8.1. Phases often have a rigidity or stiffness, which is usually associated with a spontaneously broken symmetry. Understanding what phases are and how to describe their properties, excitations, and topological defects will be the themes of chapters 7, 2 and 9. 2 Chapter 7 focuses on quantum sta- tistical mechanics: quantum statistics, metals, insulators, superfluids, Bose condensation, . . . To keep the presenta- tion accessible to a broad audience, the rest of the text is not dependent upon knowing quantum mechanics. Computational Methods: Monte–Carlo methods use simple rules to allow the computer to find ensemble averages in systems far too com- plicated to allow analytical evaluation. These tools, invented and sharp- ened in statistical mechanics, are used everywhere in science and tech- nology – from simulating the innards of particle accelerators, to studies of traffic flow, to designing computer circuits. In chapter 8, we introduce the Markov–chain mathematics that underlies Monte–Carlo. Emergent Laws. Statistical mechanics allows us to derive the new 1 Water remains a liquid, with only perturbative changes in its properties, as one changes the temperature or adds alcohol. Indeed, it is likely that all liquids are connected to one another, and indeed to the gas phase, through paths in the space of composition and external conditions. To be pub. Oxford UP, ∼Fall’05 www.physics.cornell.edu/sethna/StatMech/ 5 Fig. 1.2 Temp erature: the Ising mo del at the critical temperature. Traditional statistical mechanics fo- cuses on understanding phases of mat- ter, and transitions between phases. These phases – solids, liquids, mag- nets, superfluids – are emergent prop- erties of many interacting molecules, spins, or other degrees of free- dom. Pictured here is a simple two-dimensional model at its mag- netic transition temperature T c .At higher temperatures, the system is non-magnetic: the magnetization is on average zero. At the temperature shown, the system is just deciding whether to magnetize upward (white) or downward (black). While predict- ing the time dependence of all these degrees of freedom is not practical or possible, calculating the average be- havior of many such systems (a statis- tical ensemble) is the job of statistical mechanics. laws that emerge from the complex microscopic behavior. These laws be- come exact only in certain limits. Thermodynamics – the study of heat, temperature, and entropy – becomes exact in the limit of large numbers of particles. Scaling behavior and power laws – both at phase transitions and more broadly in complex systems – emerge for large systems tuned (or self–organized) near critical points. The right figure 1.1 illustrates the simple law (the diffusion equation) that describes the evolution of the end-to-end lengths of random walks in the limit where the number of steps becomes large. Developing the machinery to express and derive these new laws are the themes of chapters 9 (phases), and 12 (critical points). Chapter 10 systematically studies the fluctuations about these emergent theories, and how they relate to the response to external forces. Phase Transitions. Beautiful spatial patterns arise in statistical mechanics at the transitions between phases. Most of these are abrupt phase transitions: ice is crystalline and solid until abruptly (at the edge of the ice cube) it becomes unambiguously liquid. We study nucleation and the exotic structures that evolve at abrupt phase transitions in chap- ter 11. Other phase transitions are continuous. Figure 1.2 shows a snapshot of the Ising model at its phase transition temperature T c .TheIsing model is a lattice of sites that can take one of two states. It is used as a simple model for magnets (spins pointing up or down), two component crystalline alloys (A or B atoms), or transitions between liquids and gases (occupied and unoccupied sites). 3 All of these systems, at their critical 3 The Ising model has more far-flung applications: the three–dimensional Ising model has been useful in the study of quantum gravity. c James P. Sethna, April 19, 2005 Entropy, Order Parameters, and Complexity [...]... James P Sethna, April 19, 2005 Entropy, Order Parameters, and Complexity Solving the Diffusion Equation 23 21 It’s useful to remember that the Fourier transform of a normalized Gaussian √ 1 exp(−x2 /2σ2 ) is an2πσ other Gaussian, exp(−σ2 k 2 /2) of standard deviation 1/σ and with no prefactor 24 Random Walks and Emergent Properties Exercises Exercises 2.1, 2.2, and 2.3 give simple examples of random... individual particle as it random walks through space: if the particles are non-interacting, the probability distribution of one particle describes the density of all particles c James P Sethna, April 19, 2005 Entropy, Order Parameters, and Complexity S&P 500 Index / avg return 2.2 2 The Diffusion Equation 17 S&P Random 1.5 1 1985 1990 1995 Year 2000 2005 Fig 2.3 S&P 500, normalized Standard and Poor’s 500 stock... interactions between electrons disappear near the Fermi energy: the fixed point has an emergent gauge symmetry c James P Sethna, April 19, 2005 Entropy, Order Parameters, and Complexity 12 Why Study Statistical Mechanics? To be pub Oxford UP, ∼Fall’05 www.physics.cornell.edu /sethna/ StatMech/ Random Walks and Emergent Properties What makes physics possible? Why are humans able to find simple mathematical laws that... derived this law of thermal conductivity from random walks of phonons, but we haven’t yet done so 26 Some seem to define the persistence length with a different constant factor c James P Sethna, April 19, 2005 Entropy, Order Parameters, and Complexity 28 Random Walks and Emergent Properties (c) Measure for a reasonable length of time, print out the current state, and enclose it Did the simulation give √ R... identical, and that each question is answered at random with a probability 0.7 of getting it right (a) What is the expected mean and standard deviation for the exam? (Work it out for one question, and then use our theorems for a random walk with ten steps.) A typical exam with a mean of 70 might have a standard deviation of about 15 (b) What physical interpretation do you make of the ratio of the random standard... which accompany the motor moving one base pair: the motor burns up an NTP molecule into a PPi molecule, and attaches a nucleotide onto the RNA The Entropy, Order Parameters, and Complexity 26 Random Walks and Emergent Properties net energy from this reaction depends on details, but varies between about 2 and 5 times 10−20 Joule This is actually a Gibbs free energy difference, but for this exercise treat... this is also true, and is called the equipartition theorem (section 3.2.2) The constants in the (non–equilibrium) diffusion equation are related to one another, because the density must evolve toward the equilibrium distribution dictated by statistical mechanics c James P Sethna, April 19, 2005 Entropy, Order Parameters, and Complexity 18 One should note that much of quantum field theory and many-body quantum... Q)) /Ω(E) in phase space c James P Sethna, April 19, 2005 Entropy, Order Parameters, and Complexity 10 Air is a mixture of gases, but most of the molecules are diatomic: O2 and N2 , with a small admixture of triatomic CO2 and monatomic Ar The properties of diatomic ideal gases are almost as simple: but one must keep track of the internal rotational degree of freedom (and, at high temperatures, the vibrational... eralize our argument that the RMS distance scales as N to simultaneously cover both coin flips and drunkards; with more work we could = 2 sN −1 c James P Sethna, April 19, 2005 2 Entropy, Order Parameters, and Complexity Fig 2.1 The drunkard takes a series of steps of length L away from the lamppost, but each with a random angle 5 More generally, if two variables are uncorrelated then the average of their... shall define entropy and temperature for equilibrium systems, and argue from the microcanonical ensemble that heat flows to maximize the entropy and equalize the temperature In section 3.4 we will derive the formula for the pressure in terms of the entropy, and define the chemical potential Finally, in section 3.5 we calculate the entropy, temperature, and pressure for the ideal gas, and introduce some . Series and Gibbs Phenomenon. . . . . . . . 284 c James P. Sethna, April 19, 2005 Entropy, Order Parameters, and Complexity 2 CONTENTS To be pub. Oxford UP, ∼Fall’05 www.physics.cornell.edu /sethna/ StatMech/ Why. electrons disappear near the Fermi energy: the fixed point has an emergent gauge symmetry. c James P. Sethna, April 19, 2005 Entropy, Order Parameters, and Complexity 12 Why Study Statistical Mechanics? To. Statistical Mechanics: Entropy, Order Parameters and Complexity James P. Sethna, Physics, Cornell University, Ithaca, NY c April 19, 2005 Q B b SYSTEM x F E δE+ E Water Alcohol Oil Two-Phase