www.it-ebooks.info www.it-ebooks.info MySQL High Availability www.it-ebooks.info www.it-ebooks.info MySQL High Availability Charles Bell, Mats Kindahl, and Lars Thalmann Beijing • Cambridge • Farnham • Köln • Sebastopol • Taipei • Tokyo www.it-ebooks.info MySQL High Availability by Charles Bell, Mats Kindahl, and Lars Thalmann Copyright © 2010 Charles Bell, Mats Kindahl, and Lars Thalmann. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://my.safaribooksonline.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Editor: Andy Oram Production Editor: Teresa Elsey Copyeditor: Amy Thomson Proofreader: Sada Preisch Indexer: Lucie Haskins Cover Designer: Karen Montgomery Interior Designer: David Futato Illustrator: Robert Romano Printing History: July 2010: First Edition. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. MySQL High Availability, the image of an American robin, and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trademark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information con- tained herein. ISBN: 978-0-596-80730-6 [M] 1277482774 www.it-ebooks.info Table of Contents Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Part I. Replication 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 What’s This Replication Stuff Anyway? 5 So, Backups Are Not Needed Then? 6 What’s with All the Monitoring? 7 Is There Anything Else I Can Read? 8 Conclusion 8 2. MySQL Replication Fundamentals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Basic Steps in Replication 12 Configuring the Master 13 Configuring the Slave 15 Connecting the Master and Slave 15 A Brief Introduction to the Binary Log 17 What’s Recorded in the Binary Log 17 Watching Replication in Action 18 The Binary Log’s Structure and Content 20 Python Support for Managing Replication 23 Basic Classes and Functions 25 Operating System 26 Server Class 26 Server Roles 28 Creating New Slaves 30 Cloning the Master 31 Cloning the Slave 33 v Download from Library of Wow! eBook <www.wowebook.com> www.it-ebooks.info Scripting the Clone Operation 35 Performing Common Tasks with Replication 36 Reporting 37 Conclusion 43 3. The Binary Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Structure of the Binary Log 46 Binlog Event Structure 48 Logging Statements 50 Logging Data Manipulation Language Statements 50 Logging Data Definition Language Statements 51 Logging Queries 51 LOAD DATA INFILE Statements 57 Binary Log Filters 59 Triggers, Events, and Stored Routines 61 Stored Procedures 66 Stored Functions 69 Events 71 Special Constructions 71 Nontransactional Changes and Error Handling 72 Logging Transactions 75 Transaction Cache 76 Distributed Transaction Processing Using XA 79 Binary Log Management 81 The Binary Log and Crash Safety 82 Binlog File Rotation 83 Incidents 85 Purging the Binlog File 86 The mysqlbinlog Utility 87 Basic Usage 88 Interpreting Events 94 Binary Log Options and Variables 98 Conclusion 100 4. Replication for High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Redundancy 104 Planning 106 Slave Failures 106 Master Failures 106 Relay Failures 107 Disaster Recovery 107 Procedures 107 Hot Standby 111 vi | Table of Contents Download from Library of Wow! eBook <www.wowebook.com> www.it-ebooks.info Dual Masters 115 Semisynchronous Replication 124 Slave Promotion 127 Circular Replication 142 Conclusion 146 5. MySQL Replication for Scale-Out . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Scaling Out Reads, Not Writes 149 The Value of Asynchronous Replication 150 Managing the Replication Topology 152 Example of an Application-Level Load Balancer 155 Hierarchal Replication 159 Setting Up a Relay Server 160 Adding a Relay in Python 161 Specialized Slaves 162 Filtering Replication Events 162 Using Filtering to Partition Events to Slaves 164 Data Sharding 165 Shard Representation 168 Partitioning the Data 170 Balancing the Shards 171 A Sharding Example 173 Managing Consistency of Data 184 Consistency in a Nonhierarchal Deployment 185 Consistency in a Hierarchal Deployment 187 Conclusion 193 6. Advanced Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Replication Architecture Basics 196 The Structure of the Relay Log 196 The Replication Threads 200 Starting and Stopping the Slave Threads 201 Running Replication over the Internet 202 Setting Up Secure Replication Using Built-in Support 204 Setting Up Secure Replication Using Stunnel 204 Finer-Grained Control over Replication 206 Information About Replication Status 206 Options for Handling Broken Connections 214 How the Slave Processes Events 215 Housekeeping in the I/O Thread 216 SQL Thread Processing 217 Slave Safety and Recovery 222 Syncing, Transactions, and Problems with Database Crashes 222 Table of Contents | vii Download from Library of Wow! eBook <www.wowebook.com> www.it-ebooks.info Rules for Protecting Nontransactional Statements 225 Multisource Replication 226 Row-Based Replication 229 Options for Row-Based Replication 230 Mixed-Mode Replication 231 Events for Handling Row-Based Replication 232 Event Execution 236 Events and Triggers 238 Filtering 240 Conclusion 241 Part II. Monitoring and Disaster Recovery 7. Getting Started with Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 Ways of Monitoring 246 Benefits of Monitoring 247 System Components to Monitor 247 Processor 248 Memory 249 Disk 250 Network Subsystem 251 Monitoring Solutions 252 Linux and Unix Monitoring 253 Process Activity 253 Memory Usage 259 Disk Usage 261 Network Activity 265 General System Statistics 266 Automated Monitoring with cron 268 Mac OS X Monitoring 268 System Profiler 268 Console 271 Activity Monitor 273 Microsoft Windows Monitoring 276 The Windows Experience 277 The System Health Report 278 The Event Viewer 281 The Reliability Monitor 283 The Task Manager 285 The Performance Monitor 285 Monitoring as Preventive Maintenance 288 Conclusion 288 viii | Table of Contents Download from Library of Wow! eBook <www.wowebook.com> www.it-ebooks.info [...]... using MySQL for various jobs, and also a lot of literature about high- availability systems Here is a list of books that we strongly recommend if you are going to work with MySQL: MySQL by Paul DuBois (Addison-Wesley) This is the reference to MySQL and consists of 1,200 pages (really!) packed with everything you want to know about MySQL (and probably a lot that you don’t want to know) High Performance MySQL, ... to Go from Here MySQL in the Cloud MySQL Replication and EC2 Best Practices for Using MySQL in EC2 Open Source Cloud Computing Conclusion 487 488 492 492 496 511 516 517 517 520 522 523 15 MySQL Cluster 525 What Is MySQL Cluster? Terminology and Components How Does MySQL Cluster Differ from MySQL? Typical Configuration Features of MySQL Cluster...www.it-ebooks.info 8 Monitoring MySQL 291 What Is Performance? MySQL Server Monitoring How MySQL Communicates Performance Performance Monitoring SQL Commands The mysqladmin Utility MySQL GUI Tools MySQL Administrator MySQL Query Browser Server Logs Third-Party Tools The MySQL Benchmark Suite Database Performance Measuring Database... and Distributed Data Architecture of MySQL Cluster How Data Is Stored Partitioning Transaction Management Online Operations Example Configuration Getting Started Starting a MySQL Cluster Testing the Cluster Shutting Down the Cluster Achieving High Availability System Recovery Node Recovery Replication Achieving High Performance Considerations for High Performance High Performance Best Practices Conclusion... Backups Conclusion 412 413 419 420 424 425 428 430 432 432 437 438 438 439 446 449 13 MySQL Enterprise 451 Getting Started with MySQL Enterprise Subscription Levels Installation Overview MySQL Enterprise Components MySQL Enterprise Server MEM MySQL Production Support Using MySQL Enterprise Installation Fixing Monitoring Agent Problems Monitoring Query... job working for a company that has just started using MySQL You will observe Joel as he learns his way around MySQL and tackles some of the toughest problems facing MySQL professionals We hope you find this aspect of the book entertaining Audience This book is for MySQL professionals We expect you to have a basic background in SQL, administering MySQL, and the operating system you are running We will... replication, disaster recovery, system monitoring, and other key topics of high availability See Chapter 1 for other books that offer useful background xvii Download from Library of Wow! eBook www.it-ebooks.info Organization of This Book This book is written in three parts Part I encompasses MySQL replication, including high availability and scale-out Part II examines monitoring and performance... framework used by MySQL Dr Lars Thalmann is the development manager and technical lead of the MySQL Replication and Backup team and has designed many of the replication and backup features He has worked with development of MySQL clustering, replication, and backup technologies We wrote this book to fill a gap we noticed among the many books on MySQL There are many excellent books on MySQL, but few that... creating parts of MySQL and working with it for many years Charles Bell is a senior developer working on replication and backup His interests include all things MySQL, database theory, software engineering, and agile development practices Dr Mats Kindahl is the lead developer for replication and a member of the MySQL Backup and Replication team He is the main architect and implementor of the MySQL row-based... 410 410 411 411 www.it-ebooks.info High Availability Versus Disaster Recovery Disaster Recovery The Importance of Data Recovery Backup and Restore Backup Utilities and OS-Level Solutions The InnoDB Hot Backup Application Physical File Copy The mysqldump Utility XtraBackup Logical Volume Manager Snapshots Comparison of Backup Methods Backup and MySQL Replication Backup and Recovery with Replication . www.it-ebooks.info www.it-ebooks.info MySQL High Availability www.it-ebooks.info www.it-ebooks.info MySQL High Availability Charles Bell, Mats Kindahl, and Lars Thalmann Beijing • Cambridge • Farnham • Köln • Sebastopol • Taipei • Tokyo www.it-ebooks.info MySQL. Monitoring 293 SQL Commands 294 The mysqladmin Utility 300 MySQL GUI Tools 302 MySQL Administrator 302 MySQL Query Browser 312 Server Logs 313 Third-Party Tools 316 The MySQL Benchmark Suite 318 Database. Cluster 546 Achieving High Availability 547 System Recovery 550 Node Recovery 551 Replication 552 Achieving High Performance 557 Considerations for High Performance 557 High Performance Best