MySQL Enterprise Solutions phần 9 ppsx

The backup taken with mysqldump can be restored on a fresh installation of MySQL by feeding the dump to the standard command line client; for example: mysql -u root -p secret < db1.sql Incremental Backup In addition to a regular (e.g., nightly) physical and logical backup, it is also advisable to perform frequent incremental backups. Incremental backups put very little demand on system resources. If performed frequently, incremental backups greatly limit the amount of data you can lose in the case of a cata- strophic failure. To perform incremental backups with MySQL, you should enable the log-bin option on the server being backed up. Set up a cron job that will periodically connect to the server, execute FLUSH LOGS, and then archive the logs created since the previous run. Once the logs have been archived, dispose of them with the PURGE MASTER LOGS TO command, giving it the first log you are not going to purge as a string argument. If you have replication slaves, make sure they are done processing the logs you are about to purge. For a more detailed discussion of PURGE MASTER LOGS TO, see Chapter 16. To restore an incremental backup, you must first restore the full physical or log- ical backup. You will then need mysqlbinlog, a binary log dump utility included in the MySQL distribution. Use mysqlbinlog to process all of the backup logs in the correct sequence, piping the output to the standard command line client. For example: mysqlbinlog master-bin.001 master-bin.002 master-bin.003 master-bin.004 | mysql -u root -p secret One disadvantage of the incremental backup is that if the amount of log data becomes large or the queries in it are slow, restoring it will take a long time. Another problem is that temporary table operations are not fully supported, and updates with user variables are not supported at all. If you plan to use incremental backup, it is recommended that you disable binary logging for the creation of temporary tables with SET SQL_LOG_BIN=0. Backup through Replication If you can afford to buy another machine, you can back up your data by repli- cating it. The slave thread can be periodically stopped (or you can just take the slave completely offline), and the data on the slave can be backed up to tape or Backup and Table Maintenance 316 other archival media without disturbing the live system. This method allows for more frequent full backups. It also provides you with a hot spare server that you can bring online with very little downtime and data loss. The disadvantages of this backup method are the cost of an extra machine and the few limitations on what kind of queries you can run on the master. See the replications caveats section in Chapter 16 for details. Table Maintenance With MyISAM tables, the most common sources of trouble are data fragmenta- tion and index corruption. Record fragmentation can occur when the record is dynamic length, and you have a high volume of insertions and deletions. Frag- mented records cause deteriorated performance. Another form of fragmenta- tion is unfilled holes from deleted records. This can happen if you delete a large number of records and do not insert new ones to occupy the freed space. To address the fragmentation issues on MyISAM tables, you should periodically run the OPTIMIZE TABLE command. Altering the table, repairing it (with REPAIR TABLE), or dumping it out with mysqldump and reloading the data will also defragment it. Theoretically, MyISAM table corruption should never happen, but in reality it does. Corruption may happen if the server is not shut down cleanly (e.g., some- body kills it with signal 9, the power is cut off, or the operating system panics), the operating system has a bug, the hardware is faulty, or MySQL itself hits a bug and fatally crashes. When a corrupted table is discovered, the server will give an error message to the client that says “Error from table handler”. To check if the table is corrupted, use the CHECK TABLE command. For example: CHECK TABLE account; It is also possible to specify a method of checking as an argument after the name of the table; for example: CHECK TABLE account QUICK; CHECK TABLE supports several checking methods: FAST (check only if the table was not closed properly), QUICK ( just check the index file), CHANGED (check only if changed since last check), MEDIUM ( check the index file and perform some basic sanity checks in the data file), and EXTENDED ( check both the index and the data file as thoroughly as possible). FAST and CHANGED options are used primarily during automated checks. The default method is MEDIUM. Table Maintenance 317 If you discover a corrupt table, you can repair it with the REPAIR TABLE command. For example: REPAIR TABLE account; Just like CHECK TABLE, REPAIR TABLE supports method options. For example: REPAIR TABLE account QUICK; REPAIR TABLE methods are QUICK (repair the index without looking at the data file), MEDIUM (use data file, create keys by sorting), and EXTENDED (use data file, update keys once per record insertion as opposed to creating them by sort). MySQL 4.0.2 and later supports the USE_FRM option, in which case only the data file and the table definition file are being used. Both CHECK TABLE and REPAIR TABLE produce a table-like output with four columns: Table, Op, Msg_type, and Msg_text. For example: Table Op Msg_type Msg_text test.t2 repair status OK Both commands can be invoked from any client language using the exact same semantics as you would when executing SELECT. It is also possible to check a table offline with myisamchk. It is recommended that you check your physical backup this way immediately after you have copied the data. Offline repair can be done with myisamchk -r. When using myisamchk, give it the path to the directory where the table files are located, followed by the name of the table. For example: myisamchk -r /var/data/mysql/sales/account will repair a table with the name account residing in the directory /var/data/mysql/sales/. myisamchk is capable of performing several other operations, documented in the output of myisamchk help (Listing 17.3): Backup and Table Maintenance 318 myisamchk Ver 1.53 for pc-linux-gnu at i686 By Monty, for your professional use This software comes with NO WARRANTY: see the PUBLIC for details. Description, check and repair of ISAM tables. Used without options all tables on the command will be checked for errors Usage: myisamchk [OPTIONS] tables[.MYI] Listing 17.3 myisamchk output. (continues) Table Maintenance 319 Global options: -#, debug= Output debug log. Often this is 'd:t:o,filename' -?, help Display this help and exit. -O, set-variable var=option Change the value of a variable. -s, silent Only print errors. One can use two -s to make myisamchk very silent -v, verbose Print more information. This can be used with describe and check. Use many -v for more verbosity! -V, version Print version and exit. -w, wait Wait if table is locked. Check options (check is the default action for myisamchk): -c, check Check table for errors -e, extend-check Check the table VERY throughly. Only use this in extreme cases as myisamchk should normally be able to find out if the table is ok even without this switch -F, fast Check only tables that hasn't been closed properly -C, check-only-changed Check only tables that has changed since last check -f, force Restart with -r if there are any errors in the table. States will be updated as with update-state -i, information Print statistics information about table that is checked -m, medium-check Faster than extended-check, but only finds 99.99% of all errors. Should be good enough for most cases -U update-state Mark tables as crashed if you find any errors -T, read-only Don't mark table as checked Repair options (When using -r or -o) -B, backup Make a backup of the .MYD file as 'filename-time.BAK' -D, data-file-length=# Max length of data file (when recreating data file when it's full) -e, extend-check Try to recover every possible row from the data file. Normally this will also find a lot of garbage rows; Don't use this option if you are not totally desperate. -f, force Overwrite old temporary files. -k, keys-used=# Tell MyISAM to update only some specific keys. # is a bit mask of which keys to use. This can be used to get faster inserts! -l, no-symlinks Do not follow symbolic links. Normally myisamchk repairs the table a symlink points at. -r, recover Can fix almost anything except unique keys that aren't unique. -n, sort-recover Force recovering with sorting even if the temporary file would be very big. Listing 17.3 myisamchk output. (continues) Backup and Table Maintenance 320 -o, safe-recover Uses old recovery method; Slower than '-r' but can handle a couple of cases where '-r' reports that it can't fix the data file. character-sets-dir= Directory where character sets are set-character-set=name Change the character set used by the index -t, tmpdir=path Path for temporary files -q, quick Faster repair by not modifying the data file. One can give a second '-q' to force myisamchk to modify the original datafile in case of duplicate keys -u, unpack Unpack file packed with myisampack. Other actions: -a, analyze Analyze distribution of keys. Will make some joins in MySQL faster. You can check the calculated distribution by using ' describe verbose table_name'. -d, description Prints some information about table. -A, set-auto-increment[=value] Force auto_increment to start at this or higher value. If no value is given, then sets the next auto_increment value to the highest used value for the auto key + 1. -S, sort-index Sort index blocks. This speeds up 'read-next' in applications -R, sort-records=# Sort records according to an index. This makes your data much more localized and may speed up things (It may be VERY slow to do a sort the first time!) Default options are read from the following files in the given order: /etc/my.cnf /var/lib/mysql/my.cnf ~/.my.cnf The following groups are read: myisamchk The following options may be given as the first argument: print-defaults Print the program argument list and exit no-defaults Don't read default options from any options file defaults-file=# Only read default options from the given file # defaults-extra-file=# Read this file after the global files are read Possible variables for option set-variable (-O) are: key_buffer_size current value: 520192 read_buffer_size current value: 262136 write_buffer_size current value: 262136 sort_buffer_size current value: 2097144 sort_key_blocks current value: 16 decode_bits current value: 9 Listing 17.3 myisamchk output. (continued) To perform a full check of all tables in a database or on the entire server, you can use the mysqlcheck utility included in the MySQL distribution. Below are some examples of mysqlcheck use: mysqlcheck -A -C auto-repair -u root -p secret -h db1 Connect to host db1 as user root with the password secret. Check all tables that have changed since the last time they were checked or that were not closed properly. Repair tables that failed the check. mysqlcheck -C auto-repair -u root -p secret -h db1 sales Check all tables, repairing when necessary, on server db1. Although it is possible to repair corrupted MyISAM tables, you should rarely need to. If you are experiencing frequent and unexplained corruptions, you should investigate the cause of the problem immediately. First, check your hardware and ensure that your operating system is using the latest patches. With Linux, watch out for non-standard kernel patches and new hardware driv- ers, as well as the known buggy ones. With other operating systems, ensure that all virtual memory and I/O functions are operating correctly. After the initial hardware and operating system sanity checks, if the corruption persists, chances are you have found a bug. If you are able to produce a test case that demonstrates the problem, chances are the bug will be fixed in the next release of MySQL. However, if you cannot reproduce it, your best bet is to try convert- ing your tables to InnoDB. MyISAM tables that you are willing to make read-only can be compressed with myisampack. For example: myisampack /var/lib/mysql/personnel/employee will compress the employee table residing in the personnel database assuming that the data directory is /var/lib/mysql. A table can be packed even without shutting the server down if you lock it. For example: LOCK TABLES personnel.employee READ; FLUSH TABLE personnel.employee; /* run myisampack here */ UNLOCK TABLES; Compression is done on a per-record basis and will be particularly effective if you have long records. As far as the issues of corruption are concerned, InnoDB tables do not require much maintenance. Part of the reason is that they are transactional, and even if the server crashes, in most cases the table space is brought into a consistent state during recovery. Another factor is that InnoDB tables have more safety checks. We have noticed that InnoDB tables have fewer stability problems than MyISAM tables, especially in applications with large tables. Table Maintenance 321 NOTE CHECK TABLE is supported for InnoDB tables, but REPAIR TABLE is not. MyISAM tables, especially the ones with many keys, should periodically be processed by ANALYZE TABLE. For example: ANALYZE TABLE t1; Using this command updates the statistics about key distribution, which helps the optimizer make better decisions concerning which key to use. Some queries may perform better if the records in the data file are in a certain order (for both MyISAM and InnoDB tables). If you have such queries, you may want to experiment with ALTER TABLE ORDER BY command, for example: ALTER TABLE book ORDER BY author; This will physically sort records in the book table in the order of the value of the author column. The ORDER BY expression does not have to be a key—it can be anything MySQL is capable of evaluating. Backup and Table Maintenance 322 O ne of the great advantages of MySQL is that its source is publicly avail- able. This means that the opportunities for customization are limited only by your programming skills. This task is not for the timid, though. MySQL source is not easy to understand at first glance, and many very good programmers have been frustrated in the past trying to do it. However, the chal- lenge can be conquered. This chapter serves as the first steppingstone for the brave souls who dare to defy the beast. Getting Started There could be several reasons why you want to look at the source of MySQL. You may be simply interested in learning how it works. Or you may want to fix a bug, or write and maintain a patch. Perhaps you would like to add a small fea- ture or try porting MySQL to a new platform. The nature of your goal will affect your method of getting the source. If you just want to fix something in the cur- rent version and do not plan to keep up with the current development or upgrade for a long time, a regular source distribution is the way to go. However, if you plan to introduce a change and then maintain it, keeping up with all the fixes and updates from MySQL AB, you should use the BitKeeper source tree. BitKeeper is a revision control tool produced by BitMover, Inc. that MySQL AB uses to manage MySQL source. The tool is available from www.bitmover.com. If you do not plan to make any internal proprietary modification to the MySQL source, the Open Logging license will allow you to use it for free. Open Logging Exploring MySQL Server Internals CHAPTER 18 323 means that whenever you commit your changes, the description is forwarded to the central BitMover repository and logged there. You can find the details of the Open Logging license at www.bitmover.com/Sales.Licencing.Free.html. If the Open Logging license does not meet your needs, you can obtain a commercial license from BitMover. If you’re interested, send a request to sales@bitmover.com. If you plan to use the BitKeeper source tree, download BitKeeper from www.bit- mover.com and install it on your system. Once you get it up and running, it is rec- ommended that you run bk helptool and study the basic BitKeeper concepts and commands. One of the strengths of BitKeeper is a very gentle learning curve—the concepts are easy to grasp, and the commands are intuitive. If you do not expect to use the BitKeeper tree and prefer to just use the raw source distribution, see Chapter 3 for the source build instructions. To access the files in a remote BitKeeper repository, you need to clone it, or in other words, make a duplicate copy of it locally. A group of changed files is called a changeset. Once a group of files have been modified, the user will com- mit the changeset. The committed changes can optionally be pushed to the central repository if no other user has pushed his or her changes that the cur- rent local repository does not have. Otherwise, those changes must be pulled first. Now let’s get the tree. If you want to be on the bleeding edge of the MySQL 4.0 branch, type bk clone bk://work.mysql.com:7001 To start with a release version as opposed to mid-release, type bk clone -rmysql-version bk://work.mysql.com:7001 mysql replacing version with the actual version you are trying to get. For example: bk clone -rmysql-4.0.2 bk://work.mysql.com:7001 mysql For 3.23, the hot mid-release tree is available with bk clone bk://work.mysql.com:7000 mysql and to get the release version: bk clone -rmysql-version bk://work.mysql.com:7000 mysql replacing version with the actual version. For example: bk clone -rmysql-3.23.52 bk://work.mysql.com:7000 mysql There is a small chance that the name of the repository host and the ports can change in the future. The most current information on accessing the BitKeeper tree is available at www.mysql.com/doc/en/Installing_source_tree.html. Exploring MySQL Server Internals 324 The clone command creates a directory called mysql with the MySQL source plus the internal revision control BitKeeper files. Your next step is to change the current directory to the root of the tree: cd mysql And check out the files: bk -r get -Sq Before you edit a file, you must use bk edit to notify BitKeeper that you are going to do so. You can tell it that you are going to edit everything by using bk -r edit. The first thing you should do—unless you want the MySQL world to know about every modification you make in your tree—is edit BitKeeper/triggers/post- commit and change the mail addresses to your company’s internal ones, or get rid of the script altogether with bk rm BitKeeper/triggers/post-commit. Otherwise, every time you commit a changeset, your changes will be e-mailed to inter- nals@lists.mysql.com and to an internal MySQL developer alias. We recommend that you subscribe to internals@lists.mysql.com. The primary reason is that it notifies you of updates made to the tree so that you know when to pull the changes. You can also follow some of the development discussions. To subscribe, visit www.mysql.com/documentation/lists.php and follow the instructions on the Web page. To be able to compile MySQL from the BitKeeper tree, you need a few extra tools in addition to the ones required for a regular source build: autoconf, automake, and bison. If you do not have those tools installed already, you can obtain them from ftp://ftp.gnu.org/pub/gnu/. You need autoconf to generate the configure script and a few supplementary files; automake generates Makefile.in files used by the configure script to generate Makefiles; and you need bison to generate the SQL parser code. Now that we have taken care of the logistics, let’s get down to business and do the build. In the tree you see the BUILD directory, which contains scripts for building MySQL on several different platforms and architectures. As of this writing, the following scripts are available: ■■ compile-alpha ■■ compile-alpha-ccc ■■ compile-alpha-cxx ■■ compile-alpha-debug ■■ compile-ia64-debug-max ■■ compile-pentium Getting Started 325 [...]... SQLCOM_CREATE_TABLE mysql_ create_table() sql_table.cc SQLCOM_ALTER_TABLE mysql_ alter_table() sql_table.cc SQLCOM_UPDATE mysql_ update() sql_update.cc SQLCOM_INSERT mysql_ insert() sql_insert.cc SQLCOM_DELETE mysql_ delete() sql_delete.cc SQLCOM_DROP_TABLE mysql_ rm_table() sql_table.cc SQLCOM_SHOW_PROCESSLIST mysqld_list_processes() sql_show.cc SQLCOM_SHOW_STATUS mysqld_show() sql_show.cc SQLCOM_SHOW_VARIABLES mysqld_show()... client API call mysql_ select_db() ■ ■ COM_QUERY: Executes a query, sent by the client API call mysql_ real_query() ■ ■ COM_FIELD_LIST: Lists fields in a table, sent by the client API call mysql_ list_fields() ■ ■ COM_CREATE_DB: Creates a database, sent by the client API call mysql_ create_db() 330 Exploring MySQL Server Internals ■ ■ COM_DROP_DB: Drops a database, sent by the client API call mysql_ drop_db()... suite ■ ■ sql-bench: The MySQL benchmark test suite written in Perl ■ ■ strings: The MySQL string library ■ ■ libmysql_r: An empty directory used for building the thread-safe client library ■ ■ libmysqld: The library for stand-alone MySQL applications with access to server functionality See www .mysql. com/doc/en/libmysqld_overview.html for details As of this writing, this is still a work in progress... record the result: /mysql- test-run local record custom If you do not have any fatal bugs in the code that will crash the server on the test, the command produces the following output: 348 Exploring MySQL Server Internals Installing Test Databases Removing Stale Files Installing Master Databases 020828 5:02: 09 /sql/mysqld: Shutdown Complete Installing Slave Databases 020828 5:02: 09 /sql/mysqld: Shutdown... tables with other table handlers ■ ■ client: Client utilities such as mysql, mysqladmin, mysqldump, and mysqlcheck ■ ■ readline: GNU readline library for the command-line client ■ ■ innobase: The InnoDB table handler implementation ■ ■ dbug: The debugging library ■ ■ heap: The HEAP (in-memory) table handler implementation 328 Exploring MySQL Server Internals ■ ■ isam: The ISAM (3.22 compatibility) table... which is defined in sql/mysqld.cc main() performs various initializations; then calls load_ defaults() (defined in mysys/default.c) to read the options from my.cnf, followed by more initializations and a call to get_options() (defined in sql/mysqld.cc as well) to parse the command-line arguments More initializations follow; then open the logs with a call to open_log() (defined in sql/mysqld.cc) once per... example: BUILD/compile-pentium-debug This builds a debugging version of MySQL on an x86 Linux or FreeBSD system After the build finishes, you should run the new binary through the test suite to make sure that you are starting out with a problem-free binary (or at least, no problems that the test suite checks for): cd mysql- test /mysql- test-run If all tests pass, you can start the new development Otherwise,... SLAVE] commands based on the argument bitmask Sent by the client API call mysql_ refresh() ■ ■ COM_SHUTDOWN: Shuts down the server ■ ■ COM_STATISTICS: Sends a short server statistics report string to the client Sent by the client API call mysql_ stat() ■ ■ COM_PROCESS_INFO: The equivalent of SHOW PROCESSLIST Sent by the client API call mysql_ list_processes() ■ ■ COM_CONNECT: The Command field of the THD object... equivalent of KILL Sent by the client API call mysql_ kill() ■ ■ COM_DEBUG: Dumps some debugging information into the error log The level of detail depends on the server compilation options Sent by the client API call mysql_ debug() ■ ■ COM_PING: Responds with the OK packet to the client, reassuring it that the server is still alive Sent by the client API call mysql_ ping() ■ ■ COM_TIME: A special value to... is handled in the case COM_QUERY statement, which does some white space pruning, and then passes the query on to the parser with a call to mysql_ parse(), which is also defined in sql/sql_parse.cc Tour of the Source 331 The mysql_ parse() method starts by calling mysql_ init_query(), which performs some initializations Then you initialize thd->query_length and proceed to call query_cache_send_result_to_client() . clone -rmysql-4.0.2 bk://work .mysql. com:7001 mysql For 3.23, the hot mid-release tree is available with bk clone bk://work .mysql. com:7000 mysql and to get the release version: bk clone -rmysql-version. edge of the MySQL 4.0 branch, type bk clone bk://work .mysql. com:7001 To start with a release version as opposed to mid-release, type bk clone -rmysql-version bk://work .mysql. com:7001 mysql replacing. clone -rmysql-version bk://work .mysql. com:7000 mysql replacing version with the actual version. For example: bk clone -rmysql-3.23.52 bk://work .mysql. com:7000 mysql There is a small chance that