Linux Server Hacks Volume Two phần 6 pot

found that taking a nightly snapshot of the logical volume that contains the users' home directories and automatically mounting it enables most users to satisfy their own restore requests by simply retrieving the original copies of lost or incorrectly modified files from the snapshot. This makes them happier and also lightens my workload. Not a bad combination! This hack explains how to create a snapshot of an existing volume and mount it, and provides some examples of how the snapshot preserves your original files when they are modified in the parent volume. 5.4.1. Kernel Support for Snapshots Snapshots of logical volumes are created and maintained with the help of the dm_snapshot filesystem driver. This is built as a loadable kernel module on most modern Linux distributions. If you cannot find this module or snapshots simply do not work on your system, cd to your kernel source directory (typically /usr/src/linux) and check your kernel configuration file to make sure this module is either built in or available as a kernel module, as in the following example: $ cd /usr/src/linux $ grep i DM-SNAPSHOT .config CONFIG_SM_SNAPSHOT=m In this case, the dm-snapshot driver is available as a loadable kernel module. If the value of the CONFIG_DM_SNAPSHOT configuration variable is n, this option is not available in your kernel. You will have to rebuild your kernel with this driver built in (a value of y) or as a loadable kernel module (a value of m) in order to take advantage of logical volume snapshots as discussed in this hack. Even if the dm_snapshot module is available on your system, you may need to manually load it using the standard modprobe command, as in the following example: # modprobe dm_snapshot 5.4.2. Creating a Snapshot This section explains how to create a snapshot of an existing filesystem. The filesystem that you are taking a snapshot of must reside on a logical volume, as shown by the presence of the device mapper directory in the following example: # df -Ph /test Filesystem Size Used Avail Use% Mounted on /dev/mapper/testvg-testvol 485M 18M 442M 4% /test Next we'll use the dd command to create a few sample files in the test volume for use in testing later in this hack: # dd if=/dev/zero of=/test/5M bs=1048576 count=5 5+0 records in 5+0 records out # dd if=/dev/zero of=/test/10M bs=1048576 count=10 10+0 records in 10+0 records out 204 204 To create a snapshot of the testvol volume, execute a command like the following: # lvcreate -s -L 100M -n testsnap /dev/testvg/testvol Logical volume "testsnap" created In this example, I allocated 100 MB for the snapshot. This means that we can make 100 MB in changes to the original volume before the snapshot is full. Snapshots eventually fill up because they are preserving old data, and there is no way to purge the files that it has preserved because it is a snapshot of another volume, not an original logical volume itself. Once a snapshot is 100% used, it becomes uselessyou must remove it and create a new snapshot. To confirm that the snapshot was created correctly, use the lvs command to display logical volume status information: # lvs LV VG Attr LSize Origin Snap% Move Copy% testsnap testvg swi-a- 100.00M testvol 0.02 testvol testvg owi-ao 500.00M 5.4.3. Mounting a Snapshot Having a snapshot of a logical volume is fairly useless unless you enable people to access it. To mount the sample testsnap snapshot, use a standard mount command such as the following: # mount /dev/testvg/testsnap /testsnap # df -Ph /test* Filesystem Size Used Avail Use% Mounted on /dev/mapper/testvg-testvol 485M 18M 442M 4% /test /dev/mapper/testvg-testsnap 485M 18M 442M 4% /testsnap Note that a snapshot volume always lives in the same volume group as the logical volume of which it is a copy. Just to be sure, you can use the ls command to verify that both the snapshot and the original volume are available: # ls -l /test total 15436 -rw-r r 1 root root 10485760 Apr 21 23:48 10M -rw-r r 1 root root 5242880 Apr 21 23:48 5M drwx 2 root root 12288 Apr 21 23:15 lost+found # ls -l /testsnap/ total 15436 -rw-r r 1 root root 10485760 Apr 21 23:48 10M -rw-r r 1 root root 5242880 Apr 21 23:48 5M drwx 2 root root 12288 Apr 21 23:15 lost+found 205 205 Now, create a 50-MB file in the /test filesystem and examine what happens to the /testsnap filesystem and the snapshot usage (using our favorite lvs command): # dd if=/dev/zero of=/test/50M bs=1048576 count=50 50+0 records in 50+0 records out # df -Ph /test* Filesystem Size Used Avail Use% Mounted on /dev/mapper/testvg-testvol 485M 68M 392M 15% /test /dev/mapper/testvg-testsnap 485M 18M 442M 4% /testsnap # ls -l /test total 66838 -rw-r r 1 root root 10485760 Apr 21 23:48 10M -rw-r r 1 root root 52428800 Apr 22 00:09 50M -rw-r r 1 root root 5242880 Apr 21 23:48 5M drwx 2 root root 12288 Apr 21 23:15 lost+found # ls -l /testsnap/ total 15436 -rw-r r 1 root root 10485760 Apr 21 23:48 10M -rw-r r 1 root root 5242880 Apr 21 23:48 5M drwx 2 root root 12288 Apr 21 23:15 lost+found # lvs LV VG Attr LSize Origin Snap% Move Copy% testsnap testvg swi-ao 100.00M testvol 50.43 testvol testvg owi-ao 500.00M Notice that the 50-MB file does not immediately show up in /testsnap, but some of the snapshot space has been used up (50.43%). Next, simulate a user accidentally removing a file by removing /test/10M and examine the results: # rm /test/10M rm: remove regular file `/test/10M'? y # df -Ph /test* Filesystem Size Used Avail Use% Mounted on /dev/mapper/testvg-testvol 485M 58M 402M 13% /test /dev/mapper/testvg-testsnap 485M 18M 442M 4% /testsnap Note that disk space utilization in your snapshot increased slightly: # lvs LV VG Attr LSize Origin Snap% Move Copy% testsnap testvg swi-ao 100.00M testvol 50.44 testvol testvg owi-ao 500.00M When using the lvs command after significant file operations, you may need to wait a few minutes for the data that lvs uses to be updated. If you now need to recover the file 10M, you can get it back by simply copying it out of the snapshot (to somewhere safe). Say goodbye to most of your restore headaches! 206 206 Remember, once the snapshot is 100% full, its contents can no longer be relied upon, because no new files can be written to it and it is therefore no longer useful for tracking recent updates to its parent volume. You should monitor the size of your snapshots and recreate them as needed. I find that recreating them once a week and remounting them keeps them up to date and also usually prevents "snapshot overflow." 5.4.4. See Also Snapshot section of the LVM HWOTO: http://www.tldp.org/HOWTO/LVM-HOWTO/snapshots_backup.html• "Create Flexible Storage with LVM" [Hack #46]• "Combine LVM and Software RAID" [Hack #47]• Lance Tost Hack 49. Clone Systems Quickly and Easily Once you've customized and fine-tuned a sample machine, you can quickly and easily deploy other systems based on its configuration by simply cloning it. Now that Linux is in widespread use, many businesses that don't want to roll their own Linux systems simply deploy out-of-the-box systems based on supported distributions from sources such as SUSE, Mandriva, Turbo Linux, and Red Hat. Businesses that need a wider array of system or application software than these distributions provide often spend significant effort adding this software to their server and desktop systems, fine-tuning system configuration files, setting up networking, disabling unnecessary services, and setting up their corporate distributed authentication mechanisms. All of this takes a fair amount of time to get "just right"it also takes time to replicate on multiple systems and can be a pain to recreate if this becomes necessary. You do have backups, don't you? To speed up deploying multiple essentially identical systems, the classic Unix approach that I used to take in the "bad old days" was to purchase large numbers of disks that were the same size, use the Unix dd utility to clone system disks containing my tricked out systems to new disks, and then deploy the cloned disks in each new system of the specified type. This still works, but the downside of this approach is that the dd utility copies every block on a disk, regardless of whether it's actually in use or not. This process can take hours, even for relatively small disks, and seems interminable when cloning today's larger (200-GB and up) drives. Thanks to the thousands of clever people in the open source community, faster and more modern solutions to this classic problem are now readily available for Linux. The best known are Ghost for Linux (a.k.a. g4l, http://sourceforge.net/projects/g4l/), which takes its name from the commercial Ghost software package from Symantec (formerly Norton) for Windows systems, and partimage, the popular GNU Partition Image application (http://www.partimage.org). Both of these are open source software packages that are designed to create compressed images of partitions on your systems and make it easy for you to restore these partition images on different drives. The Ghost for Linux software is largely targeted for use on bootable system disks and provides built-in support for transferring the compressed filesystem or disk images that it creates to central servers using FTP. It is therefore extremely useful when you need to boot and back up a system that won't boot on its own. This hack focuses on partimage because it is easier to build, deploy, and use as an application on a system that is currently running. Of course, you have to have enough local disk space to store the compressed filesystem images, but that's easy enough to dig up nowadays. Like Ghost for Linux, you can't use partimage to create an image of a filesystem that is currently mounted, because a mounted filesystem may 207 207 change while the image is being created, which would be "a bad thing." The ability to create small, easily redeployed partition images is growing in popularity thanks to virtual machine software such as Xen, where each virtual machine requires its own root filesystem. Though many people use a loopback filesystem for this, those consume memory on both the host and client. partimage makes it easy to clone existing partitions that have been customized for use with Xen, which is something you can easily do while your system is running if you have already prepared a Xen root filesystem on its own partition. partimage easily creates optimal, compressed images of almost any type of filesystem that you'd find on a Linux system (and even many that you would not). It supports ext2fs/ext3fs, FAT16/32, HFS, HPFS, JFS, NTFS, ReiserFS, UFS, and XFS partitions, though its support for both HFS (the older Mac OS filesystem) and NTFS (the Windows filesystem de jour) is still experimental. 5.5.1. Building partimage partimage is easy enough to build, but it has a fair number of dependencies. To build partimage, you must build or already have installed the following libraries: liblzo Used for fast compression. Available from http://www.oberhumer.com/opensource/lzo. libmcrypt An encryption library required for newer versions of partimage. Available from http://mcrypt.hellug.gr/lib/temp0138.html. libnewt A text-oriented, semi-graphical interface. Available from http://www.partimage.org/deps/newt-0.50.tar.gz. libslang An internationalization package used by newt. Available from http://www.s-lang.org. libssl A Secure Sockets Layer library required for newer versions of partimage. Available from http://www.openssl.org. Must be built in shared mode after configuring it using the following configure command: # ./configure prefix=/usr -shared libz 208 208 Used for gzip compression. Available from http://www.zlib.org. libbz2 Necessary for bzip2 compression. Available at http://sources.redhat.com/bzip2. Once you've built and installed any missing libraries, you can configure and compile partimage using the standard commands for building most modern open source software: # ./configure && make install The fun begins once the build and installation is complete. The final product of the make command is two applications: partimage, which is the application that you run on a system to create an image of an existing partition; and partimaged, which is a daemon that you can run on a system in order to be able to save partition images to it over the network, much like the built-in FTP support provided by Ghost for Linux. At the time that this book was written, the latest version of partimage was 0.6.4, which was not 64-bit clean and could not be compiled successfully on any of my 64-bit systems. If you need to run partimage on a 64-bit system and no newer version is available by the time that you read this (or if you're just in a hurry), you can always download precompiled static binaries for your Linux system. Precompiled static binaries are available from the partimage download page listed at the end of this hack. 5.5.2. Cloning Partitions Using partimage Using partimage to create a copy of an existing unmounted partition is easy. Because partimage needs raw access to partitions, you must execute the partimage command as root or via sudo. As shown in Figure 5-1, the initial partimage screen enables you to select the partition of which you want to create an image, the full pathname to which you want to save the partition image, and the operation that you want to perform (in this case, saving a partition into a file). To move to the next screen, press F5 or use the Tab key to select the Next button and press Enter. Figure 5-1. Selecting a partition to image and specifying the output file 209 209 The second partimage backup screen, shown in Figure 5-2, enables you to specify the compression mechanism that you want to use in the image file. Here you can specify that you want to check the consistency of the partition that you are imaging before creating the partition image file, which is always a good idea since you don't want to clone an inconsistent filesystem. You can also optionally specify that you want to add a descriptive comment to the file, which is often a good idea if you are going to be saving and working with a large number of partition image files. You can also specify what partimage should do after the image file has been created: wait for input, quit automatically, halt the machine, and so on. (The latter is probably only useful if you've booted from a rescue disk containing partimage in order to image one of the system partitions on your primary hard drive.) Press F5 to proceed to the next screen. Note that the existing type of the partition in /dev/hdb6 is ReiserFS. The existing type of the target partition and the size of the partition that was backed up do not matter (as long as the target partition can hold the uncompressed contents of the partition image file). When restoring a partition image, the partition that is being populated with its contents is automatically created using the same type of filesystem as was used in the filesystem contained in the image file, but using all available space on the target partition. If you specified that you wanted to check the consistency of the filesystem before imaging it, partimage checks the filesystem and displays a summary screen that you can close after reviewing it by pressing Enter. partimage then proceeds to create an image file of the specified partition, as shown in Figure 5-3, displaying a summary screen when the image has been successfully created. If you specified Wait (i.e., wait for inputthe default) as the action to perform after creating the image file, you will have to press Enter to close the summary screen and exit partimage. Figure 5-2. Specifying compression methods and other options 210 210 Figure 5-3. Creating the partition image file 5.5.3. Restoring Partitions Using partimage Using partimage to restore a partition image to an existing partition is even simpler than creating the image in the first place. The initial partimage restore screen, shown in Figure 5-4, is the same as that shown in Figure 5-1. It enables you to identify the partition to which you want to restore the partition image, the name of the image file that you want to restore from, and the action that you want to perform (in this case, restoring a partition from a file). To move to the next screen, press F5 or use the Tab key to select the Next button and press Enter. 211 211 Figure 5-4. Selecting a partition to restore to and the partition image file The second partimage restore screen, shown in Figure 5-5, enables you to run a consistency check by performing a dry run of restoring from the image file and also enables you to zero out unused blocks on the target filesystem when it is created. As with the image-creation process, you can also specify what partimage should do after the image file has been restored: wait for input, quit automatically, halt or reboot the machine, and so on. Press F5 to proceed to the next screen. partimage then proceeds to restore the partition image file to the specified partition, as shown in Figure 5-6, displaying a summary screen by default when the image has been successfully restored. If you specified Wait (i.e., wait for inputthe default) as the action to perform after creating the image file, you will have to press Enter to close the summary screen and exit partimage. Figure 5-5. Specifying restore options and completion behavior 212 212 Figure 5-6. Restoring the partition image 5.5.4. Summary Creating partition image files of customized, optimized, and fine-tuned desktop and server partitions provides a quick and easy way of cloning those systems to new hardware. You can always clone partitions containing applications, such as /opt,/var,/usr, and /usr/local. (Your actual partition scheme is, of course, up to you.) If your new systems have the same devices as the system on which the image file was created, you can even easily copy preconfigured system partitions such as /boot and / itself. Either way, applications such as partimage can save you lots of time in configuring additional hardware by enabling you to reuse your existing customizations as many times as you want to. 5.5.5. See Also "Make Disk-to-Disk Backups for Large Drives" [Hack #50]• Ghost for Linux home page: http://sourceforge.net/projects/g4l/• Ghost for Linux download page: ftp://fedoragcc.dyndns.org• partimage home page: http://www.partimage.org• partimage download page: http://www.partimage.org/download.en.html• System Rescue CD home page: http://www.sysresccd.org• Hack 50. Make Disk-to-Disk Backups for Large Drives Today's hard drives are large enough that you could spend the rest of your life backing them up to tape. Putting drive trays in your servers and using removable drives as a backup destination provides a modern solution. 213 213 [...]... Warshawsky Chapter 6 Standardizing, Sharing, and Synchronizing Resources Section 6. 1 Hacks 566 2: Introduction Hack 56 Centralize Resources Using NFS Hack 57 Automount NFS Home Directories with autofs Hack 58 Keep Filesystems Handy, but Out of Your Way Hack 59 Synchronize root Environments with rsync Hack 60 Share Files Across Platforms Using Samba Hack 61 Quick and Dirty NAS Hack 62 Share Files and... top-secret 80 2005-07-04 16: 02 drwxr-xr-x 8 root root 184 2005-07-04 15:57 -rw-r r 1 wvh top-secret 53 86 2005-07-04 16: 02 wmd_overview.sxw At this point, creating any file in this directory gives it the same group ownership as the directory, as in the following example: touch $ testfile.txt ls -al$ total 8 drwxrws - 2 drwxr-xr-x 8 -rw-r r 1 -rw-rw-r 1 ts top-secret 112 2005-07-04 16: 06 root root 184 2005-07-04... cpio, and pax If you're using logical volumes, "Create a Copy-on-Write Snapshot of an LVM Volume" [Hack #48] explained how to create a copy-on-write snapshot of a volume that automatically picks up a copy of any file that's modified on its parent volume That's fine for providing a mechanism that enables people to recover copies of files that they've just deleted, 215 2 16 which satisfies the majority of... $ testfile.txt ls -al$ total 8 drwxrws - 2 drwxr-xr-x 8 -rw-r r 1 -rw-rw-r 1 ts top-secret 112 2005-07-04 16: 06 root root 184 2005-07-04 15:57 wvh top-secret 0 2005-07-04 16: 06 testfile.txt wvh top-secret 53 86 2005-07-04 16: 02 wmd_overview.sxw Because of the umask settings discussed earlier, this file was created with a mode that made it both user- and group-writable, which is exactly what you want... in a Linux filesystem These are a special type of metadata, which is the term for data about data, such as modification and access times, user and group ownership, protections, and so on Extended attributes can be associated with any object in a Linux filesystem that has an inode The names of extended attributes can be up to 2 56 bytes long, are usually standard ASCII text, and (like standard Linux. .. newfile=`touch $1; chmod 66 0 $1` Any of these solutions works fine if the group that you want to be able to share files with is the group that you initially belong to when you log in, known as your login group Linux enables users to belong to multiple groups at the same time, in order to let people work on multiple projects that are protected at the group level For the purposes of creating files, Linux users function... so that the file is also writable by group members, which can be done with either of the following commands: chmod $ 66 0 juser_comments.txt chmod $ g+w,o-r juser_comments.txt You find out a user's default umask setting by issuing the umask command, which is a built-in command in most Linux shells By default, most users' umasks are set to 0022 so that newly created files are writable only by their owners,... users leave or move between projects ACLs, which are supported in most modern Linux distributions, eliminate this hassle by providing a fine-grained set of permissions that users can impose on their own directories, going far beyond the permissions and protections provided by standard Linux groups Simply put, an ACL is a list of Linux users and/or groups and the access rights that they have to a specific... special-purpose Linux groups ACLs as implemented on Linux systems today are defined by the draft Portable Operating System Interface (POSIX) standard 1003.1e, draft 17, from the Institute of Electrical and Electronics Engineers (IEEE) This is not an official standard, but it is publicly available and has become the foundation for ACL implementations for modern operating systems such as Linux (See the... modern Linux distributions provide support for ACLs in the default kernels that they deliver If you have access to the configuration file used to build your kernel, you can use the grep utility to check to make sure that the POSIX_ACL configuration variable associated with the types of filesystems that you are using is set to y, as in the following example: grep POSIX_ACL /boot/config-2 .6. 8-24. 16- default . top-secret 112 2005-07-04 16: 06 . drwxr-xr-x 8 root root 184 2005-07-04 15:57 -rw-r r 1 wvh top-secret 0 2005-07-04 16: 06 testfile.txt -rw-rw-r 1 wvh top-secret 53 86 2005-07-04 16: 02 wmd_overview.sxw Because. over the network, much like the built-in FTP support provided by Ghost for Linux. At the time that this book was written, the latest version of partimage was 0 .6. 4, which was not 64 -bit clean. snapshot volume always lives in the same volume group as the logical volume of which it is a copy. Just to be sure, you can use the ls command to verify that both the snapshot and the original volume