ruby cookbook recipes for object oriented scripting (2nd ed ) carlson richardson 2015 03 25 (draft)

SECOND EDITION Ruby Cookbook, 2E Lucas Carlson and Leonard Richardson Ruby Cookbook, 2E, Second Edition by Lucas Carlson and Leonard Richardson Copyright © 2010 Lucas Carlson and Leonard Richardson All rights reserved Printed in the United States of America Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472 O’Reilly books may be purchased for educational, business, or sales promotional use Online editions are also available for most titles (http://my.safaribooksonline.com) For more information, contact our corporate/ institutional sales department: 800-998-9938 or corporate@oreilly.com Editors: Brian Anderson and Allyson MacDonald Production Editor: FIX ME! Copyeditor: FIX ME! Proofreader: FIX ME! November 2014: Indexer: FIX ME! Cover Designer: Karen Montgomery Interior Designer: David Futato Illustrator: Rebecca Demarest Second Edition Revision History for the Second Edition: 2014-04-30: First Early Release revision See http://oreilly.com/catalog/errata.csp?isbn=9781449373719 for release details Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc !!FILL THIS IN!! and related trade dress are trademarks of O’Reilly Media, Inc Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks Where those designations appear in this book, and O’Reilly Media, Inc was aware of a trademark claim, the designations have been printed in caps or initial caps While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein ISBN: 978-1-449-37371-9 [?] For Yoscelina, my muse and inspiration for everything great I have ever accomplished For Hugh and Valentina, the most incredible miracles ever For Tess, who sat by me the whole time —Lucas Carlson For Sumana —Leonard Richardson Table of Contents Preface xvii Ruby 2.1 1.1 What’s Different Between Ruby 1.8 and 2.1? 1.2 YARV (Yet Another Ruby VM) Bytecode Interpreter 1.3 Syntax Changes 1.4 Keyword Arguments 1.5 Performance Enhancements 1.6 Refinements 1.7 Debugging with DTrace and TracePoint 1.8 Module Prepending 1.9 New Methods 1.10 New Classes 1.11 New Standard Libraries 1.12 What’s Next? 11 12 14 15 17 18 21 23 25 Strings 27 2.1 Building a String from Parts 2.2 Substituting Variables into Strings 2.3 Substituting Variables into an Existing String 2.4 Reversing a String by Words or Characters 2.5 Representing Unprintable Characters 2.6 Converting Between Characters and Values 2.7 Converting Between Strings and Symbols 2.8 Processing a String One Character at a Time 2.9 Processing a String One Word at a Time 2.10 Changing the Case of a String 2.11 Managing Whitespace 2.12 Testing Whether an Object Is String-Like 31 32 35 37 38 41 42 43 45 46 48 50 v 2.13 Getting the Parts of a String You Want 2.14 Word-Wrapping Lines of Text 2.15 Generating a Succession of Strings 2.16 Matching Strings with Regular Expressions 2.17 Replacing Multiple Patterns in a Single Pass 2.18 Validating an Email Address 2.19 Classifying Text with a Bayesian Analyzer 51 52 54 57 59 61 64 Numbers 67 3.1 Parsing a Number from a String 3.2 Comparing Floating-Point Numbers 3.3 Representing Numbers to Arbitrary Precision 3.4 Representing Rational Numbers 3.5 Generating Random Numbers 3.6 Converting Between Numeric Bases 3.7 Taking Logarithms 3.8 Finding Mean, Median, and Mode 3.9 Converting Between Degrees and Radians 3.10 Multiplying Matrices 3.11 Solving a System of Linear Equations 3.12 Using Complex Numbers 3.13 Simulating a Subclass of Fixnum 3.14 Doing Math with Roman Numbers 3.15 Generating a Sequence of Numbers 3.16 Generating Prime Numbers 3.17 Checking a Credit Card Checksum 68 71 74 77 79 80 82 84 87 88 93 95 98 102 107 110 115 Date and Time 117 4.1 Finding Today’s Date 4.2 Parsing Dates, Precisely or Fuzzily 4.3 Printing a Date 4.4 Iterating Over Dates 4.5 Doing Date Arithmetic 4.6 Counting the Days Since an Arbitrary Date 4.7 Converting Between Time Zones 4.8 Checking Whether Daylight Saving Time Is in Effect 4.9 Converting Between Time and DateTime Objects 4.10 Finding the Day of the Week 4.11 Handling Commercial Dates 4.12 Running a Code Block Periodically 4.13 Waiting a Certain Amount of Time vi | Table of Contents 120 124 127 131 133 135 137 139 141 144 146 147 149 4.14 Adding a Timeout to a Long-Running Operation 152 Arrays 155 5.1 Iterating Over an Array 5.2 Rearranging Values Without Using Temporary Variables 5.3 Stripping Duplicate Elements from an Array 5.4 Reversing an Array 5.5 Sorting an Array 5.6 Ignoring Case When Sorting Strings 5.7 Making Sure a Sorted Array Stays Sorted 5.8 Summing the Items of an Array 5.9 Sorting an Array by Frequency of Appearance 5.10 Shuffling an Array 5.11 Getting the N Smallest Items of an Array 5.12 Building Up a Hash Using Injection 5.13 Extracting Portions of Arrays 5.14 Computing Set Operations on Arrays 5.15 Partitioning or Classifying a Set 157 161 163 164 165 167 168 174 175 177 179 181 183 187 189 Hashes 195 6.1 Using Symbols as Hash Keys 6.2 Creating a Hash with a Default Value 6.3 Adding Elements to a Hash 6.4 Removing Elements from a Hash 6.5 Using an Array or Other Modifiable Object as a Hash Key 6.6 Keeping Multiple Values for the Same Hash Key 6.7 Iterating Over a Hash 6.8 Iterating Over a Hash in Insertion Order 6.9 Printing a Hash 6.10 Inverting a Hash 6.11 Choosing Randomly from a Weighted List 6.12 Building a Histogram 6.13 Remapping the Keys and Values of a Hash 6.14 Extracting Portions of Hashes 6.15 Searching a Hash with Regular Expressions 198 199 201 203 205 207 208 211 212 214 216 218 220 221 222 Files and Directories 225 7.1 Checking to See If a File Exists 7.2 Checking Your Access to a File 7.3 Changing the Permissions on a File 7.4 Seeing When a File Was Last Used Problem 7.5 Listing a Directory 228 230 232 235 237 Table of Contents | vii 7.6 Reading the Contents of a File 7.7 Writing to a File 7.8 Writing to a Temporary File 7.9 Picking a Random Line from a File 7.10 Comparing Two Files 7.11 Performing Random Access on “Read-Once” Input Streams 7.12 Walking a Directory Tree 7.13 Locking a File 7.14 Backing Up to Versioned Filenames 7.15 Pretending a String Is a File 7.16 Redirecting Standard Input or Output 7.17 Processing a Binary File 7.18 Deleting a File 7.19 Truncating a File 7.20 Finding the Files You Want 7.21 Finding and Changing the Current Working Directory 240 244 245 247 248 252 254 257 260 263 266 268 272 273 275 277 Code Blocks and Iteration 279 8.1 Creating and Invoking a Block 8.2 Writing a Method That Accepts a Block 8.3 Binding a Block Argument to a Variable 8.4 Blocks as Closures: Using Outside Variables Within a Code Block 8.5 Writing an Iterator Over a Data Structure 8.6 Changing the Way an Object Iterates 8.7 Writing Block Methods That Classify or Collect 8.8 Stopping an Iteration 8.9 Looping Through Multiple Iterables in Parallel 8.10 Hiding Setup and Cleanup in a Block Method 8.11 Coupling Systems Loosely with Callbacks 282 284 287 289 291 294 296 298 300 304 307 Objects and Classes 311 9.1 Managing Instance Data 9.2 Managing Class Data 9.3 Checking Class or Module Membership 9.4 Writing an Inherited Class 9.5 Overloading Methods 9.6 Validating and Modifying Attribute Values 9.7 Defining a Virtual Attribute 9.8 Delegating Method Calls to Another Object 9.9 Converting and Coercing Objects to Different Types 9.10 Getting a Human-Readable Printout of Any Object 9.11 Accepting or Passing a Variable Number of Arguments viii | Table of Contents 313 316 319 321 324 326 328 329 332 337 339 9.12 Keyword Arguments 9.13 Calling a Superclass’s Method 9.14 Creating an Abstract Method 9.15 Freezing an Object to Prevent Changes 9.16 Making a Copy of an Object 9.17 Declaring Constants 9.18 Implementing Class and Singleton Methods 9.19 Controlling Access by Making Methods Private 341 343 346 348 351 354 356 358 10 Modules and Namespaces 363 10.1 Simulating Multiple Inheritance with Mixins 10.2 Extending Specific Objects with Modules 10.3 Mixing in Class Methods 10.4 Implementing Enumerable: Write One Method, Get 48 Free 10.5 Avoiding Naming Collisions with Namespaces 10.6 Automatically Loading Libraries as Needed 10.7 Including Namespaces 10.8 Initializing Instance Variables Defined by a Module 10.9 Automatically Initializing Mixed-In Modules 10.10 Prepending Modules 364 367 369 371 375 376 378 380 381 384 11 Reflection and Metaprogramming 387 11.1 Finding an Object’s Class and Superclass 11.2 Listing an Object’s Methods 11.3 Listing Methods Unique to an Object 11.4 Getting a Reference to a Method 11.5 Fixing Bugs in Someone Else’s Class 11.6 Listening for Changes to a Class 11.7 Checking Whether an Object Has Necessary Attributes 11.8 Responding to Calls to Undefined Methods 11.9 Automatically Initializing Instance Variables 11.10 Avoiding Boilerplate Code with Metaprogramming 11.11 Metaprogramming with String Evaluations 11.12 Evaluating Code in an Earlier Context 11.13 Undefining a Method 11.14 Aliasing Methods 11.15 Doing Aspect-Oriented Programming 11.16 Enforcing Software Contracts 388 389 392 393 396 397 400 402 406 408 410 413 414 417 420 423 12 XML and HTML 429 12.1 Checking XML Well-Formedness 12.2 Extracting Data from a Document’s Tree Structure 430 432 Table of Contents | ix Find.rename('./') { |file| file + '.txt' unless file.index('.') } # Renaming /tmp_files/subdir/i_am_shouting to /tmp_files/subdir/i_am_shouting.txt # Renaming /tmp_files/subdir/i_am_SHOUTING to /tmp_files/subdir/i_am_SHOUTING.txt # # => [] Discussion Renaming files in bulk is a very common operation, but there’s no standard commandline application to it because renaming operations are best described algorithmically The Find.rename method makes several simplifying assumptions It assumes that you want to rename regular files and not directories It assumes that you can decide on a new name for a file based solely on its filename, not on its full path It assumes that you’ll handle in some other way the files it couldn’t rename Another implementation might make different assumptions: it might yield both path and name, and use autoversioning to guarantee that it can rename every file, although not necessary to the exact filename returned by the code block It all depends on your needs Perhaps the most common renaming operation is modifying the extensions of files Here’s a method that uses Find.rename to make this kind of operation easier: module Find def change_extensions(extension_mappings, *paths) rename(*paths) |file| base, extension = file.split(/(.*)\./)[1 2] new_extension = extension extension_mappings.each |re, ext| if re.match(extension) new_extension = ext break end end "#{base}.#{new_extension}" end end module_function(:change_extensions) end This code uses Find.change_extensions to normalize a collection of images All JPEG files will be given the extension “.jpg”, all PNG files the extension “.png”, and all GIF files the extension “.gif ” Again, we’ll create some dummy image files to test: tmp_dir = 'tmp_graphics' Dir.mkdir(tmp_dir) 25.6 Renaming Files in Bulk | 927 ['my.house.jpeg', 'Construction.Gif', 'DSC1001.JPG', '52.PNG'].each |f| FileUtils.touch(File.join(tmp_dir, f)) end Now, let’s rename: Find.change_extensions({/jpe?g/i => 'jpg', /png/i => 'png', /gif/i => 'gif'}, tmp_dir) # Renaming tmp_graphics/52.PNG to tmp_graphics/52.png # Renaming tmp_graphics/DSC1001.JPG to tmp_graphics/DSC1001.jpg # Renaming tmp_graphics/Construction.Gif to tmp_graphics/Construction.gif # Renaming tmp_graphics/my.house.jpeg to tmp_graphics/my.house.jpg See Also • Some Unix installations come with a program or Perl script called rename, which can your renaming if you can represent it as a string substitution or a regular expression; you may not need anything else • Recipe 7.14, “Backing Up to Versioned Filenames” • Recipe 7.20, “Finding the Files You Want” 25.7 Finding Duplicate Files Problem You want to find the duplicate files that are taking up all the space on your hard drive Solution The simple solution is to group the files by size and then by their MD5 checksum Two files are presumed identical if they have the same size and MD5 sum The following program takes a list of directories on the command line, and prints out all sets of duplicate files You can pass a different code block into each_set_of_ dupli cates for different behavior: for instance, to prompt the user about which of the du‐ plicates to keep and which to delete #!/usr/bin/ruby # find_duplicates.rb require 'find' require 'digest/md5' def each_set_of_duplicates(*paths) 928 | Chapter 25: System Administration sizes = {} Find.find(*paths) |f| (sizes[File.size(f)] ||= []) md5s = {} files.each |f| digest = Digest::MD5.hexdigest(File.read(f)) (md5s[digest] ||= []) } end end each_set_of_ duplicates(*ARGV) |f| puts " Duplicates: #{f.join(", ")}" end Discussion This is one task that can’t be handled with a simple Find.find code block, because it’s trying to figure out which files have certain relationships to each other Find.find takes care of walking the file tree, but it would be very inefficient to try to make a single trip through the tree and immediately spit out a set of duplicates Instead, we group the files by size and then by their MD5 checksum The MD5 checksum is a short binary string used as a stand-in for the contents of a file It’s commonly used to verify that a huge file was downloaded without errors It’s not impossible for two different files to have an MD5 sum, but unless someone is deliberately trying to trick you, it’s almost impossible to have two files with the same size and the same MD5 sum Calculating a MD5 sum is very expensive: it means performing a mathematical calcu‐ lation on the entire contents of the file Grouping the files by size beforehand greatly reduces the number of sums that must be calculated, but that’s still a lot of I/O Even if two similarly sized files differ in the first byte, the code above will read the entire files Here’s a different version of the same program that takes an incremental approach like that seen in Recipe 7.10 When it thinks a set of files might contain duplicates, it makes repeated calls to a method called eliminate_non_duplicates The duplicates are yiel‐ ded and the nonduplicates discarded over the course of these calls #!/usr/bin/ruby # find_duplicates2.rb require 'find' 25.7 Finding Duplicate Files | 929 BLOCK_SIZE = 1024*8 def each_set_of_duplicates(*paths, &block) sizes = Hash.new {|h, k| h[k] = [] } Find.find(*paths) { |f| sizes[File.size(f)] offset = files = [files] while !files.empty? && offset = size # We know these are duplicates yield files else # We suspect these are duplicates, but we need to compare # more blocks of data possible_duplicates true) return to end # This method copied from "Backing Up to Versioned Filenames" 25.8 Automating Backups | 931 class File def File.versioned_filename(base, first_suffix=".0") suffix = nil filename = base while File.exists?(filename) suffix = (suffix ? suffix.succ : first_suffix) filename = base + suffix end return filename end end # Create a dummy directory Dir.mkdir('recipes') # And back it up backup('recipes', '/tmp/backup') # => "/tmp/backup/recipes-20061031" backup('recipes', '/tmp/backup') # => "/tmp/backup/recipes-20061031.0" backup('recipes', '/tmp/backup', '-%Y%m%d-%H.%M.%S') # => "/tmp/backup/recipes-20061031-20.48.56" Discussion The backup method recursively copies the contents of a directory into another directory, possibly on another filesystem It uses the time-based scheme you specify along with versioned_filename to uniquely name the destination directory As written, the backup method uses a lot of space: every time you call it, it creates an entirely new copy of every file in the source directory Fortunately, the technique has many variations Instead of copying the files, you can make a timestamped tarball with the techniques from Recipe 13.10 You can archive the files to another computer with the techniques from Recipe 15.11 (although to save space, you should use the rsync program instead) You could even automatically check your work into a version control system every so often; this works better with text than with binary files See Also • Recipe 7.14, “Backing Up to Versioned Filenames” • Recipe 13.10, “Compressing and Archiving Files with Gzip and Tar” • Recipe 15.11, “Copying a File to Another Machine” 932 | Chapter 25: System Administration 25.9 Normalizing Ownership and Permissions in User Directories Problem You want to make make sure your users’ home directories don’t contain world-writable directories, directories owned by other users, or other potential security problems Solution Use the etc library to look up a user’s home directory and UID from the username Then use Find.find to walk the directory trees, and File methods to check and modify access to each file We are looking out for any case where one user’s home directory can be modified by some other user Whenever we find such a case, we fix it with a File.chmod or File.chown call In this program, the actual calls are commented out, so that you don’t accidentally change your permissions when you just want to test out the program #!/usr/bin/ruby -w # normalize_homes.rb require 'etc' require 'find' require 'optparse' def normalize_home(pwd_entry, maximum_perms=0775, dry_run=true) uid, home = pwd_entry.uid, pwd_entry.dir username = pwd_entry.name puts "Scanning #{username}'s home of #{home}." Find.find(home) |f| next unless File.exists? f stat = File.stat(f) file_uid, file_gid, mode = stat.uid, stat.gid, stat.mode The most obvious thing we want to check is whether the user owns every file in their home directory With occasional exceptions (such as files owned by the web server), a user should own the files in his or her home directory: # Does the user own the file? if file_uid != uid begin current_owner = Etc.getpwuid(file_uid).name rescue ArgumentError # No such user; just use UID current_owner = "uid #{file_uid}" end puts " CHOWN #{f}" 25.9 Normalizing Ownership and Permissions in User Directories | 933 puts " Current owner is #{current_owner}, should be #{username}" # File.chown(uid, nil, f) unless dry_run end A less obvious check involves the Unix group that owns the file A user can let other people work on a file in their home directory by giving ownership to a user group But you can only give ownership to a group if you’re a member of that group If a user’s home directory contains a file owned by a group the user doesn’t belong to, something fishy is probably going on # Does the user belong to the group that owns the file? begin group = Etc.getgrgid(file_gid) group_name = group.name rescue ArgumentError # No such group group_name = "gid #{file_gid}" end unless group && (group.mem.member?(username) || group.name == username) puts " CHGRP #{f}" puts " Current group is #{group_name}, and #{username} doesn't belong." # File.chown(nil, uid, f) unless dry_run end Finally, we’ll check each file’s permissions and make sure they are no more permissive than the value passed in as maximum_perms The default value of 0775 allows any kind of file except a world-writable file If normalize_home finds a world-writable file, it will flip the world-writable bit and leave the rest of the permissions alone: # Does the file have more than the maximum allowed permissions? perms = mode & 0777 # Drop non-permission bits should_be = perms & maximum_perms if perms != should_be puts " CHMOD #{f}" puts " Current perms are #{perms.to_s(8)}, " + "should be #{should_be.to_s(8)}" # File.chmod(perms & maximum_perms, f) unless dry_run end end end All that’s left to is a simple command-line interface to the normalize_home method: dry_run = false opts = OptionParser.new |opts| opts.on("-D", " dry-run", "Display changes to be made, don't make them.") dry_run = true end opts.on_tail("-h", " help", "display this help and exit") puts opts exit 934 | Chapter 25: System Administration end end opts.banner = "Usage: #{ FILE } [ dry-run] username [username2, …]" opts.parse!(ARGV) # Make sure all the users exist pwd_entries = ARGV.collect { |username| Etc.getpwnam(username) } # Normalize all given home directories pwd_entries.each { |p| normalize_home(p, 0775, dry_run ) } Discussion Running this script on my home directory shows over 2,500 problems These are mostly files owned by root, files owned by UIDs that don’t exist on my system (these come from tarballs), and world-writable files Below I give a sample of the embarrassment: $ ruby -D normalize_homes.rb leonardr Scanning leonardr's home of /home/leonardr CHOWN /home/leonardr/writing/Ruby Cookbook/sys-proctable-0.7.3/proctable.so Current owner is root, should be leonardr CHGRP /home/leonardr/writing/Ruby Cookbook/sys-proctable-0.7.3/proctable.so Current group is root, and leonardr doesn't belong … CHOWN /home/leonardr/writing/Ruby Cookbook/rubygems-0.8.4/lib/rubygems.rb Current owner is uid 501, should be leonardr CHGRP /home/leonardr/writing/Ruby Cookbook/rubygems-0.8.4/lib/rubygems.rb Current group is gid 501, and leonardr doesn't belong … CHMOD /home/leonardr/SORT/gogol-home-2002/mail Current perms are 722, should be 720 … Running the script as root (and with the File.chmod and File.chown calls uncom‐ mented) fixes all the problems You can run the script as yourself to check your own home directory, and it’ll fix per‐ mission problems on files you own But if a file is owned by someone else, you can’t take it back just because it’s in your home directory—that’s part of the problem with having a file owned by someone else in your home directory As usual with system administration scripts, normalize.homes.rb is only a starting point You’ll probably need to adapt this program to your specific purposes For instance, you may want to leave certain files alone, especially files owned by root (who can modify anyone’s home directory anyway) or by system processes such as the web server (usually user apache, httpd, or nobody) 25.9 Normalizing Ownership and Permissions in User Directories | 935 See Also • Recipe 3.6, “Converting Between Numeric Bases” • Recipe 7.2, “Checking Your Access to a File” • Recipe 7.3, “Changing the Permissions on a File” • Recipe 7.12, “Walking a Directory Tree” 25.10 Killing All Processes for a Given User Problem You want an easy way to kill all the running processes of a user whose processes get out of control Solution #!/usr/bin/ruby -w # banish.rb def signal_all(username, signal) lookup_uid(username) killed = %x{ps -u#{username}}.each_with_index |proc, i| next if i == # Skip the header provided by ps pid = proc.split[0].to_i begin Process.kill(signal, pid) rescue SystemCallError => e raise e unless e.errno == Errno::ESRCH end killed += end return killed end There are a couple things to look out for here • ps dumps a big error message if we pass in the name of a nonexistent user It would look better if we could handle that error ourselves That’s what the call to look up_uid will • ps prints out a header as its first line We want to skip that line because it doesn’t represent a process • Killing a process also kills all of its children This can be a problem if the child process shows up later in the ps list: killing it again will raise a SystemCallError 936 | Chapter 25: System Administration We deal with that possibility by catching and ignoring that particular SystemCal lError We still count the process as “killed,” though Here’s the implementation of lookup_id: def lookup_uid(username) require 'etc' begin user = Etc.getpwnam(username) rescue ArgumentError raise ArgumentError, "No such user: #{username}" end return user.uid end Now all that remains is the command-line interface: require 'optparse' signal = "SIGHUP" opts = OptionParser.new |opts| opts.banner = "Usage: #{ FILE } [-9] [USERNAME]" opts.on("-9", " with-extreme-prejudice", "Send an uncatchable kill signal.") { signal = "SIGKILL" } end opts.parse!(ARGV) if ARGV.size != $stderr.puts opts.banner exit end username = ARGV[0] if username == "root" $stderr.puts "Sorry, killing all of root's processes would bring down the system." exit end puts "Killed #{signal_all(username, signal)} process(es)." As root, you can some serious damage with this tool: $ /banish.rb peon process(es) killed Discussion The main problem with banish.rb as written is that it depends on an external program What’s worse, it depends on parsing the human-readable output of an external program For a quick script this is fine, but this would be more reliable as a self-contained program You can get a Ruby interface to the Unix process table by installing the sysproctable library This makes it easy to treat the list of currently running processes as a Ruby data structure Here’s an alternate implementation of signal_all that uses sys-proctable 25.10 Killing All Processes for a Given User | 937 instead of invoking a separate program Note that, unlike the other implementation, this one actually uses the return value of lookup_uid: def signal_all(username, signal) uid = lookup_uid(username) require 'sys/proctable' killed = Sys::ProcTable.ps.each |proc| if proc.uid == uid begin Process.kill(signal, proc.pid) rescue SystemCallError => e raise e unless e.errno == Errno::ESRCH end killed += end end return killed end See Also • sys-proctable is a useful RubyGem for sysadmins; it’s one of the sysutils packages: see http://rubyforge.org/projects/sysutils for the others • To write an equivalent program for Windows, you’d either use WMIthrough Ruby’s win32ole standard library, or install a native binary of GNU’s ps and use win32process 25.11 DevOps System Administration with Puppet Problem You want to manage your systems using a DevOps tool that can ensure that the same commands uniformly across all your machines Solution To install Puppet, you can simply install the puppet gem Then you will want to setup a puppet user and group, which you can with Puppet $ # # # # $ # 938 | sudo puppet resource group puppet ensure=present Notice: /Group[puppet]/ensure: created group { 'puppet': ensure => 'present', } sudo puppet resource user puppet ensure=present gid=puppet shell='/sbin/nologin' Notice: /User[puppet]/ensure: created Chapter 25: System Administration # user { 'puppet': # ensure => 'present', # } Now you need to create a Puppet manifest # rails.pp package { "rails": ensure => present, provider => gem, } To run an individual Puppet manifest, you just run puppet apply $ sudo puppet apply rails.pp Notice: Compiled catalog for my-machine.local in environment production in 0.71 seconds Notice: /Stage[main]/Main/Package[rails]/ensure: created Notice: Finished catalog run in 0.30 seconds Discussion Puppet is a very complex and popular DevOps tools that provides a lot of value for system administrators Puppet abstracts the setup and administration commands into a declarative manifest format This allows you not to have to worry about the underlying operating system Whether your operating system uses apt-get or emerge or homebrew, Puppet will know how to install and manage the software Although Puppet is written in Ruby, the manifest system configuration system uses its own syntax format The other advantage that Puppet brings to DevOps is the ability to understand the state of the overall system If you run a set of manifest files on two machines in various states, Puppet will work on bringing the machines to the same state For example, if one ma‐ chine had Rails installed already and the other didn’t, the above puppet apply would only install Rails on the machine that didn’t have it already If you are on a Mac, you can play with the power of Puppet interactively before using it on your servers $ puppet module install bjoernalbers-homebrew Now you can install various homebrew packages with a simple manifest # homebrew.pp class { 'homebrew': user => $user } class hightower::packages { package { 'tmux': ensure => installed, provider => brew, } } 25.11 DevOps System Administration with Puppet | 939 Then you can puppet apply the file to install Homebrew and use Homebrew to install tmux $ sudo puppet apply homebrew.pp The main way that Puppet is intended to be run is as a server-client relationship, where puppetmaster is the main central daemon that distributes the proper manifests to the proper machines There is a world of fun as you dig into the capabilities of Puppet See Also • To learn more about Puppet, see “Puppet Types and Providers” book by O’Reilly, see http://shop.oreilly.com/product/0636920026860.do • Learn more about Puppet manifest file format, see http://docs.puppetlabs.com/ learning/manifests.html • To setup a Puppet server on Mac OS X, see http://lashomb.tumblr.com/post/ 30438782998/getting-started-with-puppet 940 | Chapter 25: System Administration About the Authors Lucas Carlson founded AppFog, a PaaS company that leveraged Cloud Foundry and was acquired by CenturyLink Lucas has been a professional developer for 20 years and specializes in Ruby on Rails development Lucas has authored Programming for PaaS and the Ruby Cookbook and has written half a dozen libraries in various programming languages and contributed to many others, including Rails and RedCloth Leonard Richardson (http://www.crummy.com/) is the author of the Ruby Cookbook (O’Reilly) and of several open source libraries, including Beautiful Soup A California native, he currently lives in New York Colophon The animal on the cover of Ruby Cookbook is a side-striped jackal (Canis adustus), found mostly in central and southern Africa These jackals avoid the open, preferring thickly wooded areas on the edge of savannas and forests They occasionally make their way into cities Side-striped jackals are rare but not considered endangered There are re‐ serves for these jackals at the Serengeti National Park in Tanzania and at the Akagera National Park in Rwanda Side-striped jackals are about 15 inches tall and weigh between 16 and 26 pounds This jackal has a light grey coat with a white stripe from shoulder to hip, and a white-tipped tail The diet of side-striped jackals consists largely of wild fruits, small mammals, and insects They also eat carrion and are adept scavengers; they will follow a lion or other big cat to a kill The jackals usually live singly or in pairs, but they sometimes gather in family units of up to six members Their lifespan is about 10 to 12 years Jackals have been an object of superstition because of their association with carrion and death, and because of their eerie nocturnal noises: they hoot, yap, and make a kind of screaming yell Perhaps because jackals were often found prowling and hunting the edges of the desert near cemeteries, the ancient Egyptian god of embalming and gate‐ keeper of the path of the dead, Anubis, was depicted as a jackalheaded man Anubis served as a psychopomp, conducting souls to the underworld, where he weighed their hearts on a scale to determine whether they would be admitted to the underworld or cast to the crocodile-headed demon, Ammit The cover image is from Lydekker’s Royal History The cover font is Adobe ITC Gara‐ mond The text font is Linotype Birka; the heading font is Adobe Myriad Condensed; and the code font is LucasFont’s TheSans Mono Condensed [...]... Removed Method Array#nitems Removed Removed Method Array#indexes Removed in favor of Array#values_at Removed Method Array#indeces Removed in favor of Array#values_at Removed Method Hash#indexes Removed in favor of Hash#select Removed Method Hash#indeces Removed in favor of Hash#select Removed Method Object# id Removed in favor of Object# object_id Removed Method Object# type Removed in favor of Object# class... Moved to Core thread Now part of core, no need to require Moved to RubyGems soap You can use gem install soap Moved to RubyGems curses You can use gem install curses Moved to RubyGems iconv You can use gem install iconv Moved to RubyGems parsedate You can use gem install rubysl-parsedate Moved to RubyGems rinda You can use gem install rubysl-rinda Removed Library finalize Replaced by objspace Removed... (O’Reilly) and Rails Recipes by Chad Fowler (Pragmatic Programmers) Some common Ruby pitfalls are explained in the Ruby FAQ (http://www.rubycen tral.com/faq/, starting in Section 4) and in “Things That Newcomers to Ruby Should Know” (http://www.glue.umd.edu/~billtj /ruby. html) Many people come to Ruby already knowing one or more programming languages You might find it frustrating to learn Ruby with... better for comprehension .) $irb irb(main):001:0> 1 + 2 # => 3 => 3 irb(main):002:0> irb(main): 003: 0* # On a long line, the expected value goes on a new line: irb(main):004:0* Math.sqrt(1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 1 0) => 7.41619848709566 irb(main):005:0> # => 7.41619848709566 irb(main):006:0* irb(main):007:0* puts "This string is self-referential." This string is self-referential => nil irb(main):008:0>... in a scoped way New Method IO#wait_writable Wait until a file becomes writable New Method Object# !~ Returns true if two objects do not match (using the =~ method) New Method Object# singleton_class Returns the singleton class of obj New Method Object# untrust Marks obj as untrusted New Method Object# untrusted? Returns true if the object is untrusted New Method Object# trust Removes the untrusted mark from... library Removed Library Win32API No longer a standard library 6 | Chapter 1: Ruby 2.1 Type About Note Removed Library xsd No longer a standard library 1.2 YARV (Yet Another Ruby VM) Bytecode Interpreter Problem You want to understand more about the Ruby interpreter changes between Ruby 1.8 and 2.1 Solution Since Ruby started in 1995, it originally used the MRI (Matz’s Ruby Interpreter) to interpret Ruby code... Pro‐ grammers) The first edition is available online in HTML format (http://www.rubycen tral.com/book /), but it’s out of date The second edition is much better and is available Preface | xxv as a printed book or as PDF (http://www.pragmaticprogrammer.com/titles /ruby/ ) It’s a much better idea to buy the second edition and use the first edition as a handy reference than to try to read the first edition “Why’s... the package ruby- [version]: for instance, ruby- 1.8 or ruby- 1.9 Red Hat Linux calls it ruby; so does the DarwinParts system on Mac OS X If all else fails, download the Ruby source code and compile it yourself You can get the Ruby source code through FTP or HTTP by visiting http://www .ruby- lang.org/ Many of the recipes in this book require that you install third-party libraries in the form of Ruby gems... solutions (using only the Ruby standard library) to solutions that use gems, and gem-based solutions to ones that require other kinds of third-party software If you’re not familiar with gems, consult Chapter 20 as needed To get started, all you need to know is that you first download the Rubygems library from http://rubyforge.org/ projects/rubygems/ (choose the latest release from that page) Unpack the tarball... Writing a C Extension for Ruby 24.2 Using a C Library from Ruby 24.3 Calling a C Library Through SWIG 24.4 Writing Inline C in Your Ruby Code 24.5 Using Java Libraries with JRuby 898 902 905 908 910 25 System Administration 915 25. 1 Scripting an External Program 25. 2 Managing Windows Services 25. 3 Running Code as Another User 25. 4 Running Periodic ... capital ize_first_letter_of_string instead of reopening String and defining capital ize_first_letter inside it Other Resources If you need to learn Ruby, the standard reference is Programming Ruby: ... program’s output when run from the Unix command line: $ ruby sample _ruby_ file.rb This string is self-referential Note that the output of sample _ruby_ file.rb looks different from the same code en‐ tered... and Ruby For such people, we recommend Ruby creator Yukihiro Mat‐ sumoto’s Ruby User’s Guide” (http://www .ruby- doc.org/docs/UsersGuide/rg/) It’s a short read, and it focuses on what makes Ruby