ruby for system administration, apress (2007)

this print for content only—size & color not accurate spine = 0.62" 264 page count Books for professionals By professionals ® Practical Ruby for System Administration Dear Reader, Ruby’s growth has been astronomical. While it is forgivably easy to be put off by the hype surrounding the language, I wrote this book because Ruby makes a real and lasting difference to my work as a system administrator. Whether I need to rotate a log, swap out some commas, automate some SSL workflow, issue batched commands over the network, analyze congestion data, implement a domain-specific language, or build an entire administrative portal, Ruby is always there supporting my efforts. This book’s approach is to cover topics that I wish I had understood better when I was first coming to grips with Ruby as an administrator. This book places an equal emphasis on step-by-step examples and conceptual discussions. In reading this book, you’ll explore subjects as diverse as safe file handling, general- ized object storage, socket manipulation, directory service interaction, database wrangling, data presentation techniques, and the power of metaprogramming. You will also pick up practical tips on documentation, testing, task-oriented scripting, performance analysis, and coding style. Whether you come from a shell script or a Perl background, a data center or a classroom, in this book I aspire to convince you of the power and efficacy of this remarkable programming language in system administration and to give you a head start on your Ruby journey. André Ben Hamou MSci (Hons) ARCS MRes DIC US $44.99 Shelve in Programming Languages User level: Beginner–Intermediate Ben Hamou Practical Ruby for System Administration The eXperT’s Voice ® in open source Practical Ruby for System Administration cyan MaGenTa yelloW Black panTone 123 c André Ben Hamou Companion eBook Available THE APRESS ROADMAP Practical Ruby for System Administration Beginning Google Maps Applications with Rails And Ajax Beginning Ruby Beginning Ruby on Rails E-Commerce Beginning Rails Practical Ruby Gems www.apress.com Companion eBook See last page for details on $10 eBook version ISBN-13: 978-1-59059-821-4 ISBN-10: 1-59059-821-0 9 781590 598214 5 4 4 9 9 Apply the power and elegance of Ruby to the job of system administration Practical Ruby for System Administration ■■■ André Ben Hamou Hamou_821-0 FRONT.fm Page i Thursday, June 7, 2007 4:36 AM Practical Ruby for System Administration Copyright © 2007 by André Ben Hamou All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-13 (pbk): 978-1-59059-821-4 ISBN-10 (pbk): 1-59059-821-0 Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1 Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. Lead Editor: Jonathan Gennick Technical Reviewer: Dee Zsombor Editorial Board: Steve Anglin, Ewan Buckingham, Gary Cornell, Jonathan Gennick, Jason Gilmore, Jonathan Hassell, Chris Mills, Matthew Moodie, Jeffrey Pepper, Ben Renow-Clarke, Dominic Shakeshaft, Matt Wade, Tom Welsh Project Manager: Denise Santoro Lincoln Copy Edit Manager: Nicole Flores Copy Editor: Nicole Flores Assistant Production Director: Kari Brooks-Copony Production Editor: Ellie Fountain Compositor: Susan Glinert Proofreader: April Eddy Indexer: Broccoli Information Management Artist: Kinetic Publishing Services, LLC Cover Designer: Kurt Krames Manufacturing Director: Tom Debolski Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax 201-348-4505, e-mail orders-ny@springer-sbm.com, or visit http://www.springeronline.com. For information on translations, please contact Apress directly at 2855 Telegraph Avenue, Suite 600, Berkeley, CA 94705. Phone 510-549-5930, fax 510-549-5939, e-mail info@apress.com, or visit http:// www.apress.com. The information in this book is distributed on an “as is” basis, without warranty. Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work. The source code for this book is available to readers at http://www.apress.com in the Source Code/Download section. Hamou_821-0 FRONT.fm Page ii Thursday, June 7, 2007 4:36 AM For François. Hamou_821-0 FRONT.fm Page iii Thursday, June 7, 2007 4:36 AM Hamou_821-0 FRONT.fm Page iv Thursday, June 7, 2007 4:36 AM v Contents at a Glance About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xiii About the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix ■CHAPTER 1 What Ruby Can Do for You . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 ■CHAPTER 2 Common Tasks, Quick Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 ■CHAPTER 3 A Practical Look at Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 ■CHAPTER 4 The Power of Metaprogramming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 ■CHAPTER 5 Building Files the Smart Way . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 ■CHAPTER 6 Object Storage and Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 ■CHAPTER 7 Working with Enterprise Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 ■CHAPTER 8 Networking for Fun and Profit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 ■CHAPTER 9 Network Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 ■CHAPTER 10 Extending Ruby: A Fistful of Gems . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 ■CHAPTER 11 Testing and Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 ■CHAPTER 12 The Future of Ruby . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 ■APPENDIX Ruby Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 ■INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 Hamou_821-0 FRONT.fm Page v Thursday, June 7, 2007 4:36 AM Hamou_821-0 FRONT.fm Page vi Thursday, June 7, 2007 4:36 AM vii Contents About the Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xiii About the Technical Reviewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix ■CHAPTER 1 What Ruby Can Do for You . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Hello World . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Ruby in a Nutcracker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Objects at Rest: The Theory of Object Orientation . . . . . . . . . . . . . . . 3 Objects in Motion: The Ruby View of OO . . . . . . . . . . . . . . . . . . . . . . . 5 By Invitation Only: Accessors Made Easy. . . . . . . . . . . . . . . . . . . . . . . 8 Blocks and the Magic of yield . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 It Takes All Sorts: A Sensible Approach to Types . . . . . . . . . . . . . . . 12 Ointment for the Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 ■CHAPTER 2 Common Tasks, Quick Solutions . . . . . . . . . . . . . . . . . . . . . . . . . 15 One-liners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Grepping with Ruby . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Working with Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Using Line Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Playing with Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Smart Record Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Creating a Customized Directory Listing . . . . . . . . . . . . . . . . . . . . . . 18 Watching Commands Over Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Larger Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Rolling Logs: A Scheduled One-liner . . . . . . . . . . . . . . . . . . . . . . . . . 19 A Ruby Springboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Quick to Write Meets Quick to Run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Hamou_821-0 FRONT.fm Page vii Thursday, June 7, 2007 4:36 AM viii ■CONTENTS ■CHAPTER 3 A Practical Look at Performance . . . . . . . . . . . . . . . . . . . . . . . . . 23 Scripts Can Be Faster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 The Numbers Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 A Script vs. Standard Binaries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Analyzing Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 The UNIX time Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 The Benchmark Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 The Profiler Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Algorithmic Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Linguistic Optimization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Side Effect Reduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Dropping the C Bomb. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Ramming Speed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 ■CHAPTER 4 The Power of Metaprogramming . . . . . . . . . . . . . . . . . . . . . . . . . 43 Flexible Method Signatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Default Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Parameter Hashes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Missing Method Dynamic Dispatch . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Macros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Module Inclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Object Extension. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Domain-Specific Languages (DSLs) . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Plug-in API: Macros for Adding Macros . . . . . . . . . . . . . . . . . . . . . . . 54 Heavy Meta . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 ■CHAPTER 5 Building Files the Smart Way . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Safety First . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 File Locking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Safe File Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 The Pen Is Mightier Than the Words . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Mob the Builder: Program-Driven File Creation . . . . . . . . . . . . . . . . 66 ThundERbolts and Lightning: Template-Driven File Creation . . . . . 70 When Flat Files Fall Flat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Hamou_821-0 FRONT.fm Page viii Thursday, June 7, 2007 4:36 AM ■CONTENTS ix ■CHAPTER 6 Object Storage and Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Local Disk Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Inspection Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Marshaling Your Thoughts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 YAML Ain’t Markup Language. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Benchmarking the Alternatives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 Network-Aware Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 General Design Principals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 memcached: A Great Big Hash in the Sky . . . . . . . . . . . . . . . . . . . . . 83 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Object-Relational Mapping with ActiveRecord . . . . . . . . . . . . . . . . . 89 Playing with the Big Boys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 ■CHAPTER 7 Working with Enterprise Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Parsing Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Separation Is Such Sweet Sorrow: Delimited Values . . . . . . . . . . . 100 XML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Network Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 Lightweight Directory Access Protocol. . . . . . . . . . . . . . . . . . . . . . . 116 XML Remote Procedure Call . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 Simple Object Access Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Representational State Transfer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 Back to Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 ■CHAPTER 8 Networking for Fun and Profit . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Basic Network I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Socket to Me . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Socket Errors and Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Clockwatching: Timing Out on Purpose . . . . . . . . . . . . . . . . . . . . . . 135 Socket-Based Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 Higher-Level Network Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 An Embarrassment of Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Building a Web Robot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Throwing Together a Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Control and Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 Taking Command with SSH. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 Packet Monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 End of Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Hamou_821-0 FRONT.fm Page ix Thursday, June 7, 2007 4:36 AM [...]... ■ WHAT RUBY C AN DO FOR YOU ■Tip If your platform doesn’t support the #!/usr/bin/env convention, then don’t forget that the script can be run by passing it to the Ruby interpreter manually: -w hello.rb Let the rejoicing begin Our first Ruby script is written and does what it’s supposed to— nothing that is of any conceivable use whatsoever One approach to executing Ruby code... pontification, and the busy system administrator has time for neither It is as though the spirit of the Turing test for intelligence has been applied to the Ruby type system The result of this eminently commonsense approach is that types work for you (in the form of subclassing and variation), rather than the other way around Type purists may argue that strong types make for strong code, but such purists... options of the Ruby interpreter, do a man ruby Anyone who’s ever programmed in C will recognize the 1 Hamou_821-0 C01.fm Page 2 Friday, May 18, 2007 4:20 AM 2 CHAPTER 1 ■ WHAT RUBY C AN DO FOR YOU puts command It is shorthand for “put string” (the string being the collection of characters inside the double quotes) RI: THE RUBY REFERENCE Before we delve too deeply into Ruby s built-in methods, classes, and... of using Ruby Hello World Why don’t we jump straight in with the example we all know and love? Here’s the Ruby to dump the phrase “hello world” in the terminal: $ ruby -e 'puts "hello world"' hello world As you might have guessed, the -e flag instructs the Ruby interpreter to execute whatever script follows (inside the single quotes in this case) For more on the command line options of the Ruby interpreter,... day to make systems function at the ISP where I work You would have thought that this combination of facts would make it straightforward to distill a few salient chapters on the matter It doesn’t Indeed it took me nearly a month of trying to build a skeleton structure for the book before I realized that the problem was one of context You see, the target audience for this book is obviously system administrators,... leave only two mainstream languages standing: Ruby and Python For my money, Python comes very close but only Ruby hits the mark xix Hamou_821-0 FRONT.fm Page xx Thursday, June 7, 2007 4:36 AM xx ■I N T R O D U C T I O N When I started to use Ruby, I did what I suspect quite a few have done in the past I wrote in Ruby and thought in Perl This does not make for convincing scripts (in much the same way... WHAT RUBY C AN DO FOR YOU simple methods just shown Since it would be a pain and rather noisy to have to write these in full every time, Ruby provides some convenience macros (which are actually just class methods themselves), as shown in Table 1-1 Table 1-1 Ruby s Built-in Accessor Macros Function Purpose attr_reader Creates a getter method for a given variable attr_writer Creates a setter method for. .. CHAPTER 1 ■■■ What Ruby Can Do for You A s I mentioned in the book’s introduction, Ruby is my language of choice It is the tool I instinctively reach for first when solving system administration problems To recap, it has principally achieved this place in my affections by • Making it simple to write nicely readable code • Emphasizing convention over configuration, so a little effort goes a long way... is essential that you be comfortable using the rather spiffy command line Ruby reference tool called ri that ships as part of the standard Ruby distribution This is one of the best ways to learn about what the various parts of the core and standard Ruby libraries do Want to know what the String class does? Simply invoke ri String What if you know you want the reference for puts but can’t remember which... enduring miracles I must thank mum as well for being so many things that I both admire and love at the same time It is also important to thank the wonderful people at Apress whose skill, experience, and patience have made this book possible You guys rock Finally, I want to express my gratitude to and respect for Yukihiro Matsumoto for the creation and nurturing of the Ruby language Together with a brilliant . Available THE APRESS ROADMAP Practical Ruby for System Administration Beginning Google Maps Applications with Rails And Ajax Beginning Ruby Beginning Ruby on Rails E-Commerce Beginning Rails Practical Ruby. to the job of system administration Practical Ruby for System Administration ■■■ André Ben Hamou Hamou_821-0 FRONT.fm Page i Thursday, June 7, 2007 4:36 AM Practical Ruby for System Administration Copyright. print for content only—size & color not accurate spine = 0.62" 264 page count Books for professionals By professionals ® Practical Ruby for System Administration Dear Reader, Ruby s