Expert Shell Scripting
Expert Shell Scripting ■■■ Ron Peters Expert Shell Scripting Copyright © 2009 by Ron Peters All rights reserved No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher ISBN-13 (pbk): 978-1-4302-1841-8 ISBN-13 (electronic): 978-1-4302-1842-5 Printed and bound in the United States of America Trademarked names may appear in this book Rather than use a trademark symbol with every occurrence of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark Lead Editor: Frank Pohlmann Technical Reviewer: Brian Culp Editorial Board: Clay Andres, Steve Anglin, Mark Beckner, Ewan Buckingham, Tony Campbell, Gary Cornell, Jonathan Gennick, Michelle Lowman, Matthew Moodie, Jeffrey Pepper, Frank Pohlmann, Ben Renow-Clarke, Dominic Shakeshaft, Matt Wade, Tom Welsh Project Manager: Sofia Marchant Copy Editor: Candace English Associate Production Director: Kari Brooks-Copony Production Editor: Liz Berry Compositor: Pat Christenson Proofreader: Lisa Hamilton Indexer: Julie Grady Artist: April Milne Cover Designer: Kurt Krames Manufacturing Director: Tom Debolski Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor, New York, NY 10013 Phone 1-800-SPRINGER, fax 201-348-4505, e-mail orders-ny@springer-sbm.com, or visit http://www.springeronline.com For information on translations, please contact Apress directly at 2855 Telegraph Avenue, Suite 600, Berkeley, CA 94705 Phone 510-549-5930, fax 510-549-5939, e-mail info@apress.com, or visit http:// www.apress.com Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use eBook versions and licenses are also available for most titles For more information, reference our Special Bulk Sales–eBook Licensing web page at http://www.apress.com/info/bulksales The information in this book is distributed on an “as is” basis, without warranty Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work The source code for this book is available to readers at http://www.apress.com You will need to answer questions pertaining to this book in order to successfully download the code Contents at a Glance About the Author xv About the Technical Reviewer xvii Acknowledgments xix Introduction xxi PART Basic Scripting Techniques ■CHAPTER Shell Script Debugging ■CHAPTER Standard Function Library 13 ■CHAPTER Date and Time Manipulation 19 ■CHAPTER Comparisons and Tests 25 ■CHAPTER Accepting Command-Line Options, Switches, and Parameters 31 ■CHAPTER Testing Variables and Assigning Defaults 37 ■CHAPTER Indirect Reference Variables 43 ■CHAPTER Shell Process Tree 49 ■CHAPTER Data Redirection 59 ■CHAPTER 10 Piping Input to read 65 PART iv ■■■ ■■■ System Interaction and Advanced Techniques ■CHAPTER 11 Math from the Shell 75 ■CHAPTER 12 cron 81 ■CHAPTER 13 Self-Linked Scripts 87 ■CHAPTER 14 Throttling Parallel Processes 91 ■CHAPTER 15 Command-Line Editing and History 99 ■CHAPTER 16 Scripting from the Command Line 103 ■CHAPTER 17 Automating User Input with expect 107 ■CHAPTER 18 User Input Timeout 115 ■CHAPTER 19 Instant Keyboard Response 121 ■CHAPTER 20 Directory Copying 125 ■CHAPTER 21 A Brief Tour of the X Display Environment 131 ■CHAPTER 22 X Navigation Window 141 ■CHAPTER 23 Command-Line E-mail Attachments 151 ■CHAPTER 24 Text-Processing One-Liners 157 ■CHAPTER 25 Editing Files in Place 173 ■CHAPTER 26 Evaluating Variables in a Flat File 181 ■CHAPTER 27 Read Piped Input 183 ■CHAPTER 28 Free-Format Output Using cat 185 ■CHAPTER 29 Automating Interactive Processes 187 PART ■■■ Useful Scripts ■CHAPTER 30 Automating E-Mail with procmail 193 ■CHAPTER 31 Process-Management Monitor 201 ■CHAPTER 32 Managing File Counts 211 ■CHAPTER 33 Processes Running from inittab 215 ■CHAPTER 34 Automatic RCS 219 ■CHAPTER 35 Colorful /proc Reporting 223 ■CHAPTER 36 Password-Aging Notification 231 ■CHAPTER 37 A Pseudo–shadow File 241 ■CHAPTER 38 Linux Gold-System Build 245 ■CHAPTER 39 System Snapshots 251 ■CHAPTER 40 Removing Large Files and Log Rolling 261 ■CHAPTER 41 Core Finder 265 ■CHAPTER 42 Network Adapter Failover 267 ■APPENDIX A Test Switches 273 ■APPENDIX B Special Parameters 275 ■APPENDIX C Other Shell-Scripting Resources 277 ■INDEX 281 Contents About the Author xv About the Technical Reviewer xvii Acknowledgments xix Introduction xxi PART ■CHAPTER ■■■ Basic Scripting Techniques Shell Script Debugging Shell Trace Options Simple Output Statements Controlling Output with Debug Levels Simplifying Error Checking with a Function Manual Stepping 11 ■CHAPTER Standard Function Library 13 The Library File 13 Some Useful Functions 14 Using Your Library 16 ■CHAPTER ■CHAPTER Date and Time Manipulation 19 Date in Days Days Since Epoch Alternatives for Finding the Date in Seconds Evaluating for the Current Day and Time 19 20 22 22 Comparisons and Tests 25 The Basics of Comparisons 25 vii viii ■C O N T E N T S ■CHAPTER ■CHAPTER ■CHAPTER Accepting Command-Line Options, Switches, and Parameters 31 Testing Variables and Assigning Defaults 37 Setting Defaults Variable Substitution := Syntax = Syntax :- Syntax - Syntax :? Syntax ? Syntax :+ Syntax + Syntax 37 38 39 40 40 40 41 41 42 42 Indirect Reference Variables 43 Log File Monitoring with Indirect Variables 43 The Main Monitor Loop 44 ■CHAPTER Shell Process Tree 49 Process Tree Implemented Using Arrays 50 Process Tree Implemented Using Indirect Variables 55 Bourne Shell Implementation of a Process Tree 56 ■CHAPTER ■CHAPTER 10 Data Redirection 59 Avoiding Confusion Common Redirection Access to User-Specified File Handles Descriptor Access from the Shell 59 60 62 64 Piping Input to read 65 Line-by-Line Option 66 Line-by-Line Option 67 Line-by-Line Option 68 ■C O N T E N T S Line-by-Line Option 68 Pipe to read Directly 70 Process Input Word-by-Word 70 PART ■■■ ■CHAPTER 11 ■CHAPTER 12 System Interaction and Advanced Techniques Math from the Shell 75 expr Internal Shell Math bc dc 75 76 78 78 cron 81 crontab Entries 81 Environment Problems 82 Output Redirection 84 ■CHAPTER 13 Self-Linked Scripts 87 ■CHAPTER 14 Throttling Parallel Processes 91 Parallel Processing with ksh 92 Parallel Processing with bash 97 ■CHAPTER 15 ■CHAPTER 16 Command-Line Editing and History 99 Setting Up vi Editing bash ksh Command and File Completion 100 100 101 101 Scripting from the Command Line 103 A Few Examples 103 ix x ■C O N T E N T S ■CHAPTER 17 Automating User Input with expect 107 A Shell Script to Customize Parameters for an expect Script 108 An expect Script to Automate telnet 111 ■CHAPTER 18 User Input Timeout 115 Manual Timeout Method 115 Timeout Using stty 117 General Timeout Utility 118 ■CHAPTER 19 Instant Keyboard Response 121 ■CHAPTER 20 Directory Copying 125 Using cp Using tar Using find Using rsync 125 126 126 127 A Brief Tour of the X Display Environment 131 The Display X Traffic Through ssh X Applications Through a Third-Party System User-Profile Entry Root-Profile Entry Throw a Temporary Root Window 131 133 133 135 137 138 X Navigation Window 141 ■CHAPTER 21 ■CHAPTER 22 Navigation Window Usage 141 Navigation Setup 142 Navigation Window 145 ■CHAPTER 23 Command-Line E-mail Attachments 151 uuencode 151 MIME Encoding 153 ■C O N T E N T S ■CHAPTER 24 ■CHAPTER 25 ■CHAPTER 26 Text-Processing One-Liners 157 Displaying Specific Fields Specifying the Field Separator Simple Pattern-Matching Matching Fields Against Several Values Determining the Number of Fields Determining the Last Field Determining the Second-to-Last Field Passing Variables to awk Using a Variable Passed to awk in a Condition Displaying a Range of Fields (Main Method) Displaying a Range of Fields (Alternate Method) Determining the Length of a String Using awk Determining the Length of a String Using expr Displaying a Substring with awk Displaying a Substring with expr Conducting Simple Search and Replace with sed Disregarding Blank and Commented Lines from a File Conducting Dual Search and Replace with sed Filtering Lines with sed Searching for Multiple Strings with egrep A Clean Method of Searching the Process Table Summing Columns Using awk Generating Random Numbers Using awk Generating Random Numbers from the Shell Displaying Character-Based Fields with sed Escaping Special Characters Returning Trailing Lines from a Pattern Match Using grep Returning Preceding Lines to a Pattern Match Using grep 157 158 158 159 159 159 160 160 161 161 162 163 163 163 164 164 164 165 165 166 166 167 167 168 168 169 170 171 Editing Files in Place 173 Simple Search and Replace with ed Search and Replace Using ed, Dissected Examples of ed Commands Escaping Special Characters in a File 174 174 175 178 Evaluating Variables in a Flat File 181 xi .. .Expert Shell Scripting ■■■ Ron Peters Expert Shell Scripting Copyright © 2009 by Ron Peters All rights reserved No part of... grow continually You might be aware of the large number of shell- scripting books and online resources aiding in the mastery of shell scripting Many are excellent and cover a wide range of topics... Other Shell- Scripting Resources 277 Manual Pages Books Scripting