perl for sysadmins - o'reilly

The Five−Minute RCS Tutorial (Perl for System Administration) Table of Contents Appendix A. The Five−Minute RCS Tutorial 1 A.1. References for More Information 2 Appendix B. The Ten−Minute LDAP Tutorial 4 B.1. LDAP Data Organization 5 Appendix C. The Eight−Minute XML Tutorial 9 C.1. XML Is a Markup Language 9 C.2. XML Is Picky 10 C.3. Two Key XML Terms 12 C.4. Leftovers 13 Appendix D. The Fifteen−Minute SQL Tutorial 14 D.1. Creating /Deleting Databases and Tables 15 D.2. Inserting Data into a Table 16 D.3. Querying Information 17 D.3.1. Retrieving All of the Rows in a Table 18 D.3.2. Retrieving a Subset of the Rows in a Table 18 D.3.3. Simple Manipulation of Data Returned by Queries 19 D.3.4. Adding the Query Results to Another Table 20 D.4. Changing Table Information 21 D.5. Relating Tables to Each Other 22 D.6. SQL Stragglers 23 D.6.1. Views 23 D.6.2. Cursors 24 D.6.3. Stored Procedures 24 Appendix E. The Twenty−Minute SNMP Tutorial 26 E.1. SNMP in Practice 31 Preface 37 0.1. How This Book Is Structured 37 0.2. Typographical Conventions 38 0.3. How to Contact Us 39 0.4. Acknowledgments 40 1.1. System Administration Is a Craft 43 1.2. How Perl Can Help 43 1.3. This Book Will Show You How 43 1.4. What You Need 45 1.5. Locating and Installing Modules 46 1.5.1. Installing Modules on Unix 48 1.5.2. Installing Modules on Win32 48 1.5.3. Installing Modules on MacOS 49 1.6. It's Not Easy Being Omnipotent 49 1.6.1. Don't Do It 50 1.6.2. Drop Your Privileges as Soon as Possible 50 1.6.3. Be Careful When Reading Data 50 1.6.4. Be Careful When Writing Data 51 1.6.5. Avoid Race Conditions 52 1.6.6. Enjoy 53 1.7. References for More Information 53 2.1. Perl to the Rescue 54 The Five−Minute RCS Tutorial (Perl for System Administration) i Table of Contents Preface 2.2. Filesystem Differences 55 2.2.1. Unix 55 2.2.2. Microsoft Windows NT/2000 56 2.2.3. MacOS 56 2.2.4. Filesystem Differences Summary 56 2.2.5. Dealing with Filesystem Differences from Perl 58 2.3. Walking or Traversing the Filesystem 58 2.4. Walking the Filesystem Using the File::Find Module 59 2.5. Manipulating Disk Quotas 59 2.5.1. Editing Quotas with edquota Trickery 64 2.5.2. Editing Quotas Using the Quota Module 72 2.6. Querying Filesystem Usage 73 2.7. Module Information for This Chapter 77 2.8. References for More Information 78 3.1. Unix User Identity 80 3.1.1. The Classic Unix Password File 80 3.1.2. Extra Fields in BSD 4.4 passwd Files 82 3.1.3. Binary Database Format in BSD 4.4 82 3.1.4. Shadow Passwords 83 3.2. Windows NT/2000 User Identity 87 3.2.1. NT/2000 User Identity Storage and Access 87 3.2.2. NT/2000 User ID Numbers 88 3.2.3. NT/2000 Passwords 89 3.2.4. NT Groups 89 3.2.5. NT/2000 User Rights 90 3.3.1. The Backend Database 91 3.3.1.1. Writing XML from Perl 92 3.3.1.2. Reading XML using XML::Parser 94 3.3.1.3. Reading XML using XML::Simple 97 3.3.1.4. Writing XML using XML::Simple 98 3.3.2. The Low−Level Component Library 100 3.3.2.1. Unix account creation and deletion routines 102 3.3.2.2. Windows NT/2000 account creation and deletion routines 105 3.3.3. The Process Scripts 107 3.3.4. Account System Wrap−Up 110 3.3. Building an Account System to Manage Users 110 3.4. Module Information for This Chapter 113 3.5. References for More Information 116 3.5.1. Unix Password Files 120 3.5.2. NT User Administration 122 3.5.3. XML 122 3.5.4. Other 122 4.1. MacOS Process Control 123 4.2. NT/2000 Process Control 123 4.2.1. Using the Microsoft Resource Kit Binaries 124 4.2.2. Using the Win32::IProc Module 125 4.2.3. Using the Win32::Setupsup Module 125 4.2.4. Using Window Management Instrumentation (WMI) 127 4.3. Unix Process Control 127 4.3.1. Calling an External Program 128 4.3.2. Examining the Kernel Process Structures 131 The Five−Minute RCS Tutorial (Perl for System Administration) ii Table of Contents Preface 4.3.3. Using the /proc Filesystem 135 4.3.4. Using the Proc::ProcessTable Module 140 4.4. Tracking File and Network Operations 141 4.4.1. Tracking Operations on Windows NT/2000 141 4.4.2. Tracking Operations in Unix 142 4.5. Module Information for This Chapter 142 4.5.1. Installing Win32::IProc 146 4.5.2. Installing Win32::Setupsup 146 4.6. References for More Information 149 5.1. Host Files 153 5.1.1. Generating Host Files 154 5.1.2. Error Checking the Host File Generation Process 154 5.1.3. Improving the Host File Output 155 5.1.4. Incorporating a Source Code Control System 157 5.2.1. NIS+ 157 5.2.2. Windows Internet Name Server ( WINS) 159 5.2. NIS, NIS+, and WINS 161 5.3. Domain Name Service (DNS) 162 5.3.1. Generating DNS Configuration Files 165 5.3.1.1. Creating the administrative header 168 5.3.1.2. Generating multiple configuration files 170 5.3.2. DNS Checking: An Iterative Approach 171 5.3.2.1. Using nslookup 171 5.3.2.2. Working with raw network sockets 172 5.3.2.3. Using Net::DNS 173 5.4. Module Information for This Chapter 175 5.5. References for More Information 180 6.1. What's a Directory? 181 6.2. Finger: A Simple Directory Service 182 6.3. The WHOIS Directory Service 186 6.4. LDAP: A Sophisticated Directory Service 188 6.4.1. LDAP Programming with Perl 188 6.4.2. The Initial LDAP Connection 190 6.4.3. Performing LDAP Searches 190 6.4.4. Entry Representation in Perl 191 6.4.5. Adding Entries with LDIF 194 6.4.6. Adding Entries with Standard LDAP Operations 196 6.4.7. Deleting Entries 197 6.4.8. Modifying Entry Names 198 6.4.9. Modifying Entry Attributes 199 6.4.10. Putting It All Together 203 6.5. ADSI (Active Directory Service Interfaces) 205 6.5.1. ADSI Basics 208 6.5.2. Using ADSI from Perl 209 6.5.3. Dealing with Container/Collection Objects 210 6.5.4. Identifying a Container Object 210 6.5.5. So How Do You Know Anything About an Object? 213 6.5.6. Searching 218 6.5.7. Performing Common Tasks Using the WinNT and LDAP Namespaces 218 6.5.8. Working with Users via ADSI 220 6.5.9. Working with Groups via ADSI 222 The Five−Minute RCS Tutorial (Perl for System Administration) iii Table of Contents Preface 6.5.10. Working with File Shares via ADSI 222 6.5.11. Working with Print Queues and Print Jobs via ADSI 223 6.5.12. Working with NT/2000 Services via ADSI 225 6.6. Module Information for This Chapter 227 6.7. References for More Information 228 6.7.1. Finger 229 6.7.2. WHOIS 230 6.7.3. LDAP 230 6.7.4. ADSI 232 7.2.1. DBI Leftovers 233 7.1. Interacting with an SQL Server from Perl 234 7.2. Using the DBI Framework 234 7.3. Using the ODBC Framework 234 7.4. Server Documentation 234 7.4.1. MySQL Server via DBI 235 7.4.2. Sybase Server via DBI 237 7.4.3. MS−SQL Server via ODBC 238 7.5. Database Logins 240 7.6. Monitoring Server Health 244 7.6.1. Space Monitoring 245 7.6.2. Monitoring the CPU Health of a SQL Server 248 7.7. Module Information for This Chapter 249 7.8. References for More Information 250 7.8.1. SQL 251 7.8.2. DBI 253 7.8.3. ODBC 255 7.8.4. Other Topics 255 8.1.1. Getting sendmail (or Similar Mail Transport Agent) 258 8.1.2. Using the OS−Specific IPC Framework 261 8.1.3. Speaking to the Mail Protocols Directly 261 8.1. Sending Mail 261 8.2. Common Mistakes in Sending Email 261 8.2.1. Overzealous Message Sending 262 8.2.1.1. Controlling the frequency of mail 262 8.2.1.2. Controlling the amount of mail 263 8.2.2. Subject Line Waste 263 8.2.3. Insufficient Information in the Message Body 264 8.3. Receiving Mail 264 8.3.1. Dissecting a Single Message 266 8.3.2. Dissecting a Whole Mailbox 268 8.3.3. Tracking Down Spam 269 8.3.3.1. Checking against a local blacklist 269 8.3.3.2. Checking against Internet−wide blacklists 271 8.3.4. Support Mail Augmentation 277 8.4. Module Information for This Chapter 277 8.5. References for More Information 279 9.1. Text Logs 279 9.2. Binary Log Files 280 9.2.1. Using unpack( ) 281 9.2.2. Calling an OS (or Someone Else's) Binary 284 9.2.3. Using the OS's Logging API 287 The Five−Minute RCS Tutorial (Perl for System Administration) iv Table of Contents Preface 9.3. Stateful and Stateless Data 291 9.4. Disk Space Problems 296 9.4.1. Log Rotation 297 9.4.2. Circular Buffering 299 9.4.2.1. Input blocking in log processing programs 299 9.4.2.2. Security in log processing programs 300 9.5. Log Analysis 300 9.5.1. Stream Read−Count 302 9.5.1.1. A simple stream read−count variation 303 9.5.2. Read−Remember−Process 305 9.5.3. Black Boxes 307 9.5.4. Using Databases 307 9.5.4.1. Using Perl−only databases 309 9.5.4.2. Using Perl−cliented SQL databases 312 9.6. Module Information for This Chapter 312 9.7. References for More Information 313 10.1. Noticing Unexpected or Unauthorized Changes 313 10.1.1. Local Filesystem Changes 317 10.1.2. Network Service Changes 318 10.2. Noticing Suspicious Activities 326 10.2.1. Local Signs of Peril 328 10.2.2. Finding Problematic Patterns 328 10.3. SNMP 333 10.3.1. Using SNMP from Perl 335 10.4. Danger on the Wire 336 10.4.1. Perl Saves the Day 337 10.5. Preventing Suspicious Activities 338 10.6. Module Information for This Chapter 338 10.7. References for More Information 342 10.7.1. Change Detection Tools 344 10.7.2. SNMP 344 10.7.3. Other Resources 345 Colophon 350 Copyright © 2001 O'Reilly & Associates, Inc. All rights reserved 351 Logos and Trademarks 358 Disclaimer 358 Table of Contents 366 Chapter 1. Introduction 370 Chapter 2. Filesystems 370 Chapter 3. User Accounts 371 Chapter 4. User Activity 371 Chapter 5. TCP/IP Name Services 372 Chapter 6. Directory Services 373 The Five−Minute RCS Tutorial (Perl for System Administration) v Table of Contents Chapter 7. SQL Database Administration 374 Chapter 8. Electronic Mail 374 Chapter 9. Log Files 374 Chapter 10. Security and Network Monitoring 375 The Five−Minute RCS Tutorial (Perl for System Administration) vi Appendix A. The Five−Minute RCS Tutorial Contents: References for More Information This quick tutorial will teach you everything you need to know about how to use Revision Control System (RCS) for system administration. RCS has considerably more functionality than we'll discuss here, so be sure to take a look at the manual pages and the reference at the end of this appendix if you plan to use it heavily. RCS functions like a car rental agency. Only one person at a time can actually rent a particular car and drive it off the lot. A new car can only be rented after the agency has added it to their pool. Customers can browse the list of cars (and their features) at any time, but if two people want to rent the same car, the second must wait for the car to be returned to the lot before renting it. Finally, car rental agencies inspect cars very carefully after they have been returned and record any changes to the car during the rental. All of these properties hold true for RCS as well. In RCS, a file is like a car. If you wish to keep track of a file using RCS (i.e., add it to the rental lot) you "check it in" for the first time: $ ci −u filename ci stands for "check in," and the −u tells RCS to leave the file in place during the check−in. When a file is checked in (i.e., made available for rental), RCS does one of two things to remind the user that the file is under RCS's control: 1. Deletes the original file, leaving only the RCS archive file behind. This archive file is usually called filename,v and is either kept in the same directory as the original file or in a subdirectory called RCS (if the user creates it). 2. If −u is used as we showed above, it checks the file out again, leaving the permissions on the file to be "read−only." To modify a file under RCS's control (i.e., rent a car), you first need to "check−out" that file: $ co −l filename The −l switch tells RCS to "strictly lock" that file (i.e., do not allow any other user to check out the file at the same time). Other switches that are commonly used with co are: • −r <revision number>: to check out an older revision of a file. • −p: to print a past revision to the screen without actually checking it out. Once you are done modifying a file, you need to check it back in using the same command we used above to put the file under RCS in the first place (ci −u filename). The check−in process stores any changes made to this file in a space−efficient manner. Each time a file that has been modified is checked in, it is given a new revision number. At check−in time, RCS will prompt you for a comment to be placed in the change log it automatically keeps for each file. This Appendix A. The Five−Minute RCS Tutorial 1 log and the listing of the current person who has checked out a file can be viewed using rlog filename. If someone neglects to check their changes to a particular file back into RCS (e.g., they've gone home for the day and you have a real need to change the file yourself ), you can break their lock using rcs−u filename. This command will prompt for a break−lock message that is mailed to the person who owns the lock. After breaking the lock, you should check to see how the current copy differs from the RCS archive revision. rcsdiff filename will show you this information. If you wish to preserve these changes, check the file in (with an appropriate change−log comment), and then check it back out again before working on it. rcsdiff, like our co example above, can also take a −r <revision number> flag to allow you to compare two past revisions. Table A−1 lists some command RCS operations and their command lines. Table A−1. Common RCS Operations RCS Operation Command Line Initial check−in of file (leaving file active in filesystem) ci −u filename Check out with lock co −l filename Check in and unlock (leaving file active in filesystem) ci −u filename Display version x.y of a file co −px.y filename Undo to version x.y (overwrites file active in filesystem with the specified revision) co −rx.y filename Diff file active in filesystem and last revision rcsdiff filename Diff versions x.y and x.z rcsdiff −rx.y −rx.z filename View log of checkins rlog filename Break an RCS lock held by another person on a file rcs −u filename Believe it or not, this is really all you need to get started using RCS. Once you start using it for system administration, you'll find it pays off handsomely. A.1. References for More Information ftp://ftp.gnu.org/pub/gnu/rcs has the latest source code for the RCS package. Applying RCS and SCCS: From Source Control to Project Control, by Don Bolinger and Tan Bronson (O'Reilly, 1995). http://www.sourcegear.com/CVS is where to go if you find you need features not found in RCS. The next step up is the very popular Concurrent Versions System (CVS). This is its main distribution point. 10.7. References for More Information B. The Ten−Minute LDAP Tutorial The Five−Minute RCS Tutorial (Perl for System Administration) Appendix A. The Five−Minute RCS Tutorial 2 Copyright © 2001 O'Reilly & Associates. All rights reserved. Perl for System Administration The Five−Minute RCS Tutorial (Perl for System Administration) Appendix A. The Five−Minute RCS Tutorial 3 [...]... its "information model." This is the part that sets the rules for how information is represented But for the answer to our question we B.1 LDAP Data Organization 5 The Five−Minute RCS Tutorial (Perl for System Administration) need to look to LDAP's "naming model," which dictates how information is organized If you look at Figure B−1, you can see we've discussed all of the parts of an entry except for. .. ldap://ldap.itd.umich.edu/o=University%20of%20Michigan,c=US?postalAddress B.1 LDAP Data Organization 7 The Five−Minute RCS Tutorial (Perl for System Administration) A.1 References for More Information C The Eight−Minute XML Tutorial Copyright © 2001 O'Reilly & Associates All rights reserved Perl for System Administration B.1 LDAP Data Organization 8 Appendix C The Eight−Minute XML Tutorial Contents: XML Is a Markup... optimized, post−parsed form for faster execution These uploads are known as stored procedures Stored procedures are often a critical component of SQL for administrators because large parts of server administration for some servers rely on them For example, to change the owner of the sysadm database in Sybase, you might do this: USE sysadm D.6.2 Cursors 24 The Five−Minute RCS Tutorial (Perl for System Administration)... Tutorial Copyright © 2001 O'Reilly & Associates All rights reserved Perl for System Administration C.4 Leftovers 13 Appendix D The Fifteen−Minute SQL Tutorial Contents: Creating /Deleting Databases and Tables Inserting Data into a Table Querying Information Changing Table Information Relating Tables to Each Other SQL Stragglers Relational databases can be an excellent tool for system administration... Five−Minute RCS Tutorial (Perl for System Administration) The set of available conditional operators for WHERE clauses are the standard programming fare: = > >= < . Databases 307 9.5.4.1. Using Perl only databases 309 9.5.4.2. Using Perl cliented SQL databases 312 9.6. Module Information for This Chapter 312 9.7. References for More Information 313 10.1. Noticing. from Perl 335 10.4. Danger on the Wire 336 10.4.1. Perl Saves the Day 337 10.5. Preventing Suspicious Activities 338 10.6. Module Information for This Chapter 338 10.7. References for More Information. its "information model." This is the part that sets the rules for how information is represented. But for the answer to our question we The Five−Minute RCS Tutorial (Perl for System