S E C U R I T Y T O O L S O N C D - R O M ® PRESS ® Linux Solutions from the Experts at Red Hat Mohammed J. Kabir ® ® ™ Red Hat Linux Security and Optimization Mohammed J. Kabir Hungry Minds, Inc. New York, NY ● Indianapolis, IN ● Cleveland, OH Trademarks: are trademarks or registered trademarks of Hungry Minds, Inc. All other trademarks are the property of their respective owners. Hungry Minds, Inc., is not associated with any product or vendor mentioned in this book. LIMIT OF LIABILITY/DISCLAIMER OF WARRANTY: THE PUBLISHER AND AUTHOR HAVE USED THEIR BEST EFFORTS IN PREPARING THIS BOOK. THE PUBLISHER AND AUTHOR MAKE NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE ACCURACY OR COMPLETENESS OF THE CONTENTS OF THIS BOOK AND SPECIFICALLY DISCLAIM ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. THERE ARE NO WARRANTIES WHICH EXTEND BEYOND THE DESCRIPTIONS CONTAINED IN THIS PARAGRAPH. NO WARRANTY MAY BE CREATED OR EXTENDED BY SALES REPRESENTATIVES OR WRITTEN SALES MATERIALS. THE ACCURACY AND COMPLETENESS OF THE INFORMATION PROVIDED HEREIN AND THE OPINIONS STATED HEREIN ARE NOT GUARANTEED OR WARRANTED TO PRODUCE ANY PARTICULAR RESULTS, AND THE ADVICE AND STRATEGIES CONTAINED HEREIN MAY NOT BE SUITABLE FOR EVERY INDIVIDUAL. NEITHER THE PUBLISHER NOR AUTHOR SHALL BE LIABLE FOR ANY LOSS OF PROFIT OR ANY OTHER COMMERCIAL DAMAGES, INCLUDING BUT NOT LIMITED TO SPECIAL, INCIDENTAL, CONSEQUENTIAL, OR OTHER DAMAGES. FULFILLMENT OF EACH COUPON OFFER IS THE SOLE RESPONSIBILITY OF THE OFFEROR. Red Hat Linux Security and Optimization Published by Hungry Minds, Inc. 909 Third Avenue New York, NY 10022 www.hungryminds.com Copyright © 2002 Hungry Minds, Inc. All rights reserved. No part of this book, including interior design, cover design, and icons, may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording, or otherwise) without the prior written permission of the publisher. Library of Congress Control Number: 2001092938 ISBN: 0-7645-4754-2 Printed in the United States of America 10 9 8 7 6 5 4 3 2 1 1B/SX/RR/QR/IN Distributed in the United States by Hungry Minds, Inc. Distributed by CDG Books Canada Inc. for Canada; by Transworld Publishers Limited in the United Kingdom; by IDG Norge Books for Norway; by IDG Sweden Books for Sweden; by IDG Books Australia Publishing Corporation Pty. Ltd. for Australia and New Zealand; by TransQuest Publishers Pte Ltd. for Singapore, Malaysia, Thailand, Indonesia, and Hong Kong; by Gotop Information Inc. for Taiwan; by ICG Muse, Inc. for Japan; by Intersoft for South Africa; by Eyrolles for France; by International Thomson Publishing for Germany, Austria, and Switzerland; by Distribuidora Cuspide for Argentina; by LR International for Brazil; by Galileo Libros for Chile; by Ediciones ZETA S.C.R. Ltda. for Peru; by WS Computer Publishing Corporation, Inc., for the Philippines; by Contemporanea de Ediciones for Venezuela; by Express Computer Distributors for the Caribbean and West Indies; by Micronesia Media Distributor, Inc. for Micronesia; by Chips Computadoras S.A. de C.V. for Mexico; by Editorial Norma de Panama S.A. for Panama; by American Bookshops for Finland. For general information on Hungry Minds’ products and services please contact our Customer Care department within the U.S. at 800-762-2974, outside the U.S. at 317-572-3993 or fax 317-572-4002. For sales inquiries and reseller information, including discounts, premium and bulk quantity sales, and foreign-language translations, please contact our Customer Care department at 800-434-3422, fax 317-572-4002 or write to Hungry Minds, Inc., Attn: Customer Care Department, 10475 Crosspoint Boulevard, Indianapolis, IN 46256. For information on licensing foreign or domestic rights, please contact our Sub-Rights Customer Care department at 212-884-5000. For information on using Hungry Minds’ products and services in the classroom or for ordering examination copies, please contact our Educational Sales department at 800-434-2086 or fax 317-572-4005. For press review copies, author interviews, or other publicity information, please contact our Public Relations department at 317-572-3168 or fax 317-572-4168. For authorization to photocopy items for corporate, personal, or educational use, please contact Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, or fax 978-750-4470. is a trademark of Hungry Minds, Inc. About the Author Mohammed Kabir is the founder and CEO of Evoknow, Inc. His company specializes in open-source solutions and customer relationship management software develop- ment. When he is not busy managing software projects or writing books, he enjoys traveling around the world. Kabir studied computer engineering at California State University, Sacramento. He is also the author of Red Hat Linux Server and Apache Server Bible. He can be reached at kabir@evoknow.com. Credits ACQUISITIONS EDITOR Debra Williams Cauley PROJECT EDITOR Pat O’Brien TECHNICAL EDITORS Matthew Hayden Sandra “Sam” Moore COPY EDITORS Barry Childs-Helton Stephanie Provines EDITORIAL MANAGER Kyle Looper RED HAT PRESS LIAISON Lorien Golaski, Red Hat Communications Manager SENIOR VICE PRESIDENT, TECHNICAL PUBLISHING Richard Swadley VICE PRESIDENT AND PUBLISHER Mary Bednarek PROJECT COORDINATOR Maridee Ennis GRAPHICS AND PRODUCTION SPECIALISTS Karl Brandt Stephanie Jumper Laurie Petrone Brian Torwelle Erin Zeltner QUALITY CONTROL TECHNICIANS Laura Albert Andy Hollandbeck Carl Pierce PERMISSIONS EDITOR Carmen Krikorian MEDIA DEVELOPMENT SPECIALIST Marisa Pearman PROOFREADING AND INDEXING TECHBOOKS Production Services This book is dedicated to my wife, who proofs my writing, checks my facts, and writes my dedications. Preface This book is focused on two major aspects of Red Hat Linux system administration: performance tuning and security. The tuning solutions discussed in this book will help your Red Hat Linux system to have better performance. At the same time, the practical security solutions discussed in the second half of the book will allow you to enhance your system security a great deal. If you are looking for time saving, practical solutions to performance and security issues, read on! How This Book is Organized The book has five parts, plus several appendixes. Part I: System Performance This part of the book explains the basics of measuring system performance, cus- tomizing your Red Hat Linux kernel to tune the operating system, tuning your hard disks, and journaling your filesystem to increase file system reliability and robustness. Part II: Network and Service Performance This part of the book explains how to tune your important network services, including Apache Web server, Sendmail and postfix mail servers, and Samba and NFS file and printer sharing services. Part III: System Security This part of the book covers how to secure your system using kernel-based Linux Intrusion Detection System (LIDS) and Libsafe buffer overflow protection mecha- nisms. Once you have learned to secure your Red Hat Linux kernel, you can secure your file system using various tools. After securing the kernel and the file system, you can secure user access to your system using such tools as Pluggable Authentication Module (PAM), Open Source Secure Socket Layer (OpenSSL), Secure Remote Password (SRP), and xinetd. Part IV: Network Service Security This part of the book shows how to secure your Apache Web server, BIND DNS server, Sendmail and postfix SMTP server, POP3 mail server, Wu-FTPD and ProFTPD FTP servers, and Samba and NFS servers. vi Part V: Firewalls This part of the book shows to create packet filtering firewall using iptables, how to create virtual private networks, and how to use SSL based tunnels to secure access to system and services. Finally, you will be introduced to an wide array of security tools such as security assessment (audit) tools, port scanners, log monitoring and analysis tools, CGI scanners, password crackers, intrusion detection tools, packet filter tools, and various other security administration utilities. Appendixes These elements include important references for Linux network users, plus an explanation of the attached CD-ROM. Conventions of This Book You don’t have to learn any new conventions to read this book. Just remember the usual rules: ◆ When you are asked to enter a command, you need press the Enter or the Return key after you type the command at your command prompt. ◆ A monospaced font is used to denote configuration or code segment. ◆ Text in italic needs to be replaced with relevant information. Watch for these icons that occasionally highlight paragraphs. The Note icon indicates that something needs a bit more explanation. The Tip icon tells you something that is likely to save you some time and effort. Preface vii The Caution icon makes you aware of a potential danger. The cross-reference icon tells you that you can find additional information in another chapter. Tell Us What You Think of This Book Both Hungry Minds and I want to know what you think of this book. Give us your feedback. If you are interested in communicating with me directly, send e-mail messages to kabir@evoknow.com. I will do my best to respond promptly. viii Red Hat Linux Security and Optimization Acknowledgments While writing this book, I often needed to consult with many developers whose tools I covered in this book. I want to specially thank a few such developers who have generously helped me present some of their great work. Huagang Xie is the creator and chief developer of the LIDS project. Special thanks to him for responding to my email queries and also providing me with a great deal of information on the topic. Timothy K. Tsai, Navjot Singh, and Arash Baratloo are the three members of the Libsafe team who greatly helped in presenting the Libsafe information. Very special thanks to Tim for taking the time to promptly respond to my emails and providing me with a great deal of information on the topic. I thank both the Red Hat Press and Hungry Minds teams who made this book a reality. It is impossible to list everyone involved but I must mention the following kind individuals. Debra Williams Cauley provided me with this book opportunity and made sure I saw it through to the end. Thanks, Debra. Terri Varveris, the acquisitions editor, took over in Debra’s absence. She made sure I had all the help needed to get this done. Thanks, Terri. Pat O’Brien, the project development editor, kept this project going. I don’t know how I could have done this book without his generous help and suggestions every step of the way. Thanks, Pat. Matt Hayden, the technical reviewer, provided numerous technical suggestions, tips, and tricks — many of which have been incorporated in the book. Thanks, Matt. Sheila Kabir, my wife, had to put up with many long work hours during the few months it took to write this book. Thank you, sweetheart. ix [...]... run the command mv linux linux.oldversion (oldversion is the version number of the current kernel) This renames the old kernel source directory, clearing the way for the installation of the new kernel source 3 Run the command ln -s /usr/src /linux- 2.4.1 linux This creates a new symbolic link, linux, that points to the /usr/src /linux- 2.4.1 directory 4 Change your directory path to /usr/src /linux At this... These days (and in this book) I try to overturn that mistaken notion; when I refer to Linux 2.4, I say Linux kernel 2.4, in distribution 7.x” to be as clear as possible Chapter 2: Kernel Tuning drwxrwxrwx — not rwxrwxrwx — is in the ls -l output 2 Run one of these commands: I If /usr/src /linux is a symbolic link, run the rm -f linux command This removes the symbolic link I If /usr/src /linux is a directory,... performance.” Today’s hardware and bandwidth — fast and relatively cheap — has spoiled many of us The long-running craze to buy the latest computer “toy” has lowered hardware pricing; the push to browse the Web faster has lowered bandwidth pricing while increasing its carrying capacity Today, you can buy 1.5GHz systems with 4GB of RAM and hundreds of GB of disk space (ultra-wide SCSI 160, at that) without taking... you can custom-compile your own kernel and tweak the installation process when you find the time When you do reach that point, however, the topics discussed in this chapter come in handy Compiling and Installing a Custom Kernel Thanks to the Linux kernel developers, creating a custom kernel in Linux is a piece of cake A Linux kernel is modular — the features and functions you want can be installed... /usr/src /linux is a symbolic link to the current source distribution of the kernel For example, on my system, ls -l reports this: lrwxrwxrwx 2.4.0 1 root root 11 Feb 13 16:21 linux -> linux- Distribution versus kernel — what’s the “real” version? New Linux users often get confused when the version numbers of the distribution and the kernel mismatch Why (they ask) do I keep talking about Linux 2.4 when what... programmers have developed the basic kernel of Linux code in diverse directions — like variations on a theme Each variation has a series of distributions and a body of users to whom it is distributed Thanks to popular, easy-to-recognize distributions like Red Hat Linux, many newcomers think distribution 7.x of Linux is the “only” — or the “latest” — version (and that everything in it is uniformly “version... Security 399 E-Mail Server Security 415 FTP Server Security 443 Samba and NFS Server Security 473 Part V Firewalls Chapter 20 Chapter 21 Firewalls, VPNs, and SSL Tunnels 491 Firewall Security Tools 541 Appendix Appendix Appendix Appendix Appendix IP Network Address Classification Common Linux. .. huge amount of bandwidth in the U.S — even in most metropolitan homes Hardware and bandwidth have become commodities in the last few years — but are we all happy with the performance of our systems? Most users are likely to agree that even with phenomenal hardware and bandwidth, their computers just don’t seem that fast anymore — but how many people distinguish between two systems that seem exactly... /proc/sys/fs/file-nr /proc/sys/fs/inode-nr every 30 seconds Summary Knowing how to measure system performance is critical in understanding bottlenecks and performance issues Using standard Red Hat Linux tools, you can measure many aspects of your system’s performance Tools such as ps, top, and vmstat tell you a lot of how a system is performing Mastering these tools is an important step for anyone interested... new kernel N Allocating file handles for demanding applications IF YOU HAVE INSTALLED THE BASIC Linux kernel that Red Hat supplied, probably it isn’t optimized for your system Usually the vendor-provided kernel of any OS is a “generalist” rather than a “specialist” — it has to support most installation scenarios For example, a run-of-the-mill kernel may support both EIDE and SCSI disks (when you need . S E C U R I T Y T O O L S O N C D - R O M ® PRESS ® Linux Solutions from the Experts at Red Hat Mohammed J. Kabir ® ® ™ Red Hat Linux Security and Optimization Mohammed J. Kabir Hungry Minds,. respond promptly. viii Red Hat Linux Security and Optimization Acknowledgments While writing this book, I often needed to consult with many developers whose tools I covered in this book. I want. writing, checks my facts, and writes my dedications. Preface This book is focused on two major aspects of Red Hat Linux system administration: performance tuning and security. The tuning solutions