hack proofing sun solaris 8 - protect your solaris network from attack

From the authors of the bes-selling HACK PROOFING ™ YOUR NETWORK ™ 1YEAR UPGRADE BUYER PROTECTION PLAN From the authors of the bestselling HACK PROOFING ™ YOUR NETWORK Protect Your Solaris Network from Attack • Complete Coverage of Solaris 8 C2 and Trusted Solaris 8 • Hundreds of Damage & Defense,Tools & Traps,and Notes from the Underground Sidebars,Security Alerts,and FAQs • Step-by-Step Instructions for Making the Most of Solaris 8 Security Enhancements Wyman Miles Ed Mitchell F. William Lynch Randy Cook Technical Editor 158_hack_sun_FC 11/11/01 2:46 PM Page 1 solutions@syngress.com With more than 1,500,000 copies of our MCSE, MCSD, CompTIA, and Cisco study guides in print, we continue to look for ways we can better serve the information needs of our readers. One way we do that is by listening. Readers like yourself have been telling us they want an Internet-based ser- vice that would extend and enhance the value of our books. Based on reader feedback and our own strategic plan, we have created a Web site that we hope will exceed your expectations. Solutions@syngress.com is an interactive treasure trove of useful infor- mation focusing on our book topics and related technologies. The site offers the following features: ■ One-year warranty against content obsolescence due to vendor product upgrades. You can access online updates for any affected chapters. ■ “Ask the Author”™ customer query forms that enable you to post questions to our authors and editors. ■ Exclusive monthly mailings in which our experts provide answers to reader queries and clear explanations of complex material. ■ Regularly updated links to sites specially selected by our editors for readers desiring additional reliable information on key topics. Best of all, the book you’re now holding is your key to this amazing site. Just go to www.syngress.com/solutions, and keep this book handy when you register to verify your purchase. Thank you for giving us the opportunity to serve your needs. And be sure to let us know if there’s anything else we can do to help you get the maximum value from your investment. We’re listening. www.syngress.com/solutions 158_HPsun_FM 10/5/01 5:07 PM Page i 158_HPsun_FM 10/5/01 5:07 PM Page ii Wyman Miles Ed Mitchell F. William Lynch Randy Cook Technical Editor ™ 1YEAR UPGRADE BUYER PROTECTION PLAN 158_HPsun_FM 10/5/01 5:08 PM Page iii Syngress Publishing, Inc., the author(s), and any person or firm involved in the writing, editing, or production (collectively “Makers”) of this book (“the Work”) do not guarantee or warrant the results to be obtained from the Work. There is no guarantee of any kind, expressed or implied, regarding the Work or its contents.The Work is sold AS IS and WITHOUT WARRANTY. You may have other legal rights, which vary from state to state. In no event will Makers be liable to you for damages, including any loss of profits, lost savings, or other incidental or consequential damages arising out from the Work or its contents. Because some states do not allow the exclusion or limitation of liability for consequential or incidental damages, the above limitation may not apply to you. You should always use reasonable care, including backup and other appropriate precautions, when working with computers, networks, data, and files. Syngress Media®, Syngress®, and “Career Advancement Through Skill Enhancement®,” are registered trademarks of Syngress Media, Inc. “Ask the Author UPDATE™,” “Mission Critical™,”“Hack Proofing™,” and “The Only Way to Stop a Hacker is to Think Like One™” are trademarks of Syngress Publishing, Inc. Brands and product names mentioned in this book are trademarks or service marks of their respective companies. KEY SERIAL NUMBER 001 EAFRET4KDG 002 23PVFDAT5Q 003 VZPE43GHBA 004 MNFT6Y456F 005 QL3R3BNM65 006 KMXV94367H 007 NSE4T63M5A 008 P3JR9DF9GD 009 XP93QNFTY6 010 VK495YDR45 PUBLISHED BY Syngress Publishing, Inc. 800 Hingham Street Rockland, MA 02370 Hack Proofing Sun Solaris 8 Copyright © 2001 by Syngress Publishing, Inc.All rights reserved. Printed in the United States of America. Except as permitted under the Copyright Act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written permission of the publisher, with the exception that the program listings may be entered, stored, and executed in a computer system, but they may not be reproduced for publication. Printed in the United States of America 1 2 3 4 5 6 7 8 9 0 ISBN: 1-928994-44-X Technical Editor: Randy Cook Freelance Editorial Manager: Maribeth Corona-Evans Technical Reviewer: Ryan Ordway Cover Designer: Michael Kavish Co-Publisher: Richard Kristof Page Layout and Art by: Shannon Tozier Acquisitions Editor: Catherine B. Nolan Copy Editors: Alexandra Kent and Darlene Bordwell Developmental Editor: Jonathan Babcock Indexer: Claire A. Splan Distributed by Publishers Group West in the United States and Jaguar Book Group in Canada. 158_HPsun_FM 10/5/01 5:08 PM Page iv v Acknowledgments v We would like to acknowledge the following people for their kindness and support in making this book possible. Richard Kristof and Duncan Anderson of Global Knowledge, for their generous access to the IT industry’s best courses, instructors, and training facilities. Ralph Troupe, Rhonda St. John, and the team at Callisma for their invaluable insight into the challenges of designing, deploying, and supporting world-class enterprise networks. Karen Cross, Lance Tilford, Meaghan Cunningham, Kim Wylie, Harry Kirchner, Kevin Votel, Kent Anderson, and Frida Yara of Publishers Group West for sharing their incredible marketing experience and expertise. Mary Ging, Caroline Hird, Simon Beale, Caroline Wheeler,Victoria Fuller, Jonathan Bunkell, and Klaus Beran of Harcourt International for making certain that our vision remains worldwide in scope. Anneke Baeten and Annabel Dent of Harcourt Australia for all their help. David Buckland,Wendi Wong, Daniel Loh, Marie Chieng, Lucy Chong, Leslie Lim, Audrey Gan, and Joseph Chan of Transquest Publishers for the enthusiasm with which they receive our books. Kwon Sung June at Acorn Publishing for his support. Ethan Atkin at Cranbury International for his help in expanding the Syngress program. 158_HPsun_FM 10/5/01 5:08 PM Page v 158_HPsun_FM 10/5/01 5:08 PM Page vi vii Contributors Hal Flynn is a Threat Analyst at SecurityFocus, the leading provider of Security Intelligence Services for Business. Hal functions as a Senior Analyst, performing research and analysis of vulnerabilities, malicious code, and network attacks. He provides the SecurityFocus team with UNIX and network expertise. He is also the manager of the UNIX Focus Area and moderator of the Focus-Sun, Focus-Linux, Focus-BSD, and Focus-GeneralUnix mailing lists. Hal has worked the field in jobs as varied as the Senior Systems and Network Administrator of an Internet Service Provider, to contracting the United States Defense Information Systems Agency, to Enterprise-level consulting for Sprint. He is also a proud veteran of the United States Navy Hospital Corps, having served a tour with the 2nd Marine Division at Camp Lejeune, NC as a Fleet Marine Force Corpsman. Hal is mobile, living between sunny Phoenix,AZ and wintry Calgary,Alberta, Canada. Rooted in the South, he currently calls Montgomery,AL home. Ido Dubrawsky (CCNA, SCSA) is a Network Security Engineer and a member of Cisco’s Secure Consulting Services in Austin,TX. He cur- rently conducts security posture assessments for clients as well as provides technical consulting for security design reviews. His strengths include Cisco routers and switches, PIX firewall, Solaris systems, and freeware intrusion detection systems. Ido holds a bachelor’s and a master’s degree from the University of Texas at Austin and is a member of USENIX and SAGE. He has written several articles covering Solaris security and net- work security for Sysadmin magazine as well as SecurityFocus.com. He lives in Austin,TX with his family. Drew Simonis (CCNA, SCSA, SCNA, CCSA, CCSE, IBM CS) is co- author of Hack Proofing Your Web Applications (ISBN: 1-928994-31-8) and is a Senior Security Engineer with the RL Phillips Group, LLC. He cur- rently provides senior level security consulting to the United States Navy, working on large enterprise networks. He considers himself a security 158_HPsun_FM 10/5/01 5:08 PM Page vii viii generalist, with a strong background in system administration, Internet application development, intrusion detection and prevention, and penetra- tion testing. Drew’s background includes a consulting position with Fiderus, serving as a Security Architect with AT&T and as a Technical Team Lead with IBM. Drew has a bachelor’s degree from the University of South Florida and is also a member of American MENSA. Drew cur- rently lives in Suffolk,VA with his wife Kym and daughters Cailyn and Delaney. Mike Lickey is a Senior Engineer for IPC Technologies in Richmond, VA. He has 20 years experience in systems administration working with the real-time production server environment, specializing in critical up- time systems. He has worked for IPC Technologies for almost ten years, providing broad support for all platforms.As a consultant, he has worked almost exclusively with Fortune 100 companies working with multiple systems and networking architectures. He has extensive experience with system security starting in 1985 when he got his first systems administra- tion position. Mike has lived in Richmond with his wife Deborah for almost 25 years. He received his bachelor’s degree in English from Virginia Commonwealth University. F. William Lynch (SCSA, CCNA, MCSE, MCP,A+) is an Independent Security and Systems Administration consultant in Denver, CO. His spe- cialties include firewalls,VPNs, security auditing, documentation, systems performance analysis, Solaris and open source operating systems such as OpenBSD, FreeBSD, and Linux. He has served as a consultant to multina- tional corporations and the Federal government including the Centers for Disease Control and Prevention headquarters in Atlanta, GA as well as various airbases of the United States Air Force.William is also the founder and director of the MRTG-PME project, which uses the MRTG engine to track systems performance of various UNIX operating systems.William holds a bachelor’s degree in Chemical Engineering from the University of Dayton in Dayton, OH and a master’s degree in Business Administration from Regis University in Denver, CO. 158_HPsun_FM 10/5/01 5:08 PM Page viii ix Edward Mitchell is the Network Operations Manager for ADC Telecommunication’s Enhanced Services Division in San Jose, CA. He oversees a large multi-platform UNIX environment with a Cisco-based infrastructure and is responsible for all aspects of network and system security. Prior to ADC, Edward spent time with the State of California as an independent consultant for a variety of network security projects. Edward also provides security and disaster recovery consulting services for a variety of clients and actively participates in various incident response teams and events. He currently resides in California’s Central Valley and appreciates the patience and understanding his wife displayed during his contribution to this book. Wyman Miles is the Senior Systems Administrator and Technical Manager for Educational Technology at Rice University. In this role, Wyman handles Solaris security for a large, distributed network. He also advises on security matters for other divisions within Information Technology. Some of his developments in security technology, including Kerberos deployment tools, SSL proxies, and wireless network security have been presented at academic conferences around the country.Though the focus of his work has been cryptography,Wyman handles all aspects of network and host-based security for the academic network.Wyman holds a bachelor’s degree in Physics with a minor in English. He resides in Houston,TX with his wife Erica. 158_HPsun_FM 10/5/01 5:08 PM Page ix [...]... Content-Length Header Summary Solutions Fast Track Frequently Asked Questions 265 266 266 266 267 267 269 271 272 274 274 274 275 276 277 Chapter 10 Dissecting Hacks Introduction Securing against Denial of Service Hacks Ping of Death Syn Flood E-Mail Flood 287 288 288 289 290 294 277 2 78 279 279 281 281 282 282 283 284 284 286 1 58_ HPsun_toc xviii 10 /8/ 01 10:56 AM Contents Securing against Brute Force Hacks... DNS Services on Solaris 173 Using BIND 174 Setting Up a chroot Jail for BIND 174 Securing Zone Transfers in BIND 8 180 Configuring Solaris to Provide Anonymous FTP Services 181 Using X-Server Services Securely 182 Using Host-Based Authentication 183 Using User-Based Authentication 183 Using X-Windows Securely with SSH 186 Using Remote Commands 187 Using Built-In Remote Access Methods 187 Using SSH for... Analyzing Trusted Solaris 8 Solaris 8 Security Enhancements Using SunScreen Secure Net Utilizing SunScreen SKIP Utilizing SKIP’s VPN Capabilities Using the Solaris Security Toolkit Working with the Solaris Security Toolkit’s System Files Using OpenSSH Summary Solutions Fast Track Frequently Asked Questions 33 34 35 38 40 42 43 44 45 47 48 50 53 54 55 55 56 56 58 58 59 61 61 63 1 58_ HPsun_toc 10 /8/ 01 10:56... variants, including Solaris Figure 1.3 details an Nmap scan of a default Solaris host from a Linux-based host (Scanning from a Solaris host would yield an identical output.) Figure 1.3 An Nmap Scan of a Default Solaris Host from a Linux-Based Host www.syngress.com 9 1 58_ HPsun_01 10 10/4/01 5:06 PM Page 10 Chapter 1 • Introducing Solaris Security: Evaluating Your Risk As you can see, Solaris includes a... truth:The hackers are out there and they want your sun ower seeds —Randy Cook, SCSA Technical Editor www.syngress.com 1 58_ HPsun_fore 10/4/01 5: 38 PM Page xxiv 1 58_ HPsun_01 10/4/01 5:06 PM Page 1 Chapter 1 Introducing Solaris Security: Evaluating Your Risk Solutions in this chapter: s Exposing Default Solaris Security Levels s Evaluating Current Solaris Security Configurations s Monitoring Solaris Systems... Fast Track Frequently Asked Questions Chapter 5 Securing Your Files Introduction Establishing Permissions and Ownership Access Control Lists Role-Based Access Control /etc/user_attr user:qualifier:res1:res2:attr xiii 67 68 71 76 77 79 81 82 83 86 88 93 94 96 99 100 101 103 104 107 109 115 122 122 125 127 1 28 129 132 135 136 1 58_ HPsun_toc xiv 10 /8/ 01 10:56 AM Page xiv Contents /etc/security/auth_attr... Frequently Asked Questions xix 347 347 349 349 350 350 351 351 352 353 357 3 58 359 Hack Proofing Sun Solaris 8 Fast Track 361 Index 381 1 58_ HPsun_toc 10 /8/ 01 10:56 AM Page xx 1 58_ HPsun_fore 10/4/01 5: 38 PM Page xxi Foreword Many years ago, my father decided to put a birdfeeder in our backyard It was great From our breakfast table we could see all kinds of birds visiting our yard However, it soon became... installed Solaris 8 system.We also go over the basics of testing, monitoring, and documenting security procedures Next, in Chapter 2, we cover the standard security tools available from Sun Microsystems.This includes an overview of Sun s BSM product and a look at the features of Sun s Trusted Solaris 8 In Chapter 3, we introduce third-party security tools which are commonly used to secure and monitor Solaris. .. authorized users and still deny unauthorized access? xxi 1 58_ HPsun_fore xxii 10/4/01 5: 38 PM Page xxii Foreword Luckily, as Solaris System Administrators, we have some excellent tools available to us Sun Microsystems has spent a great deal of effort in designing Solaris to be both stable and secure.This book is your reference guide for not only securing your Solaris systems, but also for securing the environment... system indicates your consent LOG OFF IMMEDIATELY if you do not agree to these conditions www.syngress.com 1 58_ HPsun_01 10/4/01 5:06 PM Page 9 Introducing Solaris Security: Evaluating Your Risk • Chapter 1 Evaluating Current Solaris Security Configurations When hardening a default Solaris installation, it is crucial to examine services running both on the network and on the local host itself .Your goal during . From the authors of the bes-selling HACK PROOFING ™ YOUR NETWORK ™ 1YEAR UPGRADE BUYER PROTECTION PLAN From the authors of the bestselling HACK PROOFING ™ YOUR NETWORK Protect Your Solaris. BIND 8 180 Configuring Solaris to Provide Anonymous FTP Services 181 Using X-Server Services Securely 182 Using Host-Based Authentication 183 Using User-Based Authentication 183 Using X-Windows. 284 Frequently Asked Questions 286 Chapter 10 Dissecting Hacks 287 Introduction 288 Securing against Denial of Service Hacks 288 Ping of Death 289 Syn Flood 290 E-Mail Flood 294 Configuring Squid Services Q: