intrusion detection with snort

Intrusion Detection with Snort Sams Publishing,800 East 96th Street,Indianapolis,Indiana 46240 Jack Koziol 00 157870281x FM.qxd 4/30/03 12:36 PM Page i Intrusion Detection with Snort Copyright © 2003 by Sams Publishing All rights reserved. No part of this book shall be reproduced, stored in a retrieval system, or transmitted by any means, electronic, mechanical, photocopying, recording, or otherwise, without written permission from the publisher. No patent liability is assumed with respect to the use of the information contained herein.Although every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omis- sions. Nor is any liability assumed for damages resulting from the use of the information contained herein. International Standard Book Number: 1-578-70281-X Library of Congress Catalog Card Number: 2002110728 Printed in the United States of America First Printing: May 2003 06 05 04 03 4 3 2 Trademarks All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capitalized. Sams Publishing cannot attest to the accuracy of this information. Use of a term in this book should not be regarded as affecting the validity of any trademark or service mark. Wa r ning and Disclaimer Every effort has been made to make this book as complete and as accurate as possible, but no warranty or fitness is implied.The infor- mation provided is on an “as is” basis. Bulk Sales Sams Publishing offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales. For more information, please contact: U.S. Corporate and Government Sales 1-800-382-3419 corpsales@pearsontechgroup.com For sales outside of the U.S., please contact: International Sales +1-317-581-3793 international@pearsontechgroup.com Acquisitions Editors Linda Bump Jenny Watson Development Editor Mark Cierzniak Managing Editor Charlotte Clapp Project Editor George E. Nedeff Copy Editor Margo Catts Indexer Kelly Castell Proofreader Leslie Joseph Technical Editors Stephen Halligan Bryce Alexander Team Coordinator Vanessa Evans Multimedia Developer Dan Scherf Designer Gary Adair Page Layout Julie Parks 00 157870281x FM.qxd 6/5/03 10:28 AM Page ii ❖ For Paul Noeldner, who first aroused my interest in computing ❖ 00 157870281x FM.qxd 4/30/03 12:36 PM Page iii Contents at a Glance Introduction xix 1 Intrusion Detection Primer 1 2 Intrusion Detection with Snort 23 3 Dissecting Snort 43 4 Planning for the Snort Installation 69 5 The Foundation—Hardware and Operating Systems 89 6 Building the Server 105 7 Building the Sensor 143 8 Building the Analyst’s Console 173 9 Additional Installation Methods 189 10 Tuning and Reducing False Positives 207 11 Real-Time Alerting 233 12 Basic Rule Writing 251 13 Upgrading and Maintaining Snort 279 14 Advanced Topics in Intrusion Prevention 293 A Troubleshooting 313 B Rule Documentation 319 Index 325 00 157870281x FM.qxd 4/30/03 12:36 PM Page iv Table of Contents 1Intrusion Detection Primer 1 IDSs Come in Different Flavors 2 Host-Based IDS 2 Network-Based IDS 3 A Mixed Approach 5 Methods of Detecting Intrusions 5 Signature Detection 5 Anomaly Detection 6 Integrity Verification 7 Origin of Attacks 8 External Threats 8 Internal Threats 9 Orchestrating an Attack 10 Planning Phase 11 The Reconnaissance Phase 11 The Attack Phase 15 Post-Attack Phase 19 The IDS Reality 20 IDSs Cannot Detect Every Attack 20 Intrusion Detection is Reactive 20 Deploying and Maintaining Is Difficult 20 Summary 21 2Network Intrusion Detection with Snort 23 Snort’s Specifications 24 Requirements 24 Bandwidth Considerations 25 Snort Is an Open Source Application 25 Detecting Suspicious Traffic via Signatures 26 Out of Spec Traffic 27 Detecting Suspicious Payloads 27 Detecting Specific Protocol Elements 28 Extending Coverage with Custom Rules 28 Detecting Suspicious Traffic via Heuristics 29 00 157870281x FM.qxd 4/30/03 12:36 PM Page v vi Contents Gathering Intrusion Data 29 Assessing Threats 30 Preprocessors 30 Non-Signature-Matching Detection 31 Alerting via Output Plug-ins 32 Aggregating Data 32 Logging with the Unified Format and Barnyard 33 Alerting 33 Prioritizing Alerts 34 No Prioritization 34 Hard-coded Prioritization 34 Customizable Prioritization 34 Distributed Snort Architecture 35 First Tier—The Sensor Tier 35 Second Tier—The Server Tier 37 The Third Tier—The Analyst’s Console 38 Securing Snort 38 Shortcomings 38 Flexibility Breeds Complexity 38 Problems with False Positives 39 Marketplace Factors 40 Summary 40 3Dissecting Snort 43 Feeding Snort Packets with Libpcap 44 Packet Decoder 45 Preprocessors 46 frag2 46 stream4 49 stream4_reassemble 51 HTTP_decode 52 RPC_decode 54 BO 55 Te lnet_decode 55 ARPspoof 56 ASN1_decode 57 00 157870281x FM.qxd 4/30/03 12:36 PM Page vi vii Contents fnord 57 conversation 58 portscan2 59 SPADE 60 The Detection Engine 61 Output Plugins 62 Alert_fast 62 Alert_full 62 Alert_smb 62 Alert_unixsock 63 Log_tcpdump 63 CSV 63 XML 64 Alert_syslog 65 Database 65 Unified 67 Summary 67 4Planning for the Snort Installation 69 Defining an IDS Policy 70 Malicious Activity 71 Suspicious Activity 71 Abnormal Activity 72 Inappropriate Activity 73 Deciding What to Monitor 74 External Network Connections 74 Internal Network Chokepoints 76 Critical Computing Resources 76 Designing Your Snort Architecture 76 Three-Tier 77 Single Tier 78 Monitoring Segment 78 Planning for Maintenance 79 Incident Response Plan 80 The Objective 81 Establishing a Notification Chain 82 00 157870281x FM.qxd 4/30/03 12:36 PM Page vii viii Contents Responding to an Incident 83 Identifying an Incident 84 Classifying the Incident 85 Gathering Evidence 85 Restoring to a Normal State 86 Te sting the Plan 86 Summary 87 5The Foundation—Hardware and Operating Systems 89 Hardware Performance Metrics 89 Ruleset and Configuration Settings 89 Picking a Platform 92 The Monitoring Segment 94 Inline Hub 95 SPAN Ports 98 Taps 100 Distributing Traffic to Multiple Sensors 101 Summary 102 6 Building the Server 105 Installation Guide Notes 105 Red Hat Linux 7.3 105 Partitioning Strategy 106 Network Configuration 106 Firewall Configuration 106 Time Zone Selection 107 Account Configuration 107 Package Group Selection 107 Post-Installation Tasks 108 Bastille Linux 108 Installing the Snort Server Components 111 Installing OpenSSL 112 Installing Stunnel 114 Installing OpenSSH 117 Downloading Apache 120 Installing MySQL 121 00 157870281x FM.qxd 4/30/03 12:36 PM Page viii ix Contents Configuring mod_ssl 124 Installing gd 125 PHP 127 Installing Apache 129 Installing ADODB 133 Installing ACID 134 Summary 140 7 Building the Sensor 143 Installation Guide Notes 143 Red Hat Linux 7.3 144 Post-Installation Tasks 145 Installing the Snort Sensor Components 147 Installing libpcap 147 Installing tcpdump 148 Installing OpenSSL 149 Installing Stunnel 150 Installing OpenSSH 151 Installing the MySQL Client 152 Installing NTP 152 Installing Snort 153 Configuring snort.conf 155 Running Snort 166 Implementing Barnyard 166 Configuring barnyard.conf 167 Running Barnyard 169 Automating with barnyard.server 171 Summary 171 8 Building the Analyst’s Console 173 Windows 174 Installing SSH 174 Web Browser 175 Linux 175 Installing OpenSSH 175 Web Browser 175 Te sting the Console 176 00 157870281x FM.qxd 4/30/03 12:36 PM Page ix x Contents Wor king with ACID 177 Searching 178 Alert Groups 186 Summary 188 9 Additional Installation Methods 189 The Hybrid Server/Sensor 189 Snort on OpenBSD 191 SnortSnarf 192 Snort on Windows 193 Setting Up the Windows Installation 193 Installing the Underlying Programs 195 Installing the Snort Application 201 Installing IDScenter 202 Summary 205 10 Tuning and Reducing False Positives 207 Pre-Tuning Activities 208 Tuning the Network for Snort 210 Filtering Traffic with Snort 211 Network Variables 211 BPFs 212 Tuning the Preprocessors 213 bo 213 arpspoof, asn1_decode, and fnord 213 frag2 214 stream4 217 stream4_reassemble 218 http_decode, rpc_decode, and telnet_decode 218 portscan2 and conversation 219 Refining the Ruleset 219 chat.rules 221 ddos.rules 221 ftp.rules 221 icmp-info.rules 222 icmp-info.rules 222 00 157870281x FM.qxd 4/30/03 12:36 PM Page x [...]... documentation.While there is definitely a large amount of documentation on Snort, it is often inadequate and assumes the reader has some prior experience with Snort or Intrusion Detection (usually as a profession) The goal of this book is to arm you with an arsenal of open source intrusion detection tools centered on Snort Snort makes an excellent Intrusion Detection System (IDS), but this is where it ends It lacks... WRITING INTRUSION DETECTION WITH SNORT has been to deliver the first comprehensive guide to using Snort in a real-world environment Having worked in the field of intrusion detection in both small and large organizations, and having used a wide variety of intrusion detection technologies, I felt it was necessary to provide a book that covers one of the best kept secrets in the security industry Snort Snort... hosts Most organizations start their foray into intrusion detection with an NIDS After growing accustomed to intrusion detection they gradually place HIDSs on hosts that are critical to day-to-day operation.This methodology gives complete intrusion detection coverage for an organization Methods of Detecting Intrusions IDSs have several methods of detecting intrusions at their disposal Certain techniques... That Never Create a False Positive 296 Snort Inline Patch 297 Installing Snort Inline Patch 298 Configuring 299 Writing Rules for Inline Snort 300 Building the Ruleset 301 SnortSam 303 Installing SnortSam 304 Configuring 305 Inserting Blocking Responses into Rules 310 Summary 312 A Troubleshooting 313 Snort Issues 313 How Do I Run Snort on Multiple Interfaces? 313 Snort Complains About Missing References... Overview of Real-Time Alerting with Snort 233 Prioritization of Alerts 234 Incidents 235 Targeted Attacks 235 Custom Rules 235 Prioritizing with classification.config 236 The priority Option 237 Alerting with the Hybrid 237 Installing Swatch 238 Configuring Swatch 239 -c 240 -input-record-separator 240 -p 241 -t 241 -daemon 241 Alerting with Distributed Snort 241 Configuring Snort and Installing Sendmail... book would not be complete without a meticulous discussion of how Snort works from the inside out Chapter 3, “Dissecting Snort, ” is dedicated to Snort s internal functions and sparsely documented components, such as the preprocessors that dictate how Snort behaves After you have developed strong working knowledge of how Snort works, I dedicated Chapter 4, “Planning for the Snort Installation,” to guide... Positives.” Another important configuration task, getting Snort to send out alerts in real time, is covered in Chapter 11, “Real-Time Alerting.” Chapters 12 through 14 deal with more advanced issues, such as writing custom Snort signatures (termed rules), upgrading Snort, and using Snort as an Intrusion Prevention device One of the greatest assets of Snort that separates it from closed source, commercial,... 4/30/03 12:35 PM Page 1 1 Intrusion Detection Primer I NTRUSION DETECTION SYSTEMS (IDSS) HAVE EVOLVED into a critical component in secure network architecture Nonetheless, IDSs are a foreign concept to many security practitioners and systems administrators.This chapter offers a brief synopsis of intrusion detection, and illustrates why IDS is an important technology An Intrusion Detection System is any... 280 IDS Policy Manager 280 Installing 280 Configuring 282 SnortCenter 284 Installing SnortCenter 285 The SnortCenter Sensor Agent 287 Configuring 288 Upgrading Snort 289 Summary 291 14 Advanced Topics in Intrusion Prevention 293 A Warning Concerning Intrusion Prevention 294 00 157870281x FM.qxd 4/30/03 12:36 PM Page xiii Contents Planning an Intrusion Prevention Strategy 295 Unpatched Servers 296 New... negatives are a distinct possibility Even with the issues with signature detection, IDSs that utilize it are the most prominent and reliable on the market today Anomaly Detection Anomaly detection detects misuse by measuring a norm over time and then generating an alert when patterns differ from the norm Anomaly detection comes in many different forms Anomaly detection can be used at the application . experience with Snort or Intrusion Detection (usually as a profession). The goal of this book is to arm you with an arsenal of open source intrusion detection tools centered on Snort. Snort makes. iii Contents at a Glance Introduction xix 1 Intrusion Detection Primer 1 2 Intrusion Detection with Snort 23 3 Dissecting Snort 43 4 Planning for the Snort Installation 69 5 The Foundation—Hardware. Intrusion Detection with Snort Sams Publishing,800 East 96th Street,Indianapolis,Indiana 46240 Jack Koziol 00 157870281x FM.qxd 4/30/03 12:36 PM Page i Intrusion Detection with Snort Copyright