Network Traffic Analysis Using tcpdump

1 1 Network Traffic Analysis Using tcpdump Judy Novak Judy Novak Johns Hopkins University Applied Physics Laboratory jhnovak@ix.netcom.com Introduction to tcpdump All material Copyright  Novak, 2000, 2001. All rights reserved. 2 2 Table of Contents Topics Introduction to tcpdump Writing tcpdump Filters Examination of Datagram Fields Beginning Analysis Real World Examples Step by Step Analysis References 3 3 Course Objectives • Introduce the fundamentals of tcpdump • Explain how to write tcpdump filters • Examine fields in datagram for uses/misuses • Analyze traffic by placing it in categories • Demonstrate “real-world” analysis using tcpdump • Let you participate in the analysis process The objectives of this course are to introduce you to the fundamentals and benefits of using tcpdump as a tool to analyze your network traffic. We’ll start with introducing concepts and output of tcpdump. One of the most important aspects of using tcpdump is being able to write tcpdump filters to look for specific traffic. Filter writing is fairly basic unless you want to examine fields in an IP datagram that don’t fall on byte boundaries. So, that is why an entire section is devoted to the art of writing filters. Before we start to use tcpdump to analyze traffic, we’ll examine many of the fields found in the IP datagram. This is done to familiarize you with those fields in theory and also how they might be used in practice. We’ll study how and why fields might be changed and for what purpose. Next, we’ll start the basic analysis process by looking at tcpdump output and categorizing the kind of traffic that you can see. Then, we’ll take a look at some real-world examples and of how tcpdump was used on monitored networks to discover what was happening. Next, the analysis process will be inspected step by step often with missteps to get you comfortable with it. As a note, all tcpdump output shown in this course is activity that actually occurred. Source and destination hosts/IP’s have been altered to obfuscate the true identities. 4 4 Overview • Introduction to tcpdump • Writing tcpdump filters • Examination of Datagram Fields • Beginning Analysis • Real World Examples • Step by Step Analysis This page intentionally left blank. 5 5 Introduction to tcpdump • Introduction to tcpdump • Writing tcpdump Filters • Examination of Datagram Fields • Beginning Analysis • Real World Examples • Step by Step Analysis This page intentionally left blank. 6 6 Objectives • Examine the strengths/weaknesses of tcpdump • Organize collection/analysis process of tcpdump data via Shadow • Examine tcpdump output • Standard • Hexadecimal • Length fields and how to convert them to bytes • Application layer • Interpretation of payload/hex output This page intentionally left blank. 7 7 Introduction This page intentionally left blank. 8 8 Strengths • Provides audit trail/historical record of network activity • Provides absolute fidelity • Universally available and used A One of the most important parts of an arsenal in your security infrastructure is at least one tool or software package that captures an audit trail or a historical record of the traffic that enters or leaves your network. There will be times when you will be required to examine activity or connections that occurred in your network – not just traffic that caused an alarm to sound. For instance, what if you suspect that your packet filtering router that acts as your perimeter defense was acting strangely after some major network changes were made. You would have to examine the traffic that was allowed into your network to assist in determining the problem. That is where tcpdump is invaluable. Also, many tools - even logs from firewalls will display suspicious traffic, yet only partial data is displayed. What if you get a log of rejected traffic, but it doesn’t display or keep TCP flags? You’ll never know what kind of connection was attempted. tcpdump allows the analyst to examine all the bits and fields that are collected. If nothing is “wrong” with the connection, examination at the bit level is unnecessary. Yet, if you suspect something “foul” with the traffic, you really need access to all the data down to the bit level. And tcpdump is a tool that is universally used and very portable. If you become familiar with this software or its Windows counterpart, windump, it can be used on just about any platform to assist you in analysis of traffic. 9 9 Weaknesses • By default, doesn’t collect all the payload • Does not scale well on large networks • No idea of state • Limited operations • Do-it-yourself interpretations tcpdump will capture 68 bytes of data from the network interface. Some of this data might be used for the link layer frame header. For Ethernet, 14 bytes of the data are used to capture fields like the source and destination MAC address, along with the type of embedded data. That leaves only 54 bytes to capture the IP header and embedded protocol header as well as any data. Most of the time this size will allow you to capture the IP header and embedded protocol header. But, sometimes protocol headers or data will be truncated. And, if you are interested in the data payload, tcpdump is really not the tool to use for this. tcpdump can collect a large volume of data for larger networks. This can be alleviated by not collecting all the data on the network – perhaps omit web traffic (port 80). Or, another way to deal with this is more disk space and faster processors to analyze all the collected data. But, at some point, the volume gets unwieldy. tcpdump blindly collects packet after packet. It has no idea of state or being able to know that a given packet is anomalous because it does not follow the flow of a normal connection. And while tcpdump has some primitive arithmetic operations or ways to manipulate bits, it cannot do complex operations for analyzing data. Finally, while it is an excellent way to collect data, tcpdump does not attempt to make interpretations of what it sees. It does have some integrity checking operations for certain data to make sure that the data is not irregular, but the analyst has to have the training and savvy to interpret the data. For the sophisticated analyst, this is a bonus because she or he can make the correct call. Compare this with a tool that is prone to false positives that gives no way of verifying the alarmed event. But, for an analyst who has little training, tcpdump can be daunting since it does not interpret events. 10 10 tcpdump Versions • tcpdump: Unix version; official current version 3.4 • ftp://ftp.ee.lbl.gov/tcpdump.tar.Z • ftp://ftp.ee.lbl.gov/libpcap.tar.Z • windump: Windows version • http://netgroup-serv.polito.it/windump • http://netgroup-serv.polito.it/winpcap • Collective effort; current version 3.5: www.tcpdump.org • tcpdump-3.5.tar.gz • libpcap-0.5.tar.gz tcpdump is officially supported by the Lawrence Berkeley Labs. The current version is 3.4. There is an effort to improve tcpdump and patch known problems with tcpdump and libpcap that appears to be a collective effort of anyone interested. The software for this effort can be found at www.tcpdump.org. Their current version is 3.5 For the Unix versions of tcpdump, you need to download software known as libpcap that implements a portable framework for capturing low-level network traffic. windump is a Windows variant of tcpdump. It also requires an application program interface to collect the traffic known as winpcap. The unofficial version of tcpdump has some nice enhancements. It decrypts more of the applications at the application layer and has a very nice capability of converting hexadecimal payload to character output. [...]... hour 02 data DMZ analysis host sensor html output 17 The Shadow architecture is a two-host system Typically, the sensor resides on the DMZ, but it can be placed anywhere on the network It collects the traffic from the network interface and stores the data in hourly files which are in raw tcpdump compressed format Each hour, the analysis host securely copies the files from the sensor Using perl scripts... filters will extract the following types of inbound traffic: • Traffic sent to broadcast address • Traffic from reserved private networks • Fragmentation • Initial SYN connections • Particular UDP ports • Specific ICMP traffic • Scans • Traffic to core infrastructure hosts A 18 Shadow comes with several tcpdump filters to examine each hour’s traffic All records extracted by the filters are processed and... interpret output • Freeware available from www.nswc.navy.mil 15 Shadow is a Unix based intrusion detection system It has a sensor and analysis component The sensor component collects network traffic and the analysis component fetches that traffic and analyzes it Both the sensor and analysis host process data in an hourly timeframe The entire IP datagram is not captured because Shadow is mostly concerned with... destination IP traffic Shadow will format into html all the events of interest detected by the tcpdump filters and processed by some perl programs The analyst can examine the output with a browser and further investigate activity using some additional perl scripts to look through an hour’s or day’s worth of data Using Shadow relieves the analyst from having to worry about the collection of tcpdump data;.. .tcpdump in Action Network 0101001110 111010010011000 00100011011 packets tcpdump running on a host “sniffing” network packets tcpdump output 07:00:48.036746 ping.net > myhost.com: icmp: echo request (DF) 07:00:48.036776 myhost.com > ping.net: icmp: echo reply (DF)... examine the actual traffic at the IP datagram level Shadow, by virtue of tcpdump, will allow the user a very intimate view of the data collected You will maintain fidelity of data and you can use all fields for interpretation and analysis If the traffic you are analyzing is corrupted in some way, you want to be able to inspect the entire datagram 16 Shadow Architecture hour 00 data tcpdump filters secure... on this slide, a host running tcpdump and gathering records from the network interface We see the records that tcpdump has collected below tcpdump has a default standard output based on the protocol (TCP, UDP, ICMP) of the record that is displayed While each of the various protocols has a similar format to the other, they are also distinct in what is displayed By default, tcpdump will collect and print,... http://www.nswc.navy.mil/ISSEC/CID Shadow uses tcpdump as its underlying collection and processing tool Shadow turns tcpdump from a packet collecting tool into an intrusion detection system Shadow collects data from the network interface and stores it in hourly files in raw tcpdump compressed format It analyzes each hour’s collected data after-the-fact and runs a series of tcpdump filters against it looking for... the output displayed in html for the analyst makes it easier for the analyst to examine the hour’s traffic 19 Examining tcpdump Output 20 This page intentionally left blank 20 Default tcpdump Output Command: tcpdump Default display 11:55:52.069484 192.168.143.5 > 192.168.143.101: icmp: echo request 21 tcpdump will display any collected or processed output to standard output – typically the console or... system, it requires a savvy analyst to accurately interpret the output However, since it is predicated upon tcpdump, the analyst has the ability to examine all the collected data down to the bit level 14 What is Shadow? • Intrusion detection system based on tcpdump • Unix-based • Performs traffic analysis • Primary focus on datagram headers • Pull-based architecture • Analyst reviews hourly events of . 1 1 Network Traffic Analysis Using tcpdump Judy Novak Judy Novak Johns Hopkins University Applied. uses/misuses • Analyze traffic by placing it in categories • Demonstrate “real-world” analysis using tcpdump • Let you participate in the analysis process The