www.it-ebooks.info www.it-ebooks.info Understanding LINUX NETWORK INTERNALS www.it-ebooks.info Other Linux resources from O’Reilly Related titles Linux in a Nutshell Linux Network Administrator’s Guide Running Linux Linux Device Drivers Understanding the Linux Kernel Building Secure Servers with Linux LPI Linux Certification in a Nutshell Learning Red Hat Linux Linux Server Hacks TM Linux Security Cookbook Managing RAID on Linux Linux Web Server CD Bookshelf Building Embedded Linux Systems Linux Books Resource Center linux.oreilly.com is a complete catalog of O’Reilly’s books on Linux and Unix and related technologies, including sample chapters and code examples. ONLamp.com is the premier site for the open source web plat- form: Linux, Apache, MySQL, and either Perl, Python, or PHP. Conferences O’Reilly brings diverse innovators together to nurture the ideas that spark revolutionary industries. We specialize in document- ing the latest tools and systems, translating the innovator’s knowledge into useful skills for those in the trenches. Visit conferences.oreilly.com for our upcoming events. Safari Bookshelf (safari.oreilly.com) is the premier online refer- ence library for programmers and IT professionals. Conduct searches across more than 1,000 books. Subscribers can zero in on answers to time-critical questions in a matter of seconds. Read the books on your Bookshelf from cover to cover or sim- ply flip to the page you need. Try it today with a free trial. www.it-ebooks.info Understanding LINUX NETWORK INTERNALS Christian Benvenuti Beijing • Cambridge • Farnham • Köln • Paris • Sebastopol • Taipei • Tokyo www.it-ebooks.info Understanding Linux Network Internals by Christian Benvenuti Copyright © 2006 O’Reilly Media, Inc. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (safari.oreilly.com). For more information, contact our corporate/insti- tutional sales department: (800) 998-9938 or corporate@oreilly.com. Editor: Andy Oram Production Editor: Philip Dangler Cover Designer: Karen Montgomery Interior Designer: David Futato Printing History: December 2005: First Edition. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. The Linux series designations, Understanding Linux Network Internals, images of the American West, and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc. was aware of a trademark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein. [M] ISBN: 978-0-596-00255-8 [5/08] www.it-ebooks.info v Table of Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Part I. General Background 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Basic Terminology 3 Common Coding Patterns 4 User-Space Tools 18 Browsing the Source Code 19 When a Feature Is Offered as a Patch 20 2. Critical Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 The Socket Buffer: sk_buff Structure 22 net_device Structure 43 Files Mentioned in This Chapter 57 3. User-Space-to-Kernel Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Overview 58 procfs Versus sysctl 60 ioctl 67 Netlink 70 Serializing Configuration Changes 71 www.it-ebooks.info vi | Table of Contents Part II. System Initialization 4. Notification Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Reasons for Notification Chains 75 Overview 77 Defining a Chain 78 Registering with a Chain 78 Notifying Events on a Chain 79 Notification Chains for the Networking Subsystems 81 Tuning via /proc Filesystem 82 Functions and Variables Featured in This Chapter 83 Files and Directories Featured in This Chapter 83 5. Network Device Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 System Initialization Overview 84 Device Registration and Initialization 86 Basic Goals of NIC Initialization 86 Interaction Between Devices and Kernel 87 Initialization Options 93 Module Options 93 Initializing the Device Handling Layer: net_dev_init 94 User-Space Helpers 96 Virtual Devices 100 Tuning via /proc Filesystem 103 Functions and Variables Featured in This Chapter 104 Files and Directories Featured in This Chapter 105 6. The PCI Layer and Network Interface Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Data Structures Featured in This Chapter 106 Registering a PCI NIC Device Driver 108 Power Management and Wake-on-LAN 109 Example of PCI NIC Driver Registration 110 The Big Picture 112 Tuning via /proc Filesystem 114 Functions and Variables Featured in This Chapter 114 Files and Directories Featured in This Chapter 115 www.it-ebooks.info Table of Contents | vii 7. Kernel Infrastructure for Component Initialization . . . . . . . . . . . . . . . . . . . . 116 Boot-Time Kernel Options 116 Module Initialization Code 122 Optimized Macro-Based Tagging 125 Boot-Time Initialization Routines 128 Memory Optimizations 130 Tuning via /proc Filesystem 134 Functions and Variables Featured in This Chapter 134 Files and Directories Featured in This Chapter 135 8. Device Registration and Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 When a Device Is Registered 137 When a Device Is Unregistered 138 Allocating net_device Structures 138 Skeleton of NIC Registration and Unregistration 140 Device Initialization 141 Organization of net_device Structures 145 Device State 147 Registering and Unregistering Devices 149 Device Registration 154 Device Unregistration 156 Enabling and Disabling a Network Device 159 Updating the Device Queuing Discipline State 161 Configuring Device-Related Information from User Space 166 Virtual Devices 169 Locking 171 Tuning via /proc Filesystem 171 Functions and Variables Featured in This Chapter 172 Files and Directories Featured in This Chapter 173 Part III. Transmission and Reception 9. Interrupts and Network Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 Decisions and Traffic Direction 178 Notifying Drivers When Frames Are Received 178 Interrupt Handlers 183 softnet_data Structure 206 www.it-ebooks.info viii | Table of Contents 10. Frame Reception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 Interactions with Other Features 211 Enabling and Disabling a Device 211 Queues 212 Notifying the Kernel of Frame Reception: NAPI and netif_rx 212 Old Interface Between Device Drivers and Kernel: First Part of netif_rx 219 Congestion Management 225 Processing the NET_RX_SOFTIRQ: net_rx_action 228 11. Frame Transmission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 Enabling and Disabling Transmissions 241 12. General and Reference Material About Interrupts . . . . . . . . . . . . . . . . . . . . . 261 Statistics 261 Tuning via /proc and sysfs Filesystems 262 Functions and Variables Featured in This Part of the Book 263 Files and Directories Featured in This Part of the Book 265 13. Protocol Handlers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 Overview of Network Stack 266 Executing the Right Protocol Handler 274 Protocol Handler Organization 278 Protocol Handler Registration 279 Ethernet Versus IEEE 802.3 Frames 281 Tuning via /proc Filesystem 293 Functions and Variables Featured in This Chapter 293 Files and Directories Featured in This Chapter 294 Part IV. Bridging 14. Bridging: Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 Repeaters, Bridges, and Routers 297 Bridges Versus Switches 299 Hosts 300 Merging LANs with Bridges 300 Bridging Different LAN Technologies 302 Address Learning 302 Multiple Bridges 305 www.it-ebooks.info [...]... first introduced to the beautiful world of networking, I started playing with the tools available on Linux I also had the fortune to work for a UNESCO center in Italy where I helped develop their networking courses, based entirely on Linux boxes That gave me access to a good lab equipped with all sorts of network devices and documentation, plus plenty of Linux enthusiasts to learn from and to collaborate... most of the networking features I cover At the Netfilter home page, http://www.netfilter.org, you can find some interesting documentation about its kernel internals Network filesystems Several network filesystems are implemented in the kernel, among them NFS (versions 2, 3, and 4), SMB, Coda, and Andrew You can read a detailed description of the Virtual File System layer in Understanding the Linux Kernel,... lack of space, I had to select a subset of the Linux networking features to cover No selection would make everyone happy, but I think I covered the core of the networking code, and with the knowledge you can gain with this book, you will find it easier to study on your own any other networking feature of the kernel In this book, I decided to focus on the networking code, from the interface between device... documentation about the networking code of the Linux kernel and the availability of good books for other parts of the kernel, I decided to try filling in the gap—or at least part of it I hope this book will give you the starting documentation that I would have loved to have had years ago I believe that this book, together with O’Reilly’s other two kernel books (Understanding the Linux Kernel and Linux Device... and kernel preemption This makes the networking code of the Linux kernel a very good gym in which to train and keep your networking knowledge in shape Moreover, if you are like me and want to learn everything, you will find enough details in this book to keep you satisfied for quite a while Background Information Some knowledge of operating systems would help The networking code, like any other component... more than ever before, networking is a hot topic Any electronic gadget in its latest generation embeds some kind of networking capability The Internet continues to broaden in its population and opportunities It should not come as a surprise that a robust, freely available, and feature-rich operating system like Linux is well accepted by many producers of embedded devices Its networking capabilities... implemented in the Linux kernel Besides the two wellknown ones, UDP and TCP, Linux has the newer Stream Control Transmission Protocol (SCTP) A good description of the implementation of those protocols would require a new book of this size, all on its own Traffic Control This is the Quality of Service (QoS) layer of Linux, another interesting and powerful component of the kernel’s networking code Traffic... between dedicated hardware and general-purpose CPUs However, Linux can definitely compete with low-end commercial products that are entirely software-based Of course, simple extensions to the Linux kernel allow vendors to use Linux on hybrid systems as well (software and hardware); it is only a matter of writing the necessary device drivers Linux is also often used as the operating system of choice for... The code samples are covered by a dual BSD/GPL license We appreciate, but do not require, attribution An attribution usually includes the title, author, publisher, and ISBN For example: Understanding Linux Network Internals, by Christian Benvenuti Copyright 2006 O’Reilly Media, Inc., 0-59600255-6.” xxii | Preface This is the Title of the Book, eMatter Edition www.it-ebooks.info Copyright © 2008 O’Reilly... see your contribution to the Linux kernel being used by potentially millions of users? There is only one drawback: if your contribution is really appreciated, you may not be able to cope with the numerous emails of thanks or requests for help The momentum for Linux has been growing continually over the past years, and apparently it can only keep growing I first encountered Linux at the University of . www.it-ebooks.info www.it-ebooks.info Understanding LINUX NETWORK INTERNALS www.it-ebooks.info Other Linux resources from O’Reilly Related titles Linux in a Nutshell Linux Network Administrator’s. Guide Running Linux Linux Device Drivers Understanding the Linux Kernel Building Secure Servers with Linux LPI Linux Certification in a Nutshell Learning Red Hat Linux Linux