building and integrating virtual private networks with openswan (2006)

Building and Integrating Virtual Private Networks with Openswan Learn from the developers of Openswan how to build industry-standard, military-grade VPNs and connect them with Windows, Mac OS X, and other VPN vendors Paul Wouters Ken Bantoft BIRMINGHAM - MUMBAI Building and Integrating Virtual Private Networks with Openswan Copyright © 2006 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, Packt Publishing, nor its dealers or distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. First published: February 2006 Production Reference: 1010206 Published by Packt Publishing Ltd. 32 Lincoln Road Olton Birmingham, B27 6PA, UK. ISBN 1-904811-25-6 www.packtpub.com Cover Design by www.visionwt.com Credits Authors Paul Wouters Ken Bantoft Reviewers Michael Stelluti Tuomo Soini Nate Carlson James Eaton-Lee Technical Editor Richard Deeson Editorial Manager Dipali Chittar Development Editor Louay Fatoohi Indexer Abhishek Shirodkar Proofreader Chris Smith Production Coordinator Manjiri Nadkarni Cover Designer Helen Wood About the Authors Paul Wouters has been involved with Linux networking and security since he co-founded the Dutch ISP Xtended Internet back in 1996, where he started working with FreeS/WAN IPsec in 1999 and with DNSSEC for the .nl domain in 2001. He has been writing since 1997, when his first article about network security was published in Linux Journal in 1997. Since then, he has written mostly for the Dutch spin-off of the German c't magazine, focusing on Linux, networking, and the impact of the digital world on society. He has presented papers at SANS, OSA, CCC, HAL, BlackHat, and Defcon, and several other smaller conferences. He started working for Xelerance in 2003, focusing on IPsec, DNSSEC, Radius, and training delivery. Over a year ago, we wrote a proposal for an Openswan book. Without knowing about this proposal, Louay Fatoohi of Packt Publishing asked us if we were interested in publishing just such a book. We are very happy with the result of that collaboration. We would like to thank everyone who is or has been part of the Linux IPsec and Openswan communities, without whom neither Openswan nor this book would have been possible. Many thanks to John Gilmore for founding the FreeS/WAN Project, and to XS4ALL for hosting it. Many people contributed to FreeS/WAN, but we would like to especially thank Hugh Daniel, Michael Richardson, Hugh Redelmeier, and Richard Guy Briggs. The FreeS/WAN and Openswan community contributed some important features. Thanks to Andreas Steffen of StrongSec for the X.509 patches, JuanJo Ciarlante for the original ALG patches that included AES, Mattieu Lafon of Arkoon Systems for the NAT-Traversal patches, and Hendrik Nordstrom of MARA Systems for the Aggressive Mode patches. Further thanks are due to Rene Mayrhofer of Debian and Robert-Jan Cornelissen of Xtended Internet as early adopters of Openswan. Xtended Internet also graciously hosted the Openswan servers for two years. We are especially grateful to Herbert Xu for his tremendous work on integrating Openswan with the Linux 2.6 NETKEY stack, and Michael Richardson for maintaining and enhancing tcpdump. Thanks also to Jacco de Leeuw for his excellent work on documenting L2TP, and Nate Carlson for his elaborate X.509 configuration guide. They have invested a large amount of time in helping the community with Openswan configuration. Everyone knows how important a cute logo is, but the logo that Nana Manojlovic spontaneously gave us surpasses even the penguin. Thank you Nana! And of course, thanks to all the Linux distributions that have included Openswan in their packages. You have truly caused the widespread use and acceptance of Openswan. Over the course of a year, quite a few people have helped to create this book. Many thanks to Louay Fatoohi and Richard Deeson of Packt Publishing. This book would have been filled with errors, had it not been for our reviewers, Tuomo Soini, Nate Carlson, and James Eaton- Lee. Extra praise goes to Mike Stelluti who, without ever having touched a Linux computer, went through the book verifying every single command, which included setting up and testing entire X.509, L2TP, and UML setups from scratch. And a special thanks goes to Michael Richardson for writing the section on debugging Openswan using tcpdump. Ken Bantoft started programming in 1988, and successfully avoided it as a full-time job until 2002. Before that, he opted instead to focus on Unix, Networking, and Linux integration. Beginning at OLS2002, he started working alongside the FreeS/WAN project, integrating various patches into his own fork of its code—Super FreeS/WAN, which is now known as Openswan. He currently lives in Oakville, ON, Canada, with his wife Van, two cats, and too many computers. Ken started working for Xelerance in 2003 where he works mostly on IPsec, BGP/OSPF, Asterisk, LDAP, and Radius. I'd like to thank: My father, who put a computer in front of me 20 years ago, and who has supported my digital addiction for all those years; My wife Van, who puts up with the large amount of hardware in the basement, and the power bills it generates; Kyle Schustyk, with whom I set up my first IPsec tunnel; Jim Alton, Alex Bichuch, and Rob Rankin who kept me busy building VPNs for various people; Michael Richardson—without his ROT13- encrypted party invitation I'd have never starting hacking IPsec code; Sam Sgro, with whom a bet started Super FreeS/WAN, which in turn begat Openswan; D. Hugh Reidelmier, who still answers any C question I have. About the Reviewers Michael Stelluti is completing his studies in Computer Science and has been an intern at Xelerance Corporation since 2005. As part of the Xelerance support group, Michael reproduces client environments in the labs and also moderates the Openswan mailing lists. To relax, he enjoys watching Battlestar Gallactica with a pint of Guinness well in hand. Michael currently resides in Kelowna, British-Columbia, in Canada. Nate Carlson is currently a full time systems administrator for Internet Broadcasting, and also does occasional Linux consulting on the side. He's been using IPSec under Linux since the early FreeS/WAN days, and has written a popular guide on using Windows XP in a RoadWarrior configuration. He lives near Minneapolis, Minnesota with his wonderful wife Tiffany. He can be reached via his website, www.natecarlson.com James Eaton-Lee works as an Infrastructure Security Consultant for a firm whose clients range from small businesses with a handful of employees to multinational banks. He has formerly worked for an Internet Service Provider and at a call center, as well as providing independent consultancy in the areas of forensics and security. James has extensive experience of traditional and IP telephony, as well as how these technologies can be integrated into existing IT infrastructure. He has been involved in a variety of work in his present role, ranging from simple IT and infrastructure work for small clients to security work across infrastructure comprising thousands of servers for a large bank. He is a strong advocate of the relevancy of open-source and free software, and—wherever appropriate—uses it for himself and his clients. Table of Contents Preface 1 Chapter 1: Introduction 5 The Need for Cryptography 5 Privacy 5 Security 6 A History of the Internet 6 Holding the Internet Together 7 The Creation of ICANN 7 ICANN Bypassed 8 The Root Name Servers 8 Running the Top-Level Domains 8 History of Internet Engineering 9 The Internet Engineering Task Force (IETF) 9 RFCs—Requests For Comments 10 IETF and Crypto 11 The War on Crypto 12 Dual Use 12 Public Cryptography 12 The Escrowed Encryption Standard 13 Export Laws 13 The Summer of '97 14 The EFF DES Cracker 14 Echelon 14 The End of the Export Restrictions 15 Free Software 15 The GPL 15 Free as in Verifiable 16 The Open Source Movement 16 The History of Openswan 17 IETF Troubles over DNS 17 Super FreeS/WAN 17 The Arrival of Openswan 18 NETKEY 18 Table of Contents Further Reading 19 Using Openswan 19 Copyright and License Conditions 20 Writing and Contributing Code 20 Legality of Using Openswan 21 International Agreements 21 International Law and Hosting Openswan 22 Unrecognized International Claims 22 Patent Law 23 Expired and Bogus Patents 23 Useful Legal Links 24 Summary 25 Chapter 2: Practical Overview of the IPsec Protocol 27 A Very Brief Overview of Cryptography 27 Valid Packet Rewriting 28 Ciphers 28 DES, 3DES, and AES 29 Algorithms 29 Uniqueness 30 Public-Key Algorithms 30 Exchanging Public Keys 30 Digital Signatures 30 Diffie-Hellman Key Exchange 30 Avoiding the Man in the Middle 31 Session Keys 31 Crypto Requirements for IPsec 32 IPsec: A Suite of Protocols 32 Kernel Mode: Packet Handling 32 Authentication Header (AH) 33 Encapsulated Security Payload (ESP) 34 Transport and Tunnel Mode 34 Choosing the IPsec Mode and Type 35 The Kernel State 35 Encryption Details 36 Manual Keying 36 Final Note on Protocols and Ports 37 Usermode: Handling the Trust Relationships 37 The IKE Protocol 37 Phase 1: Creating the ISAKMP SA 37 ii [...]... Making the Choice GPL Compliance and KLIPS Binary Installation of the Openswan Userland Checking for Old Versions Installing the Binary Package for Openswan Building from Source Using RPM-based Distributions Rebuilding the Openswan Userland Building src.rpm from Scratch Openswan Options Building the Openswan Userland from Source Downloading the Source Code Configuring the Userland Tools Optional Features... and respected by the community They know the ins and outs of a wide range of setups, and also know the caveats and pitfalls that can obstruct successful Openswan deployment What This Book Covers Chapter 1 presents the historical context of IPsec and Openswan, and discusses the legal aspects involved with using and selling cryptography tools such as Openswan Chapter 2 explains in non-mathematical terms... IPsec WaveSEC software, as used to encrypt the wireless networks at IETF, BlackHat, and DefCon Chapter 11 discusses advanced Openswan techniques, such as how to set up a robust fail-over VPN Openswan server, and how to deal with the bottlenecks that large enterprise deployments can experience, as well as how to handle BGP and OSPF using IPsec and Openswan Chapter 12 is the distillation of two years of... L2TP on your Openswan VPN server, and explains how to configure X.509 or L2TP on your Microsoft Windows or Apple MacOSX clients We also look at the pros and cons of some commonly used third-party software packages that work with Openswan Chapter 9 deals with getting Openswan to properly interoperate with third-party IPsec VPN servers such as Cisco, Check Point, NetScreen, WatchGuard, and various other... A famous Green Paper and White Paper with recommendations were written, leading to a Memorandum of Understanding (MoU) between ICANN and the DoC The 'ICANN at large' program, which allowed every individual to participate with ICANN and elect three board members, took two years to set up and was launched in 2000 Two of these newly elected directors—Karl Auerbach, a legal scholar and Internet veteran... cryptographic technology that either: (a) impose restrictions by implementing export controls; and/ or (b) restrict commercial and private users to weak and inadequate mechanisms such as short cryptographic keys; and/ or (c) mandate that private decryption keys should be in the hands of the government or of some other third party; and/ or (d) prohibit the use of cryptology entirely, or permit it only to specially... otherwise been Public Cryptography One by one, all inventions made secretly within the military were being re-invented by non-military cryptanalysts And new algorithms and ciphers were being designed at universities and private companies Rivers, Shamir, and Adelman invented RSA public key encryption In 1976 Diffie and Hellman came up with a technique which has become known as DH key exchange, enabling the... through the process of designing, building, and configuring Openswan as your VPN gateway, covering these topics with the detail and depth of explanation you would expect from key members of the Openswan development team You should note that Openswan is not restricted to only Linux clients, but can support all common operating systems such as Microsoft Windows and Mac OS X Furthermore, we look at some... Check Point, NetScreen, and others As official developers of the Openswan code, the authors give you the inside view on essential techniques This book includes the latest developments and upcoming issues With their experience in answering queries from users on the mailing lists since the creation of Openswan, and its predecessor FreeS/WAN, the authors are authority figures well known and respected by the... Connection Known Issues with WaveSEC WaveSEC for Windows Design Limitations Building a WaveSEC for Windows Server Obtaining the Certificate and Client Software Our Prototype Experiences Openswan Issues Windows Kernel Issues Summary x 246 246 247 247 248 248 248 249 250 251 252 253 253 254 Table of Contents Chapter 11: Enterprise Implementation 255 Cipher Performance Handling Thousands of Tunnels Managing . Building and Integrating Virtual Private Networks with Openswan Learn from the developers of Openswan how to build industry-standard, military-grade VPNs and connect them with. with Windows, Mac OS X, and other VPN vendors Paul Wouters Ken Bantoft BIRMINGHAM - MUMBAI Building and Integrating Virtual Private Networks with Openswan Copyright ©. Rebuilding the Openswan Userland 58 Building src.rpm from Scratch 58 Openswan Options 59 Building the Openswan Userland from Source 59 Downloading the Source Code 59 Configuring the Userland