Planning for Survivable Networks Table of Contents Planning for Survivable Networks—Ensuring Business Continuity 1 Foreword 3 Chapter 1: Introduction 5 Overview 5 Network Continuity 5 Define Survival 6 In Defense of Paranoia 7 By the Numbers 8 Borrow from Einstein 9 Think the Unthinkable 9 Plan to Survive 10 Choice versus Chance 11 Chapter 2: Network Threats 12 Overview 12 Kinds of Attacks 13 Immature Hands 13 Deliberate Attackers 17 Mature Hands 23 Externalities 28 Chapter 3: Tactics of Mistake 29 Overview 29 TCP/IP 29 Probes 35 Viruses 37 Worms 38 Trojan Horses 39 Denial of Service/Distributed DoS 40 Sample Attack 41 Means 44 Opportunity 45 Chapter 4: Murphy's Revenge 47 Overview 47 System Is Not a Dirty Word 47 Complexity 48 Interaction 48 Emergent Properties 48 Bugs 48 Where Opportunity Knocks 49 Top General Vulnerabilities 49 Top Windows Vulnerabilities 53 Top UNIX Vulnerabilities 54 Common Threads 56 Design Your Way Out of Trouble 57 Topology 57 Defense in Depth 60 i Table of Contents Chapter 4: Murphy's Revenge The Price of Defense 62 Olive−Drab Networks 63 Benefits 63 Costs 63 Converged Networks 64 The Catch 66 Operator Error 67 Chapter 5: "CQD MGY" 68 Overview 68 A Classic Disaster 68 Lessons from Failure 70 A Trophy Property 70 Warning Noted 71 Train the Way You Will Fight 71 What Did You Say? 72 A Scarcity of Heroes 72 Lessons from Success 73 Organization 73 Training 74 Attitude 74 A Plan 75 What Are You Planning For? 76 Adequate Warning 76 Modest Warning 80 No Real Warning at All 82 It's a Scary World, Isn't It? 87 Chapter 6: The Best−Laid Plans 88 Overview 88 Three Main Points 88 Operational Continuity 88 Getting the People Out 94 Network Assets 95 Example: Data Services 97 Lessons Actually Learned 102 Lessons Potentially Learned 104 Kudos 104 Extending the Example 105 Chapter 7: Unnatural Disasters (Intentional) 107 Overview 107 Physical Attacks 109 Bombs 109 Electromagnetic Pulse 110 Sabotage 110 CBR Attacks 111 World Trade Center Examples 113 Successes 114 ii Table of Contents Chapter 7: Unnatural Disasters (Intentional) Lost Access 118 Less Than Successes 120 Cyber−Attacks 123 Cyber−Kidnapping 123 Extortion 124 Easier Targets 124 Combined Attacks 125 Chapter 8: Unnatural Disasters (Unintentional) 127 Overview 127 Unfortunate Opportunities 127 Reportable Outages: They're Everywhere 128 Route Diversity in Reality 129 Fire 130 Required Evacuations 131 Unfortunate Planning 132 Yours 132 Theirs 134 Unfortunate Implementation 138 Equipment 1, Plan 0 138 Solving the Wrong Problem 139 Chapter 9: Preparing for Disaster 141 Overview 141 Define Survival 141 What Must Roll Downhill 141 Survival Requirements 143 Network Continuity Requirements 144 Threat Analysis 149 Operational Analysis 151 Survival Planning 152 Fixes 152 Remedies 154 Procedures 155 Survivability Today 156 Don't Get Too Close 157 Talk Is Cheap 158 Data Currency 159 Trade−offs 159 Chapter 10: Returning From the Wilderness 161 Overview 161 Cyber−Recovery 161 Operational Procedures 161 Forensic Procedures 162 Physical Recovery 166 Immediate Operations 166 Sustained Operations 166 Restoration 167 iii Table of Contents Chapter 10: Returning From the Wilderness Undress Rehearsal 169 Exercise Scenario 1: Cyber−Problems 171 Exercise Scenario 2: Physical Problems 172 Evolution 173 Chapter 11: The Business Case 178 Overview 178 Understanding Costs 178 Fixed and Variable Costs 178 Direct Costs versus Indirect Costs 179 Explicit and Implicit Costs 180 Valid Comparisons 181 Understanding Revenues 182 Expected Values 183 Presenting Your Case 184 CDG Example 186 Alternatives Considered 187 Disaster Summary 187 Alternatives Summary 188 Risks Not Mitigated 190 Finally 190 Chapter 12: Conclusion 191 Overview 191 Necessity 192 Basic Defenses You Must Implement 192 The Deck Is Stacked Against You 193 Catastrophes Happen 193 Your Recovery 194 Trade−offs 196 Systemic Behavior 196 Standardization versus Resiliency 197 Pay Me Now or Pay Me Later 198 Appendix A: References 200 Books 200 Web Sites 200 Disaster Planning 200 Earthquake Hazard 200 Other Government Information (U.S.) 201 Miscellaneous 201 Natural Hazard Costing 202 Terrorism 202 UPS Capabilities 203 Volcanic Eruption Data 203 Weather Planning 203 iv Table of Contents Appendix B: Questions to Ask Yourself 204 Appendix C: Continuity Planning Steps 206 Network Requirements 206 Threat Analysis 206 Operational Analysis 206 Survival Planning 206 Reality Check 207 Recovery 207 Appendix D: Post−Mortem Questions 209 Appendix E: Time Value of Money 210 Appendix F: Glossary 211 A−L 211 N−W 212 List of Figures 214 List of Tables 216 List of Sidebars 217 v Planning for Survivable Networks—Ensuring Business Continuity Annlee Hines Wiley Publishing, Inc. Publisher: Robert Ipsen Editor: Carol A. Long Developmental Editor: Adaobi Obi Managing Editor: Micheline Frederick Text Design & Composition: Wiley Composition Services Designations used by companies to distinguish their products are often claimed as trademarks. In all instances where Wiley Publishing, Inc., is aware of a claim, the product names appear in initial capital or ALL CAPITAL LETTERS. Readers, however, should contact the appropriate companies for more complete information regarding trademarks and registration. This book is printed on acid−free paper. Copyright © 2002 by Annlee Hines. All rights reserved. Published by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per−copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750−8400, fax (978) 750−4470. Requests to the Publisher for permission should be addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspointe Blvd., Indianapolis, IN 46256, (317) 572−3447, fax (317) 572−4447, E−mail: <permcoordinator@wiley.com>. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. 1 For general information on our other products and services please contact our Customer Care Department within the United States at (800) 762−2974, outside the United States at (317) 572−3993 or fax (317) 572−4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Library of Congress Cataloging−in−Publication Data: ISBN: 0−471−23284−X Printed in the United States of America 10 9 8 7 6 5 4 3 2 1 For Eric and Aylyffe sine qua non ANNLEE A. HINES is a systems engineer for Nortel Networks (Data Networks Engineering). Prior to Nortel, Hines was an engineer in the U.S. Air Force working with command, control, communications, and intelligence systems. She has also worked for a defense contractor, owned two small businesses, and taught economics and political science at a community college. Hines has written three white papers for publication by CertificationZone.com on network management, switched WAN technologies, and an introduction to telephony. 2 Foreword It is a mistake to try to look too far ahead. The chain of destiny can only be grasped one link at a time. Winston Churchill It's true that the events of September 11, 2001 crystallized my thoughts about network survivability, but the thoughts go back much further than that. I became very interested in terrorism while serving in the USAF in Europe, where it was a very real threat, especially to those of us in an American uniform. That interest had been somewhat dormant, but it never really went away. I stayed aware of the threats and how they were evolving; where once terrorists struck only where they could melt away into the populace to live and strike another day, they no longer care about that. This is a watershed, for it changes the nature of the threat: Delivery need no longer be safe for the deliverer. That turns previously untouchable locations into targets. Since I left the service, I have become a network engineer after owning two businesses, and the bottom−line responsibility I held there changed the way I thought about business; it has also affected how I look at network operations. The network exists only because it brings value to its business. But if it brings value, that value must continue or the business itself may suffer such a degradation of its financial condition that it is in danger of failing. That statement was not always true, but it has become true in the past two decades. Almost unnoticed, networks have indeed become integral to the operations of all major businesses, all around the world. What is more, we do operate in a global economy, with costs held to their barest minimum in the face of competition from other companies, some of whom operate in other countries, where cost structures are different. If the network is a major factor in your firm's competitiveness, whether from a perspective of increasing productivity or a perspective of minimizing the cost of timely information transfer, its continuity is critical to business continuity. The networking community was as mutually supportive as ever during and after the terrorist attacks of September 11. The NANOG (North American Network Operators Group) mailing list was flooded with advisories of where the outages were, who was able to get around them, offers of available bandwidth and even temporary colocation, if needed. There were also dire thoughts concerning how much worse the situation would have been had a couple of other locations been hit as well. Many of the first responders who died lost their lives due to communications failures—they did not notify the command center of their presence or location, but rushed in to help because lives were at stake right now. When the command center decided to evacuate because senior officials knew the buildings could not stand much longer, radio coverage was so spotty that some who lost their lives did so because they simply never got the word to get out. The communication network that day was inadequate to the task. After the collapse of the World Trade Center, much of the information dissemination was made via email and Internet; those hubs were the ones referred to on NANOG in the what−if discussions. Networks have always been about communications—moving the information from where it is already known to where it needs to be known to add value. "Rejoice! For victory is ours," gasped Phaedippides with his dying breath after running from the battlefield at Marathon to Athens. His message had value because Athens expected to lose the battle, and the city fathers were preparing to surrender when they saw the Persian fleet approach. 3 On a more business−centric note, the time to buy, said Lord Rothschild, is when the blood is running in the streets. He used his superior communications to cause that to happen, after the Battle of Waterloo, and he made a financial killing in the London markets his better information had manipulated. Your network is the nervous system of your business—the connector between its brains and direction and the actual execution of business decisions. If the nervous system is damaged or disrupted, bad decisions may ensue (from bad information), or good decisions may be ordered but never executed. Either way, it might be your company's blood that is running in the streets. Business continuity implies that the organization continues to operate as a business; for this, the nervous system must continue to be there. It may not be there in all its ordinary glory, but the essential services it provides must continue to be present. Getting those defined and finding ways to ensure their continuity are the subjects of this book. The threats to continued network operation range from the dramatic (major terrorist attack) through the more common, but still not frequent (natural disasters), to the threat attacking you every day (hackers). The tools that protect you from the first two are quite similar; there is also considerable overlap with the tools to protect you from the third. As with anything in either networking or business in general, you are going to have to make compromises. If you learn from the principles addressed here, rather than blindly answering the lists of questions presented, you will be prepared to make the hard choices on a knowledgeable basis. They won't be any more pleasant, but the consequences are less likely to be an unpleasant surprise. No book ever springs full−blown from the author's forehead, like the fully armed Athena. I have had so much help I cannot begin to thank those people. From years ago, I owe Colonel Richard W. Morain (USAF, Retired) for his patience and support. Even after I left the service, he maintained contact, and I am better for it. More recently, I've wound up doing this through the intervention, after a fashion, of Howard W. Berkowitz, who liked my comments on a mailing list, and offered me the opportunity to write about networking for publication. Then it was a review of his manuscript that put me in contact with his editor, Carol Long, at John Wiley and Sons. During the hashing−out process of what this book would actually become, and the grind of getting it all down in bytes in a lot of files, Carol's support has been invaluable. Likewise, my friends at Nortel have maintained an enthusiasm for the project when my energy flagged; chief among them have been Ann Rouleau and John Gibson. My manager at the time, Mark Wilson, massaged the administrative system to propitiate the intellectual−property gods; he had more patience than I, lots of times. And, of course, sine qua non, have been my family, who now expect me to do this again. With their help, I will. 4 [...]... unaffected, for it surely was Military information systems, though, were robust enough to avoid serious disruption to any of the command and control functions—the networks delivered, with a little help from the human elements We will examine a few exemplary stories from the attack on the WTC (civilian networks are more directly comparable for our purposes); in these cases, the companies' networks were... or she was not supposed to do Most networks, including the overwhelming majority of business networks, are unbounded, if for no other reason than, at least at one point, they connect to another network outside the company's administration (such as the Internet) While bounded networks are not inherently safe, unbounded networks (to be delicate) are less safe Unbounded networks suffer from several problems:... manufacturing techniques for gallium arsenide chips and the code for computer−aided design and manufacturing Hess, who did much of the actual stealing, got a portion of the money and no cocaine Hess had become a sophisticated hacker for the challenge; it took very little money to reward him for doing what he was doing anyway Suppose it cost you (a competitor) as much as $100,000 for the information And suppose... information about an unbounded network A corporate network, even one composed of many LANs, is bounded until it has a single connection (authorized or not) to a network outside its administrative control, such as an extranet or the Internet As a result, it is common for a system to be composed of both bounded and unbounded networks For instance, a firm may have three regional domains, all bounded networks, ... your network Every demarc is on a link and is therefore an entry point for incoming traffic, or an ingress Of course, it is also an egress, or exit point, for traffic as well Like a traffic intersection for surface transportation, both directions must be considered After all, if the network voyeur can gain entry but never realizes it because the flow of information back to her is suppressed, she will not... than that for financial fraud Remember this when you estimate how much you can afford to spend on protection Some of the theft may have occurred whether or not you use a network Social engineering is the term used for getting information from people they really shouldn't have told you Examples abound, from the story of a firm, well ahead of its competitors, whose employees supported all the information... side of a river) Fortunately, your preparations to deal with natural disasters form a good foundation for preparation to deal with a terrorist attack In both cases, you are preparing to lose the use of a major networking location for an indeterminate period of time You are concerned about saving your people first—equipment is far easier to replace, and arrangements can be made quickly for new desktops... principle, to forgery, that is appropriate Why? The short answer to Why? is money The Sting was a wildly popular movie glamorizing an elaborate scheme to doctor the information used by organized crime to make money from betting on horse racing Organized crime delayed the information long enough to ensure their bets won; the scheme altered the information on which the criminals placed their bets in order for. .. vendor and your firm The contract contains performance guarantees, premiums for exceeding requirements, and penalties for failure to meet minimum requirements The vendor, of course, would prefer to receive the largest possible payment while you would prefer to tender the smallest possible The size of the payment depends on information—delivery dates, performance characterization, payment characterization... worked with IP for a while or have used IP networks but never needed to know how the information moved around in them, this will introduce the principles on which that movement is based These pieces are what attackers use against you TCP/IP TCP/IP (Transmission Control Protocol/Internet Protocol) was developed for the U.S Department of Defense in the early 1970s The goal was to have a flexible, survivable . Planning for Survivable Networks Table of Contents Planning for Survivable Networks Ensuring Business Continuity 1 Foreword 3 Chapter 1: Introduction 5 Overview. 4 3 2 1 For Eric and Aylyffe sine qua non ANNLEE A. HINES is a systems engineer for Nortel Networks (Data Networks Engineering). Prior to Nortel, Hines was an engineer in the U.S. Air Force working. Glossary 211 A−L 211 N−W 212 List of Figures 214 List of Tables 216 List of Sidebars 217 v Planning for Survivable Networks Ensuring Business Continuity Annlee Hines Wiley Publishing, Inc. Publisher: