End-to-End QoS Network Design Second Edition Tim Szigeti, CCIE No 9794 Robert Barton, CCIE No 6660 Christina Hattingh Kenneth Briley, Jr., CCIE No 9754 Cisco Press 800 East 96th Street Indianapolis, IN 46240 ii End-to-End QoS Network Design End-to-End QoS Network Design Quality of Service for Rich-Media & Cloud Networks Second Edition Tim Szigeti, CCIE No 9794 Robert Barton, CCIE No 6660 Christina Hattingh Kenneth Briley Jr., CCIE No 9754 Copyright © 2014 Cisco Systems, Inc Published by: Cisco Press 800 East 96th Street Indianapolis, IN 46240 USA All rights reserved No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without written permission from the publisher, except for the inclusion of brief quotations in a review Printed in the United States of America First Printing November 2013 Library of Congress Control Number: 2013950000 ISBN-13: 978-1-58714-369-4 ISBN-10: 1-58714-369-0 Warning and Disclaimer This book is designed to provide information about designing a network with end-to-end quality of service Every effort has been made to make this book as complete and as accurate as possible, but no warranty or fitness is implied The information is provided on an “as is” basis The authors, Cisco Press, and Cisco Systems, Inc., shall have neither liability nor responsibility to any person or entity with respect to any loss or damages arising from the information contained in this book or from the use of the discs or programs that may accompany it The opinions expressed in this book belong to the author and are not necessarily those of Cisco Systems, Inc Trademark Acknowledgments All terms mentioned in this book that are known to be trademarks or service marks have been appropriately capitalized Cisco Press or Cisco Systems, Inc cannot attest to the accuracy of this information Use of a term in this book should not be regarded as affecting the validity of any trademark or service mark iii Corporate and Government Sales The publisher offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales, which may include electronic versions and/or custom covers and content particular to your business, training goals, marketing focus, and branding interests For more information, please contact: U.S Corporate and Government Sales 1-800-382-3419 corpsales@pearsontechgroup.com For sales outside of the U.S please contact: International Sales international@pearsoned.com Feedback Information At Cisco Press, our goal is to create in-depth technical books of the highest quality and value Each book is crafted with care and precision, undergoing rigorous development that involves the unique expertise of members from the professional technical community Readers’ feedback is a natural continuation of this process If you have any comments regarding how we could improve the quality of this book, or otherwise alter it to better suit your needs, you can contact us through e-mail at feedback@ciscopress.com Please make sure to include the book title and ISBN in your message We greatly appreciate your assistance Publisher: Paul Boger Business Operation Manager, Cisco Press: Jan Cornelssen Associate Publisher: Dave Dusthimer Executive Editor: Brett Bartow Senior Development Editor: Christopher Cleveland Managing Editor: Sandra Schroeder Copy Editor: Keith Cline Project Editor: Seth Kerney Technical Editors: John Johnston, Roland Saville Editorial Assistant: Vanessa Evans Proofreader: Jess DeGabriele Cover Designer: Mark Shirar Indexer: Christine Karpeles Composition: Jake McFarland iv End-to-End QoS Network Design About the Authors Tim Szigeti, CCIE No 9794, is a senior technical leader in the Systems Design Unit at Cisco Systems, where his role is to design network architectures for enterprise mobility solutions He has specialized in quality of service technologies for the past 15 years, during which time he has authored many technical papers, design guides, and two Cisco Press books: End-to-End QoS Network Design (version 1) and Cisco TelePresence Fundamentals Robert Barton, CCIE No 6660, is located in Vancouver, where he lives with his wife and two children He graduated from the University of British Columbia with a degree in engineering physics, and is a registered professional engineer Rob holds dual CCIEs, in Routing and Switching and Security, and was also the first CCDE in Canada Rob joined Cisco from ArrowPoint Communications, where he worked as a data center specialist supporting many of the largest corporations in Canada In the time since ArrowPoint was acquired by Cisco, Rob has worked as a public sector systems engineer, primarily focused on wireless and security architectures Currently, Rob is working on SmartGrid network technologies, including smart meter and intelligent substation design Christina Hattingh spent 13 years as a senior member of the technical staff in Unified Communications (UC) in the Enterprise Networking Routing Group (formerly Services Routing Technology Group or SRTG) at Cisco Systems The SRTG products, including the Cisco 2900/3900 and 2800/3800 series ISR platforms and their predecessors, were the first Cisco platforms to converge voice, data, and video traffic and services on IP networks by offering TDM gateway interfaces, WAN interfaces, call control, and QoS features The ISR series of routers often live at smaller remote offices and therefore at the edge of the WAN, where the need for QoS services is most sensitive In this role, Christina spoke at Cisco Live conferences, trained Cisco sales staff and Cisco resale partners on router-based UC technologies, authored several Cisco Press books, and advised customers on UC network deployment and design, including QoS designs and helping them through the TDM to SIP trunk industry transition Kenneth Briley, Jr., CCIE No 9754 is a technical lead in the Network Operating Systems Technology Group at Cisco Systems For over 10 years, he has specialized in quality of service design and implementation in customer environments, alignment of QoS features and functions, and the marketing of new products that leverage QoS technologies During this time, he has written several deployment guides and whitepapers, presented at Cisco Live, and most recently has focused on the convergence of wired and wireless quality of service v About the Technical Reviewers John Johnston, previously CCIE No 5232, is a technical marketing engineer for Cisco Systems His focus is on mobile security technology and design validation John has more than 19 years of experience in IP internetworking, including the design and implementation of enterprise networks Before joining Cisco Systems, John provided network design support for Fortune 500 companies He holds a BSEE from the UNC-Charlotte Roland Saville is a Technical Leader for the Systems Development Unit (SDU) at Cisco, focused on developing best-practice design guides for enterprise network deployments He has more than 18 years of experience at Cisco as a Systems Engineer, Product Manager, Consulting Systems Engineer, Technical Marketing Engineer, and Technical Leader During that time, he has focused on a wide range of technology areas, including the integration of voice and video onto network infrastructures, network security, wireless LAN networking, RFID, energy management, Cisco TelePresence, and BYOD He has also spent time focusing on the retail market segment Prior to Cisco, he spent eight years as a Communications Analyst for Chevron Corporation Roland holds a bachelor’s of science degree in electrical engineering from the University of Idaho and an MBA degree from Santa Clara University He co-authored the book Cisco TelePresence Fundamentals, is a member of the IEEE, and has 12 U.S patents vi End-to-End QoS Network Design Dedications Tim Szigeti: I find myself in a dedication dilemma On the one hand, I already went to great lengths to explain why not dedicating the first edition of this book to my wife would have been a fatal mistake Since then, I’ve gone on to dedicate my second book to my son, and now I have a beautiful daughter who deserves a dedication too (and whose arrival, incidentally, actually delayed the release of this edition by a couple of months) So, the question becomes: Are dedications—as their definition implies—exclusive to a given book? Or can these be edition-specific? Or perhaps the more important question is: Do I really think it wise to get into a debate over semantics with my wife, who has a double-major in both English and philosophy? So I’ll play the political game and try to weakly rationalize a compromise: The first edition of this book was dedicated to Lella The second will be to Lella 2.0, or as she’s more commonly known, Isla Besides, I’ve already witnessed how much my daughter values my books For example, over the past few months, she’s had two copies of my previous book under her crib, slightly elevating one end to alleviate nighttime gas Since she wakes up happy and smiling every morning, I’ll infer from this her appreciation of the practical benefits of my work Furthermore, she’s always ready to gnaw and drool on my books until they’re nice and soggy, and since pure happiness is expressed during the process, I’ll attribute this to her esteem of the quality of the authorship And so, to my beautiful little girl, I wish to dedicate to you this work I really don’t know how I ever managed to finish it, seeing as how little you let me sleep over the past few months! I know you’ll probably never read it, but that’s not the point I just want you to know you were always on my mind and made working on it virtually impossible! And I’m so very happy it’s all done with now, so that I can spend more time playing with you and letting you continue wrapping me tightly around your tiny little finger! vii Rob Barton: This book is dedicated to my two wonderful boys, Adrian and Matthew It’s not that I expect you to actually pick up the book and try to become QoS experts, or that I am even trying to encourage you toward a career in network design or engineering, although these are noble pursuits Rather, the lesson that writing this book has reminded me of is that you only grow as a person when you recognize the space you are in and make the decision to something new Oftentimes, we don’t know what direction our efforts will take us in, but when you make the mindful choice to something that is difficult, challenging, and can cause you more than a little pain along the way, you grow No muscle ever grew without the fibers being damaged through exercise, and so is it too with all aspects of life My hope is that this book will inspire you throughout your life to look for opportunities for growth—be it artistic, mental, professional, physical, or spiritual This book is for you Christina Hattingh: To Robert Verkroost and my parents for their unfailing encouragement and support Kenneth Briley, Jr.: As this is my first book, I’d like to heed Tim’s advice and dedicate it to my beautiful wife Mirah for fear of the aforementioned transgression To Mirah, who incidentally read and approved this dedication, and her countless hours devoted to resolving numerous grammatical errors and listening to me drone on about how incredibly interesting QoS is To our growing family; Lukas, Erik and Max: please don’t grow up too fast, and remember that all things are possible viii End-to-End QoS Network Design Acknowledgments Tim Szigeti: First, I’d like to thank all the readers of the first edition who made it the success that it has become There aren’t many technology books that are still being steadily purchased nearly 10 years after their release And a special thanks to the reviewers who have posted comments for it; I cannot express the pride and appreciation I felt when I saw five-star ratings for our book on Amazon Thank you! Thanks to my director, Neil Anderson, for long recognizing the critical role of QoS across all our networking systems and solutions and ensuring that it was always properly addressed Thanks, too, to Greg Edwards in helping to define and articulate various endto-end QoS strategies Thank you Fred Baker for your guidance and direction in both defining and interpreting various QoS standards Thanks, too, to James Polk for continuing to push the envelope in the IETF to define what tomorrow’s QoS models are going to look like I’d like to thank the Vancouver Cisco office lab administrator, Mojan Mobasser, for all her diligence in sourcing and arranging access to equipment Similar thanks are extended to Dawid Brink for letting me use his Nexus boxes—no questions asked! Farther east, I’d also like to extend thanks to the Toronto Bell Canada team for allowing me extended access to their ASR and CRS labs Similar thanks, but in the opposite geographic direction, go out to Lim Fung in our Singapore office for providing me access to his labs also I’d like to extend sincere thanks to Tim Stevenson for his amazing technical expertise, particularly relating to data center platforms You really helped demystify a lot of hardware architectural questions I was grappling with Thanks, Tim! Also I’d like to thank Lukas Krattiger in Switzerland for hours of research, testing, and correspondence to ensure that we properly wrapped our arms around Nexus 7000 QoS Thanks for all your insight, patience, and hard work, Lukas! Additionally, I’d like to thank Lucien Avramov for sharing his work on data center QoS and allowing me to borrow from it Thank you too, Mike Herbert—wherever you are—for getting the ball rolling in this area I’d like to thank also the Cisco product teams that listened to the feedback we offered as a result of producing this book so as to continue to improve our QoS feature sets This includes Albert Mitchell and the Catalyst 2K/3K team for implementing our latest designs into a new version of AutoQoS Thanks also to Sarath Subrahmanya and Ramachandra Murthy in India for taking to heart our suggestions on WLC QoS feature enhancements Kudos also go out to Oz Ben-Rephael and team in Israel for continuing to develop NBAR2 application signatures, including for our own Jabber application Thanks to the Cisco Press team Brett Bartow: Thanks for taking on this project and allowing us to thoroughly update and expand on our original work in a comprehensive manner We appreciate that you didn’t blow a gasket when we exceeded our targeted ix page count again, again, and again—to a final tune of target +50%! Thanks also for delaying this publication by a couple of months, letting me focus on my family as my daughter was born Thank you Chris Cleveland for making the review process so easy Your comments and accommodation were very much appreciated and really helped polish this work Thank you, too, Seth Kerney for coordinating the copy review And also thanks to Vanessa Evans for ensuring that we always had everything we needed at every step of the way I’d like to extend exceptional thanks to our technical editors Roland Saville and John Johnston Roland: You’re one of the smartest persons I’ve had the pleasure of working with—and in so many fields I don’t know how your brain doesn’t explode! You know I like to think of you as a “philosopher engineer,” because you can take almost any design recommendation and find the corner-case counterargument where it breaks downs and doesn’t apply That’s critically important to the process because by seeing from a distance where things can break you continually save us tremendous amounts of time in the lab, as well as ensuring the overall quality of the final designs Thank you, too, JJ! You allowed me unfettered access to your massive labs in RTP and helped me along every step of the way Your attention to detail is so impressive that I’m nearly spooked by your ability to catch the tiniest errors while reviewing hundreds of pages of configurations! Finally, I owe a great deal of gratitude to my co-authors: Ken: Thanks for your impressive knowledge and flexibility that you demonstrated by being able to jump right in and seamlessly adapt your research to our work in such an intuitive and cohesive manner I’ve enjoyed working with you on many projects for the past decade and look forward to many more collaborations Thanks again, Ken! Christina: Thanks so much for coming out of retirement to work on one more project Even though you’re on the road more than Jack Kerouac these days, it was a real pleasure working with you again! Thanks for donning your QoS hat for us once again and bringing all your knowledge and experience to the table to help make this such a solid work Rob: Over the past 20 years we’ve been friends, classmates, roommates, workmates, “second-best” men at each other’s weddings, and now co-authors Your courage and determination are very inspiring I honestly don’t know if I would have taken my CCIE if I hadn’t watched you it Same goes with running half-marathons (and one-day marathons!) Thanks for all your tremendous work on this project It certainly was not for the faint-hearted, as every time we turned around we seemed to uncover yet another rabbit hole of technical issues that required yet more research and testing to be done Thanks for sticking with it and seeing it through, Rob But then again, that’s just the kind of friend you are Rob Barton: To begin, I would like to thank my very forgiving colleagues in the Cisco Vancouver office who have suffered through two years of trying to depend on an attention divided systems engineer who was more interested in solving theoretical QoS problems Appendix B: Control Plane Policing to treat traffic that is not explicitly associated with any other user-defined classes It is desirable to give such traffic access to the RP, but at a highly reduced rate With a default classification in place, statistics can be monitored to determine the rate of otherwise unidentified traffic destined to the control plane After this traffic is identified, further analysis can be performed to classify it If needed, the other CPP/ CoPP policy entries can be updated to account for this traffic Deploying Control Plane Policing Policies Because CPP/CoPP filters traffic, it is critical to gain an adequate level of understanding about the legitimate traffic destined to the RP prior to deployment CPP/CoPP policies built without proper understanding of the protocols, devices, or required traffic rates involved can block critical traffic, which has the potential of creating a DoS condition Determining the exact traffic profile needed to build the CPP/CoPP policies might be difficult in some networks The following steps follow a conservative methodology that facilitates the process of designing and deploying CPP/CoPP This methodology uses iterative access control list (ACL) configurations to help identify and to incrementally filter traffic To deploy CPP/CoPP, you should perform the six steps that follow Step 1: Determine the Classification Scheme for Your Network Identify the known protocols that access the RP and divide them into categories using the most useful criteria for your specific network As an example of classification, the eight categories template presented earlier in this section (BGP, IGP, Interactive Management, File Management, Reporting, Critical Applications, Undesirable, and Default) use a combination of relative importance and traffic type Select a scheme suited to your specific network, which might require a larger or smaller number of classes Step 2: Define Classification Access Lists Configure each ACL to permit all known protocols in its class that require access to the RP At this point, each ACL entry should have both source and destination addresses set to any In addition, the ACL for the default class should be configured with a single entry, permit ip any any This matches traffic not explicitly permitted by entries in the other ACLs After the ACLs have been configured, create a class map for each class defined in Step 1, including one for the default class Then assign each ACL to its corresponding class map End-to-End QoS Network Design Note In this step, you should create a separate class map for the default class, instead of using the class default available in some platforms Creating a separate class map and assigning a permit ip any any ACL allows you to identify traffic not yet classified as part of another class Each class map should then be associated with a policy map that permits all traffic, regardless of classification The policy for each class should be set as conform-action transmit exceed-action transmit Step 3: Review the Identified Traffic and Adjust the Classification Ideally, the classification performed in Step identified all required traffic destined to the router However, realistically, not all required traffic is identified before deployment, and the permit ip any any entry in the default class ACL logs a number of packet matches Some form of analysis is required to determine the exact nature of the unclassified packets For example, you can use the show access-lists command to see the entries in the ACLs that are in use and to identify any additional traffic sent to the RP However, to analyze the unclassified traffic, you can use one of these techniques: Q General ACL classification based on traffic characterization and tracing by Cisco routers Note For more information on this technique, see http://www.cisco.com/en/US/tech/ tk59/technologies_tech_note09186a0080149ad6.shtml Q Packet analyzers When traffic has been properly identified, adjust the class configuration accordingly Remove the ACL entries for those protocols that are not used Add a permit ip any any entry for each protocol just identified Step 4: Restrict a Macro Range of Source Addresses Refine the classification ACLs by only allowing the full range of the allocated CIDR block to be permitted as the source address For example, if the network has been allocated 172.68.0.0/16, permit source addresses from 172.68.0.0/16 where applicable This step provides data points for devices or users from outside the CIDR block that might be accessing the equipment An external BGP (eBGP) peer requires an exception because the permitted source addresses for the session lies outside the CIDR block This phase might be left on for a few days to collect data for the next phase of narrowing the ACL entries Appendix B: Control Plane Policing Step 5: Narrow the ACL Permit Statements to Authorized Source Addresses Increasingly limit the source address in the classification ACLs to permit only sources that communicate with the RP For example, only known network management stations should be permitted to access the SNMP ports on a router Step 6: Refine CPP/CoPP Policies by Implementing Rate Limiting Use the show policy-map control-plane command to collect data about the actual policies in place Analyze the packet count and rate information and develop a rate-limiting policy accordingly At this point, you might decide to remove the class map and ACL used for the classification of default traffic If so, you should also replace the previously defined policy for the default class by the class default policy Note Table AB-1 shows a set of tested and validated CPP/CoPP rates Note that the values presented here are solely for illustration purposes, as every environment has different baselines Note At the time of this writing, the Catalyst 4500 supports policing rates only in bits per second (bps), but the Catalyst 6500 and Cisco IOS support both bps and packets per second (pps) rate limits When configuring CPP, rate limiting using a pps rate is generally preferred because it is the per-packet processing that more significantly impacts CPU utilization than the bps rate Table B-1 summarizes control plane policing rate examples along with recommended conforming and exceeding actions Table B-1 Control Plane Policing Rate Limits and Actions Examples Traffic Class Rate (pps) Rate (bps) Conform Action Exceed Action Border Gateway Protocol 500 4,000,000 Transmit Drop Interior Gateway 50 Protocol 300,000 Transmit Drop Interactive Management 500,000 Transmit Drop 6,000,000 Transmit Drop 100 File Management 500 End-to-End QoS Network Design Traffic Class Rate (pps) Rate (bps) Conform Action Exceed Action Monitoring 125 900,000 Transmit Drop Critical Applications 125 900,000 Transmit Drop Undesirable 101 32 Kbps1 Drop Drop Default 100 500,000 Transmit Drop The policing rate for the Undesirable CPP class is effectively 0—regardless of whether pps or bps rates are used and also regardless of what values these rates are set to—since both the conforming and exceeding actions for this class are set to drop However, the policer still performs a metering operation of the rates of these flows which is often useful for management purposes Cisco Catalyst 3850 Control Plane Policing The Catalyst 3850 switch supports control plane policing, but at the time of this writing, this feature is not configurable Cisco Catalyst 4500 Control Plane Policing On Cisco IOS Catalyst switches, CoPP comes into play right after the switching or the routing decision and before traffic is forwarded to the control plane When CoPP is enabled, the sequence of events (at a high level) is as follows: A packet enters the switch configured with CoPP on the ingress port The port performs any applicable input port and QoS services The packet gets forwarded to the switch CPU The switch CPU makes a routing or a switching decision, determining whether the packet is destined for the control plane Packets destined for the control plane are processed by CoPP and are dropped or delivered to the control plane according to each traffic class policy Packets that have other destinations are forwarded normally The Catalyst 4500 and Catalyst 6500 series switches implement CoPP similarly However, CoPP has been enhanced on both platforms to leverage the benefits of their hardware architectures, and as a result each platform provides unique features Therefore, the CoPP implementations on Catalyst 4500 and Catalyst 6500 series switches are discussed in platform-specific detail in their respective sections within this appendix The Catalyst 4500 series switches support CoPP in hardware in a centralized, nondistributed fashion CoPP policies are centrally configured under the control plane configuration mode and then enforced in hardware by the classification ternary content-address- Appendix B: Control Plane Policing able memory (TCAM) and QoS policers of the Supervisor Engine Figure B-1 shows this CoPP model The Catalyst 4500 implementation of CoPP uses MQC to define traffic classification criteria and to specify the configurable policy actions for the classified traffic MQC Switch CPU 16 CPU … Queues User-Defined CoPP Policies Control and CPU Bound Traffic Ingress Control Plane • Pre-configured System Traffic Types Apply: and/or • User-Configurable Traffic Types Forwarding ASICs Data Traffic Backplane Linecard Figure B-1 Linecard Catalyst 4500 Control Plane Policing Implementation uses class maps to define packets for a particular traffic class After you have classified the traffic, you can create policy maps to enforce policy actions for the identified traffic The control plane global configuration command allows the CoPP service policy to be directly attached to the control plane Catalyst 4500 CoPP supports the definition of non-IP traffic classes in addition to IP traffic classes With this, instead of using the default class for handling all non-IP traffic, you can define separate policies for non-IP traffic This results in better and more granular control over non-IP protocols, such as Address Resolution Protocol (ARP), Internetwork Packet Exchange (IPX), bridge protocol data units (BPDUs), Cisco Discovery Protocol (CDP), and Secure Socket Tunneling Protocol (SSTP) One particular characteristic of (Multi-Layer Switch [MLS]-based) Catalyst 4500 CoPP is that the CoPP policy must be named system-cpp-policy In fact, on these systems, the system-cpp-policy is the only policy map that can be attached to the control plane Note This restriction has been removed on MQC-based Catalyst 4500 series switches, beginning with the Supervisor 6-E However, to maintain backward compatibility, the system-cpp-policy name has been used in this configuration example To facilitate the configuration of system-cpp-policy, Catalyst 4500’s CoPP provides a global macro function (called system-cpp) that automatically generates and applies CoPP 10 End-to-End QoS Network Design policies to the control plane The resulting configuration uses a collection of systemdefined class maps for common Layer and Layer control plane traffic The names of all system-defined CoPP class maps and their matching ACLs contain the prefix systemcpp By default, no action is specified on any of the system predefined traffic classes Table B-2 lists the predefined system ACLs Table B-2 Catalyst 4500 System Predefined CoPP ACLs Predefined Named ACL Description system-cpp-dot1x MAC DA = 0180.C200.0003 system-cpp-lldp MAC DA=0180.c200.000E system-cpp-mcast-cfm MAC DA=0100.0ccc.ccc0 - 0100.0ccc.ccc7 system-cpp-ucast-cfm MAC DA=0100.0ccc.ccc0 system-cpp-bpdu-range MAC DA = 0180.C200.0000 - 0180.C200.000F system-cpp-cdp MAC DA = 0100.0CCC.CCCC (UDLD/DTP/VTP/Pagp) system-cpp-sstp MAC DA = 0100.0CCC.CCCD system-cpp-cgmp MAC DA = 01-00-0C-DD-DD-DD system-cpp-ospf IP Protocol = OSPF, IP DA matches 224.0.0.0/24 system-cpp-igmp IP Protocol = IGMP, IP DA matches 224.0.0.0/3 system-cpp-pim IP Protocol = PIM, IP DA matches 224.0.0.0/24 system-cpp-all-systems-on-subnet IP DA = 224.0.0.1 system-cpp-all-routers-on-subnet IP DA = 224.0.0.2 system-cpp-ripv2 IP DA = 224.0.0.9 system-cpp-ip-mcast-linklocal IP DA = 224.0.0.0/24 system-cpp-dhcp-cs IP Protocol = UDP, L4SrcPort = 68, L4DstPort = 67 system-cpp-dhcp-sc IP Protocol = UDP, L4SrcPort = 67, L4DstPort = 68 system-cpp-dhcp-ss IP Protocol = UDP, L4SrcPort = 67, L4DstPort = 67 In addition to the predefined classes, you can configure your own class maps matching other control plane traffic To take effect, these user-defined class maps need to be added to the system-cpp-policy policy map CoPP can be deployed on the Catalyst 4500 in one of two main ways: Q You can use the global macro macro global apply system-cpp to preconfigure CoPP access lists, class maps, and a system-cpp-policy policy map (with no class actions configured); an administrator can then modify this template and tune it to suit specific environments Appendix B: Control Plane Policing 11 Q The CoPP policy can be generated manually (as shown in the Example B-1) In Example B-1, CoPP has been deployed manually (to keep the policy as consistent as possible between the other CoPP/CPP configuration examples), inline with the previously defined recommendations Note It is recommended to include a CPP or CoPP prefix in the ACL and class map names to prevent any potential classification errors for similarly named ACLs and class maps used in data plane policies Example B-1 Catalyst 4500 Control Plane Policing Configuration Example ! This section defines the access lists for the CoPP traffic classes C4500-E(config)# ip access-list extended COPP-BGP C4500-E(config-ext-nacl)# remark BGP C4500-E(config-ext-nacl)# permit tcp host 192.168.1.1 host 10.1.1.1 eq bgp C4500-E(config-ext-nacl)# permit tcp host 192.168.1.1 eq bgp host 10.1.1.1 C4500-E(config)# ip access-list extended COPP-IGP C4500-E(config-ext-nacl)# remark IGP (OSPF) C4500-E(config-ext-nacl)# permit ospf any host 224.0.0.5 C4500-E(config-ext-nacl)# permit ospf any host 224.0.0.6 C4500-E(config-ext-nacl)# permit ospf any any C4500-E(config)# ip access-list extended COPP-INTERACTIVE-MANAGEMENT C4500-E(config-ext-nacl)# remark TACACS (return traffic) C4500-E(config-ext-nacl)# permit tcp host 10.2.1.1 host 10.1.1.1 established C4500-E(config-ext-nacl)# remark SSH C4500-E(config-ext-nacl)# permit tcp 10.2.1.0 0.0.0.255 host 10.1.1.1 eq 22 C4500-E(config-ext-nacl)# remark SNMP C4500-E(config-ext-nacl)# permit udp host 10.2.2.2 host 10.1.1.1 eq snmp C4500-E(config-ext-nacl)# remark NTP C4500-E(config-ext-nacl)# permit udp host 10.2.2.3 host 10.1.1.1 eq ntp C4500-E(config)# ip access-list extended COPP-FILE-MANAGEMENT C4500-E(config-ext-nacl)# remark (initiated) FTP (active and passive) C4500-E(config-ext-nacl)# permit tcp 10.2.1.0 0.0.0.255 eq 21 host 10.1.1.1 gt 1023 established C4500-E(config-ext-nacl)# permit tcp 10.2.1.0 0.0.0.255 eq 20 host 10.1.1.1 gt 1023 C4500-E(config-ext-nacl)# permit tcp 10.2.1.0 0.0.0.255 gt 1023 host 10.1.1.1 gt 1023 established C4500-E(config-ext-nacl)# remark (initiated) TFTP C4500-E(config-ext-nacl)# permit udp 10.2.1.0 0.0.0.255 gt 1023 host 10.1.1.1 gt 1023 12 End-to-End QoS Network Design C4500-E(config)# ip access-list extended COPP-MONITORING C4500-E(config-ext-nacl)# remark PING-ECHO C4500-E(config-ext-nacl)# permit icmp any any echo C4500-E(config-ext-nacl)# remark PING-ECHO-REPLY C4500-E(config-ext-nacl)# permit icmp any any echo-reply C4500-E(config-ext-nacl)# remark TRACEROUTE C4500-E(config-ext-nacl)# permit icmp any any ttl-exceeded C4500-E(config-ext-nacl)# permit icmp any any port-unreachable C4500-E(config)# ip access-list extended COPP-CRITICAL-APPLICATIONS C4500-E(config-ext-nacl)# remark HSRP C4500-E(config-ext-nacl)# permit ip any host 224.0.0.2 C4500-E(config-ext-nacl)# remark DHCP C4500-E(config-ext-nacl)# permit udp host 0.0.0.0 host 255.255.255.255 eq bootps C4500-E(config-ext-nacl)# permit udp host 10.2.2.8 eq bootps any eq bootps C4500-E(config)# ip access-list extended COPP-UNDESIRABLE C4500-E(config-ext-nacl)# remark UNDESIRABLE TRAFFIC C4500-E(config-ext-nacl)# permit udp any any eq 1434 ! This section defines the CoPP policy class maps C4500-E(config)# class-map match-all COPP-BGP C4500-E(config-cmap)# match access-group name COPP-BGP ! Associates COPP-BGP ACL with class map C4500-E(config)# class-map match-all COPP-IGP C4500-E(config-cmap)# match access-group name COPP-IGP ! Associates COPP-IGP ACL with class map C4500-E(config)# class-map match-all COPP-INTERACTIVE-MANAGEMENT C4500-E(config-cmap)# match access-group name COPP-INTERACTIVE-MANAGEMENT ! Associates COPP-INTERACTIVE-MANAGEMENT ACL with class map C4500-E(config)# class-map match-all COPP-FILE-MANAGEMENT C4500-E(config-cmap)# match access-group name COPP-FILE-MANAGEMENT ! Associates COPP-FILE-MANAGEMENT with class map C4500-E(config)# class-map match-all COPP-MONITORING C4500-E(config-cmap)# match access-group name COPP-MONITORING ! Associates COPP-MONITORING ACL with class map C4500-E(config)# class-map match-all COPP-CRITICAL-APPLICATIONS C4500-E(config-cmap)# match access-group name COPP-CRITICAL-APPLICATIONS ! Associates COPP-CRITICAL-APPLICATIONS ACL with class map C4500-E(config)# class-map match-all COPP-UNDESIRABLE C4500-E(config-cmap)# match access-group name COPP-UNDESIRABLE ! Associates COPP-UNDESIRABLE ACL with class map Appendix B: Control Plane Policing 13 ! This section defines the CoPP policy C4500-E(config-cmap)# policy-map system-cpp-policy C4500-E(config-pmap)# class COPP-BGP C4500-E(config-pmap-c)# police cir 4000000 bc 400000 be 400000 C4500-E(config-pmap-c-police)# conform-action transmit C4500-E(config-pmap-c-police)# exceed-action drop ! Polices BGP to Mbps C4500-E(config-pmap)# class COPP-IGP C4500-E(config-pmap-c)# police cir 300000 bc 3000 be 3000 C4500-E(config-pmap-c-police)# conform-action transmit C4500-E(config-pmap-c-police)# exceed-action drop ! Polices IGP to 300 Kbps C4500-E(config-pmap)# class COPP-INTERACTIVE-MANAGEMENT C4500-E(config-pmap-c)# police cir 500000 bc 5000 be 5000 C4500-E(config-pmap-c-police)# conform-action transmit C4500-E(config-pmap-c-police)# exceed-action drop ! Polices Interactive Management to 500 Kbps C4500-E(config-pmap)# class COPP-FILE-MANAGEMENT C4500-E(config-pmap-c)# police cir 6000000 bc 60000 be 60000 C4500-E(config-pmap-c-police)# conform-action transmit C4500-E(config-pmap-c-police)# exceed-action drop ! Polices File Management to Mbps C4500-E(config-pmap)# class COPP-MONITORING C4500-E(config-pmap-c)# police cir 900000 bc 9000 be 9000 C4500-E(config-pmap-c-police)# conform-action transmit C4500-E(config-pmap-c-police)# exceed-action drop ! Polices Monitoring to 900 Kbps C4500-E(config-pmap)# class COPP-CRITICAL-APPLICATIONS C4500-E(config-pmap-c)# police cir 900000 bc 9000 be 9000 C4500-E(config-pmap-c-police)# conform-action transmit C4500-E(config-pmap-c-police)# exceed-action drop ! Polices Critical Applications to 900 Kbps C4500-E(config-pmap)# class COPP-UNDESIRABLE C4500-E(config-pmap-c)# police cir 32000 bc 3000 be 3000 C4500-E(config-pmap-c-police)# conform-action drop C4500-E(config-pmap-c-police)# exceed-action drop ! Polices all Undesirable traffic (conform action is drop) C4500-E(config-pmap)# class class-default C4500-E(config-pmap-c)# police cir 500000 bc 5000 be 5000 C4500-E(config-pmap-c-police)# conform-action transmit C4500-E(config-pmap-c-police)# exceed-action drop ! Polices all other Control Plane traffic to 500 Kbps 14 End-to-End QoS Network Design ! This section attaches the CoPP policy to the control plane C4500-E(config)#control-plane C4500-E(config-cp)# service-policy input system-cpp-policy ! Attaches CoPP policy to control plane You can verify the configuration in Example B-1 with the following commands: Q show class-map Q show policy-map Q show policy-map control-plane Cisco Catalyst 6500 Control Plane Policing As previously stated, the Catalyst 4500 and Catalyst 6500 series switches implement CoPP similarly However, CoPP has been enhanced on both platforms to leverage the benefits of their hardware architectures, and as a result each platform provides unique features In the Catalyst 6500 series switches, CoPP takes advantage of the processing power present on linecards by implementing a distributed CoPP model In this platform, the class QoS policies are centrally configured under the control plane configuration mode When configured, these policies are first applied at the route processor (Multilayer Switch Feature Card [MSFC]) level, and then they get automatically pushed to the Policy Feature Card (PFC) and each Distributed Forwarding Card (DFC) Figure B-2 illustrates this CoPP model CoPP is enabled by default on the Catalyst 6500 These default settings provide adequate protection to the control plane in most deployment scenarios To create manual CoPP policies, the default CoPP policy needs to be disabled To disable the default CoPP configuration, enter the no service-policy input policy-default-autocopp control plane configuration mode command Example B-2 shows the corresponding CoPP configuration for the Catalyst 6500 series switches, based on the recommendations previously defined Note As previously mentioned, Cisco 6500 CoPP allows the ability to configure rate limits based on either bits per second or packets per second When configuring CPP, rate limiting using a pps rate is preferred because it is the per-packet processing that more significantly impacts CPU utilization than the bps rate The pps rates in the following example are based on Table B-1 Appendix B: Control Plane Policing 15 EOBC Distributed Control Plane Protection (CPP) in S/W Dist Line Card Line Card Switch Fabric Switch Fabric Line Card Line Card Distributed Control Plane Protection (CPP) in H/W Dist Line Card Line Card MSFC PFC4 EARL DBase Supervisor SP Figure B-2 Control Plane Protection (CPP) in H/W RP Control Plane Protection (CPP) in S/W Catalyst 6500 Control Plane Policing Implementation Note To minimize redundancy, ACLs and class maps—which are identical to the previous example—are not repeated here Example B-2 Catalyst 6500 Control Plane Policing Configuration Example ! This section defines the CoPP policy C6500-E(config)# policy-map COPP-POLICY C6500-E(config-pmap)# class COPP-ACL-BGP C6500-E(config-pmap-c)# police rate 500 pps burst 50 C6500-E(config-pmap-c-police)# conform-action transmit C6500-E(config-pmap-c-police)# exceed-action drop ! Polices BGP to Mbps C6500-E(config-pmap)# class COPP-ACL-IGP C6500-E(config-pmap-c)# police rate 50 pps burst C6500-E(config-pmap-c-police)# conform-action transmit C6500-E(config-pmap-c-police)# exceed-action drop ! Polices IGP to 300 Kbps C6500-E(config-pmap)# class COPP-ACL-INTERACTIVE-MANAGEMENT C6500-E(config-pmap-c)# police rate 100 pps burst 10 C6500-E(config-pmap-c-police)# conform-action transmit C6500-E(config-pmap-c-police)# exceed-action drop ! Polices Interactive Management to 500 Kbps C6500-E(config-pmap)# class COPP-ACL-FILE-MANAGEMENT 16 End-to-End QoS Network Design C6500-E(config-pmap-c)# police rate 500 pps burst 50 C6500-E(config-pmap-c-police)# conform-action transmit C6500-E(config-pmap-c-police)# exceed-action drop ! Polices File Management to Mbps C6500-E(config-pmap)# class COPP-ACL-MONITORING C6500-E(config-pmap-c)# police rate 125 pps burst 12 C6500-E(config-pmap-c-police)# conform-action transmit C6500-E(config-pmap-c-police)# exceed-action drop ! Polices Monitoring to 900 Kbps C6500-E(config-pmap)# class COPP-ACL-CRITICAL-APPLICATIONS C6500-E(config-pmap-c)# police rate 125 pps burst 12 C6500-E(config-pmap-c-police)# conform-action transmit C6500-E(config-pmap-c-police)# exceed-action drop ! Polices Critical Applications to 900 Kbps C6500-E(config-pmap-)# class COPP-ACL-UNDESIRABLE C6500-E(config-pmap-c)# police rate 10 pps burst C6500-E(config-pmap-c-police)# conform-action drop C6500-E(config-pmap-c-police)# exceed-action drop ! Polices all Undesirable traffic (conform action is drop) C6500-E(config-pmap)# class class-default C6500-E(config-pmap-c)# police rate 100 pps burst 10 C6500-E(config-pmap-c-police)# conform-action transmit C6500-E(config-pmap-c-police)# exceed-action drop ! Polices all other Control Plane traffic to 500 Kbps ! This section attaches the CoPP policy to the control plane C6500-E(config)#control-plane C6500-E(config-cp)# service-policy input COPP-POLICY ! Attaches CoPP policy to control plane You can verify the configuration in Example B-2 with the following commands: Q show class-map Q show policy-map Q show policy-map control-plane Cisco IOS Control Plane Policing (for ASR and ISR Routers) In Cisco IOS, CPP is implemented in software and comes into play right after the routing decision and before traffic is forwarded to the control plane, as shown in Figure B-3 In addition, Cisco IOS CPP allows for a CPP policy to be applied in the input/output directions Appendix B: Control Plane Policing 17 Process Level Control Plane Services Ingress CPP Policies Egress CPP Policies Interrupt Level Feature Checks and Switching Input Interface Figure B-3 Output Interface Cisco IOS Control Plane Policing Operation When CPP is enabled, the sequence of events (at a high level) is as follows: A packet enters the router (configured with CPP) on the ingress interface The interface performs any applicable input port and QoS services The packet gets forwarded to the router CPU The CPU makes a routing or switching decision, determining whether the packet is destined for the control plane Packets destined for the control plane are processed by CPP and are dropped or delivered to the control plane according to each traffic class policy Packets that have other destinations are forwarded normally (Optional) Packets originating from the control plane may also be subject to outbound CPP policies, to prevent the router from being utilized as a source of network reconnaissance or DoS attack The corresponding CPP configuration for the Cisco IOS routers, based on the recommendations previously defined, is shown in Example B-3 Note As previously mentioned, Cisco IOS CPP allows the ability to configure rate limits based on either bits per second or packets per second When configuring CPP, rate limiting using a pps rate is preferred because it is the per-packet processing that more significantly impacts CPU utilization than the bps rate The pps rates in the following example are based on Table B-1 18 End-to-End QoS Network Design Note To minimize redundancy, ACLs and class maps—which are nearly identical to those in Example B-1—are not repeated here The only difference is that the prefix CPP is used in ACL and class map and policy map names in this example, as compared to CoPP in the previous examples Example B-3 Cisco IOS Control Plane Policing Configuration Example ! This section defines the CPP-Policy policy map Router(config)# policy-map CPP-POLICY Router(config-pmap)# class CPP-ACL-BGP Router(config-pmap-c)# police rate 500 pps Router(config-pmap-c-police)# conform-action transmit Router(config-pmap-c-police)# exceed-action drop ! Polices BGP control plane traffic to 500 pps Router(config-pmap)# class CPP-ACL-IGP Router(config-pmap-c)# police rate 50 pps Router(config-pmap-c-police)# conform-action transmit Router(config-pmap-c-police)# exceed-action drop ! Polices IGP control plane traffic to 50 pps Router(config-pmap)# class CPP-ACL-INTERACTIVE-MANAGEMENT Router(config-pmap-c)# police rate 100 pps Router(config-pmap-c-police)# conform-action transmit Router(config-pmap-c-police)# exceed-action drop ! Polices Management control plane traffic to 100 pps Router(config-pmap)# class CPP-ACL-FILE-MANAGEMENT Router(config-pmap-c)# police rate 500 pps Router(config-pmap-c-police)# conform-action transmit Router(config-pmap-c-police)# exceed-action drop ! Polices File-Mgmt control plane traffic to 500 pps Router(config-pmap)# class CPP-ACL-MONITORING Router(config-pmap-c)# police rate 125 pps Router(config-pmap-c-police)# conform-action transmit Router(config-pmap-c-police)# exceed-action drop ! Polices Monitoring control plane traffic to 125 pps Router(config-pmap)# class CPP-ACL-CRITICAL-APPLICATIONS Router(config-pmap-c)# police rate 125 pps Router(config-pmap-c-police)# conform-action transmit Router(config-pmap-c-police)# exceed-action drop ! Polices Critical Applications control plane traffic to 125 pps Router(config-pmap)# class CPP-ACL-UNDESIRABLE Router(config-pmap-c)# police rate 10 pps Router(config-pmap-c-police)# conform-action drop Router(config-pmap-c-police)# exceed-action drop Appendix B: Control Plane Policing 19 ! Polices Undesirable control plane traffic to (effectively) pps ! As both the conform and exceed action are set to drop Router(config-pmap)# class class-default Router(config-pmap-c)# police rate 100 pps Router(config-pmap-c-police)# conform-action transmit Router(config-pmap-c-police)# exceed-action drop ! Polices all other control plane traffic to 100 pps ! This section applies the CPP policy to the control plane ! The CPP policy is applied in both directions Router(config)# control-plane Router(config-cp)# service-policy input CPP-POLICY ! Attaches the CPP-POLICY to the control plane in the input direction Router(config-cp)# service-policy output CPP-POLICY ! Attaches the CPP-POLICY to the control plane in the output direction You can verify the configuration in Example B-3 with the following commands: Q show class-map Q show policy-map Q show policy-map control-plane { input | output } Additional Reading Cisco Enterprise Medianet Campus QoS Design 4.0: http://www.cisco.com/en/US/ docs/solutions/Enterprise/WAN_and_MAN/QoS_SRND_40/QoSCampus_40.html Cisco Enterprise Medianet WAN QoS Design 4.0: http://www.cisco.com/en/US/ docs/solutions/Enterprise/WAN_and_MAN/QoS_SRND_40/QoSWAN_40.html Cisco White Paper: Deploying Control Plane Policing: http://www.cisco.com/en/ US/prod/collateral/iosswrel/ps6537/ps6586/ps6642/prod_white_paper0900aecd804fa16a.html Characterizing and Tracing Packet Floods Using Cisco Routers: http://www.cisco com/en/US/tech/tk59/technologies_tech_note09186a0080149ad6.shtml Cisco Catalyst 4500 Control Plane Policing Configuration Guide: http://www cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/15.02SG/configuration/guide/ cntl_pln.html Cisco Catalyst 6500 Control Plane Policing Configuration Guide: http://www cisco.com/en/US/docs/switches/lan/catalyst6500/ios/15.0SY/configuration/guide/ control_plane_policing_copp.html Cisco IOS Control Plane Policing Configuration Guide: http://www.cisco.com/en/ US/docs/ios-xml/ios/qos_plcshp/configuration/15-0m/qos-plcshp-ctrl-pln-plc.html ... LAN Controller) QoS Design 435 Chapter 21 Converged Access QoS Design Case Study 477 xii End- to -End QoS Network Design Part V: Data Center QoS Design Chapter 22 Data Center QoS Design Considerations... 800 East 96th Street Indianapolis, IN 46240 ii End- to -End QoS Network Design End- to -End QoS Network Design Quality of Service for Rich-Media & Cloud Networks Second Edition Tim Szigeti, CCIE No... ASR 1000) QoS Design 697 Chapter 29 Branch Router (Cisco ISR G2) QoS Design 735 Chapter 30 WAN and Branch QoS Design Case Study 759 Part VII: MPLS VPN QoS Design Chapter 31 MPLS VPN QoS Design Considerations