RIPE65 – Amsterdam, NL September 24, 2012 SCALING MPLS – SEAMLESSLY RESILIENT SERVICE ENABLEMENT AT MASSIVE SCALE USING STANDARD PROTOCOLS Christian Martin Sr Director, Network Architecture Office of the CTO – Platform Systems Division, Juniper Networks ACKNOWLEDGEMENTS Many thanks to Maciek Konstantynowicz, Kireeti Kompella, Yakov Rekhter, Nitin Bahadur and many others from Juniper for their contribution to the developments of technologies described in this presentation Copyright © 2009 Juniper Networks, Inc www.juniper.net AGENDA Network design evolution “Seamless” MPLS § Architecture § Design use cases § MPLS in the access Universal Edge with MPLS access Copyright © 2009 Juniper Networks, Inc www.juniper.net NEW NETWORK GOALS STRATEGY: § Create an architecture for network integration, self automation and programmability § Simplify control and operations § Reduce TCO and enable new services Mega Data Centers Converged Supercore NGCOs Last 20 Miles Data Centers POPs Intermediate Offices COs Rea Last Miles liz e of th l a i tent o p e e tru h t e Future New Network Value Proposition Present ne ork etw n w Functional Integration Enable Simpler IT Systems Programmable Dynamic Network Breakthrough Economics Highly Scalable and Reliable Value Creation and Innovation Copyright © 2011 Juniper Networks, Inc www.juniper.net NEW NETWORK TOPOLOGY Legacy & Hub COs Priv DC HQ Next-Generation Central Offices (NGCO) METRO [AGGR] NETWORK ACCESS NETWORK BACKBONE NETWORK Remote Cabinets Universal Services Metro-Aggr Universal Edge Long-Haul Packet Optical Universal Edge (DSL & Cable) Broadband Access Home or SOHO Optical Access Cell Sites Universal Services Fabric Universal Edge Servers & Storage Packet Optical Long-reach Fiber (CO consolidation) Mobile Terminals Service Subscribers Mega Data Centers (or service POPs) TRANSMISSION/OPTICAL NETWORK Optical/TDM Access Branch Office Supercore INFRASTRUCTURE FOR NEW NETWORK Mega Data Centers Supercore NGCOs Access & Aggregation The All-IP NGN new network vision: § Eliminate silos, consolidate and streamline the access & metropolitan part of the SP networks § Optimize service delivery (network, content, applications) § Simplify network and service control and operation, enable streamlined IT Systems § Service innovation with software programmable network, leverage self-organizing network § Further integrate packet and optical network layers Copyright © 2009 Juniper Networks, Inc www.juniper.net SEAMLESS MPLS - ARCHITECTURE Copyright © 2009 Juniper Networks, Inc www.juniper.net FIRSTLY - WHY IS MPLS USEFUL ? Control plane and data plane separation Unified data plane § Universal platform for Services Support for arbitrary hierarchy § Stack of MPLS labels § Used for Services, Scaling and fast service Restoration Copyright © 2009 Juniper Networks, Inc www.juniper.net IMPLEMENTATION: SEAMLESS MPLS FOUNDATION FOR THE CONVERGED NETWORK Network Scale and End-to-End service restoration § MPLS in the access, 100,000s of devices in ONE packet network § Seamless service recovery from any failure event (Sub-50ms) Decoupled network and service architectures § Complete virtualization of network services § Flexible topological placement of services – enabler for per service de-centralization § Minimized number of provisioning points, simplified end-to-end operation Clients Access Metro Aggregation Edge Core Seamless MPLS Networking at scale without boundaries Copyright © 2009 Juniper Networks, Inc www.juniper.net Data Center SEAMLESS MPLS FUNCTIONAL BLUEPRINT Seamless MPLS Network SH SH SN EN AN TN TN Metro-1 Region SN BN TN TN BN WAN Backbone Region TN TN AN EN Metro-2 Region Devices and their roles § Access Nodes – terminate local loop from subscribers (e.g DSLAM, MSAN) § Transport Nodes – packet transport within the region (e.g Metro LSR, Core LSR) § Border Nodes – enable inter-region packet transport (e.g ABR, ASBR) § Service Nodes – service delivery points, with flexible topological placement (e.g.BNG, IPVPN PE) § Service Helpers – service enablement or control plane scale points (e.g Radius, BGP RR) § End Nodes – represent customer network, located outside of service provider network Regions § A single network divided into regions: multiple Metro regions (leafs) interconnected by WAN backbone (core) § Regions can be of different types: (i) IGP area, (ii) IGP instance, (iii) BGP AS § All spanned by a single MPLS network, with any to any MPLS connectivity blueprints (AN to SN, SN to SN, AN to AN, etc) Decoupled architectures § Services architecture – defines where & how the services are delivered, incl interaction between SNs and SHs § Network architecture – provides underlying connectivity for services Copyright © 2009 Juniper Networks, Inc www.juniper.net SEAMLESS MPLS ARCHITECTURE CONNECTIVITY AND SERVICES BLUEPRINT “Seamless” MPLS Network SH SH SN EN AN TN TN SN BN Metro-1 Region TN AN TN S Pseudowire EN S AN Pseudowire SN AN EN 10 S AN Pseudowire SN EN Centralized Business edge L3 or L2 VPN Services SN Any2Any S SN C Pseudowire Content / hosted app Services AN SN Connectivity – provisioned Any2Any C by NMS or AAA L3/L3+ Services – S provisioned by NMS or AAA Internet Access Services SN Any2Any Copyright © 2009 Juniper Networks, Inc EN Network service provisioning and operation points: De-centralized residential edge C EN C De-centralized residential edge C AN Metro-2 Region Pseudowire C AN TN Basic Pt-to-Pt Connectivity Services Centralized Business edge EN BN WAN Backbone Region C EN TN www.juniper.net Internet RESTORATION FROM EDGE FAILURES P1 PE1 P3 PE3 CE2 CE1 PE2 P2 P4 edge failures edge failures Global-repair CE3 PE4 Local-repair Fast IGP - IS-IS, OSPF PE-CE link failure § As for transit failures § (1) Vrf-table-label with IP lookup § Used as a trigger for BGP next-hop § (2) PE-CE link protection change Hierarchical FIB § Hierarchical FIB with pre-programmed alternate BGP next-hops Egress PE node protection ! § LSP tailend protection with context label lookup § Local-repair by PLR* transit router § Based on the Junos indirect- and composite-next-hop technologies 54 *PLR – point of local repair Copyright © 2009 Juniper Networks, Inc www.juniper.net PSEUDOWIRE HEADEND TERMINATION (PHT) FOR BUSINESS AND BROADBAND SERVICES Service Node PW Type Encapsulation and PHT classification IP-payload or PW-lbl MAC IPH or PW-lbl MAC VLAN IPH 11 PW-lbl IPH pht-1 PW-1 IP-payload PW-2 VLAN-1 VLAN-2 pht-2.1 pht-2.2 IP-payload vrf-g vrf-b PW-lbl – Pseudowire label MAC – MAC header VLAN – 802.1Q or QinQ/.3AD tag IPH – IP header vrf-r Business Edge § Pseudowire per subscriber (customer) line, carries a single service or bundle of services (service per VLAN, multiple VLANs) § Implementation based on JUNOS LT, later on Pseudowire Services IFD Broadband Edge § Pseudowire per access node (DSLAM), carries multiple subscriber lines and sessions § Implementation based on JUNOS Pseudowire Services IFD 55 Copyright © 2009 Juniper Networks, Inc www.juniper.net PHT FAILURE HANDLING LOCAL LINK FAILURE Local link failure is handled by native local-repair CPE AGN1a § IS-IS LFA with MPLS LDP AGN2a LT-IFD § RSVP TE-FRR AN § L2 LAG PE pri-RE bkp-RE vpn core ECMP path vpn PFE1.1 Edge LC1 PFE3.1 Core LC3 PHT traffic forwarding in case of link failure § via backup link to PE AGN1b § then internally via fabric to PFE core ECMP path PFE4.1 Core LC4 PFE2.1 Edge LC2 AGN2b hosting associated LT § apart from local-repair no other impact on service traffic IP redundancy - once IS-IS converges traffic directed by global-repair § no impact on access PW traffic LFA local-‐ repair path PE Failed link CPE AGN1a AGN2a AN X PFE1.1 Edge LC1 Same scheme applies to the adjacent AGN2 node failure AGN1b 56 LT-IFD Copyright © 2009 Juniper Networks, Inc AGN2b www.juniper.net PFE2.1 Edge LC2 pri-RE bkp-RE vpn core ECMP path vpn PFE3.1 Core LC3 core ECMP path PFE4.1 Core LC4 PHT FAILURE HANDLING LER LINECARD WITH PW REDUNDANCY (ACT/BKP) LER L2 aAachment circuit Linecard with LT failure handled by pre-provisioned backup LT and backup PW Primary PW AGN1a CPE AGN2a pri-RE bkp-RE core ECMP path LT-IFD VPN AN PFE1.1 Edge LC1 PHT traffic forwarding in case of LT linecard failure PFE2.1 Core LC3 LT-IFD VPN § via backup PW to backup LT AGN1b § restoration time dependent on PW AGN2b core ECMP path PFE2.1 Core LC4 PFE1.1 Edge LC2 Backup PW down detection time, activation of backup PW and routing convergence to backup LT-IFD LER s/over to bkp PW failed LC AGN1a CPE AGN2a pri-RE X LT-IFD bkp-RE core ECMP path VPN AN PFE1.1 Edge LC1 PFE2.1 Core LC3 LT-IFD AGN1b 57 Copyright © 2009 Juniper Networks, Inc AGN2b www.juniper.net VPN PFE1.1 Edge LC2 core ECMP path PFE2.1 Core LC4 PHT FAILURE HANDLING PE EDGE LINECARD FAILURE – PFE REDUNDANCY PE pri-RE CPE AGN1a Linecard with LT failure handled by pre-programmed redundant LT (rLT) and native local-repair AGN2a PFE1.1 Edge LC1 vpn AGN1b PHT traffic forwarding in case of LT linecard failure AGN2b core ECMP path PFE4.1 Core LC4 PFE2.1 Edge LC2 Failed edge linecard § via backup link to PE § then to rLT-IFD § no impact on service traffic PFE3.1 Core LC3 vpn stb rLT-IFD § L2 LAG IP redundancy - once IS-IS converges traffic directed by global-repair core ECMP path vpn § RSVP TE-FRR vpn act rLT-IFD AN § IS-IS LFA with MPLS LDP bkp-RE LFA local-‐ repair path CPE AGN1a AGN2a X rLT-IFD AN PE pri-RE bkp-RE vpn core ECMP path vpn PFE1.1 Edge LC1 PFE3.1 Core LC3 vpn act rLT-IFD vpn AGN1b 58 Copyright © 2009 Juniper Networks, Inc AGN2b www.juniper.net PFE2.1 Edge LC2 core ECMP path PFE4.1 Core LC4 E2E RESTORATION IP/MPLS LOCAL-REPAIR COVERAGE – 100%! Ingress: CE-PE link, PE node failure Egress: PE-CE link failure § ECMP, LFA § BGP PE-CE link local protection Transit: PE-P, P-P link, P node failure Egress: PE node failure (new)(*) § LFA based on IGP/LDP; if no 100% LFA § LSP tailend protection with context label coverage, delta with RSVP-TE § RSVP-TE FRR lookup on the backup PE § Failure repaired locally by adjacent P router using LFA (or TE-FRR) Packet based networks finally can provide E2E service protection similar to SDH 1:1 protection, regardless of network size and service scale This provides network layer failure transparency to service layers, becoming a major enabler for network consolidation (*) “High Availability for 2547 VPN Service”, Y.Rekhter, MPLS&Ethernet World Congress, Paris 2011 59 Copyright © 2009 Juniper Networks, Inc www.juniper.net PROTECTING A (SERVICE) TUNNEL ENDPOINT Customer region Service Provider region Customer region PE3 10.0.0.3 vrf CE1 172.16.0.0/16 vrf PLR PE1 vrf PE4 BGP LDP Traffic PLR: Point of Local Repair – this is one hop from the point of failure 60 Copyright © 2009 Juniper Networks, Inc www.juniper.net CE2 LSP TAIL END PROTECTION – BACKUP PE LOOKUP Backup label Service LSP IP Payload Dataplane operations MPLS->IP POP mpls.0 POP, MPLS->IPv4 backup-mpls.0 backup-vrf1.inet.0 61 Copyright © 2009 Juniper Networks, Inc www.juniper.net NH 711 IFL 69 STEP #1 – L2CIRCUIT LINK PROTECTION (AVAILABLE IN 10.4) Customer region Service Provider region PE3 10.0.0.3 l2vpn vrf CE1 172.16.0.0/16 vrf l2vpn Customer region PLR PE1 l2vpn vrf CE2 PE4 BGP LDP Traffic Backup RSVP LSP Configuration on PE2: Configuration on PE1: protocols { l2circuit { protocols { neighbor 1.1.1.4 { l2circuit { interface fe-1/0/2.1001 { neighbor 1.1.1.3 { egress-protection { interface fe-1/0/1.1001 { protected-l2circuit PW31 { egress-protection { ingress-pe 1.1.1.3; protector-pe 1.1.1.2 egress-pe 1.1.1.1; context-identifier 10.0.0.3; virtual-circuit-id 13; } } } } } } } } } } } www.juniper.net Copyright © 2009 Juniper Networks, Inc 62 L3VPN PE-CE LINK PROTECTION (11.4) Customer region Service provider Region Customer region Primary l3vpn l3vpn Backup l3vpn ldp bgp Traffic Normal m-IBGP Configuration on PE1:! ! [edit routing-instances vpn-xy]! routing-options {! forwarding-table {! link-protection;! }! }! 63 [edit routing-instances vpn-xy]! routing-options {! ! l3vpn-composite-nexthop;! multipath vpn-unequal-cost equal-externalinternal;! }! ! ! Copyright © 2009 Juniper Networks, Inc www.juniper.net L3VPN PE NODE PROTECTION (TARGET 1H-2012) Customer region Service provider Region Customer region Primary l3vpn l3vpn Backup l3vpn ldp bgp Traffic Normal m-IBGP protocols {! bgp {! group internal ! family [inet-vpn|inet6-vpn|iso-vpn]! unicast {! egress-protection {! context-identifier 10.0.0.3;! }! }! }! }! Copyright © 2009 Juniper Networks, }! Inc www.juniper.net Configuration on Backup PE:! ! [edit routing-instances vpn-xy]! interface ge-0/1/0.200 {! egress-protection {! context-identifier 10.0.0.3;! }! }! 64 PE-CE LINK FAILURE LOCAL-REPAIR - SOLUTION Route Flow Traffic Flow Primary path Backup path PE1 P1 local failure detection P3 PE3-PLR CE2 CE1 PE2 P2 P4 PE4 CE3 Choices for handling egress PE-CE link failure § Use PE-CE link protection for any label allocation mode PE-CE link protection (local-repair) § Core facing nexthop(s) installed in FIB as alternate (backup) for CE facing routes § Upon local PE-CE failure FIB in-place modification of CE routes to use alternate nexthop(s), using JUNOS indirect-next-hop § Support for both BGP uni-path and multi-path 65 Copyright © 2009 Juniper Networks, Inc www.juniper.net EGRESS PE NODE FAILURE LOCAL-REPAIR – LSP TAILEND PROTECTION* Route Flow Traffic Flow Primary path Backup path PE1 P1 P3-PLR protected PE3 CE2 CE1 PE2 P2 P4 protector PE4 CE3 § Protector PE4 maintains a “mirror image” of the protected PE3 service label table – a context specific label space identified by a context-id (an IP address) present on both protected and protector PEs § Protected PE3 “owns” the context-id address, advertising it in the BGP Next_Hop attribute (context-id is never used for control plane peerings) § In case of protected PE3 failure, P3-PLR diverts the traffic destined to the context-id address to the protector PE4 using TE FRR or IP FRR procedures § Protector PE4 looks up received packets in the context-specific label table for PE3 (identified by the label associated with PE3 context-id), and forwards packets to the right destination * draft-minto-2547-egress-node-fast-protection 66 Copyright © 2009 Juniper Networks, Inc www.juniper.net IPFRR – LFA VS NOTVIA VS PQ LFA is useful and networks are being designed to improve coverage (draft-ietf-rtgwg-lfa-applicability) § but LFA doesn’t guarantee 100% coverage § Increasing Demand for IP/LDP Fast-Reroute with 100% Coverage NotVia can guarantee coverage but requires significant network state § Research done to reduce it, but nothing sufficiently practical & it’s been years PQ tunnels (aka remote LFA) cannot guarantee 100% coverage Requires explicit tunnels requires targeted LDP sessions for FEC label bindings Topologies change due to failures and growth § 100% Coverage gives protection always – § not just until the first maintenance event => Increasing Requirement and Demand for IP/LDP Fast- Reroute with 100% Coverage 67 Copyright © 2009 Juniper Networks, Inc www.juniper.net ... www.juniper.net * IP /MPLS control plane protocol stack and MPLS dataplane per “Deployment Scenario #1” in draft -mpls- seamless -mpls- 00 PW-L BGP-L LDP-L pop PW-L BGP-L pop PW-L CPE SEAMLESS MPLS – USE CASE... www.juniper.net * IP /MPLS control plane protocol stack and MPLS dataplane per “Deployment Scenario #1” in draft -mpls- seamless -mpls- 00 PW-L BGP-L LDP-L pop PW-L BGP-L pop PW-L CPE SEAMLESS MPLS – USE CASE... www.juniper.net * IP /MPLS control plane protocol stack and MPLS dataplane per “Deployment Scenario #1” in draft -mpls- seamless -mpls- 00 PW-L BGP-L LDP-L pop PW-L BGP-L pop PW-L CPE-R ENABLING IP /MPLS SCALE