proper label allocation if the interfaces are not listed under this command hierarchy. Not all default decisions are obvious. The neighbour state is verified using the show ldp neighbor command. JUNOS output You can verify the neighbour state using the show ldp neighbor detail operational level command.The output displays the session IP addresses plus the neighbour’s link IP address. hannes@Frankfurt> show ldp neighbor detail Address Interface Label space ID Hold time 10.0.0.5 so-0/1/2.0 62.154.13.8:0 11 Transport address: 62.154.13.8, Configuration sequence: 0 Up for 01:33:30 LDP is very much dependent on a working IGP. LDP itself cannot be run in stand- alone mode. Like BGP it is topology agnostic and cannot assert which label is better over another. LDP picks the label of the outgoing interface based on the best IGP distance. If the LDP topology is non-congruent than the IGP topology then LDP paths might get black holed. One of the most frequent configuration mistakes is that the list of interfaces that run IS-IS and the list of interfaces which run LDP are not the same. Consider Figure 14.15. All links in the core network have IS-IS and LDP enabled, except the link between Washington and New York, which lacks LDP due to a configuration mistake. Paris learns the /32 FEC of the New York router via the London, Frankfurt, Washington path and selects the path via Washington because it is on the shortest path tree. The traffic gets labelled to Washington where it gets black holed because no valid MPLS labelled switched paths to the FEC of New York are available. MPLS Signalling Protocols 421 10 IS-IS LDP 4 IS-IS LDP 12 IS-IS 4 IS-IS LDP 4 IS-IS LDP Frankfurt London Washington NewYork Paris 4 IS-IS LDP FIGURE 14.15. If the IS-IS and LDP topology is non-congruent Washington is black holing traffic If you are troubleshooting an MPLS reachability problem, the first thing to check is if the IS-IS adjacencies match the LDP session. It remains problematic why router vendors do not change their default behaviour. LDP should be automatically brought up as soon as you enable IS-IS on an interface. If someone does not want to run LDP, they could then explicitly turn it off. That way you can prevent a network from black holing traffic. 14.4.4 Conclusion Clients often ask what the “signalling protocol of choice” is. In 99 per cent of the cases, the answer is: both (LDP and RSVP-TE). Both protocols augment each other. LDP lacks path control, however. It is very frugal in its label usage and therefore inherently scalable. RSVP-TE is a heavyweight both from an administrative point of view as well as from a label allocation perspective; however, RSVP-TE has sound path control properties. So in general, networks use LDP, but once they need to offload some traffic from hot trunks, they use RSVP-TE in addition. There is no need to build full-mesh, explicitly configured RSVP-TE tunnels. First, pick a careful IGP metric scheme that provides good-enough routes, and then on top of that use RSVP-TE established TE-tunnels to take some heat off the hot trunks. 14.5 Complex Traffic Engineering by CSPF Computations Traffic engineering is deployed in two general ways: the first option is when the network administrator wants to have the maximum level of control and explicitly configures all the label switched paths, plus the EROs. In moderately complex topologies, however, manu- ally writing up tens to hundreds of EROs is a daunting task and almost certainly over- whelms the processing capabilities of humans. This is especially true if constraints like hop count and backup path diversity need to be considered; in these cases, automatic com- putation of EROs is the preferred choice. The computation of the EROs is done using a distributed traffic engineering database called the TED. The contents of this database are carried in IS-IS or OSPF. Figure 14.16 shows the differences between the two models. 422 14. Traffic Engineering and MPLS Extended IS-IS Routing Table Traffic Engineering Database (TED) Constrained Shortest Path First (CSPF) User constraints ERO RSVP Signalling 2 1 FIGURE 14.16. The RSVP Call Manager gets its input from the outcome of the CSPF calculation which is influenced by User Constraints and Topological Input In the first method, the network administrator supplies the ERO data, and in the second the EROs are calculated using a Constrained Shortest Path First Calculation (CSPF) based on user constrained TED input from the routers. The final result is an ERO which gets passed to RSVP-TE for LSP setup. You can display the contents of the TED database using the show mpls traffic- eng topology command in IOS and show ted database extensive com- mand in JUNOS. IOS command output London#show mpls traffic-eng topology My_System_id: 1921.6800.1008.00 (isis level-2) Signalling error holddown: 10 sec Global Link Generation 5 IGP Id: 1921.6800.1012.00, MPLS TE Id:192.168.0.12 Router Node (isis level-2) link[0]: Point-to-Point, Nbr IGP Id: 1921.6800.1008.00, nbr_node_id:1, gen:2 frag_id 0, Intf Address:172.16.0.2, Nbr Intf Address:172.16.0.1 TE metric:10, IGP metric:10, attribute_flags:0x0 physical_bw: 2488320 (kbps), max_reservable_bw_global: 2488320 (kbps) max_reservable_bw_sub: 0 (kbps) Global Pool Sub Pool Total Allocated Reservable Reservable BW (kbps) BW (kbps) BW (kbps) bw[0]: 0 2488320 0 bw[1]: 0 2488320 0 bw[2]: 0 2488320 0 bw[3]: 0 2488320 0 bw[4]: 0 2488320 0 bw[5]: 0 2488320 0 bw[6]: 0 2488320 0 bw[7]: 0 2488320 0 The TED database contains all IP addresses, links and current bandwidth reservation states. The data found here is the foundation for the CSPF calculation which produces a path described by an ERO. JUNOS command output hannes@Frankfurt> show ted database extensive TED database: 3 ISIS nodes 3 INET nodes NodeID: Frankfurt.00(192.168.0.8) Type: Rtr, Age: 189 secs, LinkIn: 1, LinkOut: 1 Protocol: IS-IS(2) To: London.00(192.168.0.8), Local: 172.16.0.1, Remote: 172.16.0.2 Color: 0 <none> Metric: 10 Static BW: 2488.32Mbps Reservable BW: 2488.32Mbps Complex Traffic Engineering by CSPF Computations 423 Available BW [priority] bps: [0] 2488.32Mbps [1] 2488.32Mbps [2] 2488.32Mbps [3] 2488.32Mbps [4] 2488.32Mbps [5] 2488.32Mbps [6] 2488.32Mbps [7] 2488.32Mbps Interface Switching Capability Descriptor(1): Switching type: Packet Encoding type: Packet Maximum LSP BW [priority] bps: [0] 2488.32Mbps [1] 2488.32Mbps [2] 2488.32Mbps [3] 2488.32Mbps [4] 2488.32Mbps [5] 2488.32Mbps [6] 2488.32Mbps [7] 2488.32Mbps Why isn’t the data for CSPF calculations taken straight from the link-state database of the routing protocol? Well, there still may be OSPF deployed in parts of the network. The TED is a unified view to the topology of the network, so no matter which IGP (OSPF, IS-IS, or even vendor-proprietary protocols) supplied the topology data. The TED is a unified, abstracted view and knows only about nodes, links and link attributes. How does IS-IS generate and encode the data in the TED output? How does it know that a certain interface is an OC-48 interface? As soon as RSVP-TE is enabled on an interface, a lot of extra information is generated and conveyed using IS-IS. Consider the following tcpdump output of a LSP before RSVP-TE has been turned on. Tcpdump output If RSVP-TE is not enabled on a core interface then no bandwidth relevant information is generated inside the Extended IS Reach TLV. 00:27:20.871975 OSI, IS-IS, length: 104 L2 LSP, hlen: 27, v: 1, pdu-v: 1, sys-id-len: 6 (0), max-area: 3 (0) lsp-id: 0620.0000.0001.00-00, seq: 0x00000030, lifetime: 1196s chksum: 0x1d9d (correct), PDU length: 104, L1L2 IS Area address(es) TLV #1, length: 4 Area address (length: 3): 49.0001 Protocols supported TLV #129, length: 1 NLPID(s): IPv4 Traffic Engineering Router ID TLV #134, length: 4 Traffic Engineering Router ID: 62.0.0.1 IPv4 Interface address(es) TLV #132, length: 4 IPv4 interface address: 62.0.0.1 Hostname TLV #137, length: 9 Hostname: Frankfurt Extended IS Reachability TLV #22, length: 23 IS Neighbor: 0621.5401.3008.00, Metric: 10, sub-TLVs present (12) IPv4 interface address subTLV #6, length: 4, 10.0.0.2 IPv4 neighbor address subTLV #8, length: 4, 10.0.0.1 Extended IPv4 Reachability TLV #135, length: 18 IPv4 prefix: 62.0.0.1/32, Distribution: up, Metric: 0 IPv4 prefix: 10.0.0.0/30, Distribution: up, Metric: 10 424 14. Traffic Engineering and MPLS Complex Traffic Engineering by CSPF Computations 425 Next, traffic engineering and RSVP-TE is configured on IOS and JUNOS and the resulting LSP structure is examined. IOS configuration In IOS you need to enable traffic-eng globally and under the router isis stanza. Additionally you need to enable it on each interface using the mpls traffic-eng tunnels command plus the ip rsvp bandwidth keyword specifies how much bandwidth can be reserved. London#sh running-config [… ] mpls traffic-eng tunnels ! interface POS4/1 [… ] ip router isis mpls traffic-eng tunnels tag-switching ip ip rsvp bandwidth 2488320 2488320 ! router isis mpls traffic-eng router-id Loopback0 mpls traffic-eng level-2 metric-style wide level-2 [… ] ! The ip rsvp bandwidth statement takes two parameters. The first is the max- imum amount of bandwidth that is reservable on the interface, and the second is the max- imum amount of bandwidth that is available for a single reservation. Typically those two values are the same, which means that a single reservation can eat up all the interface’s bandwidth. Under the router isis stanza you need to specify the IS-IS level to which you want to send traffic engineering information. Unfortunately, you need to decide for Level-1 or Level-2. Both levels are not yet supported. Typically Level-2 is configured, and that is done here. In JUNOS the sending of traffic engineering sub-TLV parameters is the default behav- iour and there is no need to configure any further global options. All that needs to be con- figured is to add the interface under the protocols rsvp stanza. JUNOS configuration In JUNOS you need to specify the interface where you want to send bandwidth and reser- vation state both under the protocols mpls {} and protocols rsvp {} stanza. Alternatively you can set the mpls interface list to all. You can change the 426 14. Traffic Engineering and MPLS oversubscription of RSVP bandwidth by changing the default value of 100% using the subscription keyword. hannes@Frankfurt# show [… ] protocols { mpls { interface all; } rsvp { interface so-0/1/2.0 { subscription 120; } } } [… ] As soon as you enable RSVP-TE on an interface on which the router has established an adjacency, then the LSP gets updated with a lot of extra information, encoded by adding several sub-TLVs to the extended IS Reachability TLV #22. Tcpdump output An RSVP-TE enabled IS-IS adjacency shows the interface speed plus current reservation state using 8 pre-emption classes. 00:28:20.760649 OSI, IS-IS, length: 156 hlen: 27, v: 1, pdu-v: 1, sys-id-len: 6 (0), max-area: 3 (0), pdu-type: L2 LSP lsp-id: 0620.0000.0001.00-00, seq: 0x00000031, lifetime: 1196s chksum: 0x2674 (correct), PDU length: 156, L1L2 IS Area address(es) TLV #1, length: 4 Area address (length: 3): 49.0001 Protocols supported TLV #129, length: 1 NLPID(s): IPv4 Traffic Engineering Router ID TLV #134, length: 4 Traffic Engineering Router ID: 62.0.0.1 IPv4 Interface address(es) TLV #132, length: 4 IPv4 interface address: 62.0.0.1 Hostname TLV #137, length: 9 Hostname: Frankfurt Extended IS Reachability TLV #22, length: 75 IS Neighbor: 0621.5401.3008.00, Metric: 10, sub-TLVs present (64) IPv4 interface address subTLV #6, length: 4, 10.0.0.2 IPv4 neighbor address subTLV #8, length: 4, 10.0.0.1 Unreserved bandwidth subTLV #11, length: 32 priority level 0: 2488.320 Mbps priority level 1: 2488.320 Mbps priority level 2: 2488.320 Mbps priority level 3: 2488.320 Mbps Complex Traffic Engineering by CSPF Computations 427 priority level 4: 2488.320 Mbps priority level 5: 2488.320 Mbps priority level 6: 2488.320 Mbps priority level 7: 2488.320 Mbps Reservable link bandwidth subTLV #10, length: 4, 2488.320 Mbps Maximum link bandwidth subTLV #9, length: 4, 2488.320 Mbps Administrative groups subTLV #3, length: 4, 0x00000000 Extended IPv4 Reachability TLV #135, length: 18 IPv4 prefix: 62.0.0.1/32, Distribution: up, Metric: 0 IPv4 prefix: 10.0.0.0/30, Distribution: up, Metric: 10 Figure 14.17 shows the contents of the Traffic Engineering Router ID TLV #134. It basically contains a single unique 32-bit ID in order to uniquely identify a router in the TED. The TE Router ID TLV #134 corresponds to the OSPF Router-ID and puts the topology gathered by the two protocols into a relationship in the TED. The underlying problem is that IS-IS identifies its nodes through System-IDs (48-bit) and OSPF does it using Router-IDs (32-bit). By issuing a TLV #134 the IS-IS speaker tells other routers what would be the corresponding OSPF router-ID in case one router is running both OSPF and IS-IS for transition purposes. In Table 14.2 there is a list of sub-TLVs to the extended Reachability TLV #22. These are used for conveying various pieces of link information like Admin (Affinity) Groups, bandwidth parameters and IPv4 endpoint addresses. Chapter 11 “TLVs and Sub-TLVs” explores more about TLVs and sub-TLV nesting. TLV Type TLV Length Traffic engineering router ID 132 Bytes 1 1 4 4 FIGURE 14.17. The Traffic Engineering TLV #134 contains a unique ID which identifies a TE speaker throughout disjoint TE domains TABLE 14.2. Sub-TLV code points. Sub-TLV Sub-TLV name 3 Administrative (Affinity) Group 4 Link Local ID 5 Link Remote ID 6 IPV4 Interface Address 8 IPV4 Remote Interface Address 9 Maximum Link Bandwidth 10 Reserve Able Bandwidth 11 Unreserved Bandwidth 18 TE-Metric 20 Link Protection Type (GMPLS) 21 Switching Capability (GMPLS) Not yet assigned by IANA Bandwidth Constraints 428 14. Traffic Engineering and MPLS After the TED has been populated with the above link-related information, the routers engage in a CSPF calculation based on the network operator’s constraints. The CSPF is a two-pass calculation where in the first pass all the links that do not fit a certain con- straint are removed, and the second pass is a vanilla SPF calculation as was described in Chapter 10, “SPF and Route Calculation”. See Figure 14.18 for an example of CSPF. The network needs to compute a label switched path between Washington and New York which can only run on links carrying the “Internet” Link Colour (Affinity Group) and must not run on links carrying the “Maintenance” Link Colour (Affinity Group). The amount of reserved bandwidth is 600 MBit/s. In the first pass of the CSPF calculation all the links that do not belong to the required “Internet” administrative group are removed. The direct link between Washington and New York does not fit the constraint because it carries the “Maintenance” Link Colour. Next, all the links that do not have sufficient bandwidth are removed. The reservation of additional 600 MBit/s would oversubscribe the link between Washington and Frankfurt and is removed as well. Based on the resulting “skeleton”, the routers run an SPF calculation and try to find the shortest path node between the source and the destination point. In our example, the path via Paris, Frankfurt and London fits all the constraints and therefore the tunnel comes up. The result of the SPF calculation does not really matter because in this case there is only a single path left which fulfils all constraints. If there are too many constraints during the first pass and there are no feasible paths at all, then the result of the SPF calculation will be that there is no shortest path between a pair of nodes. Note that in CSPF calculations, there is not any type of crank-back proced- ure where the systems try to find a path at all costs. This was common practice for voice networks, but crank-back schemes run the risk of sending traffic around the continent sev- eral times, like the overlay networks of the 1990s did. Sometimes the result of the CSPF calculation is even no result at all and then no tunnel will be signalled. 14.6 LDP over RSVP-TE Tunnelling Which signalling protocol (LDP or RSVP-TE) to use is one of the first questions that net- work operators raise when deploying MPLS. Many people like the “call-oriented” notion of RSVP-TE and the amount of control the network operator has over traffic. On the other hand, LDP works like a charm – you turn it on and seconds later you have got label switched paths to every corner of your network at almost no cost and with nice scaling properties. To achieve the same connectivity matrix that LDP creates, one would have to deploy RSVP-TE in a full-mesh fashion with a dedicated tunnel between all the MPLS edge routers. In moderate size networks full-mesh RSVP-TE may be a design choice, how- ever, in medium-to large-sized networks, this may be a scaling nightmare. Recall that in a full-mesh network with 1000 edge routers, one would need 1000 * (1000 – 1)/2 ϭ 499,500 label switched paths! The refresh noise alone from repeating each reservation every 30 sec- onds, which will be processed twice (PATH and RESV messages from all the core routers along the label switched path), would result in approximately 30,000 messages per second being processed by each core router. Although there are extensions to aggregate refreshes LDP over RSVP-TE Tunnelling 429 0 MBit/s rsvd. Internet, Maintenance 400 MBit/s rsvd. Internet 280 MBit/s rsvd. Internet 1900 MBit/s rsvd. Internet 1250 MBit/s rsvd. Internet 1100 MBit/s rsvd. Internet 4 4 6 4 4 4 400 MBit/s rsvd. Internet 280 MBit/s rsvd. Internet 1900 MBit/s rsvd. Internet 1250 MBit/s rsvd. Internet 1100 MBit/s rsvd. Internet 4 4 4 4 4 Prune all links that do not have Internet and Maintenance set 1 Prune all links that do not fit Bandwidth requirements 2 280 MBit/s rsvd. Internet 4 4 1100 MBit/s rsvd. Internet 4 1250 MBit/s rsvd. Internet Frankfurt London Washington New York Paris Frankfurt London Washington New York Paris Frankfurt London Washington New York Paris 400 MBit/s rsvd. Internet 4 FIGURE 14.18. In the CSPF calculation all paths that do not meet any of the constraints are pruned off the final topology 430 14. Traffic Engineering and MPLS LDP RSVP POP POP POP POP POP LDP over RSVP tunneling 3 Frankfurt London Washington NewYork Paris TE Tunnel 2 1 FIGURE 14.19. Traffic from Paris to London does not take the TE tunnel path (see RFC 2961 for details), and thereby reduce the refresh noise, the underlying problem (which is the familiar networking “N^2” problem) is not addressed by aggregation alone. For scalability reasons, network operators are tempted to use the more scaleable LDP, which sets up a kind of full-mesh matrix (based on sink trees). But LDP label selection is dictated by the IGP, and that translates to a lack of traffic path-control because no one wants to tweak IGP metrics anymore. So the answer to the signalling protocol question is most often to use both protocols where they fit best. LDP is used for setting up lightweight labels switched paths across the network, and RSVP-TE is used for traffic engineering. Consider Figure 14.19, where both protocols are deployed. LDP is deployed across the core for establishing label switched paths between all routers in the network (1). Additionally, there is a Traffic Engineering Tunnel between the core router in Paris and London (2). If traffic is loaded on that path, then all the traffic will be guided through LDP paths and the single RSVP-TE TE tunnel in the core is completely ignored. Why? Because MPLS is a source routing technique. The ingress router makes the choice as to which label switched path is used for traffic forwarding. If an edge (ingress) router does not know about a TE tunnel path in the core, then it will not use it. The trick is now to make LDP use the TE tunnel in the core for forwarding. A tech- nique called LDP over RSVP-TE tunnelling is used for that purpose. Previously, LDP was deployed in a hop-by-hop fashion – the LDP speakers propagate their label mapping messages from node to adjacent node. In order to make LDP use the TE tunnel, an add- itional LDP session is brought up between the Paris and London core router (3). For set- ting up a session between a pair of non-adjacent routers, an LDP option called targeted Hellos is used. Targeted Hellos are similar to internal BGP sessions. The two LDP speak- ers at the edge send a Hello across several hops. If the two speakers at the edge agree on the capabilities reported in the Hello message, then an LDP session (using TCP) is estab- lished to advertise label mappings. All label advertisements learned via the multi-hop . over another. LDP picks the label of the outgoing interface based on the best IGP distance. If the LDP topology is non-congruent than the IGP topology then LDP paths might get black holed. One of the. find the shortest path node between the source and the destination point. In our example, the path via Paris, Frankfurt and London fits all the constraints and therefore the tunnel comes up. The. switched paths to the FEC of New York are available. MPLS Signalling Protocols 421 10 IS-IS LDP 4 IS-IS LDP 12 IS-IS 4 IS-IS LDP 4 IS-IS LDP Frankfurt London Washington NewYork Paris 4 IS-IS LDP FIGURE