76 8600 50133k 8600 smart routers ethernet configuration guide

8609 Smart Router and 8611 Smart Router LAG support added in 2 Ethernet Link Aggregation.Eth lag min-link parameter description updated in2.3 LinkAggregation Configuration.. CCM Continui

Ethernet Configuration Guide

76.8600-50133K 01.06.2015

Revision History Document No Date Description of Changes

76.8600-50133K 01.06.2015 Added support for 8665 Smart Router

8.2 Ethernet Switching QoSupdated

ELP support for 8615 Smart Router and 8665 Smart Router stacked.Ethernet PWE3 redundancy support added for 8615 Smart Routerstacked

QinQ interface support added for 8615 Smart Router and 8665 SmartRouter

VPLS Scalability table added in12.5 8600 VPLS Implementation.LAG support for 8615 Smart Router and 8665 Smart Router added.Ethernet Forwarding and OAM updates for 8615 Smart Routerstacked and 8665 Smart Router

LAG interface support for MAC switching for 8602 Smart Router8602-D, 8609 Smart Router, 8611 Smart Router and ELC1 line card.76.8600-50133J 05.11.2014 Support for 8615 Smart Router added

MAC Switching support for 8602 Smart Router, 8609 Smart Routerand 8611 Smart Router added in8 Ethernet Switching

New chapter8.3 QoS Aware Flooding in MAC Switching and VPLS.New chapter9.2 Flooding Policer CLI Examples

New chapter1.1.3 Pause Frames Support in 8600 Network Elements.LACP support added to2 Ethernet Link Aggregation

ELC1 LAG support added to2 Ethernet Link Aggregationand

3.3 Joining Links to a Link Aggregation Group.LAG restrictions added to2.8 Limitations and Restrictions for LAG.New chapter

Limitations and Restrictions for electrical SFP support in

16.2 Limitations and Restrictions for ELP.76.8600-50133I 28.05.2014 Support for 8602 Smart Router added

QinQ interface updates added in1.1.2 Ethernet Logical Interfaces,

4.4.2 Ethernet PWE3 Forwarding,4.4.3 Internal Ethernet InterfaceBridging,4.4.4 Limitations and Restrictions for Ethernet PWE3,,14.5 Limitations and Restrictions for Ethernet OAMand

2.8 Limitations and Restrictions for LAG.Extended Ethernet PWE3 attachment circuit support for LAG in 8600smart routers in2 Ethernet Link Aggregationand3 Ethernet LinkAggregation CLI Configuration Examples

Updated ETH PWE3 CLI example in

Restriction removed from10.2 Limitations and Restrictions for IRB:

• Flexible IRB is supported only in configurations with a combination

of ELC1 line cards and CDC1 control cards

76.8600-50133G 13.03.2014 1x10GBASE-R HM data added in1.1.1 Ethernet Physical Interfaces

ELC1 Ethernet MAC switching support added in8 EthernetSwitching

8.5 Applications of Ethernet Switchingtitle updated

8.6 Limitations and Restrictions for Ethernet Switchingadded.Ethernet pseudowire VCCV support updated in

Ethernet pseudowire redundancy data added in

4.4.4 Limitations and Restrictions for Ethernet PWE3added.Note on ARP resolution with IFC2 IRB and flexible IRB supportadded in

10.2 Limitations and Restrictions for IRBadded

Virtual Private LAN Service (VPLS) data added in and related CLIexamples in

Ethernet Switching Service configuration example updated in

9.1 Ethernet Switching Service CLI Examples.IRB configuration examples Manual Configuration Inside One NEand Ethernet PWE3 to VRF termination via IRB/VPLS added in

11 Integrated Routing and Bridging CLI Examples.ELC1 Ethernet OAM support in

8609 Smart Router and 8611 Smart Router LAG support added in

2 Ethernet Link Aggregation.Eth lag min-link parameter description updated in2.3 LinkAggregation Configuration

IP fragmentation data in LAG load balancing updated in

2.3.4 Ethernet Link Aggregation Group

2.8 Limitations and Restrictions for LAGadded

ELC1 ELP support added in16 Ethernet Link Protection.ELP passive-if mode description and ELP group configuration rulesupdated in16.1.1 Ethernet Link Protection Group

ELP passive-if mode down added to16.1.5 Operational Examples

16.2 Limitations and Restrictions for ELPadded

CPU packet storm control data added in20 CPU Packet StormControl

© 2014 Tellabs All rights reserved.

This manual is protected by U.S and international copyright laws, conventions and treaties Your right to use this manual is subject to limitations and restrictions imposed by applicable licenses and copyright laws Unauthorized reproduction, modification, distribution, display or other use of this manual may result in criminal and civil penalties The following trademarks and service marks are owned by Tellabs Operations, Inc or its affiliates in the United States and/or other countries: TELLABS ® , TELLABS ®

logo, TELLABS and T symbol ® , T symbol ® , CORIANT TM Any other company or product names may be trademarks of their respective companies.

The specifications and information regarding the products in this manual are subject to change without notice All statements, information, and recommendations in this manual are believed to be accurate but are presented without warranty of any kind,

express or implied Users must take full responsibility for their application of any products.

Adobe ® Reader ® are registered trademarks of Adobe Systems Incorporated in the United States and/or other countries Tellabs and Coriant are joining forces You may see references to Coriant or Tellabs when doing business with us Contact

information is available athttp://www.coriant.com.

Terms and Abbreviations Term Explanation

AC Attachment CircuitARP Address Resolution ProtocolATM Asynchronous Transfer Mode

C Customer device, usually a router a LAN switch (not a 8600 element) The device is

owned by the customer and does not have a direct WAN connectivity

CCM Continuity Check MessageControl card The control card in the 8600 system consists of a Control and DC Power Card CDC

(used in 8630 Smart Router and 8660 Smart Router), Switching and Control ModuleSCM (used in 8611 Smart Router) or Control Functionality CU (used in 8615 SmartRouter and 8665 Smart Router) In 8000 Intelligent Network Manager and CLIreferred to as unit

CE Customer Edge deviceCFM Connectivity Fault ManagementCLI Command Line InterfaceCSMA/CD Carrier Sense Multiple Access with Collision DetectionC-tag Customer VLAN tag (the inner VLAN tag)

CU Control Functionality (used in 8615 Smart Router and 8665 Smart Router)C-VLAN Customer VLAN

DA Destination AddressDMM Delay Measurement MessageDMR Delay Measurement ResponseDSLAM Digital Subscriber Line Access MultiplexerE-CCM Ethernet OAM CCM for IDU-ODUELC1 Ethernet Line Card (used in 8630 Smart Router and 8660 Smart Router)ELP Ethernet Link Protection

ETH-DM Ethernet OAM Frame Delay MeasurementEthernet PDU Ethernet frame excluding preamble (7 bytes), start of frame delimiter (1 byte) and

inter-frame gap (12 bytes)

EtherType A two-octet field in an Ethernet frame It is used to indicate which protocol is

encapsulated in the payload of an Ethernet frame

ETH-LM Ethernet OAM Frame Loss Measurement

FDV Frame Delay VariationFEC Forward Equivalence ClassFIB Forwarding Information BaseFLR Frame Loss Ratio

FMC Fixed-mobile convergenceFTN Forwarding equivalence class To Next hop label forwarding entry map (for MPLS)

IFC Interface Module Concentrator is a line card baseboard and it can be equipped with

one or two IFMs There are three variants available: IFC1-A, IFC1-B and IFC2-B(used in 8630 Smart Router and 8660 Smart Router)

IFDV Inter-Frame Delay VariationIFM Interface Module A specific term of the module which can be placed on the IFC line

card or 8620 Smart Router baseboard and which consists of the physical interfaces(used in 8620 Smart Router, 8630 Smart Router and 8660 Smart Router)

IGMP Internet Group Management ProtocolILM Incoming Label Map (for MPLS)

IP Internet ProtocoliQoS Internal Quality of ServiceIRB Integrated Routing and Bridging

LACP Link Aggregation Control ProtocolLACPDU Link Aggregation Control Protocol Data UnitLAG Ethernet Link Aggregation

LAN Local Area NetworkLAN-IC LAN InterconnectionLAPF Link Access Procedure/Protocol for Frame Mode ServicesLBM Loopback Message

LBR Loopback ResponseLDP Label Distribution Protocol (MPLS)

Line card The line card in the 8600 system consists of an Ethernet Line Card (ELC1), Interface

Module Concentrator (IFC) and up to two Interface Modules (IFMs) (used in 8630Smart Router and 8660 Smart Router) or Line Unit LU (used in 8615 Smart Routerand 8665 Smart Router) In 8000 Intelligent Network Manager and CLI referred

LU Line Unit (used in 8615 Smart Router and 8665 Smart Router)

MA Maintenance AssociationMAC Media Access Control Sublayer of Data Link LayerMAID Maintenance Association Identifier

MAN Metro Area Network

MD Maintenance DomainMDL Maintenance Domain Level

ME Maintenance EntityMEP Maintenance EndpointMIP Maintenance Intermediate PointMLPPP Multi-Link Point to Point Protocol

MP Maintenance PointMPLS Multiprotocol Label Switching, a switching method that forwards IP traffic using a

labelMSP1+1 Multiplex Section Trail Protection 1+1MSTP Multiple Spanning Tree ProtocolMTU Maximum Transfer/Transmission UnitMTU Multi Tenant Unit A large building block with several rental apartment of offices

in different floors

Multiple hoppath

Involves multiple packet forwarding decisions

Node In 8000 intelligent network manager and CLI refers to network element

OAM Operation, Administration and MaintenanceODU Out Door Unit

P-CCM Ethernet OAM CCM for ODU-ODUPDU Protocol Data Unit

PE Provider Edge Network ElementPIM Protocol Independent Multicast

PM Performance MonitoringPOS Packet over SDH/SonetPRI bits User Priority field bitsProtecting

QoS Quality of Service

RAN Radio Access NetworkRDI Remote Defect IndicationRMEP Remote Maintenance EndpointRNC Radio Network ControllerRSTP Rapid Spanning Tree Protocol

SAToP Structure-Agnostic Time Division Multiplexing over PacketSCM Switching and Control Module (used in 8611 Smart Router)Single hop Involves only a single packet forwarding decision

SLI Set Link Info control messageSNAP Subnetwork Access ProtocolSSM Synchronization Status MessageS-tag Service VLAN tag (the outer VLAN tag)STP Spanning Tree Protocol

SU Switch Unit (used in 8665 Smart Router)S-VLAN Service VLAN

TDM Time Division MultiplexingTLV Type Length Value

T-PE Terminating Provider Edge Network Element (RFC4364)TTL Time-to-Live

VRF VPN Routing and Forwarding (RFC4364)VRRP Virtual Router Redundancy ProtocolVSI Virtual Switching Instance

WAN Wide Area NetworkWorking

interface

This ELP group member interface is selected by default as the active interface Theworking interface must be the right-hand side interface of the two interfaces of anELP group (in case of 8660 Smart Router the interface residing on the right-hand sideIFC, in case of 8620 Smart Router the right-hand side interface on the IFMs, and incase of 8630 Smart Router the lower interface as IFCs are in horizontal direction).Note that in CLI a working interface is referred to as primary interface

Table of Contents

About This Manual 17

Objectives 17

Audience 17

8600 Smart Routers Technical Documentation 17

Interface Numbering Conventions 21

Document Conventions 21

Documentation Feedback 21

8600 Smart Routers Discontinued Products 22

1 Ethernet Overview 23

1.1 Ethernet Interfaces 24

1.1.1 Ethernet Physical Interfaces 24

1.1.2 Ethernet Logical Interfaces 26

1.1.3 Pause Frames Support in 8600 Network Elements 27

2 Ethernet Link Aggregation 29

2.1 Application of Link Aggregation 30

2.2 References 31

2.3 Link Aggregation Configuration 31

2.3.1 Members 31

2.3.2 LACP Configuration 32

2.3.3 Attributes of Group 33

2.3.4 Ethernet Link Aggregation Group 35

2.3.5 Bandwidth Allocation in IFC2 Line Card 38

2.3.6 Synchronization 38

2.3.7 Queuing 38

2.4 Faults 39

2.5 Statistics 39

2.6 LACP States 39

2.6.1 LACPDU Explained 39

2.6.2 LACP Member Interfaces States 40

2.6.3 LACP Configuration and Protocol Status 41

2.7 LAG Troubleshooting 43

2.8 Limitations and Restrictions for LAG 45

3 Ethernet Link Aggregation CLI Configuration Examples 47

3.1 Creating a Link Aggregation Group 47

3.2 Provisioning LACP 47

3.3 Joining Links to a Link Aggregation Group 48

3.4 Removing Links From a Link Aggregation Group 48

3.5 Static LAG Operations 49

3.5.1 Minimizing Traffic Disturbance When Joining a Link to a Group 49

3.5.2 Minimizing Traffic Disturbance When Removing a Link 50

3.6 Removing a Link Aggregation Group 51

3.7 Configuring Load Balancing Parameters 51

3.8 Configuring Member Link Properties during Membership 52

3.9 Configuring L2/L3 Properties of a Link Aggregation Group 53

3.10 Creating an IP Interface on a Link Aggregation Group 54

3.11 Creating a VLAN on a Link Aggregation Group 55

3.12 Showing and Clearing of Link Aggregation Group Statistics 55

3.13 Showing and Clearing of VLAN Statistics 56

4 Pseudowire Emulation Edge-to-Edge (PWE3) 58

4.1 Overview 58

4.1.1 Terminating Provider Edge (T-PE) 59

4.1.2 Switching Provider Edge (S-PE) 59

4.2 Virtual Circuit Connectivity Verification Overview 60

4.2.1 Control Channel Methods 60

4.2.2 Connectivity Verification 62

4.2.3 Multi-Segment PWE3 VCCV LSP Ping and Traceroute 63

4.3 PWE3 Types 64

4.3.1 Ethernet PWE3 Support (Raw, Tagged) 65

4.4 Ethernet Pseudowires 66

4.4.1 Application Example of Ethernet Pseudowires 66

4.4.2 Ethernet PWE3 Forwarding 66

4.4.3 Internal Ethernet Interface Bridging 73

4.4.4 Limitations and Restrictions for Ethernet PWE3 73

4.5 PWE3 Counters 73

4.6 References 73

5 Pseudowire Redundancy 75

5.1 Overview 75

5.2 Supported Functionality 77

5.3 Limitations and Restrictions 78

5.4 Operation 79

5.4.1 Provisioning Redundancy Group 79

5.4.2 Switching Operation 80

5.4.3 Dynamically Provisioned PWE3 Redundancy 80

5.4.4 Statically Provisioned PWE3 Redundancy 81

5.4.5 VCCV BFD 81

5.4.6 PWE3 Redundancy Counters 81

5.5 PWE3 Redundancy Considerations 81

5.5.1 Specific PWE3 Types 81

5.5.2 Multi-Layer Protection 82

5.5.3 Configuration Checklist 82

5.6 References 82

6 Ethernet PWE3 Configuration Examples 83

6.1 Ethernet PWE3 Services Configuration 83

6.1.1 Configuration Summary 83

6.1.2 Configuration Prerequisites 84

6.1.3 PWE3 Configuration 85

6.1.4 Configuration Verification and Diagnostics 88

6.2 S-VLAN Tagged Raw Mode Ethernet PWE3 Configuration 89

6.3 Internal Bridging Configuration 90

6.3.1 Configuration Summary 91

6.3.2 Bridging Configuration 91

7 Ethernet PWE3 Redundancy Configuration Examples 93

7.1 Configuration Summary 95

7.2 T-PE Nodes Configuration 95

7.2.1 Node T-PE90 96

7.2.2 Node T-PE80 99

7.3 S-PE Nodes Configuration 102

7.3.1 Node S-PE76 102

7.3.2 Node S-PE77 104

7.3.3 Node S-PE79 105

7.4 Transit Nodes Configuration 107

7.4.1 NODE78 107

7.5 Configuration Verification and Diagnostics 107

8 Ethernet Switching 109

8.1 Ethernet Switching on QinQ Interface 116

8.2 Ethernet Switching QoS 117

8.3 QoS Aware Flooding in MAC Switching and VPLS 118

8.4 Ethernet Switching QoS on QinQ Interface 118

8.5 Applications of Ethernet Switching 118

8.6 Limitations and Restrictions for Ethernet Switching 120

8.7 References 120

9 Ethernet Switching CLI Configuration Examples 121

9.1 Ethernet Switching Service CLI Examples 121

9.1.1 Manual Configuration Inside One NE 121

9.2 Flooding Policer CLI Examples 122

9.2.1 Configuration Summary 123

9.2.2 Configuration Prerequisites 123

9.2.3 Configuration Check 127

10 Integrated Routing and Bridging 128

10.1 Applications of Integrated Routing and Bridging 132

10.2 Limitations and Restrictions for IRB 134

11 Integrated Routing and Bridging CLI Examples 136

11.1 Manual Configuration Inside One NE, IFC line card based IRB and 8665 Smart Router LU1 IRB 136

11.2 Manual Configuration Inside One NE, ELC1 based IRB 136

11.3 Ethernet PWE3 to VRF termination via IRB/VPLS 137

12 Virtual Private LAN Service 139

12.1 VPLS Introduction 139

12.2 Network Applications for VPLS 139

12.3 Fully Meshed VPLS Networks 140

12.4 Hierarchical VPLS Networks 140

12.5 8600 VPLS Implementation 141

12.5.1 Supported Control Plane Functionality 143

12.5.2 Supported Data Plane Functionality 144

12.6 Limitations and Restrictions for VPLS 146

12.7 References 146

13 Virtual Private LAN Service CLI Examples 147

13.1 Configuring Dual-Homed MTU-s with Redundant Spoke PWE3s and Port Mode Attachment Circuit 147

13.2 Configuring Primary Spoke PWE3 and Mesh Peers to PE1-rs 148

13.3 Configuring Secondary Spoke PWE3 and Mesh Peers to PE2-rs 148

13.4 Configuring Remote PE3-rs to Join VPLS Core Mesh 149

14 Ethernet OAM Overview 150

14.1 Ethernet OAM Interfaces 151

14.2 Connectivity Fault Management 152

14.2.1 Continuity Check Function 153

14.2.2 Loopback Function 155

14.2.3 Linktrace Function 156

14.2.4 MIP OAM Processing 156

14.3 Performance Monitoring 157

14.3.1 Ethernet Frame Loss Measurement (ETH-LM) 157

14.3.2 Ethernet Frame Delay Measurement (ETH-DM) 158

14.4 Ethernet OAM Capacity 159

14.5 Limitations and Restrictions for Ethernet OAM 161

14.6 References 162

15 Ethernet OAM CLI Configuration Examples 163

15.1 Configuring Local MEP 164

15.2 Configuring CCM 164

15.2.1 Configure MEP with CCM Send Option 165

15.2.2 Autodiscovery Configuration 165

15.2.3 Static Configuration 165

15.3 Configuring Linktrace 166

15.4 Running Linktrace 166

15.5 Displaying Ethernet OAM Information 166

15.5.1 Displaying MD 166

15.5.2 Displaying MA 167

15.5.3 Displaying Maintenance Point 168

15.6 Ethernet OAM CFM Ping 169

15.7 Configuring QoS Mapping for Ethernet OAM PM 169

15.7.1 Notes on 8600 Interface QoS Configuration for Y.1731 Ethernet OAM 171

15.7.2 Notes on QoS Mapping Configuration for Y.1731 Ethernet OAM 171

15.8 Configuring Ethernet OAM PM - ETH-DM and ETH-LM 171

15.8.1 Examples of “Clear” CLI Command for ETH-LM and ETH-DM 173

15.8.2 Examples of “Show” CLI Command for ETH-LM and ETH-DM 173

15.8.3 Provisioning CLI Commands for Direct Connection Ethernet Port Based PM 175

15.8.4 Provisioning CLI Commands for Tunneled PWE3 176

15.8.5 Provisioning CLI Commands for Tunneled VLAN PWE3 over Routed VLAN Connection 177

15.8.6 Provisioning CLI Commands for ELP and Y.1731 PM 179

15.9 Ethernet OAM ETH-DM-PING Utility 180

16 Ethernet Link Protection 182

16.1 Operation 185

16.1.1 Ethernet Link Protection Group 185

16.1.2 ELP Group Configuration 186

16.1.3 Switchover Behavior 188

16.1.4 MAC Address 189

16.1.5 Operational Examples 189

16.2 Limitations and Restrictions for ELP 194

16.3 References 194

17 Ethernet Link Protection CLI Configuration Examples 195

17.1 Point-to-Point Connection 195

17.2 Ethernet Switched Network Connection 196

18 Microwave Radio 1+1 Protection 197

19 Configuring Microwave 1+1 Protection in an ELP Group 200

19.1 Create ELP Protection Group 201

19.2 Create ODU Management VLAN 201

19.3 Create IDU-ODU Monitoring VLAN 202

19.4 Create ODU-ODU Monitoring VLAN 203

19.5 Configure Egress Rate Limiter 204

19.6 Create Protected User VLANs to ELP Group 204

20 CPU Packet Storm Control 206

About This Manual

This chapter discusses the objectives and intended audience of this manual, 8600 Smart Routers Ethernet Configuration Guide and consists of the following sections:

This manual provides an overview of the 8600 smart routers Ethernet applications and instructions

on how to configure them using command line interface (CLI) and ASCII textual commands with arouter's console or remote terminal (Telnet)

8600 Smart Routers Technical Documentation

The document numbering scheme consists of the document ID, indicated by numbers, and thedocument revision, indicated by a letter The references in the Related Documentation table beloware generic and include only the document ID To make sure the references point to the latestavailable document versions, please refer to the relevant product document program on the Tellabsand Coriant Portal by navigating towww.portal.tellabs.com> Product Documentation & Software

> Data Networking > 8600 Smart Routers > Technical Documentation

Document Title Description

8600 Smart RoutersATM and TDM Configuration Guide(76.8600-50110)

Provides an overview of 8600 NEs PWE3 applications,including types, Single-Segment and Multi-Segment; PWE3Redundancy; ATM applications, including PWE3 tunnelling,Traffic Management, Fault Management OAM, protection andTDM applications as well as instructions on how to configurethem with CLI

8600 Smart RoutersBoot and Mini-ApplicationsEmbedded Software Release Notes(76.8600-50108)

Provides information related to the boot and mini-applicationssoftware of 8605 Smart Router, 8607 Smart Router, 8609Smart Router, 8611 Smart Router, 8620 Smart Router, 8630Smart Router and 8660 Smart Router as well as the installationinstructions

8600 Smart RoutersCLI Commands Manual(76.8600-50117)

Provides commands available to configure, monitor and maintain

8600 system with CLI

8600 Smart RoutersEmbedded Software Release Notes

8600 Smart Routers SR7.0 Embedded Software Release Notes(76.8670-50177) for the following products:

Provides an overview of 8600 system HW inventory, softwaremanagement, equipment protection 1+1 (CDC and SCM) as well

as instructions on how to configure them with CLI

8600 Smart RoutersEthernet Configuration Guide (76

8600-50133)

Provides an overview of 8600 system Ethernet applications,including interfaces; Ethernet forwarding (MAC Switching,Ethernet PWE3, IRB, VLAN, VPLS); Ethernet OAM; LAG;ELP as well as instructions on how to configure them with CLI

8600 Smart Routers Smart RoutersFault Management ConfigurationGuide (76.8600-50115)

Provides an overview of 8600 system fault management,including fault source, types and status as well as instructions onhow to configure it with CLI

8600 Smart RoutersFrame Relay Configuration Guide(76.8600-50120)

Provides an overview of 8600 system Frame Relay applications,including interfaces; Performance Monitoring; protection; TrafficManagement as well as instructions on how to configure themwith CLI

8600 Smart RoutersHardware Installation Guide(76.8600-40039)

Provides guidance on mechanical installation, cooling,grounding, powering, cabling, maintenance, commissioning andESW downloading

Document Title Description

8600 Smart RoutersInterface Configuration Guides The Interface Configuration Guides provides an overview of the8600 NEs interface functions, including NE supported interface

types and equipping; interface features; configuration options andoperating modes; fault management; performance monitoring;interface configuration layers and port protocols as well asinstructions on how to configure them with CLI The followinginterface configuration guides are available:

• 8600 Smart Routers Network Interfaces ConfigurationGuide (76.8600-50161) (for 8602 Smart Router, 8615 SmartRouter and 8665 Smart Router)

• 8609 Smart Router and 8611 Smart Router FP7.0 InterfaceConfiguration Guide (76.8670-50179)

• 8600 Smart Routers FP7.0 Interface Configuration Guide(76.8670-50180) (for 8630 Smart Router and 8660 SmartRouter)

8600 Smart Routers

IP Forwarding and TrafficManagement Configuration Guide(76.8600-50122)

Provides an overview of 8600 NEs IP, forwarding and trafficmanagement functionality, including: IP addressing; IP hosting(ARP, DHCP); IP routing (static); ACL; Differentiated Services(Policing, Queue Management, Shaping) as well as instructions

on how to configure them with CLI

8600 Smart RoutersManagement CommunicationsConfiguration Guide

(76.8600-50125)

Provides an overview of 8600 system managementcommunications functions, including communication protocols:BMP; FTP; RADIUS; SNMP; SSH; TELNET as well asinstructions for configuring them with CLI

8600 Smart RoutersMobile Optimization ConfigurationGuide (76.8600-50100)

Provides an overview of 8600 system Mobile Optimizationapplications as well as instructions on how to configure themwith CLI

8600 Smart RoutersMPLS Applications ConfigurationGuide (76.8600-50123)

Provides an overview of 8600 NEs MPLS applications (includingFRR (one-to-one and facility backup); LDP; protection andTraffic Engineering), MPLS-TP applications (including OAM,linear protection), S-MPLS applications as well as instructions

on how to configure them with CLI

8600 Smart RoutersPerformance Counters ReferenceGuide (76.8600-50143)

Provides an overview of 8600 system supported performancecounters

Document Title Description

8600 Smart RoutersReference Manuals The reference manuals describe the 8600 network elementfeatures including:

• NE enclosure, baseboard, power supply modules, andinterfaces in 8602 Smart Router FP7.0 Reference Manual(76.8670-40130)

• NE enclosure, baseboard, power supply modules, interfacesand physical LM types in 8609 Smart Router FP7.0 Refer-ence Manual

• NE enclosure, baseboard, power supply modules, SCMs, HMand LM types in 8611 Smart Router FP7.0 Reference Manual

• NE enclosure, baseboard, power supply modules, and terfaces in 8615 Smart Router FP7.0 Reference Manual(76.8670-40132)

in-• NE subrack, fan modules, CDCs, line cards and IFMs in 8630Smart Router FP7.0 Reference Manual

• NE subrack, fan modules, CDCs, line cards and IFMs in 8660Smart Router FP7.0 Reference Manual

• NE subrack, fan modules, line unit and switch unit in 8665Smart Router FP7.0 Reference Manual (76.8670-40128)

8600 Smart RoutersRouting Protocols ConfigurationGuide (76.8600-50121)

Provides an overview of 8600 NEs routing protocols, includingBFD; BGP; BGP MP; ECMP; IS-IS; OSPF and VRRP as well asinstructions on how to configure them with CLI

8600 Smart RoutersScalability Reference Manual(76.8600-50160)

Provide a summary of tested scalability limits of the 8600 SmartRouters

8600 Smart RoutersSNMP MIB Support(76.8600-50116)

Describes SNMP MIB support by the 8600 NEs and providesinformation on the supported objects and traps For furtherinformation on SNMP MIBs, see the related RFCs

8600 Smart RoutersStatistic Counters Reference Guide(76.8600-50142)

Provides an overview of 8600 system supported statistic counters

8600 Smart RoutersSynchronization ConfigurationGuide (76.8600-50114)

Provides an overview of 8600 NEs synchronization functionality,including physical layer Frequency Synchronization (SEC, EEC);PTP Frequency Synchronization; Phase-Time Synchronization(L2 and L3 applications) as well as instructions on how toconfigure them with CLI

8600 Smart RoutersTest and Measurement ConfigurationGuide (76.8600-50124)

Provides an overview of 8600 NEs measurement and connectivityverification tools, including Ethernet loopback; IP ping andtraceroute; MAC swap loopback; MPLS ping and traceroute;PLT; PWE3 loopback; VCCV (BFD, LSP ping) as well asinstructions on how to configure them with CLI

8600 Smart RoutersVPNs Configuration Guide(76.8600-50128)

Provides an overview of 8600 system virtual private network(VPN) layer 3 applications as well as instructions on how toconfigure them with CLI

8000 Intelligent Network ManagerOnline Help Provides instructions on how different operations are performedwith the 8000 Intelligent Network Manager Describes also

different parameters and controls of the 8000 Intelligent NetworkManager dialogs and windows

Note that the Online Help is not available on the Portal but it isincorporated in the 8000 Intelligent Network Manager

Interface Numbering Conventions

To be able to follow more easily the feature descriptions and configuration examples given in this

document, see also the 8600 system interface numbering and related figures described in 8600 Smart Routers CLI Commands Manual.

Document Conventions

This is a note symbol It emphasizes or supplements information in the document.

This is a caution symbol It indicates that damage to equipment is possible if the instructions are not followed.

This is a warning symbol It indicates that bodily injury is possible if the instructions are not followed.

Documentation Feedback

Please contact us to suggest improvements or to report errors in our documentation:

Email: fi-documentation@tellabs.com

8600 Smart Routers Discontinued Products

8600 Smart Routers Manufacture Discontinued (MD) notifications are available on the Tellabsand Coriant Portal,www.portal.tellabs.com > Product Documentation & Software > Data Networking > [8600 Smart Router product name] > Product Notifications.

1 Ethernet Overview

Ethernet is one of the most common technologies in various types of networks Its longhistory started from the corporate LAN side and it has spread and it has been used in differenttelecommunication applications for years Today Ethernet has a prominent role in mobileback-hauling and fixed-mobile convergence (FMC) Its benefits are simplicity, flexibility,cost-effectiveness both in operational and capital expenditures, and a broad range of productsavailable

Ethernet can be applied for different network segments, such as,

• Single-hop link between two routers or other network elements

• Multi-hop network segment or operational domain like access

• WAN, and core network

• End-to-end Ethernet service

Ethernet frames can be carried over almost all physical or data link protocols, such as,

• directly over optical fiber and copper cables at various nominal rates of 10 Mbps, 100 Mbps,

1 Gbps, 10 Gbps, or higher,

• over multiple physical links using Ethernet Link Aggregation,

• over SDH/SONET encapsulated into ATM, PPP, or Frame Relay,

• over single or bundled xDSL lines,

• over MPLS using pseudowires

To keep the network operating efficiency, Ethernet service OAM [IEEE802.1ag]/[ITU-T Y.1731]provides tools for testing, monitoring, measuring, and troubleshooting the network

The 8600 Smart Routers support Ethernet in various ways and on different layers This documentdescribes these features including Ethernet pseudowires, Integrated Routing and Bridging, OAMfunctions, Ethernet layer protections, Ethernet Link Aggregation and Virtual Private LAN Service(VPLS)

Ethernet features supported by 8600 are based on IEEE, IETF, ITU-T, and MEF standards andtechnical specifications

1.1 Ethernet Interfaces

1.1.1 Ethernet Physical Interfaces

The notations used in the rows of the supported Ethernet physical interfaces tables are explained

in the following table

X The interface module (IFM), Ethernet line card (ELC), line module

(LM), high speed module (HM) or fixed Ethernet interface is supported

by the network element

IFC1, IFC2 The interface module (IFM) when used in the mentioned interface

module concentrator (IFC) is supported by the network element

– Not supported by the network element.

8620 Smart Router, 8630 Smart Router and 8660 Smart Router support the following Ethernetinterface modules and Ethernet line card:

Interface Module (IFM) 8620 Smart

Router

8630 Smart Router

8660 Smart Router

COMBO

Ethernet Line Card (ELC) 8620 Smart

Router

8630 Smart Router

8660 Smart Router

R/12x1000BASE-X ELC1

8605 Smart Router 8607 Smart Router 8609 Smart Router 8611 Smart Router

Physical Port, Line Module (LM) or High Speed Module (HM)

8605 Smart Router

8607 Smart Router

8609 Smart Router

8611 Smart Router

M34x10/100/1000BASE-TX

8665 Smart Router LU1 line unit supports the following Ethernet interfaces ‘M’ followed by

a number refers to the module slot:

Virtual Modules Physical Port

Virtual Modules Physical Port

1.1.2 Ethernet Logical Interfaces

8600 system supports the Ethernet protocol in a variety of encapsulations that can exist on differenthardware, from basic Ethernet on Ethernet ports to Ethernet PWE3 in multiservice ports Thesevarious Ethernet interfaces are collectively called logical Ethernet interfaces The following logicalEthernet interfaces are supported:

• Logical Ethernet interfaces in Ethernet physical interfaces:

• Stacked VLAN for pseudowire use

• QinQ/Ethernet supported in:

• Attachment circuits for Ethernet PWE3 in multiservice interface modules (1xchSTM-1/chOC-3

MS, 4xchSTM-1/chOC-3 MS and 24xchE1/chT1 MS IFMs):

• Ethernet/LAPF_SNAP/FR/P12s

• VLAN/Ethernet/LAPF_SNAP/FR/P12s

• Ethernet/MLPPP and Ethernet/PPP (24xchE1/chT1 MS IFM only)

• Attachment circuits for Ethernet PWE3 in multiservice interfaces in 8605 Smart Router and linemodules in 8609 Smart Router and 8611 Smart Router:

• Ethernet/MLPPP and Ethernet/PPP on fixed chE1/chT1 interfaces and 8xchE1/chT1 LMs

• Virtual IRB interfaces and Ethernet PWE3 SLIs in IRB and VPLS applications

Supported Number of VLANs

In principle any Ethernet interface, including Ethernet Link Aggregation (LAG) groups, can usethe full VLAN ID space of 4094 VLANs (12-bit VLAN IDs; the values 0 and 4095 are reserved).However, there are also the following constraints on the total number of interfaces:

• At most 4096 of logical IP interfaces (Ethernet interfaces and VLANs) are supported per linecard;

• At most 16384 logical IP interfaces are supported per network element; including virtual IRBinterfaces;

• LAG groups use resources more heavily than individual Ethernet interfaces: a group of N bers uses (N+1) times the resources of an individual Ethernet interface; thus the number of sup-ported VLANs is further limited if some of the VLANs are on LAG groups;

mem-• Line modules (LM) consume the same resources as VLANs Each LM will consume one resourcethus correspondingly decreasing the maximum number of VLANs that can be created

Because of these constraints, the maximum of 4094 VLANs cannot be realized by a large number

of Ethernet interfaces simultaneously, or by LAG groups having a large number of members Forexample,

• there can be at most 4 (basic) Ethernet interfaces that have close to 4094 VLANs, and to realizethis maximum they have to be located on different line cards;

• a LAG group of more than 3 members cannot have close to 4094 VLANs, and if a 3-membergroup has close to 4094 VLANs, there cannot be many other interfaces in the network element.These extreme cases are, however, rather theoretical More realistic configurations do not typicallysuffer from resource limitations For example, the following configuration uses only about 4% ofthe internal resources of the network element:

• 6 IFCs with two 8-port Ethernet IFMs on each, thus having 96 physical Ethernet interfaces;

• 1 IFC with one 8-port Ethernet IFM where 4 ports are members of a LAG group;

• 96 VLANs on the LAG group, which is used as a trunk into which traffic from the physicalEthernet interfaces on the 6 other IFCs is switched

Supported Number of QinQs

QinQ interfaces are also counted as VLAN interface, so the maximum number of VLANs mentionedabove indicates the sum of VLAN and QinQ interfaces

1.1.3 Pause Frames Support in 8600 Network Elements

Ethernet provides a flow control mechanism by using pause frames Handling of pause frames isnot uniformly supported by 8600 NEs The following table identifies which products support thehandling of pause The command provides the following configuration options:

Pause Frames Operation

Receive Transmit Transmit

& Receive

8602 SmartRouter Virtual M0–M5 GE

8605 SmartRouter Virtual M0

8607 SmartRouter

8x10/100BASE-TXLM

8609 SmartRouter

8x10/100BASE-TXLM

4x100/1000BASE-XHM

4x10/100/1000BASE-TX HM

8611 SmartRouter

8x10/100BASE-TXLM

8615 SmartRouter Virtual M0–M1 XE & GE

8x10/100BASE-TXIFM

8620 SmartRouter

8630 SmartRouter

8660 SmartRouter

2+6x10/100/1000BASE-COMBO IFM GE

8x10/100/1000BASE-TX R2 IFM GE

8x100/1000BASE-XR2 IFM

1x10GBASE-R R2IFM

8630 SmartRouter

8660 SmartRouter

8665 SmartRouter

2 Ethernet Link Aggregation

Ethernet Link Aggregation (LAG) [IEEE 802.1AX] is a method for bundling Ethernet links sothat they appear to upper layers as one higher-capacity Ethernet link Traffic is distributed to themember links by a load balancing algorithm

802.1AX (formerly 802.3ad) Link Aggregation Control Protocol (LACP) is a control protocolrunning over Ethernet Link Aggregation members By means of LACP, network elements canshare information about which links their local configuration allows or prefers to be aggregatedtogether, and also monitor the status of the links

The benefits of Ethernet Link Aggregation include:

• Increase of Ethernet link bandwidth beyond the bandwidth of a single physical link

• Increase of bandwidth in incremental steps, e.g from 100 Mbps to 200 Mbps instead of an of-magnitude increase from 100 Mbps to 1000 Mbps

order-• Interface protection by means of load balancing; if one of the member links fails, load balancing

is automatically re-configured so that the remaining links carry the traffic

LACP is fundamental to LAG and hence used in conjunction The user may choose to disable LACP when working with legacy connections between NEs that do not support LACP In all other cases, LACP should be enabled.

8600 Smart Routers always operate in LACP active mode when LACP is enabled In active mode the network element is sending LACPDUs to the peer without waiting for the peer

to first send an LACPDU.

If a link is not configured to be a member of any LAG group, it will not participate in the LACPprotocol and will be visible as a normal interface to the user

Pure L2 devices are capable of performing automatic aggregation However, in routers, including

8600 Smart Routers, the use of LACP usually requires manual configuration of group memberships.The role of LACP when enabled on a LAG is as follows:

• Facilitating “plug and play” operation on third party devices LACP works by sending frames(LACPDUs) down on all the links, which are part of the LAG on which the protocol is enabled

It sends frames independently along all the member links, enabling network elements to ically detect and aggregate links Creating LAG group followed by enabling LACP and addingmembers are the only steps to be performed manually, while the enabling of traffic RX/TX oneach link will happen dynamically

automat-• Ensuring interoperability of network devices Devices supporting Ethernet Link Aggregationtypically support LACP

• Detecting situations where the bundling of links into aggregates is not configured consistently inLACP partner network elements.

• Monitoring link liveliness end-to-end between LACP partner network elements

• Ensuring hitless member addition and deletion Any change in the number of active ports pergroup due to member removal, will cause the traffic patterns to reflect the new state of the group.Member addition is a hitless operation, except on an IFC2 line card where there may be somedrops due to re-arrangement in the distribution function on the IFC2 line card

LACP is supported on all products featuring LAG The following table outlines LAG support inthe 8600 network elements

LAG Support in 8600 Smart Routers

8602 Smart Router All supported Ethernet interfaces Yes Yes

8609 Smart Router All supported Ethernet interfaces Yes Yes

100 Mbps and 1 Gbps Ethernet

8611 Smart Router

10 Gbps Ethernet interfaces No No

8615 Smart Router All supported Ethernet interfaces Yes Yes

2.1 Application of Link Aggregation

Link Aggregation can be used in Ethernet interfaces facing the Metro Ethernet Fig 1illustrates theapplication of LAG

Link Aggregation provides higher capacity to the logical interface, due to increased bandwidth bycombining links, and interface protection for any kind of pseudowire and IP routed traffic It alsoprovides load sharing among the member interfaces Failure in one member will result in the trafficbeing re-distributed appropriately across the remaining members

LACP-enabled LAG provides dynamic configuration detection between participating systems,enabling interoperability between network elements of different vendors

Fig 1 LACP Application

2.2 References

[IEEE 802.1AX] IEEE Std 802.1AX-2008 – IEEE Standard for Local and metropolitan area

networks: Link Aggregation

2.3 Link Aggregation Configuration

The main configuration actions for a Link Aggregation group are group creation, LACP enabling,member addition, member removal, and group deletion

All members of the group must be full duplex and running the same speed, which can be 100 Mbps,

1000 Mbps or 10 Gbps This speed is either configured by the user before joining any members,

or determined by the speed of the first joined member if not configured by the user before Afterconfiguring the required member speed or joining the first member, other members can be addedonly if they are running the required speed

The first joined member has a special role and it cannot be removed from the group until the wholegroup is deleted

An interface cannot be added to a group if there are VLANs defined on that interface It is notpossible to have VLANs as members of an aggregate, in other words, only physical Ethernetinterfaces can be members of an aggregate

The member links of an aggregate cannot be part of an Ethernet Link Protection (ELP) group Theaggregate itself cannot be part of an ELP group either

2.3.2 LACP Configuration

For LACP to determine whether a set of links can aggregate, several identifiers are required asoutlined below These identifiers and other related configurations allow exchanges to occur betweensystems so that a maximum level of aggregation capability can be achieved

Before configuring any member or LAG interface level parameters, it is required to enable LACP by using the command “eth lag lacp”.

LACP System Priority

The LACP System Priority is a network element global level parameter that is used as one part ofsystem identification This parameter need not be configured explicitly to successfully negotiateLACP It is an optional configuration which defaults to a value of 32768

LACP System ID

The LACP System ID, encoded as a MAC address format, along with the System Priority helpsidentifying a network element participating in LACP uniquely within an administrative domain.The combination of the LACP System ID and System Priority needs to be unique for each networkelement and helps to identify a system which wishes to participate in LACP over LAG It alsohelps in identifying unintended loopbacks due to any misconfiguration The LACP System ID

is configured as a network element level global parameter This identifier does not need to beconfigured explicitly on the participating network element It defaults to the MFE MAC on 8602Smart Router, 8609 Smart Router and 8611 Smart Router and the EEPROM MAC on 8615 SmartRouter, 8630 Smart Router, 8660 Smart Router and 8665 Smart Router

LACP TX QoS

The LACP TX QoS, represented by a QoS class, is configured as a network element level globalparameter on the system It is used to indicate the QoS queue within the NE that the LACPDUscontrol packets will take to reach their partner It ensures priority of the packet being sent out.LACP TX QoS defaults to class “cs6” It is not required to be set explicitly on the NE

LACP Port Priority

The LACP Port Priority, a LAG member level parameter, is used to determine which ports areaggregated into LAG as a standby-port configuration, if more than the maximum number of portssupported in a LAG is configured Each member port participating in the aggregation is identified

by a combination of port priority and port number The LACP port priority defaults to 32678

LACP Timeout

LACP Timeout is configured at LAG member level The options are:

• Long timeout: The network element will advertise to the partner that it should transmit anLACPDU on the link every 30 seconds If no LACPDUs are received from the partner duringthree consecutive periods (i.e 90 seconds), the link is removed from the aggregate.

• Short timeout: The network element will advertise to the partner that it should transmit anLACPDU on the link every 1 second If no LACPDUs are received from the partner duringthree consecutive periods (i.e 3 seconds), the link is removed from the aggregate

LACP Key

This system generated key is the LAG group identifier within the system It is transmitted inLACPDUs on the group’s member links The links that transmit the same key can be accepted

to the same aggregate by the peer

Fig 2 LACP Configurations

Changing either the system-id or system-priority on an already established LACP session will lead to LACP states being re-negotiated This may cause cuts in data traffic.

2.3.3 Attributes of Group

The following properties of a Link Aggregation group are configurable by the user:

• Maximum Transmission Unit If a group-specific MTU has not been provided by the user, theaggregate will use the MTU of the first member link that was added to the group

• MAC address If a group-specific MAC address has not been provided by the user, the aggregatewill use the MAC address of the first member link that was added to the group

• Minimum number of links The eth lag min-links sets the minimum number of member links thathave to be in up state in order for the group to be considered to be in up state If the number ofactive links is less than the configured number in min-links, a fault is raised for the group

• In this failure condition, the remaining active link members stay active at the physical layerand are able to transmit traffic

• In this failure condition, dynamically routed IP traffic will be rerouted to use another face; VRRP switchover happens as a result of this

inter-• If the LAG group is used as the attachment circuit of a pseudowire, the minimum number oflinks required for up state should not be configured to be larger than 1, because with a valuelarger than 1 it is possible that the pseudowire still transmits traffic while a fault is on

• In 8600 smart routers the protocols to be used in hash calculation for load balancing The natives are:

alter-1 multiple protocols, meaning the following:

• For IP packets over Ethernet: use the IP SA and DA, and optionally the IP Protocolfield and layer 4 source and destination port fields

• For MPLS packets: use MPLS label(s) and IP SA/DA (if available)

2 Ethernet only: use Ethernet SA, DA and VLAN ID if it is contained in the frame, and

optionally the Ethernet Type field

3 outer VLAN id only

4 inner and outer VLAN id

• Optional choice of protocol header fields in hash calculation:

• for IP headers, whether to use the IP Protocol field

• for IP headers, if the IP Protocol field has been chosen, whether to use also the layer 4 sourceand destination port fields (if the IP Protocol field has not been chosen, layer 4 fields willnot be used)

• in the “Ethernet only” case, whether to use the Ethernet Type field

• Automatic rebalancing of the load balancer in case of persistent uneven load This functionalitycan be enabled or disabled

• Loopbacks Notice that at the member interface level loopbacks cannot be configured duringmembership For more information on

loopbacks, refer to 8600 Smart Routers Test and Measurement Configuration Guide.

• Line loopback

• Equipment loopback

• Loopback timeoutThe user can execute the following actions on an aggregate:

• Forced rebalancing of the load balancer:

The network element will reorganize the mapping of traffic flows into member interfaces, inorder to achieve a more even load balancing As a side effect this may cause packet misorderingwithin flows during a short period

• In 8615 Smart Router, 8630 Smart Router, 8660 Smart Router and 8665 Smart Router, query ofload balancing result:

the user gives the values of some packet header fields as input, and gets as output the name ofthe interface into which the load balancer maps packets having those values

• Clearing of group statistics

2.3.4 Ethernet Link Aggregation Group

In 8630 Smart Router and 8660 Smart Router, the member links of a Link Aggregation group must

be on the same IFM or virtual module, seeFig 3 Within each IFM the links can be joined intogroups in any combination In particular, the maximum number of members in a group is 8

Fig 3 Intra-IFM Link Aggregation

Because Link Aggregation groups are restricted to one IFM, they do not provide line card protection.This is achieved by using IP or MPLS layer protection as illustrated inFig 4 The primary path andbackup paths are forwarded through different Link Aggregation groups

Fig 4 Intra-IFM Link Aggregation Group Protection

In 8602 Smart Router, 8609 Smart Router and 8611 Smart Router, any of the Ethernet interfaces(except the 10GBASE-R interfaces in 8611 Smart Router) can be member links of a LinkAggregation group, i.e the member links of a group need not be in the same module (LM, HM).

In 8615 Smart Router and 8665 Smart Router, the member links of a Link Aggregation groupmust be on the same line unit This restriction is non-trivial for 8615 Smart Router stacked, wheremember links from two line units cannot be joined to the same group Within a line unit any of theEthernet interfaces can be member links of a Link Aggregation group

Link Aggregation Group Scalability

8615 Smart Router 13 (12 groups with two 1 Gbps

members each and 1 group withtwo 10 Gbps members)

Link Aggregation Group Throughput

In 8630 Smart Router and 8660 Smart Router, a single IFM can forward traffic in a typical case

of up to 2.5 Gbps of the actual physical line rate, which corresponds to the maximum bandwidth

of a Link Aggregation group A single line card (IFC) can forward traffic in a typical case of up

to 3.5 Gbps of the actual physical line rate An ELC1 line card can forward traffic in a typical

case up to 25 Gbps For more information on adjustable forwarding capacity, refer to 8600 Smart Routers Interface Configuration Guide.

In 8609 Smart Router and 8611 Smart Router, LAG throughput ranges from about 2.5 Gbps for64-byte packets to about 7 Gbps for 1500-byte packets, naturally assuming that the number ofmembers in the group is sufficient to carry the given amount of traffic

In 8615 Smart Router the total forwarding capacity ranges from about 14 Gbps for 64-byte packets

to about 44 Gbps for Simple IMIX traffic This means that the small-packet throughput of a groupconsisting of 10 Gbps member links (of which there can be a maximum of two) is limited by thetotal forwarding capacity of the network element, whereas in all other setups it is possible for thethroughput to reach the maximum allowed by the line speeds of the member links

In 8665 Smart Router the total forwarding capacity of a line unit ranges from about 60 Gbps for64-byte packets to about 150 Gbps for Simple IMIX traffic This means that the small-packetthroughput of a group consisting of seven or eight 10 Gbps member links is limited by the totalforwarding capacity of the network element, whereas in all other setups it is possible for thethroughput to reach the maximum allowed by the line speeds of the member links

To reach the throughput allowed by the member line speeds, provided it is allowed by the totalforwarding capacity of the network element, the traffic flowing through the group should consist of alarge number of streams that are relatively small in size compared to the line speed This is because

in order to avoid packet reordering, a stream cannot be split between member links

Link Aggregation Group as Ethernet Interface

After a Link Aggregation group is created, it can otherwise be configured like any Ethernet interface.The group can have e.g., an IP address and MPLS enabled VLANs can be defined on a LinkAggregation group in the same way as on any Ethernet interface

For detailed information about LAG group attributes, refer to chapter2.3 Link AggregationConfiguration

For more information about Link Aggregation group statistics and faults for, refer to chapters2.5 Statisticsand2.4 Faults, respectively

Load Balancing

Load balancing works by first applying a hash calculation algorithm to a set of protocol header fields

of each packet In 8600 Smart Routers the choice of fields to be used in the calculation can be

configured by the user The calculation produces a number called the hash result The packet is

then mapped to one of the member interfaces of the group by using a table that maps all possiblevalues of the hash result onto members All packets that contain the same values of the headerfields (e.g MPLS label), i.e belong to the same flow, are thus mapped to the same interface,which guarantees that the ordering of packets within this flow is preserved The choices of headers

to be used in a hash calculation are:

Load Balancing in 8600 Smart Routers Protocol

Configuration

MPLS overEthernet

Pseudowires over LAG interface MPLS labels and (if available) IP

DA/SA + optionally IP Protocol and L4destination/source port

IP over Ethernet IP routing over LAG interface IP DA/SA + optionally IP protocol and L4

destination/source port (if available)Anything over

Ethernet

Ethernet pseudowire endpoint

Interface between 8600 andcustomer Ethernet equipment

MAC DA/SA + optionally Eth Type +VLAN ID(s) (if available); optionally one

or two VLAN ID(s) only

If IP packet fragmentation is allowed, L4 ports should not be used in hashing because it couldlead to packet re-ordering

2.3.5 Bandwidth Allocation in IFC2 Line Card

In IFC2 line card in 8630 Smart Router and 8660 Smart Router, the allocation of the internalbandwidth resources (cell bandwidth) of a network element works in a particular way in LinkAggregation groups In the user interface, cell bandwidth is not configured directly to the groupbut the bandwidth configurations are made for the member interfaces individually The actualinternal resource allocation is done by summing the configured bandwidths of the members andallocating the sum to the group The bandwidth per member is not limited by the individualbandwidth configurations, but each member has the full line speed in use within the bandwidthlimits of the group The configured bandwidth of a member remains allocated to the group as long

as the membership continues, even if the member may be in down state

The interface bandwidth allocation is done in 20 Mbps steps With LAG, one 20 Mbps sliceper each member in addition to the master is reserved for internal signaling So for example,the effective bandwidth of a two member LAG that has 200 Mbps allocated for both members

is actually 380 Mbps (2 x 200 - 20) For a corresponding three member LAG, it is 560 Mbps(3 x 200 - 2 x 20), etc

The “Active bandwidth” shown in the user interface is the smaller one of these two:

• the total allocated cell bandwidth

• the sum of member interface speeds of the members whose link state is upThus “Active bandwidth” is the maximum bandwidth that could currently be received or transmittedfrom/to the network through the Link Aggregation group

For more information on bandwidth allocations, refer to 8600 Smart Routers Interface Configuration Guide.

2.3.6 Synchronization

The member links of an aggregate can be used as synchronization sources independently of theirmembership SSM signaling in Synchronous Ethernet (SyncE) and IEEE1588 Precision TimeProtocol is supported upon introduction of member-specific signaling for LAG This is madepossible by associating these protocol packets with a particular member interface rather than beingtransparent to the LAG interface The aggregate itself has no role in synchronization

Queuing in a LAG group is optimized in a different way depending on the network element In IFC2line card , the classification of packets into QoS classes is strictly followed in queuing, even at theexpense of leaving some of the member link bandwidth unused in some situations, depending onrandomness in packet hashing In ELC1 line card, 8602 Smart Router, 8609 Smart Router and 8611Smart Router, 8615 Smart Router and 8665 Smart Router, the bandwidth usage is maximized, even

at the expense of passing lower QoS class packets while dropping higher QoS class packets in somesituations, again depending on randomness in packet hashing

The likelihood of the above mentioned exceptional situations is increased if traffic in the groupcontains a few streams of very large bandwidth (comparable to the line speed of a member)

In all cases, the queue statistics counters of the master member show the counts for the whole group.The queue statistics counters of other members do not increment during their membership.

2.4 Faults

The following two faults are specific to LACP members:

• Ethernet LAG LACP ID Mismatch This alarm is raised, in a Link Aggregation group with LACPenabled, in the event of some member links receiving different partner system id and/or partnerkey values than the other member links, from the partner side This will keep out the links onwhich mismatch exists, out of the aggregator The LACP state remains down on links for whichmismatch exists, until the conflict is resolved by the user

This is a major communications alarm

• Ethernet LAG Link LACP Negotiation Incomplete – This alarm is raised, in a Link Aggregationgroup with LACP enabled, in the event of a member link and its peer not having reached a col-lecting and distributing state

This is a major communications alarm

The following two faults are specific to a LAG group:

• the “Member links down” fault is raised if one or more member links are down

• the “Link down” fault is raised if less than the configured minimum number of links are up Thisfault masks the “Member links down” fault

The interface-specific faults of the member interfaces are not affected by their membership

2.5 Statistics

The same statistics counters exist for Link Aggregation groups as for individual Ethernet interfaces(refer to2.8 Limitations and Restrictions for LAGfor more information) The group-level counterscontain the sum of statistics of the current member interfaces starting from the beginning of theirmembership in the group, or the latest clearing of the group-level counters The statistics of earlier,removed members are not included in the group statistics

The interface-level statistics of the member interfaces are not affected by their membership Inparticular the group statistics can be cleared without affecting the members’ interface-level statistics.Interface-level statistics can be cleared by interface-level commands

2.6 LACP States

LACP provides a means of exchanging information between the first party (also known as the Actor)and the second party (also known as the Partner) systems to determine the candidate links to beaggregated This is facilitated by exchanging LACPDUs that transmit information pertaining tothe state of each member to its partner system

LACPDUs are encapsulated in Slow Protocol IEEE 802.3 Ethernet Frames and use the well knownmulticast group MAC address of 01-80-C2-00-00-02 as the destination MAC

LACPDU exchange can happen in any of the following modes pertaining to each port in theaggregator group:

• Active – Active LACP port can participate in the protocol by initiating negotiations with remoteports Active LACP ports always transmit LACPDUs

• Passive – Passive LACP ports indicate their preference to not initiate negotiations but only mit LACPDUs whenever requested by the Active Partner port

trans-The Partner values sent in the Actor’s LACPDU are the Actor’s current view of the Partner system.LACPDUs are transmitted periodically by the Actor to the Partner This periodic transmission willoccur at a rate expressed by the Partner system

• A Long timeout value (90 seconds), advertised by the Partner, corresponds to a Slow sion rate.

• A Short timeout value (3 seconds), advertised by the Partner, corresponds to a Fast sion.

transmis-Because of this link monitoring is relatively slow as the fastest detection time is 3 seconds

Apart from the periodic transmissions, LACPDUs are also exchanged whenever there is a change inthe Actor’s state or when the Actor determines that the Partner does not have the Actor’s currentstate

Only once both Actor and Partner have come to agreement about each other’s states that the devicecan make a decision about the aggregation capabilities of a particular link

Refer section2.6.3 LACP Configuration and Protocol Statusto determine how the various LACPDUframes are displayed to the user via various LACP “show” CLI commands and how to interpretthe same

2.6.2 LACP Member Interfaces States

The various states that an Aggregator Multiplexer goes through during the Link Aggregation processare outlined in this section Initially, both Collection and Distribution are disabled for each port.Refer section2.6.3 LACP Configuration and Protocol Statusto determine how the various memberstates are displayed to the user via various LACP “show” CLI commands and how to interpretthe same

Collecting

Once Collection is enabled, each port in the aggregator becomes ready to receive incoming framessent from the Partner’s distributor This is a transient state that occurs during LACP enabled LAGestablishment only

Distributing

This is not a standalone state, but generally is attributed along with collecting state A port, when indistributing state, indicates that it is ready to send data over the link