Application Performance Optimization with 10 Gigabit Ethernet BRKAPP-2011 BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Agenda 10 Gigabit Ethernet Database Clusters Oracle RAC Implementation Financial Clusters M k tF Market Feed/Algorithmic d/Al ith i T Trading di Compute Cluster High Performance Computing Clusters HPC Applications Parallel Applications Data Delivery y NAS, Clustered NAS Block/File Parallel File Systems, Object Based Parallel File Systems Designs L2MP Technologies BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 10 Gigabit Ethernet BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 10 Gigabit Ethernet 10 GE has been historically a higher priced interconnect primarily used for inter-switch links within the data center center, within the campus backbone and for uplinks from high density closets Implementation at the server has been price prohibitive with the exception of a few very specific applications while the primary server interconnect has remained Gigabit Ethernet Many changes are driving the move towards 10GE as a commonplace p interconnect for server p platforms which have significant benefits to many applications even though the full bandwidth is not being utilized Per p port costs have dropped pp from near $12,000 two to three yyears ago to as low as $500 today 10GE adapter costs have seen reductions to 25% of what they were e e in tthe e recent ece t past BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 10 Gigabit Ethernet – Host Changes Core densities are increasing with two-way, four-way and eight-way quad core motherboards The elimination of key in-host bottlenecks – front side bus frequencies, frequencies memory bandwidth frequencies frequencies, kernel processing, context switching and message copies Gigabit Ethernet is and will continue to be an inhibitor that does not allow the the necessary throughput rates to feed more than four cores cores 10 Gigabit Ethernet LAN On Motherboard (LOM) based systems y are starting g to show on the market Needs of the virtualized se BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 10 GE Adapter Vendors BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Applications BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Database Cluster Oracle Implementation with 10 Gigabit Ethernet BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Oracle RAC in the Data Center Clustered Implementation (RAC) Latency sensitivity for inter-process communications Bandwidth sensitivityy for data delivery y Interconnect density and bandwidth Multiple p interconnects Heartbeat, IPC, Cache Fusion, Data load/unload Database Financial High Performance Computing BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Oracle RAC Optimization DB IPC communication acceleration DB to App tier potential acceleration Oracle 11 actually has the ability to leverage SDP in Asynchronous I/O mode using iWARP for IB and Ethernet with OFED 1.2 Oracle 10g uses UDP Oracle 11g RAC – RDS standard within ofed 1.4 for RDS over RDMA (iWARP) and RDS over TCP Database Financial High Performance Computing BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 10 Going Beyond Spanning Tree Today Ethernet forwarding is done according to Spanning Tree IIn trees, t going i from f the th roott toward the leaves, branches get smaller IIn 2009/2010 d datacenters most of the links will be 10GE Single size branches There is a concrete need to go beyond Spanning Tree BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 74 Meshed Network Reduced to a Tree BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 75 Example of a Possible Solution Keep all the link forwarding BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 76 Bisectional Bandwidth Bisectional Bandwidth is the most common measure of aggregated bandwidth and is loosely defined as follows: Divide the hosts into two group of equal size, size the two groups will be interconnect by some number of network links Bisectional bandwidth is the total possible bandwidth crossing these links Th The goall is i tto design d i a solution l ti th thatt maximizes i i bisectional bandwidth BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 77 Etherchannel Etherchannel is a solution to create a “fat link” between two switches Easy E to t deploy: d l upper layer protocols are unaware of it Limited to two switches BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 78 VSS/vPC/EHV Three different solutions to connect an access switch to two distribution switches Active-active uplinks VSS and vPC use Etherchannel EHV uses “pinning” BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 79 VSS (Virtual Switching System) Cluster two physical chassis Tightly coupled, they become “undistinguishable” Etherchannel from the access switches works unchanged Data Plane both switches are active Control Plane A i /S Active Stand-by db Single point of management BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 80 vPC (virtual Port Channel) Aka MCEC (Multi (Multi-Chassis Chassis Etherchannel) Etherchannel from the access switches works unchanged Loosely Coupled Data and Control Plane Both switches are active Two points of management Peer-Link is a standard link BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 81 L2MP Terminology Cisco L2MP Aka Cisco DCE An internal Cisco effort to provide L2MP Devices that support Cisco L2MP are called “DBridges” TRILL An IETF project http://www.ietf.org/html.charters/trill-charter.html D i Devices th thatt supportt Ci Cisco L2MP are called ll d “RB “RBridges” id ” Extremely similar – interoperability is possible The term “D/Rbridge” is used when there is commonality BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 82 Q and A BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 83 Interested in Data Center? Discover the Data Center of the Future Cisco booth: #617 See a simulated data center and discover the benefits including investing to save, energy efficiency and innovation Data Center Booth Come by and see what’s happening in the world of Data Center – d demos; social i l media di activities; ti iti bl bloggers; author th signings i i Demos include: Unified Computing Systems Cisco on Cisco Data Center Interactive Tour Unified Service Delivery y for Service Providers Advanced Services BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 84 Interested in Data Center? Data Center Super Session Data Center Virtualization Architectures, Road to Cloud Computing p g ((UCS)) Wednesday, July 1, 2:30 – 3:30 pm, Hall D Speakers: John McCool and Ed Bugnion Panel: 10 Gig LOM Wednesday 08:00 AM Moscone S303 Panel: Next Generation Data Center Wednesday 04:00 PM Moscone S303 Panel: Mobility in the DC Data Thursday 08:00 AM Moscone S303 BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 85 Please Visit the Cisco Booth in the World of Solutions See the technology in action Data Center and Virtualization DC1 – Cisco Unified Computing System DC2 – Data Center Switching: Cisco Nexus and Catalyst DC3 – Unified Fabric Solutions DC4 – Data Center Switching: Cisco Nexus and Catalyst DC5 – Data Center 3.0: Accelerate Your Business, Optimize Your Future DC6 – Storage Area Networking: MDS DC7 – Application Networking Systems: WAAS and ACE BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 86 Complete Your Online Session Evaluation Give us your feedback and you could win fabulous prizes prizes Winners announced daily Receive 20 Passport points for each session evaluation you complete Complete your session evaluation online now ((open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Don’t forget f to activate your Cisco Live Virtual account for access to all session material, communities, and on-demand and live activities throughout the h year A Activate i your account at the h Cisco booth in the World of Solutions or visit www.ciscolive.com 87 BRKAPP-2011_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 88 ... communications Bandwidth sensitivityy for data delivery y Interconnect density and bandwidth Multiple p interconnects Heartbeat, IPC, Cache Fusion, Data load/unload Database Financial High Performance. .. Cisco Public 17 Deterministic Performance #1 problem in financial trading environments Financials don’t care about MIN (latency) or AVG (latency) , but STDDEV (latency) at the application level A. .. more akin to robots with machine guns.’ Database IBM Report ‘Tackling latency: the algorithmic arms race’’ Financial High Performance Computing BRKAPP- 2011_ c1 © 2009 Cisco Systems, Inc All rights