Cisco networkers 2009 session BRKAPP 3006 troubleshooting cisco wide area application services DDU

Troubleshooting Cisco Wide Area Application Services BRKAPP-3006 BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Data Center Building Blocks Applications Application Networking Services Application Delivery and Application Optimization Virtualization Network, Server, Storage and Management Transport Infrastructure Eth, FC, DCE, WAN, MAN BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Compute Infrastructure OS, Hardware, Firmware Cisco Public Storage Infrastructure SAN, NAS, DAS Application Optimization Infrastructure Network Classification Application Scalability Application Networking • • • • • • • • • • • • Quality of service Network-based app recognition Queuing, policing, shaping Visibility, monitoring, control Server load-balancing Site selection SSL termination and offload Video delivery Message transformation Protocol transformation Message-based security Application visibility WAN Application Acceleration WAN Acceleration Application Optimization • • • • • • • • • • • • Latency mitigation Application data cache Meta data cache Local services BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Data redundancy elimination Window scaling LZ compression Adaptive congestion avoidance Cisco Public Delta encoding FlashForward optimization Application security Server offload Agenda ƒ Diagnostic Reports ƒ Physical Components ƒ Platform ƒ Transport Optimizations ƒ Application Acceleration BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Wide Area Application Engine (WAE) Wide Area Application Services (WAAS) Version 4.1 IOS Platform with Services and CLI CIFS AO MAPI AO HTTP AO SSL AO Video AO NFS AO TCP Proxy with Scheduler Optimizer (SO) DRE, LZ, TFO EPM WoW Virtual Virtual Blade Blade #2 #3 Configuration Management System (CMS) Virtual Blades Cisco Linux Kernel Policy Engine, Filter-Bypass, Egress Method, Directed Mode, Auto-Discovery Flash IOS Shell Linux BRKAPP-3006_c1 Application Storage © 2009 Cisco Systems, Inc All rights reserved Object Storage Cisco Public DRE Storage Virtual Blade Storage /vbspace Ethernet Network I/O Diagnostic Reports BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Self Diagnostic Tool A Good Place to Start… BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Self Diagnostic Tool BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Device Alarms WAE674# show alarms major Major Alarms: Alarm ID Module/Submodule Instance -1 core_dump sysmon core WAE674# WAE674# cd core_dir WAE674# dir size time of last change Thu Jun 19 19:12:18 2008 1074 Thu Jun 19 19:19:11 2008 1216 Thu Jun 19 19:19:11 2008 WAE674# name core.exec_show_stats… diagnostic_report.txt diagnostic_report.xml Core file causing alarm Local copy of last diagnostic report BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public WAAS System Report Help Us Help You ƒ The WAAS system report (sysreport) contains a compressed archive containing all relevant support and system health information ƒ The sysreport includes the following: CLI command output, platform configuration and logs, platform state information, print services configuration and logs, authentication configuration and logs, logs for internal services and acceleration, CMS configuration and logs, system logs, etc ƒ The sysreport can be generated from the WAE Manager GUI or CLI: WAE612# copy sysreport BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 10 SSL Accelerator WAE7326# sh policy-engine application dyn Dynamic Match Freelist Information: Allocated: 32768 In Use: Max In Use: < snip > Allocations: 1751 Dynamically created on Core and Edge WAEs Individual Dynamic Match Information: Number: Type: Any->Host (6) User Id: SSL (4) Src: ANY:ANY Dst: 171.70.150.5:443 Map Name: basic Flags: SSL Seconds: Remaining: - NA - DM Index: 32764 Hits: 25 Flows: - NA - Cookie: 0x00000001 Number: Type: Any->Host (6) User Id: EPM (3) Src: ANY:ANY Dst: 10.88.80.53:1026 Map Name: uuide3514235-4b06-11d1-ab04-00c04fc2dcd2 Flags: TIME_LMT REPLACE FLOW_CNT Seconds: 1200 Remaining: - NA - DM Index: 32765 Hits: Flows: Cookie: 0x00000000 Number: Type: Any->Host (6) User Id: SSL (4) Src: ANY:ANY Dst: 151.193.164.6:443 Map Name: basic Flags: SSL Seconds: Remaining: - NA - DM Index: 32766 Hits: Flows: - NA - Cookie: 0x00000000 WAE7326# BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 90 SSL Acceleration ƒ Check for connections br1-wae1# sho stat conn opt D:DRE,T:TCP Optimization, C:CIFS,E:EPM,H:HTTP,M:MAPI,N:NFS,S:SSL,V:VIDEO, ConnID 372 858 861 889 896 897 Local IP:Port 10.1.3.43:43436 10.1.3.100:1054 10.1.3.100:1057 12.159.148.121:4641 10.1.3.100:1090 10.1.3.100:1091 Remote IP:Port 10.1.1.41:443 10.1.1.24:139 10.1.1.29:445 10.1.3.100:65001 96.6.177.51:443 96.6.177.51:443 PeerID 0:11:25:aa:12:54 0:11:25:aa:12:54 0:11:25:aa:12:54 0:11:25:aa:12:54 0:11:25:aa:12:54 0:11:25:aa:12:54 Accelerator S,D C,D C,D D S,D,S S,D,S PeerID 0:14:5e:42:65:a0 0:14:5e:42:65:a0 0:14:5e:42:65:a0 0:14:5e:42:65:a0 0:14:5e:42:65:a0 0:14:5e:42:65:a0 Accelerator S,D,S D,C D,C D S,D,S S,D,S dc1-wae1# sho stat conn opt D:DRE,T:TCP Optimization, C:CIFS,E:EPM,H:HTTP,M:MAPI,N:NFS,S:SSL,V:VIDEO, ConnID Local IP:Port 338 10.1.3.43:43436 824 10.1.3.100:1054 827 10.1.3.100:1057 855 12.159.148.121:4641 862 10.1.3.100:1090 863 10.1.3.100:1091 br1-wae1# BRKAPP-3006_c1 Remote IP:Port 10.1.1.41:443 10.1.1.24:139 10.1.1.29:445 10.1.3.100:65001 96.6.177.51:443 96.6.177.51:443 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 91 SSL Acceleration ƒ Check accelerator statistics WAE7326# sh stat acc ssl < snip > Number of failed handshakes: Number of SSLv3 negotiated on LAN: Number of TLSv1 negotiated on LAN: Number of SSLv3 negotiated on WAN: Number of TLSv1 negotiated on WAN: Number of SSLv3 negotiated on peer: Number of TLSv1 negotiated on peer: Number of server initiated SSL renegotiations: Number of client initiated SSL renegotiations: Successful certificate verifications: Failed certificate verifications: Failed certificate verifications due to invalid certificates: Failed certificate verifications due to ocsp verification: Failed certificate verifications due to other errors: OCSP connections outstanding: OCSP requests processed since last clear/system start: Maximum number of concurrent OCSP requests ever reached: BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 393 39 1237 39 1237 1276 0 0 0 0 0 92 SSL Acceleration ƒ Check accelerator statistics Successful OCSP requests: Successful OCSP requests that returns OK status: Successful OCSP requests because of 'NONE' revocation option set: Successful OCSP requests that returns REVOKED status: Successful OCSP requests that returns UNKNOWN status: Failed OCSP requests: Failed OCSP requests due to other errors: Failed OCSP requests due to connection errors: Failed OCSP requests due to connection timeouts: Failed OCSP requests due to insufficient resources: OCSP total read bytes: OCSP total write bytes: WAE7326# BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 0 0 0 0 0 0 93 Q&A BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 94 Interested in Data Center? ƒ Discover the Data Center of the Future Cisco booth: #617 See a simulated data center and discover the benefits including investing to save, energy efficiency and innovation ƒ Data Center Booth Come by and see what’s happening in the world of Data Center – demos; social media activities; bloggers; author signings Demos include: Unified Computing Systems Cisco on Cisco Data Center Interactive Tour Unified Service Delivery for Service Providers Advanced Services BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 95 Interested in Data Center? ƒ Data Center Super Session Data Center Virtualization Architectures, Road to Cloud Computing (UCS) Wednesday, July 1, 2:30 – 3:30 pm, Hall D Speakers: John McCool and Ed Bugnion ƒ Panel: 10 Gig LOM Wednesday 08:00 AM Moscone S303 ƒ Panel: Next Generation Data Center Wednesday 04:00 PM Moscone S303 ƒ Panel: Mobility in the DC Data Thursday 08:00 AM Moscone S303 BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 96 Please Visit the Cisco Booth in the World of Solutions See the technology in action ƒ Data Center and Virtualization DC1 – Cisco Unified Computing System DC2 – Data Center Switching: Cisco Nexus and Catalyst DC3 – Unified Fabric Solutions DC4 – Data Center Switching: Cisco Nexus and Catalyst DC5 – Data Center 3.0: Accelerate Your Business, Optimize Your Future DC6 – Storage Area Networking: MDS DC7 – Application Networking Systems: WAAS and ACE BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 97 Recommended Reading… ƒ Application Acceleration and WAN Optimization Fundamentals (Cisco Press) BRKAPP-3006_c1 ƒ Deploying Cisco Wide Area Application Services (Cisco Press) Ted Grevers, Joel Christner Zach Seils, Joel Christner Overview of technologies employed in today’s acceleration systems Extensible guide for deploying and managing Cisco WAAS © 2009 Cisco Systems, Inc All rights reserved Cisco Public 98 Complete Your Online Session Evaluation ƒ Give us your feedback and you could win fabulous prizes Winners announced daily ƒ Receive 20 Passport points for each session evaluation you complete ƒ Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Don’t forget to activate your Cisco Live Virtual account for access to all session material, communities, and on-demand and live activities throughout the year Activate your account at the Cisco booth in the World of Solutions or visit www.ciscolive.com 99 BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 100 Packet Capture Debugs ƒ Packets can be captured on all WAAS interfaces using one of the following CLI tools: tethereal tcpdump ƒ The Problem? A packet capture taken on the WAE will contain packets of all TCP segments ƒ How can you differentiate between original and optimized connections? BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 101 Multiple TCP Segments header src dst header src dst IP Client Server IP Client Server eth Client Router eth Router header src dst Server Client 61 in 62 in Client exclude in header src dst IP Server Client IP eth Router Client eth header src dst header src dst IP Client Server IP Server Client eth Router WAE eth Router WAE Router WAE BRKAPP-3006_c1 header src dst header src dst IP Server Client IP Client Server eth WAE Router eth WAE Router © 2009 Cisco Systems, Inc All rights reserved Cisco Public 102 Displaying Optimized Segments header src dst header src dst IP Client Server IP Client Server eth Client Router eth Router header src dst IP Server Client 61 in 62 in Client exclude in eth Router header src dst IP Server Client eth Router WAE WAE Wireshark Display Filter: tcp && ip.src == && eth.dst == BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 103 Displaying Original Segments header src dst header src dst IP Client Server IP Client Server eth Client Router eth Router 61 in 62 in Client header src dst IP Server Client eth Router Client header src dst IP Client Server eth Router WAE header src dst IP Server Client eth WAE Router exclude in WAE Wireshark Display Filter: tcp && ip.src == && eth.src == BRKAPP-3006_c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 104 ... tftp> BRKAPP- 3006_ c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 10 Physical Components BRKAPP- 3006_ c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 11 Wide Area Application. .. (Historical) BRKAPP- 3006_ c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 20 Platform BRKAPP- 3006_ c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public 21 Wide Area Application. .. Good Place to Start… BRKAPP- 3006_ c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Self Diagnostic Tool BRKAPP- 3006_ c1 © 2009 Cisco Systems, Inc All rights reserved Cisco Public Device