1 YEAR UPGRADE BUYER PROTECTION PLAN Building SANs Brocade with Fa b r i c S w i t c h e s How to Design, Implement, and Maintain Storage Area Networks (SANs) with Brocade Fabric Switches • Step-by-step instructions for establishing your SAN requirements—such as high availability, performance, and cost savings—and translating those requirements into an effective SAN design • Detailed examples to guide you through the process of installing and troubleshooting your Brocade SAN • Practical discussions about SAN components and popular SAN configurations such as storage consolidation, disaster tolerance, and LAN-free backup Chris Beauchamp Author Josh Judd Author Benjamin Kuo Contributor 140_SANs_FM 8/14/01 3:40 PM Page i solutions@syngress.com With more than 1,500,000 copies of our MCSE, MCSD, CompTIA, and Cisco study guides in print, we continue to look for ways we can better serve the information needs of our readers One way we that is by listening Readers like yourself have been telling us they want an Internet-based service that would extend and enhance the value of our books Based on reader feedback and our own strategic plan, we have created a Web site that we hope will exceed your expectations Solutions@syngress.com is an interactive treasure trove of useful information focusing on our book topics and related technologies The site offers the following features: ■ One-year warranty against content obsolescence due to vendor product upgrades You can access online updates for any affected chapters ■ “Ask the Author”™ customer query forms that enable you to post questions to our authors and editors ■ Exclusive monthly mailings in which our experts provide answers to reader queries and clear explanations of complex material ■ Regularly updated links to sites specially selected by our editors for readers desiring additional reliable information on key topics Best of all, the book you’re now holding is your key to this amazing site Just go to www.syngress.com/solutions, and keep this book handy when you register to verify your purchase Thank you for giving us the opportunity to serve your needs And be sure to let us know if there’s anything else we can to help you get the maximum value from your investment We’re listening www.syngress.com/solutions 140_SANs_FM 8/14/01 3:40 PM Page ii 140_SANs_FM 8/14/01 3:41 PM Page iii YEAR UPGRADE BUYER PROTECTION PLAN Building SANs Brocade with Fa b r i c S w i t c h e s Chris Beauchamp Author Josh Judd Author Benjamin Kuo Contributor 140_SANs_FM 8/14/01 3:41 PM Page iv Syngress Publishing, Inc., the author(s), and any person or firm involved in the writing, editing, or production (collectively “Makers”) of this book (“the Work”) not guarantee or warrant the results to be obtained from the Work There is no guarantee of any kind, expressed or implied, regarding the Work or its contents.The Work is sold AS IS and WITHOUT WARRANTY.You may have other legal rights, which vary from state to state In no event will Makers be liable to you for damages, including any loss of profits, lost savings, or other incidental or consequential damages arising out from the Work or its contents Because some states not allow the exclusion or limitation of liability for consequential or incidental damages, the above limitation may not apply to you You should always use reasonable case, including backup and other appropriate precautions, when working with computers, networks, data, and files Syngress Media®, Syngress®, and “Career Advancement Through Skill Enhancement®,”are registered trademarks of Syngress Media, Inc “Ask the Author™,”“Ask the Author UPDATE™,”“Mission Critical™,” “Hack Proofing™,” and “The Only Way to Stop a Hacker is to Think Like One™” are trademarks of Syngress Publishing, Inc Brands and product names mentioned in this book are trademarks or service marks of their respective companies.“Brocade®,” “SilkWorm®,” and the Brocade logo are registerd trademarks of Brocade Communications Systems, Inc., in the United States and/or any other countries KEY 001 002 003 004 005 006 007 008 009 010 SERIAL NUMBER Q3G4T9U2F5 6YHQ94MLE4 VMERKJ6C4N XD7Y4B39UN 8SRT9U6N7H 3W7YRNTEP4 LHB65TR46T 4DB9R5LZMR N835M4KBAZ QT6Y4RTWFC PUBLISHED BY Syngress Publishing, Inc 800 Hingham Street Rockland, MA 02370 Building SANs with Brocade Fabric Switches Copyright © 2001 by Syngress Publishing, Inc All rights reserved Printed in the United States of America Except as permitted under the Copyright Act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written permission of the publisher, with the exception that the program listings may be entered, stored, and executed in a computer system, but they may not be reproduced for publication Printed in the United States of America ISBN: 1-928994-30-X Technical Editors: Chris Beauchamp, Josh Judd, Benjamin Kuo Acquisitions Editor: Catherine B Nolan Developmental Editor: Kate Glennon Copy Editor: Beth A Roberts Freelance Editorial Manager: Maribeth Corona-Evans Cover Designer: Michael Kavish Page Layout and Art by: Shannon Tozier Indexer: Jennifer Coker 140_SANs_FM 8/14/01 3:41 PM Page v Syngress Acknowledgments We would like to acknowledge the following people for their kindness and support in making this book possible Greg Reyes, Jack Cuthbert, Doug Wesolek, Maggie Conroy, Julie Chiu, Elaine Tite, Jeff Seltzer, and Chris Mingrone at Brocade, for championing the idea of a Brocade SANs book Also special thanks to Viet Dao, John Gareri, Mark Murphy, Jay Rafati, Ron Totah, Ezio Valdevit, John Bae, James Carpignano, Steve Daheb, Derek Granath, Jay Kidd, Omy Shani, James Bleess, Owen Higginson, Leo Kappeler, Chris M Nguyen, Mark Peluso, and Henry Robinson for their help in making this book a reality Ralph Troupe of Callisma for his invaluable insight and guidance Ralph’s expertise in SAN architecture and design solutions for next-generation storage networking implementations helped define our vision for this book Richard Kristof and Duncan Anderson of Global Knowledge, for their generous access to the IT industry’s best courses, instructors, and training facilities Karen Cross, Lance Tilford, Meaghan Cunningham, Kim Wylie, Harry Kirchner, Kevin Votel, Kent Anderson, and Frida Yara of Publishers Group West for sharing their incredible marketing experience and expertise Mary Ging, Caroline Hird, Simon Beale, Caroline Wheeler,Victoria Fuller, Jonathan Bunkell, and Klaus Beran of Harcourt International for making certain that our vision remains worldwide in scope Anneke Baeten and Annabel Dent of Harcourt Australia for all their help David Buckland,Wendi Wong, Daniel Loh, Marie Chieng, Lucy Chong, Leslie Lim, Audrey Gan, and Joseph Chan of Transquest Publishers for the enthusiasm with which they receive our books Kwon Sung June at Acorn Publishing for his support Ethan Atkin at Cranbury International for his help in expanding the Syngress program v 140_SANs_FM 8/14/01 3:41 PM Page vi This book was designed and written to provide information about storage area networking architectures Every effort has been made to make this book as complete and accurate as possible However, the information in this book is provided to you “AS IS,” without warranty of any kind, including, without limitation, any implied warranty of merchantability or fitness for a particular purpose The authors and Brocade Communications Systems, Inc., shall have no liability or responsibility to any person or entity with respect to any loss, cost, liability, or damages arising from the information contained in this book or the computer programs that accompany it, and specifically disclaim any implied vi 140_SANs_FM 8/14/01 3:41 PM Page vii Brocade Acknowledgments This book truly represents a complete Brocade team effort.We would like to acknowledge several people in particular.Without their help, dedication, and knowledge, this book would not have been possible.The thorough technical review by Viet Dao, John Gareri, Mark Murphy, Jay Rafati, Ron Totah, and Ezio Valdevit shaped our manuscripts into a book that Brocade can be proud of John Bae, James Carpignano, Steve Daheb, Derek Granath, Jay Kidd, and Omy Shani provided several timely contributions to the content.We also incorporated material written by others within Brocade: James Bleess, Owen Higginson, Leo Kappeler, Chris M Nguyen, Mark Peluso, and Henry Robinson.We would also like to thank Maggie Conroy and Doug Wesolek for their guidance and help with the publication process —Josh Judd and Chris Beauchamp vii 140_SANs_FM 8/14/01 3:41 PM Page viii 140_SANs_FM 8/14/01 3:41 PM Page ix Authors Chris Beauchamp is a Senior SAN Architect for Brocade Communications Systems, Inc Chris moved to Brocade in 1998 as a Systems Engineer supporting several strategic customers with the application and qualification of SilkWorm fabric switches Chris now focuses on SAN design and architecture, with an emphasis on scalability and troubleshooting His specialties include Sun servers, storage performance analysis and capacity planning, Fibre Channel trace analysis, scripting in various languages, and SAN administration Chris holds a Master of Science in Computer Engineering from Villanova University and a Bachelor of Science in Computer Science degree from West Chester University Chris’s background includes positions as a Systems Administrator at General Electric and a Systems Engineer at Sun Microsystems Chris currently resides outside of San Jose in the Santa Cruz Mountains with his wife Sarah and daughter Meagan Josh Judd is a Senior SAN Architect with Brocade Communications Systems, Inc In addition to writing technical literature, he provides senior-level strategic support for major OEMs and end-users of Brocade storage network products worldwide.When he first went to work for Brocade, he was the company’s Senior IT Specialist, responsible for escalations in every area of the company’s network, server, and desktop infrastructure Josh’s career as an IT consultant has given him a diverse skill set, which includes senior-level expertise in several UNIX variants,Windows 9x/NT/2k administration, RAID configuration and optimization, storage virtualization and clustering software (such as that produced by VERITAS Software), and network engineering with many vendors, including Cisco, Foundry, Lucent, and 3com Before joining Brocade four years ago, Josh worked at IBM Global Services, LSI Logic, and Taos Mountain Consulting He lives in San Jose, California ix 140_SANs_index 440 8/14/01 3:43 PM Page 440 Index Integrated Drive Electronics (IDE), Intelligent Fabric Services Architecture, 140–143 frame filtering and, 142 hardware-enforced zoning, 142 ISL Trunking, 140–142 performance analysis and, 143 Secure Fabric OS and, 143 Intelligent Peripheral Interface (IPI), 42 Inter-Switch Links (ISL; E_ports), 48–49, 87 cable layout and, 351–354 incomplete initialization of, 318–319 ISL over-subscription ratio, 231–232 load sharing through, 134 port configuration conflicts and, 322–323 trunking, 140–142 interconnecting devices, 33–34 Internal Rate of Return (IRR) calculations, 189 Internet, data storage and, Internet Protocol (IP), 2, 31, 39, 42, 96–97 Internet Service Providers (ISPs), 12–13 interoperability labs, 22–23 interviews, SAN design process and, 150, 153 backup information, identifying required, 167 business problem identification, 158 business requirements identification, 158–159 component testing needs, 166–167 current performance data, 168–172 design interview form, 175–176 host information, collecting, 160–162 identifying people to interview, 156–157, 193–194 implementation, determining acceptable downtime for, 174 initiator-to-target communications matrix, 167–168 maintenance downtime, determining acceptable, 174–175 performance, determining future needs, 172–174 SAN-enabled applications desired, 165 storage device information, 162–163 storage facility information, 164 technical requirement identification, 159–160 timeline creation from, 175–176 Iometer, Intel’s, 169, 170, 387, 389 iostat utility, Sun Solaris, 169, 386 IOzone, 387 IP addresses, setting switch, 358, 361 IP networks routing Fibre Channel across, 110 routing over Fibre Channel to Gigabit Ethernet, 110–111 IP protocol, 2, 31, 39, 42, 96–97 IP targets, 32, 33 IPFC protocol, 88, 96, 356–358 iSCSI, 57–58 ISLs (E_Ports) See Inter-Switch Links (ISLs) J Java-based Web pages, switch management by, 93–94 JNI EZ Fibre, 312 Just a Bunch Of Disks (JBOD), 32, 33, 64, 111 K K characters, 41 Key Distribution server, 50 L LAN-based backup configurations, 212–213, 214 LAN-free backup configurations, 212–216 latency, 231 LC connectors, 72 LEDs, switch, 287–289, 318, 329 140_SANs_index 8/14/01 3:43 PM Page 441 Index legacy devices, connecting to SANs, 106 Legato NetWorker, 13 license, fabric, 322, 366 licenseShow command, 286, 322, 366 LIP (Loop Initialization Primitive), 77–78, 337–338, 339 LIP process, 77, 78–79 LIP storm, 79 Lip_in count, 339, 342 Lip_out count, 339, 342 Lip_rx count, 342 load testing, I/O, 385–390 locality, performance optimization through, 266–268 logical unit number See LUN (Logical Unit Number) Login Server, 38, 50 logs, configuration, 391–393 logs, error, 292–295 loop environments, 4, 5, 33, 90 isolating marginal port faults, 339 LIP imbalances and, 339 LIP process and, 78–79 marginal GBICs and, 338 marginal loop connections, 337–338 marginal port behavior on disrupted, 338 migrating to switched fabrics, 79–80 Loop Initialization Primitive (LIP), 77, 337–338, 339 loop ports See FL_Ports loop zoning, 77 loopPortTest command, 290 LUN (Logical Unit Number) access permissions, 100 high-availability (HA) clusters and, 200 LUN-level zoning, 373, 374 Microsoft Cluster Server configurations, 200–202 selective presentation of, 108 LUN Manager, Hewlett-Packard’s, 212 LUN mapping (persistent binding), 99–100 LUN masking, 122, 205, 226 441 HBA-based, 99 hot-plug systems and, 105 storage partitioning by, 210–211 M make_zone script, 379–380 MAN technologies See Metropolitan Area Networking (MAN) manageability, high-availability cluster, 197 managed Fibre Channel hub, 34–35, 60, 76–78 Management Information Base (MIB), 92–93, 135 Management Server, 38, 85, 86, 133 mapping, LUN, 99–100 marginal GBICs, 338 marginal loop connections, 337–338 marginal point-to-point/fabric device links, 335–336 marginal ports, 335 disrupted loops and, 338 fault isolation and, 339, 340 marginal GBICs, 338 marginal loop connections, 337–338 marginal point-to-point/fabric device links, 335–336 portErrShow command, 295–297 masking, LUN, 122, 205, 226 HBA-based, 99 hot-plug systems and, 105 storage partitioning by, 210–211 media, cabling, 61, 65–68, 164 Media Interface Adapters (MIAs), 75 mesh topologies, 238 compared to other topologies, 247 full-mesh topology, 239–240 partial-mesh topology, 240–242 resiliency of, 256, 257, 258 metadata servers, 212 Metropolitan Area Networking (MAN), 15–16, 217, 219–221, 238 MIAs See Media Interface Adapters (MIAs) 140_SANs_index 442 8/14/01 3:43 PM Page 442 Index Micromuse’s Netcool, 350, 368 Microsoft Cluster Server (MSCS), 11, 200–202 Microsoft Exchange databases, 11 Microsoft SQL Server, 226 Microsoft Windows Hardware Quality Lab (WHQL), 202 migration process, 154 missing devices, troubleshooting, 279–283, 327–335 locating on Name Server with nsShow, 332–333 port configuration conflicts, 329–332 switchShow command and, 329–332 zoning mismatches and, 333–335 MQ errors, 318, 327 mqShow command, 286 MSCS See Microsoft Cluster Server (MSCS) MT-RJ connectors, 72–73 multi-LUN devices, 208 multicast groups, 52 multimode optical cables, 36, 61, 66–67 multipathing software, 199 N Name Server, 38, 41, 50–51, 85, 133–134 hard zoning and, 376 missing devices and, 280–283, 303–307, 332–335 nsAllShow command, 286, 320, 329, 382, 395 nsShow command, 281–282, 303–307, 320–321 names, switch, 360 Net Present Value (NPV) calculations, 189 NetBackup,VERITAS’, 13 Netcool, Micromuse’s, 350, 368 Network Attached Storage (NAS), 17, 18 network backups accelerating cycling of, 14 collecting information on in interviews, 167 LAN-free configurations for, 212–213 reducing network congestion from, 13 remote, 218–219 SAN-based server-free, 213–216 Network Data Management Protocol (NDMP), 214 network heartbeats, 198, 202 network protocols, 31 NetWorker, Legato’s, 13 node count, 230 node WWNs, 306–307 nodes, 230 adding to fabrics, 401–402 collecting information on in interviews, 160–164 missing from Name Server, 334–335 over-subscription of, 232 timeout of at bring up, 321–322 WWNs, 307 nonresilient dual-fabric SANs, 257 nsAllShow command, 286, 320, 329, 395 nsShow command, 286 fabric validation using, 382 troubleshooting fabrics with, 320–321 troubleshooting missing devices with, 281–282, 332–335 troubleshooting SANs with, 303–307 O Open Fibre Control (OFC), 67 OpenView, Hewlett-Packard’s, 12, 350, 368 optical cabling, 36, 61, 65 optical connectors, 62, 71–73 Oracle Parallel Server, 15, 226 out-of-band switch management, 356 over-subscription, 231 140_SANs_index 8/14/01 3:43 PM Page 443 Index P parallel bus SCSI, 2–4 parallel SCSI termination, 108 parityCheck command, 290 partial-mesh topology, 240–242, 247 PATROL, BMC’s, 350, 368 payback period, 189 payloads, 36, 39 PCI bus speed, pre-SAN, 170–171 PCI hot-plug systems, 104–106 perfmeter, Sun Solaris, 169 perfmon,Windows, 169 performance Brocade Intelligent Fabric Services Architecture and, 143 of cascade topology, 237 of complex core resilient core/edge topology, 245 determining future performance needs, 172–174 evaluating pre-SANs, 168–172 factors affecting, 270–271 of full-mesh topology, 240 of partial-mesh topology, 242 of ring topology, 237–238 Peripheral Component Interconnect (PCI) hot-plug systems, 104–106 persistent binding (LUN mapping), 99–100 persistent logging, 293 phantom devices, 302, 303 PLOGI/FLOGI timeout failures, 321–322 point-to-point connections, 87 point-to-point topology, 37, 39, 45–47 port addressing, 307–308 port-based zoning, 83 port counts, 230 port indicators, LED, 287–289 port internal loopback diagnostic, 290 Port Login (PLOGI) frame, 50 port register diagnostics, 289 port WWNs, 306–307 443 portCfgEport command, 322–323, 330 portcfgFport command, 330 portCfgLport command, 286, 330 portDisable command, 249, 302, 318, 330, 334–335 portEnable command, 302, 318, 330, 334–335 portErrShow command, 286, 295–297 portFlagsShow command, 286 portLog debugging tool, 314 portLogDump command, 286, 314 portLogShow command, 314 portLoopbackTest command, 289, 290 portRegShow command, 286 portRegTest command, 289 portRouteShow command, 286 ports, 48–49, 50 automatic bypass of by managed hubs, 78 buffer credits available per switch port, 86–87 collecting information on in interviews, 160–161 configuration conflicts, 322–323, 348 determining number needed, 182–187 display of port state information, 297–303, 318 E_Ports See E_Ports (ISLs) edge, 243 F_Ports, 48–49, 87 FL_Ports, 48–49 free, 243 manual configuration of, 330 marginal, 335–342 monitoring of by Fabric OS, 133 number on Brocade SilkWorm switches, 125 number on HBAs, 98 number on routers, 107, 108 self-configuring, 87–88 types of SCSI on routers, 108 universal port support by Fabric OS, 133 portSemShow command, 286 140_SANs_index 444 8/14/01 3:43 PM Page 444 Index portShow command, 341, 342 portStatsShow command, 278, 341, 342 power supplies, redundancy and, 86, 125 primitives, 36, 39–41 principal switches, 300 Prisa’s Visual SAN, 350, 368 private devices, 302, 303 private Host Bus Adapters (HBAs), 98 Private Loop Direct Attached (PLDA), 337 private loop drivers, 165–166 Private Loop Fabric Attach (PLFA), 337 problem descriptions, troubleshooting and, 284–285 protocol analyzers, 315 prototypes, 21, 153–154 psShow switch command, 286 Q qlDisable command, 330 qlPortDisable command, 330 qlShow command, 286, 330 QuickLoop, 79, 80, 138–139, 330, 337, 339, 366 QuickLoop CAM diagnostics, 289 R R_RDY primitive, 43 racking, 354–355 RAID storage arrays, 32, 33, 64, 111–112 controller speed requirements, 171 switch zoning and, 208 RAM (Random Access Memory), 171 ramTest command, 289 redundancy, 86, 125, 126, 256–257 redundant fabric architecture, 175, 199, 256–257, 258–260 Registered State Change Notification (RSCN), 51, 85–86 I/O pauses and, 343–344, 347 new device registration and, 284 reliability cascade topology, 236, 237 complex core resilient core/edge topology, 245 full-mesh topology, 240 partial-mesh topology, 241, 242 ring topology, 238 remote backups, 218–219 remote booting, 103–104 Remote Switch, 218 resiliency, SANs and, 126, 256 resilient core/edge fabric topology, 242–246 adding edge switches to, 248–250 defined, 229–230 upgrading core switches, 250–253 resilient dual-fabric SANs, 257 resource sharing, 19 Return Merchandise Authorization (RMA) system, 173–174 Return On Investment (ROI) proposition, 153, 159, 187–189, 190 RFT_ID frame, 50–51 ring topology, 237–238 compared to other topologies, 247 resiliency of, 256, 257, 258 ROI proposition See Return On Investment (ROI) proposition rolling upgrades, HA clusters and, 197, 198–199, 202 routers, Fibre Channel, 35, 60, 64–65, 106–109 extended copy support by, 108–109 management interfaces for, 109 number of SCSI buses on, 107 SCSI termination type and, 108 selective LUN presentation by, 108 types of SCSI ports available on, 108 run_sw_cmd script, 369, 372, 379, 384, 400 S SAN architecture, 153, 230–231, 253–256 any-to-any connectivity and, 268–269 availability models, 256–260 core/edge fabric designs, 253–256 140_SANs_index 8/14/01 3:43 PM Page 445 Index development of, 153 Intelligent Fabric Services Architecture, 140–143 leveraging tiers, 261–265 localizing traffic, 266–268 SAN-based server-free backups, 213–216 SAN-based third-party copy data movers, 214, 215–216 SAN configurations, 196–221, 226 disaster tolerance solutions, 216–221 High-Availability (HA) clusters, 196–202 LAN-free backup configurations, 212–213 SAN-based server-free backups, 217 storage consolidation configurations, 203–212 SAN grouping, 178–182 SAN installation, 21, 350 creating baseline profile at, 382–383 ISL cable layout for, 351–354 racking considerations, 354–355 setting switch parameters, 358–359, 361 zoning considerations, 372–382 SAN lifecycle, 190 architecture development phase, 153 data analysis phase, 153, 177–189 data collection phase, 153, 156–177 maintenance, 155 migration process, 154 overview of, 151–155 prototypes, 153–154 release to production, 154–155 testing, 154 SAN maintenance, 155 adding edge devices to fabrics, 401–402 adding switches to fabrics, 395–398 bringing up fabrics, 321–322, 394–395 configuration log for, 391–393 downtime requirements for, 174–175 fabric upgrades, 398–400 merging fabrics, 395, 408 replacing switches on fabrics, 395–398 switch configuration backups, 393–394 445 SAN management, 350–351 automation of administrative activities, 367–372 in-band IPFC, 356–358 out-of-band, 356 SAN port count, 230 SAN profiles, 308–311, 317–318, 382–384 SAN troubleshooting, 278–316 creating problem descriptions, 284–285 creating SAN profiles, 308–311 Fibre Channel analyzers, 314–316 gathering supportShow information, 285–287 I/O pauses, 342–343 marginal links, 335–342 missing devices, 279–283, 327–335 portLog debugging tool, 314 SAN profiles, 308–311, 317–318, 382–384 scenario describing, 279–283 switch diagnostics and, 289–308 switch LEDs and, 287–289 using information from hosts for, 312–313 “virtual cable” approach to, 278–279 See also fabric troubleshooting SAN verification fault injection techniques, 384–385 I/O load and, 385–390 SAN profiles and, 382–383 SANavigator, 350, 368 SANergy,Tivoli’s, 12, 212 SANmark Conformance Documents (SCDs), 35 SANmark suite, 8, 22, 35, 58 SANs (Storage Area Networks), 229 acceleration of backup cycles by, 14 applications best served by, 16–17 arbitrated loop topology, 4, 5, 33, 37, 39, 47–48 architecture overview, 30–36 availability levels, 257–260 cluster protocol access and, 14–15 configuration log for, 391–393 140_SANs_index 446 8/14/01 3:43 PM Page 446 Index configuration, typical, creation of, 2–7 data access speeds and, 14, 31 deployment of, 20–24, 154–155, 174 design process, 153, 156–177, 187–189, 194 disaster tolerance with, 15–16, 31, 216–221 distances supported, 4, 17 fabrics vs., 275 high availability features of, 10–11, 31 High Availability (HA) clusters and, 196–202 initiating devices, 33 installing See SAN installation lifecycle of, 151–155 maintenance requirements, 155 managing data storage with, 9–10 Network Attached Storage (NAS) vs., 18 network congestion and, 13 point-to-point topology, 37, 39, 45–47 port requirements, 182–187 prototypes, 21, 153–154 release to production, 154–155 remote booting and, 103–104 resources on, Return On Investment (ROI) proposition, 153, 159, 187–189 scalability of, 31 speed of, 17, 31 storage consolidation and data pooling through, 11–13, 203–212 storage servers vs., 31, 32 switched fabric topology, 37, 39, 48–49 target devices and, 32–33 testing, 21, 23, 154, 382–390 tiered, 261–265 when to deploy, 16–20 See also fabrics; Fibre Channel SANsymphony, DataCare’s, 12 SC connectors, 62, 71–72 SCDs See SANmark Conformance Documents (SCDs) scripting, automated of firmwareDownload, 400 SAN profiles and, 384 switch operations, 369–372 zoning operations, 379–380 SCSI buses, 107 SCSI disks, 33 SCSI Enclosure Services (SES), 94 SCSI-FCP standard, 41 SCSI protocol, 2, 28, 31, 39, 41, 42, 57 SCSI-to-Fibre Channel routers, 65 Secure Fabric OS, 143 segmented fabrics, 323–327, 348 selective LUN presentation, 108, 113 serial ports router/bridge management by, 109 switch management by, 91 Serial Storage Architecture (SSA), serialized ID GBICs, 74–75 server cycles, increasing, 14–15 server-free backups, SAN-based, 213–216 See also LAN-free backup configurations services, fabric See fabric services SES See SCSI Enclosure Services (SES) setGbicMode command, 290 setSplbMode command, 290 shared file system technique, 206 signal retiming, managed hubs and, 78 signaling interface, Fibre Channel, 40 SilkWorm series switches See Brocade SilkWorm switches simple electrical hubs, 34, 35, 60, 76 Simple Name Server, 133 See Name Server simple resilient core/edge topology, 244, 247 single fabric, nonresilient SAN architecture, 257 single fabric, resilient SAN architecture, 257 single-mode fiber-optic cable, 36, 61, 68–69 140_SANs_index 8/14/01 3:43 PM Page 447 Index Single Point Of Failure (SPOF), 233–234 single-port adapters, 98 snapshot backup volumes, 114 SNIA See Storage Networking Industry Association (SNIA) SNMP Management Information Base (MIB), 92, 121, 135 SNMP polling, 295–296 SNMP (Simple Network Management Protocol) router/bridge management by, 109 switch management by, 91–93 soft zoning, 83, 375–378, 408 software identifying existing in design phase, 165–166 selecting, 21–22, 165–166 storage partitioning, 205, 211–212, 226 Solaris, 169–170, 287, 312, 317 special character (K), 41 speed assessing need for, 17, 122 auto-negotiation of by switches, 88 HBA capabilities, 97 spinFab command, 290 spinSilk command, 290 sramRetentionTest command, 290 ST connectors, 71 star topology network, 228, 242–244 Start Of Frame (SOF) primitive, 39 State Change Registration (SCR) frame, 51 static discovery, HBAs and, 101 Storage Area Networks (SANs), Fibre Channel See SANs (storage area networks) storage arrays, 9, 11, 21, 287 See also RAID storage arrays storage consolidation, SANs and, 11–13, 203–212 partitioning software, 211–212 shared storage with Web farms, 206–207 switch zoning and, 208–210 447 storage devices, 64 collecting information on in interviews, 162–163 connecting legacy to SANs, 106 drivers for, 204 High-Availability (HA) clusters and, 200 high-end storage arrays, 113–114 individual disk drives, 33, 64, 111 Just a Bunch Of Disks (JBOD), 32, 33, 64, 111 RAID storage arrays, 64, 111–112, 208 storage consolidation SAN configurations, 203, 204 storage LUN masking, 105 storage network topologies, 37, 45–49 arbitrated loop topology, 37, 39, 47–48 point-to-point topology, 37, 39, 45–47 switched fabric topology, 37, 39, 48–49 Storage Networking Industry Association (SNIA), 8, 103 Storage Networking World Conference, storage partitioning with LUN masking, 210–211 software for, 205, 211–212, 226 with switch zoning, 51 See also switch zoning storage tier switches, 263 subordinate switches, 300 Sun Microsystems products HighGround, 350, 368 Jira management standard, 109 Solaris, 169–170, 287, 312, 317 supportShow command, 285–286, 289, 290, 318 switch beaconing, Fabric OS and, 135 switch commands, 289–290 case sensitivity of, 291 diagnostic See diagnostic switch commands show vs dump, 291 See also specific commands switch configuration file, 393–394 140_SANs_index 448 8/14/01 3:43 PM Page 448 Index switch domain IDs, 300–301, 319, 326–327, 358, 361, 397 switch LEDs device troubleshooting and, 329 fabric troubleshooting and, 318 SAN troubleshooting and, 287–289 switch ports, buffer credits per, 86–87 switch zoning, 83–84, 226, 372–382 adding new switches and, 381, 396–397 aliases and, 381 automatic scripting of activities, 379–380 backing up of configurations, 382 clearing zones, 381 conflicts with, 323–324 determining where to zone, 373–375 hard, 83–84, 374, 375–378 hardware enforced, 142 hot-plug systems and, 105–106 management of, 378–379 minimizing unwanted interactions, 381 mismatch of and missing devices, 333–334 Node and Port WWN and, 381 port-based, 83 soft, 83, 375–378, 408 storage partitioning and, 84, 205, 208–210 switchBeacon command, 302 switchDisable command, 251, 300, 343 switched fabric ports See F_Ports switched fabric topology, 5, 37, 39, 48–49 switchEnable command, 251, 318, 343 switches, Fibre Channel, 4, 33–34, 36, 60, 63, 80–83 adding new to fabrics, 395–398 adding new to zoned SAN, 381 auto-negotiation of speed by, 88 automated management of, 369–372 automatic configuration of by Fabric OS, 133–134 Brocade SilkWorm series of See Brocade SilkWorm switches buffer credits available per switch port, 86–87 cable layout and, 351–354 classes of service and, 84 configuration backups for, 393–394 core See core switches Destination ID (D_ID), 302 disabled, 300 edge, 243, 248–250 equipment redundancy and, 86 exchange of information by, 49–50 fabric licenses and, 322, 366 fabric switches, 81 FSPF compliance, 90–91 host tier, 263 hubs vs., 57 in-band switch management, 356–358 IPFC broadcasting and, 88 loop operation capabilities, 90 management interfaces, 91–94 naming, 360 out-of-band switch management, 356 principal, 300 restoring configuration of, 393–394 selecting most appropriate, 124–126 self-configuring ports and, 87–88 setting parameters for, 358–359, 361 storage tier, 263 subordinate, 300 switched fabric topology and, 48–49 upgrading, 89 zoning and See switch zoning switchShow command, 278, 279, 286 fabric troubleshooting with, 307–308, 318–319 fabric validation using, 382 SAN troubleshooting with, 297–303, 307–308 troubleshooting missing devices with, 279–280, 329 SwitchType values, 302 syslog daemon interface, Fabric OS, 135 syslogIpaAdd command, 293 syslogIpRemove command, 293 140_SANs_index 8/14/01 3:43 PM Page 449 Index syslogIpSho command, 293 system DRAM diagnostics, 289 T T11, 35 tape drives, 32, 33, 171 target devices, 32–33 Task Name, 294, 295 technical goals, identifying, 153 technical requirements document, 160, 190 technical requirements, identifying SAN, 159–160, 190 telnet, switch management by, 91, 92, 378 tempShow switch command, 285 10-switch core/edge SAN design, 253–254 termination, type of SCSI, 108 third-party copy backup systems, 214, 215–216 third-party copy devices, 14, 214, 215–216, 217 tiered SANs, 261–265 tiers, leveraging, 261–265 Time Server, 38, 41, 50, 52, 86 timeline, creation of development, 175–176, 190 timeout failures, edge device, 321–322 Tivoli products SANergy, 12, 212 Storage Network Manager, 350, 368 Token Ring protocol, 31 too_long error statistic, 297 topologyShow command, 307–308, 309, 320, 321, 329, 382, 395 traceShow command, 286 traffic patterns any-to-any connectivity and, 268–269 evaluating future traffic, 172–174 Fabric OS dynamic routing, 134 initiator-to-target mapping, 167–168 leveraging tiers, 261–265 localizing traffic, 266–268 pre-SANs performance data on, 169–172 449 SAN grouping process and, 178–182 transitioning, 154 transmission words, 36 Troika products HBA driver, 172 SAN Command utility, 101, 102, 312 troubleshooting fabrics, 316–327 missing devices, 279–283, 327–335 SANs (Storage Area Networks), 278–316 tools for, 287–316 20-switch core/edge SAN design, 254 U ULP information, 36 ULP mapping, 42 unbonded SC connectors, 72 unconfirmed domains, 300–301, 319 Unicenter TNG, 350, 368 UNIX dd tool for I/O generation, 388–389 format command, 312, 317 performance monitoring, 169–170 unmanaged Fibre Channel hub, 34, 35, 60 Unzoned Name Server, 51 upgrades acceptable downtime for, 174–175 fabric, 398–400, 408 Fibre Channel switch, 89 firmware, 89, 398–400 rolling with HA clusters, 197, 198–199, 202 upstream devices, 302, 303, 304 uptime switch command, 285 V /var/adm/messages file, Solaris’s, 287, 312, 317 VERITAS products Cluster Server, 11 Dynamic Multipathing, 172, 199 140_SANs_index 450 8/14/01 3:43 PM Page 450 Index NetBackup, 13 SANavigator, 103 SANPoint control, 103, 350, 368 Volume Manager, 12, 212 vxbench, 387 version switch command, 285 Virtual Interface (VI) protocol, 2, 15, 31, 42, 97 Visual SAN, Prisa’s, 350, 368 volume-level sharing, 19 Volume Manager,VERITAS’, 12 Vxbench,VERITAS’, 387 W Web-based router management, 109 Web-based switch management, 93–94 Web farms, storage sharing with, 206–207 Web sites Brocade, 158 Expect, 369 Intel (Iometer), 387 IOzone, 387 router management through, 109 switch management through, 93–94 WEB TOOLS, 139–140, 147, 366, 378, 408 well-known addresses, 50, 51, 52 WHQL See Microsoft Windows Hardware Quality Lab (WHQL) Windows 2000 Advanced Server, 200 Windows 2000 Data Center, 200, 202 Windows 2000 OS, 99 Windows NT, 99, 169, 200 wiring, 61, 65–68, 164 World-Wide Names (WWNs), 307, 366 World Wide Web, data storage needs and, WWN ports, 306–307 WWN spoofing, 143 X X3T11, 35 Z zero-downtime failover, 199 zoning, switch See switch zoning 140_SANs_BM 8/14/01 3:38 PM Page 451 140_SANs_BM 8/14/01 3:38 PM Page 452 Additional SAN Information Resources from Brocade To learn more about SANs and SAN technologies, you can visit the Brocade Web site at www.brocade.com.This site contains a wide range of useful resources designed for many types of audiences—from high-level managers and other key decision makers to system administrators and network architects In particular, the Brocade SAN solution center and educational Web pages offer a wealth of timely information Brocade SAN Solution Center Accessible at www.brocade.com/san, the Brocade SAN solution center includes a variety of information about Brocade products and SAN technologies in the form of: ■ White papers ■ Product data sheets ■ Brocade SOLUTIONware configuration summaries ■ Standards updates ■ Industry news ■ Tools for understanding Return On Investments Brocade Education Services Accessible at www.brocade.com/education_services, Brocade education provides information about the full range of Brocade educational offerings, including: ■ Brocade SAN certification requirements ■ Course descriptions ■ Registration forms ■ Special promotions 140_SANs_BM 8/14/01 3:38 PM Page 453 Brocade Networking Storage Conference 2002 If you want to interact with the SAN experts, just plan to attend the second annual Brocade Storage Networking Conference.Visit www.brocade.com/san to get the latest information about registration, early-bird discounts, and previews of the agenda and speakers 140_SANs_BM 8/14/01 3:38 PM Page 454 SYNGRESS SOLUTIONS… AVAILABLE NOW ORDER at www.syngress.com ASP Configuration Handbook: A Guide to ISPs Internet Service Providers (ISPs) are confronted with shrinking profit margins, fierce competition, and undifferentiated products To successfully overcome these new economic realities, ISPs have begun to expand beyond just offering bandwidth and connectivity, by providing customers with “outsource services” such as software applications and electronic commerce solutions This book is the blueprint for a successful ISP-to-ASP transition ISBN: 1-928994-26-1 Price: $49.95 US, $77.95 CAN AVAILABLE NOW ORDER at www.syngress.com Administering Cisco QoS in IP Networks Administering Cisco QoS for IP Networks discusses IP Quality of Service (QoS) and how it applies to Enterprise and Internet Service Provider (ISP) environments It reviews routing protocols and quality of service mechanisms available today on Cisco network devices and it will provide you with examples and exercises for a hands-on experience designed to give you the background to implement these capabilities in your network ISBN: 1-928994-21-0 Price: $59.95 US, $92.95 CAN AVAILABLE DECEMBER 2001 ORDER at www.syngress.com Juniper Networks Internet Routing Guide Upstart Juniper Networks has created a new generation of faster routers that are extremely price competitive with similar Cisco models Juniper Networks Internet Routing Guide is a comprehensive, hands-on guide on how to install, configure, and troubleshoot Junipers core router family This book will immediately meet the demands of network engineers responsible for the installation and configuration of Juniper Routers It also covers the latest “M-Class” routers, as well as configuration strategies, and the JUNOS software environment ISBN: 1-928994-76-8 Price: $69.95 US, $108.95 CAN solutions@syngress.com ... guidebook for implementing Fibre Channel SANs. To help fill that void, we have joined with Syngress Publishing to bring you Building SANs with Brocade Fabric Switches This book details the design,... 400 401 403 405 408 Appendix Building SANs with Brocade Fabric Switches Fast Track 409 Index 431 140 _SANs_ fore 8/14/01 3:44 PM Page xxv Foreword Why Write a Book about SANs? During the last few... N835M4KBAZ QT6Y4RTWFC PUBLISHED BY Syngress Publishing, Inc 800 Hingham Street Rockland, MA 02370 Building SANs with Brocade Fabric Switches Copyright © 2001 by Syngress Publishing, Inc All rights