John wiley sons storage networks explained basics and application of fibre channel san nas iscsi and infiniband 0470861827

Storage Networks Explained Basics and Application of Fibre Channel SAN, NAS, iSCSI and InfiniBand Ulf Troppens, Rainer Erkens IBM TotalStorage Interoperability Center, Mainz, Germany Wolfgang Muller ă IBM Software Development Open Systems, Mainz, Germany Translated by Rachel Waddington, Member of the Institute of Translating and Interpreting Storage Networks Explained Storage Networks Explained Basics and Application of Fibre Channel SAN, NAS, iSCSI and InfiniBand Ulf Troppens, Rainer Erkens IBM TotalStorage Interoperability Center, Mainz, Germany Wolfgang Muller ă IBM Software Development Open Systems, Mainz, Germany Translated by Rachel Waddington, Member of the Institute of Translating and Interpreting First published under the title Speichernetze Grundlagen und Einsatz von Fibre Channel SAN, NAS, iSCSI und InfiniBand, ISBN: 3-89864-135-X by dpunkt.verlag GmbH  dpunkt.verlag GmbH, Heidelberg, Germany, 2003 Copyright  2004 for the English translation: John Wiley & Sons Ltd The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England Telephone (+44) 1243 779777 Email (for orders and customer service enquiries): cs-books@wiley.co.uk Visit our Home Page on www.wileyeurope.com or www.wiley.com All Rights Reserved No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1T 4LP, UK, without the permission in writing of the Publisher Requests to the Publisher should be addressed to the Permissions Department, John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England, or emailed to permreq@wiley.co.uk, or faxed to (+44) 1243 770620 This publication is designed to provide accurate and authoritative information in regard to the subject matter covered It is sold on the understanding that the Publisher is not engaged in rendering professional services If professional advice or other expert assistance is required, the services of a competent professional should be sought Other Wiley Editorial Offices John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA Jossey-Bass, 989 Market Street, San Francisco, CA 94103-1741, USA Wiley-VCH Verlag GmbH, Boschstr 12, D-69469 Weinheim, Germany John Wiley & Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australia John Wiley & Sons (Asia) Pte Ltd, Clementi Loop #02-01, Jin Xing Distripark, Singapore 129809 John Wiley & Sons Canada Ltd, 22 Worcester Road, Etobicoke, Ontario, Canada M9W 1L1 Wiley also publishes its books in a variety of electronic formats Some content that appears in print may not be available in electronic books British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library ISBN 0-470-86182-7 Typeset in 10/12pt Times by Laserwords Private Limited, Chennai, India Printed and bound in Great Britain by Antony Rowe Ltd, Chippenham, Wiltshire This book is printed on acid-free paper responsibly manufactured from sustainable forestry in which at least two trees are planted for each one used for paper production For Silke, Hannah, Nina, and Julia You keep showing me what really matters in life For Christina, Edith, and Heribert For the love and constant support you have given me For Christel Only your patience and your understanding have made my contribution to this book possible Contents About the Authors xv Foreword by Tom Clark xvii Foreword to the German Edition by Hermann Strass xix Preface by the Authors xxi List of Figures and Tables xxv Introduction 1.1 Server-Centric IT Architecture and its Limitations 1.2 Storage-Centric IT Architecture and its Advantages 1.3 Case Study: Replacing a Server with Storage Networks 1.4 The Structure of the Book 1 PART I Technologies for Storage Networks 11 Intelligent Disk Subsystems 2.1 Architecture of Intelligent Disk Subsystems 2.2 Hard Disks and Internal I/O Channels 2.3 JBOD: Just a Bunch of Disks 2.4 Storage Virtualization using RAID 2.5 Different RAID Levels in Detail 2.5.1 RAID 0: block-by-block striping 2.5.2 RAID 1: block-by-block mirroring 2.5.3 RAID 0+1/RAID 10: striping and mirroring combined 2.5.4 RAID and RAID 5: parity instead of mirroring 13 14 16 19 20 22 23 24 24 28 viii CONTENTS 2.5.5 RAID and RAID 2.5.6 A comparison of the RAID levels Caching: Acceleration of Hard Disk Access 2.6.1 Cache on the hard disk 2.6.2 Write cache in the controller of the disk subsystem 2.6.3 Read cache in the RAID controller Intelligent Disk Subsystems 2.7.1 Instant copies 2.7.2 Remote mirroring 2.7.3 LUN masking Availability of Disk Subsystems Summary 33 34 35 35 36 36 37 37 39 43 45 46 I/O Techniques 3.1 The Physical I/O Path from the CPU to the Storage System 3.2 SCSI 3.2.1 SCSI basics 3.2.2 SCSI and storage networks 3.3 The Fibre Channel Protocol Stack 3.3.1 Links, ports and topologies 3.3.2 FC-0: cables, plugs and signal encoding 3.3.3 FC-1: 8b/10b encoding, ordered sets and link control protocol 3.3.4 FC-2: data transfer 3.3.5 FC-3: common services 3.3.6 Link services: login and addressing 3.3.7 Fabric services: name server and co 3.3.8 FC-4 and ULPs: application protocols 3.4 Fibre Channel SAN 3.4.1 Point-to-point topology 3.4.2 Fabric topology 3.4.3 Arbitrated loop topology 3.4.4 Hardware components for Fibre Channel SAN 3.4.5 InterSANs 3.4.6 Interoperability of Fibre Channel SAN 3.5 IP Storage 3.5.1 IP storage standards: iSCSI, iFCP, mFCP, FCIP and iSNS 3.5.2 TCP/IP and Ethernet as an I/O technology 3.5.3 Migration from SCSI and Fibre Channel to IP storage 3.6 InfiniBand 3.7 Virtual Interfaces and Remote Direct Memory Access (RDMA) 3.8 RDMA over TCP, Socket Direct Protocol (SDP) and iSCSI Extensions for RDMA (iSER) 3.9 Summary 49 50 52 52 54 56 57 60 2.6 2.7 2.8 2.9 62 65 71 71 74 75 77 77 78 81 84 88 89 91 91 97 102 103 106 108 109 386 B.3.9 Security • How can I protect resources against unauthorized access? • Which physical measures are to be taken for this? B.4 NETWORK B.4.1 Topology • Which devices are connected together how? B.4.2 Monitoring • How can I recognize the failure of connections? • Are there criteria for predicting any failures? B.4.3 Availability • • • • What level of availability of the network is required? How can I guarantee the availability of the network? Where are redundant data paths required and how can I provide these? Where single points of failure exist? B.4.4 Performance • • • • Where are there bottlenecks in the data path? How can I optimize the data path? Usage profile: when are which data paths utilized and how? Trend analysis: am I coming up against bandwidth limits? APPENDIX B Index Glossary entries are type set in italic 3rd-Party SCSI Copy Command 64b/66b-encoding 64, 353 8b/10b encoding 103, 62, 353 221, 353 Access control 353 Access handle, see Removable media management Access path 353 Active 353 Active/active 353 Active/passive 354 Adaptability 187–203 bandwidth 98 connection of server to storage 139 device replacement during runtime 139 limitations 140–141, 170 new applications 193, 197 new end devices 197 storage capacity 198 user interface 195 Agent 354 Aggregation 354 AL PA, see Fibre Channel API, see Management of storage networks Appliance 354 Application server-free back-up, see Back-up Arbitrated loop, see Fibre Channel Archive bit, see Back-up Archive log file, see Databases Archiving, see Back-up ASIC (Application Specific Integrated Circuit) 51 Asymmetric storage virtualization, see Storage virtualization ATA 52 Atomicity, see Databases Audit trail, see Removable media management, media tracking Autoloader, see Removable media, library Availability 1, 135, 175–187 auxiliary components 46 cluster, see Cluster controller 46 database 184–187 disk subsystems 45–46 dual SAN 176, 184 due to storage virtualization 149 external I/O channels 46 instant copy 46 internal I/O channel 46 multipathing software 176–179 RAID, see RAID Storage Networks Explained U Troppens R Erkens W Măuller 2004 John Wiley & Sons, Ltd ISBN: 0-470-86182-7 388 Availability (continued ) redundant data centres 186 redundant disk subsystems 181–184 redundant I/O buses 175–179, 180, 182–183, 184 redundant servers 179–180, 182–183, 186 remote mirroring 46, 181–184, 187 single point of failure 181 self-diagnosis of hard disks 21, 46 server cluster, see Cluster single point of failure 45, 181 virtualization entity 158, 184 volume manager mirroring 181–184 Back-up application server-free back-up 226, 229, 242, 354 archive bit 229, 354 archiving 207, 213, 354 back-up of desktop PCs 210 back-up of laptops 212 back-up 207, 354 back-up clients 212 time window 206 bare metal restore, see Back-up, image restore block level 212, 213, 243 block-level incremental back-up 229, 243, 355 cold back-up, see Databases, back-up consistency 240 costs of data loss 244 data monster 215 databases, see Databases error handler 209, 358 Fibre Channel over IP (FCIP) 94, 98–99 file level 212, 229 file server 227–228 metadata 228 restore 228 file systems 212, 214, 228 archive bit 229 fuzzy back-up, see Databases, back-up general conditions 206 hierarchical storage management (HSM) 207–208, 213, 359 hot back-up, see Databases, back-up image restore 228, 360 INDEX incremental forever 209, 209–210, 213, 243, 360 archive bit 229 instant copy 224–225, 229, 242 insufficient distance between production data and back-up copies 219 job scheduler 209, 361 LAN-free back-up 221–223, 243, 361 Library sharing, see Storage sharing media manager 209–212, 362 metadata 228, 230 metadata database 209, 212, 214, 217, 363 mksysb command 228 NAS server 229–238 metadata 230 network back-up 206–244 network back-up system 364 Network Data Management Protocol (NDMP), see NDMP off-site location 364 off-site storage 211–212, 219 online back-up 225 performance increase, see Performance increase performance, see Performance personnel cost 243 point-in-time restore 209, 364 remote mirroring 227 restore 226, 244 server components 208–212 server-free back-up 219–221, 367 storage hierarchy 210, 214, 218, 368 streaming, see Removable media, tape tape library sharing, see Storage sharing tape reclamation 210, 214, 219, 370 terminology 207 training scenarios 244 volume level 212, 213 Web Flash Archive (Solaris) 228 with storage virtualization 149 XCOPY, see SCSI Back-up window, see Back-up Bare metal restore, see Back-up, image restore Binary encoding 355 Block aggregation 355 Block layer, see SNIA shared storage model Block level 355 Block orientation 355 Bluefin, see CIM, SMI-S INDEX Buffer credit, see Fibre Channel Bus 355 Cache 355 Cache server 161, 202, 355 Cartridge, see Removable media CD, see Removable media, optical media CIFS, see File systems CIM 264–270, 355 association 266 attribute 266 Bluefin, see CIM, SMI-S class 266 client 269 CIM operations over HTTP 266 Common Schema 267 Core Schema 267 Extension Schema 268 instance 266 managed object 269 methods 266 object manager 269 object-oriented modelling (OOM) 266, 364 provider 269 relationship 266 schema 267 SMI-S 270–271, 368 subclass 266 trigger 270 WBEM 264–265, 269, 372 xmlCIM Encoding Specification 266 Class 1, Class 2, Class 3, see Fibre Channel, service classes Cluster 188–192, 198, 356 active/active 189 active/passive 189 definition of terms 188 enhanced shared-nothing 189–191, 357 fault-tolerance 179–180 file server 179–180, 188–192 load distribution 188–192 dynamic 190 static 190 representation server 196 shared-everything 191–192, 203, 367 shared-nothing 189–191, 203, 368 shared-null configuration 188, 368 virtual IP address 179, 191 cluster computer RS/6000 SP 126 389 CMIP, see Management of storage networks Cold back-up, see Back-up, databases 356 Co-location 356 Community name, see SNMP Computer upgrade 4–7 Consistency 5, 225, 240 Copy-on-demand 356 Cut-through routing 78, 79, 356 CWDM 62, 356 D ID, see Fibre Channel DAFS, see File systems DAS 356 Data copying, see Storage sharing Data doubling, see Storage doubling Data networks 168 Data protection, see Back-up Data sharing, see Storage sharing Databases 5, 202, 238–243 archive log files 240, 242 atomicity 238, 240 back-up 240–243 application server-free back-up 242 archive log files 240–241 block level 243 block-level incremental back-up 243 cold back-up 240 consistency 240 forward recovery 241 frequency 243 fuzzy back-up 242 hot back-up 241–242, 359 incremental forever 243 instant copy 242 LAN-free back-up 243 DAFS 123 file systems and databases 112 forward recovery 241, 359 log file 238 parallel 175 transaction 238, 240 Device driver, see I/O techniques Direct access file system, see file systems, DAFS Director, see Fibre Channel Discovery, see Management of storage networks Disk subsystems 14–46, 53, 357 auxiliary components 46 390 Disk subsystems (continued ) availability, see Availability cache 35–37, 136 controller 19–46 external I/O channel 46 fault-tolerance 17, 18, 20, 137 hard disks 16, 19 cache 35–37 capacity 21 controller 35–36 physical 16–17, 20 self-diagnosis 21, 46 virtual 20, 21, 113 instant copy 5, 37–39, 46, 135, 360 comparison snapshot 113 data consistency 5, 37 full copy 39 partial copy 38 three-way mirror 38 intelligent disk subsystems 37–45 internal I/O channel 16–19, 46 active 17 active/active 18 active/passive 17 interoperability 140–141, 170 JBOD 19–20, 361 LUN masking 43–45, 81, 362 port-based 45 server-based 45 multitasking capability 90 performance 16, 20 physical protection prefetch hit rate 37 RAID, see RAID over several disk subsystems 137 remote mirroring 5, 39–43, 46, 366 asynchronous 41–43 highly available applications 39 synchronous 41 replacement of defective hard disks 20–22 single point of failure 45 DMI, see Management of storage networks DMTF 357 Doubling of storage requirement every 4–12 months 206 Drive sharing, see Storage sharing Drive, see Removable media, drive Dual SAN, see Availability 357 INDEX Dual storage network, see Availability, dual SAN DVD, see Removable media, optical media DWDM 357 E-business 195 Element manager, see Management of storage networks 357 Emulated loop, see Fibre Channel Enhanced shared-nothing cluster, see cluster Error handler, see Back-up Escalator jitter 63 skew 60 ESCON 77, 358 Ethernet 52, 36 8b/10b-encoding 62 cut-through routing 101 gigabit 198 latency 101 level router 101 Virtual Interface Architecture (VIA) 106 Exchange, see Fibre Channel External storage, see Storage, external Fabric login (FLOGI), see Fibre Channel Fabric, see Fibre Channel Fault-tolerance, see Availability FC 358 FCIA 358 FCIP, see IP storage FCN, see Fibre Channel, Fibre Channel Name (FCN) FCP, see Fibre Channel, Fibre Channel Protocol (FCP) FDDI 56 Fiber 358 Fibre Channel 14, 17, 19, 52, 56–91, 95, 129–131, 358 64b/66b-encoding 64 8b/10b-encoding 62 addressing 73–74 AL PA 354 aliasing 79, 82, 87 application protocols 75–77 Arbitrated Loop Physical Address (AL PA) 73 arbitrated loop topology 81–84, 354 bandwidth 79, 82 INDEX buffer-to-buffer credit 67 cable types 60–62, 85 class 1, class 2, class 3, see Fibre Channel, service classes clock-pulse rate synchronization 62–64 common HBA API 255 compression 71 configuration server 256 credit 66, 356 cut-through routing 78, 79 data word 64 destination identifier (D ID) 73 director 86, 89, 357 directory service 256 dual fabric 86 emulated loop 83, 357 encryption 71 end point node information 258 end-to-end credit 67 error correction 65 error rate 62, 64 exchange 65, 358 extender 87 fabric controller 74 fabric login (FLOGI) 71, 74, 358 fabric login server 74 fabric services 74–75 fabric topology 78–81, 358 FCIP-to-Fibre-Channel-gateway 95 Fibre Channel FCP-to-iFCP gateway 93 Fibre Channel Name (FCN) 73, 358 Fibre Channel over IP (FCIP), see IP storage Fibre Channel protocol (FCP) 76, 89, 95, 358 Fibre Channel SAN 57, 76–91, 358 terminology ‘SAN’ 56 Fibre Channel-to-SCSI bridges 85 Fibre connection (FICON) 77, 358 flow control 66, 79, 359 frame 65–66, 359 frame buffer 79 generic services (FC-GS) 256 hard zoning 81, 359 hardware components 84–88 host bus adapter (HBA) 51, 57, 85 hub 82, 87, 359 interoperability 89 interSAN 88 inter-switch link (ISL) 78, 79, 80, 361 391 investment protection 102 IP backbone 56, 57 IP over Fibre Channel (IPFC) 76, 92, 360 iSCSI-to-Fibre Channel gateway 93, 102 latency 78, 81 link 57, 362 link extender 87 LIP 87, 362 login 71–73 LUN masking 81 managed hub 87, 362 management service 256 methodologies for interconnects (FC-MI) 256 mirroring 71 multipathing 71 multipathing software 177 name server 74, 79, 81, 256, 363 N-Port login (PLOGI) 72 number of ports 85 ordered set 64, 364 platform registration 258 plug types 85 point-to-point topology 77–78, 364 port 57 port address (N-Port ID) 73, 79 port login (PLOGI) 72, 365 port types 59–60 port zoning 365 private loop 83, 365 private loop device 83, 365 process login (PRLI) 72, 365 protocol mapping 365 protocol stack 57 public loop 83, 365 public loop device 83, 365 Quickloop 365 RAID 71 read link error status (RLS) 257 read port status (RPS) 257 registered state change notification (RSCN) 74, 258, 366 request node identification data (RNID) 257 request topology information (RTIN) 257 requirements 56 routing 82 RSCN, see Fibre Channel, registered state change notification (RSCN) 392 Fibre Channel (continued ) SAN islands 88 SCR 74 SCSI over Fibre Channel, see Fibre Channel, FCP sequence 65–66, 367 serial transmission 60 service classes 68–69, 355 class 68 class 68–69 class 68–69 soft zoning 81, 368 source identifier (S ID) 73 state change registration (SCR) 74 striping 71 support matrix 90 switch 85 switched hub 87, 370 translated loop 371 transmission rate 60 transmission word 64 ULP 75, 371 unmanaged hub 87, 371 upper layer protocol (ULP) 75 Virtual Interface Architecture (VIA) 77, 106 VSAN 81, 372 World Wide Name (WWN) 73 World Wide Node Name (WWNN) 73, 79 World Wide Port Name (WWPN) 79, 81 WWN 372 WWN zoning 372 WWNN 372 WWPN 372 zone server 257 zoning 79–81, 87, 258, 372 Fibre Channel over IP (FCIP), see IP storage FICON, see Fibre Channel File level 358 File orientation 359 File server, see file systems File system check 113 File systems 111–132 archive bit 229 back-up, see Back-up CIFS 116, 120, 355 DAFS 109, 120–124, 356 databases 123 fDAFS (user space) 123 INDEX Locking 121 RDMA 120 uDAFS (user space) 121 file server FTP server 116–117 HTTP server 117 NFS server 116 performance bottleneck 118–119 protection against failure 179–180 file system check 113 file systems and databases 112 GPFS 126–129, 175 configuration management 128 File System Manager 128 GPFS cluster 127 GPFS Daemon 127 Meta Data Manager 160 node set 127 striping 127 token administration 128 Virtual Shared Disk (VSD) 126 journaling 112–113, 361 local 111–115 log file 113 mount command 116 NAS server 117–124, 129–131, 202–203, 363 administrative burden 117 back-up, see Back-up performance bottleneck 118–119 scalability 117 upgrade path 117 network drive 116 network file systems 115–124, 191–192, 364 NFS 116, 120, 364 performance bottleneck 118–119 shared disk file systems 124–129, 175, 191–192, 367 snapshot 5, 112, 117, 225, 368 volume manager 113–115 File/record layer, see SNIA shared storage model Firewire 52 Five tier architecture, see IT architecture, web architecture FLOGI, see Fibre Channel, fabric login (FLOGI) Forward recovery, see databases INDEX Frame, see Fibre Channel FTP 116 Fuzzy back-up, see Databases, back-up, Gigabit Ethernet 198 GPFS, see file systems Hamming code 21, 33, 46 Hard disk, see disk subsystems Hard zoning, see Fibre Channel HBA, see Host bus adapter (HBA) HCA, see InfiniBand, host channel adapter (HCA) Hierarchical storage management, see Back-up High-availability, see availability HIPPI 52 Host bus adapter (HBA) 51, 57, 359 Host-I/O bus, see I/O techniques Hot back-up, see Back-up, databases Hot spare disks 20, 359 HSM, see Back-up, hierarchical storage management (HSM) Hub, see Fibre Channel Hyper Text Meta Language (HTML) 117 Hyper Text Transfer Protocol (HTTP) 117 I/O bus, see I/O techniques I/O path, see I/O techniques I/O techniques 50–110 controller 51 device driver, see I/O techniques, device driver 51 host bus adapter (HBA) 51 host I/O bus 51, 359 I/O bus 50–51, 359 I/O bus, availability, see Availability I/O bus, redundant, see Availability I/O path 50–51, 129, 135–139, 167, 360 latency 106 network interface controller (NIC) 51 system bus 51, 370 IEEE 1244, see Removable media management IEEE 1394 52 IETF 360 iFCP, see IP storage 360 I-link 52 Image restore, see Back-up In-band management, see Management of storage networks 393 In-band virtualization, see Storage virtualization, symmetric Incremental forever, see Back-up Infiniband 103–106, 198, 360 8b/10b-encoding 62 cable types 104 channel adapter 104 host channel adapter (HCA) 104, 359 interoperability 91 protocol stack 106 remote direct memory access (RDMA) 109 target channel adapter (TCA) 104, 370 transmission rate 103–104 Virtual Interface Architecture (VIA) 106 Instant copy, see Disk subsystems Internal storage, see Storage, internal Internet FCP (iFCP), see IP storage Internet SCSI, see iSCSI internet Storage Name Service (iSNS), see IP storage Interoperability 89 disk subsystems 140–141, 170 IP storage 102 RDMA 108 support matrix 90, 370 Inter-switch link (ISL), see Fibre Channel IP storage 52, 91–103, 360 advantages 97 collisions 101 CPU load 100 cut-through routing 101 disadvantages 97 encryption 95 Fibre Channel over IP (FCIP) 91, 94–95, 358 back-up 94, 98–99 FCIP-to-Fibre Channel gateway 95 Internet FCP (iFCP) 91, 93, 95 Fibre Channel FCP-to-iFCP gateway 93 iSNS 95 Internet SCSI (iSCSI) 14, 91–92, 95, 101, 102, 109, 124, 129–131, 360 boot 93 iSCSI HBA 92 iSCSI-to-Fibre channel gateway 93, 102 iSNS 95 software driver 92 TCP/IP offload engine (TOE), see TCP/IP offload engine 394 IP storage (continued ) internet storage name service (iSNS) 95, 361 interoperability 100, 102 investment protection 102 iSCSI extension for RDMA (iSER) 92, 101, 108, 109, 124, 361 iSCSI SAN 361 latency 99, 101 metro FCP (mFCP) 93, 363 migration path potential 97 standardization 99 storage over IP (SoIP) 92, 368 IPFC, see Fibre Channel, IP over Fibre Channel (IPFC) iSCSI, see IP storage iSER, see IP storage ISL, see Fibre Channel, inter-switch link (ISL) iSNS, see IP storage, internet Storage Name Service (iSNS) IT architecture five tier architecture, see IT architecture, web architecture server-centric 1–3, 169, 218, 367 performance bottlenecks 214–219 storage-centric 3–7, 54, 170, 369 three tier architecture 192–193 web architecture 193–195, 199, 372 Jitter 63, 361 Job scheduler, see Back-up Journaling, see File systems Just a bunch of disks (JBOD), see Disk subsystems, JBOD K28.5 symbol 361 Label 361 LAN 361 LAN-free back-up, see Back-up Latency 78, 106, 361 Level router 101 Library partitioning, see Storage sharing Library sharing, see Storage sharing Library, see Removable media Link, see Fibre Channel LIP, see Fibre Channel Load balancing, see Cluster, load distribution INDEX Load distribution, see Cluster Log file, see Databases Loop 362 Lord of the Rings 278 LUN masking, see Disk subsystems LUN zoning 362 LUN, see SCSI Magnetic tapes, see Removable media, tapes Magneto-optical media, see Removable media, optical media Maintainability 135 MAN 362 Managed hub, see Fibre Channel Management console, see Management of storage networks Management interface, see Management of storage networks Management of storage networks 247 API 253 Bluefin, see CIM, SMI-S CIM, see CIM CMIP 271–272 CMIP 356 discovery 250, 257, 263, 357 DMI 271–272, 357 element manager 254, 357 events 258 in-band management 254–258 in-band management interface 252 management agent 255 management console 250, 362 management systems 248, 249 mechanisms 252 messages 257, 264 metadata controller 362 MIB file 261 monitoring 250, 257, 263, 363 out-band management 258–275, 364 out-band management interface 252 removable media, see removable media management SMI-S, see CIM SNMP, see SNMP Telnet 254 traps 264 WBEM, see CIM Management of removable media, see Removable media management INDEX 395 Manchester encoding 63, 362 Media changer, see Removable media 362 Media manager, see Back-up Media server 203 Metadata 207, 209 Metadata controller (MDC), see Storage virtualization, metadata controller Metadata database, see Back-up Metro FCP (mFCP), see IP storage MIB, see SNMP, management information base (MIB) Mirroring, see RAID, RAID (mirroring) mksysb command (AIX) 228 Monitoring, see Management of storage networks Mount command 116 Mount request, see Removable media management Multipathing 363 Multipathing software 176–179 service 232–233 session 231–233 snapshots 237 tape service 233 translator service 235–236 Network attached storage (NAS), see File systems, NAS server Network Back-up, see Back-up Network card, see Network interface controller (NIC) Network data management protocol, see NDMP Network drive 116 Network file systems, see File systems Network interface controller (NIC) 51, 107, 364 Network Management System (NMS), see SNMP NFS, see File systems NIC, see Network interface controller (NIC) Name server, see Fibre Channel NAS, see file systems, NAS server NDMP 231–238, 363 architecture 232 back-up of metadata 237 client 232 control connection 232, 233 data connection 232 data management application (DMA) 231–232, 233 data management operation 231 data service 232 data stream compression 237 encryption 237 multiplexing 235 direct access recovery 233 file history interface 233 file locator data 233 host 232 instant copy 237 interoperability 91 LAN-free back-up 234 library sharing 234 local back-up 233–234 remote back-up 233–234 SCSI pass through service 233 server 232 Offline storage, see Removable media management, vaulting Off-site storage, see Back-up OOM, see CIM, object-oriented modelling (OOM) Open Systems 364 Optical media, see Removable media Ordered set, see Fibre Channel OSI reference model 166 Out-band management, see Management of storage networks Out-band virtualization, see Storage virtualization, asymmetric Parallel computer RS/6000 SP 126 Parity, see RAID Partition, see Removable media PCI 51, 214, 364 Performance bottleneck buses 214, 216, 219 copying to the storage hierarchy 214, 219 CPU 219 due to hot back-up 242 due to intensive testing file server 118–119 hard disk 24 I/O path 24, 139 main memory 219 396 E T AM Y L F Performance bottleneck (continued ) metadata database 214, 217 NAS 119 network file system 118–119 parity hard disk, see RAID, parity hard disk tape reclamation 214, 219 TCP/IP 119 write penalty, see RAID, write penalty Performance increase by archiving 207, 213 by back-up on block level 213 by back-up on volume level 212, 213 by back-up over the storage network 219–223 by caching 35–37 by HSM 207, 213 by incremental forever 209, 213 by installation of all servers on one computer 218–219 by installation of an additional LAN 215–216 by installation of several back-up servers 217 by instant copy 37 by network back-up 208–227 by optimisation to reduce the write penalty 31 by parallel hard disk access, see RAID by RAID, see RAID by RDMA 120 by reduction of tape mounts 210, 213 by shared disk file systems 124 by shared memory 218, 219 by smoothing of load peaks 36 by storage virtualization 148 by streaming 210, 213 by striping over several disk subsystems 139 by virtualization 135 striping 127 Point-in-time restore, see Back-up Point-to-point, see Fibre Channel Port 364 Port login (PLOGI), see Fibre Channel Port zoning, see Fibre Channel Prefetch hit rate 37, 129–131, 365 Private loop device, see Fibre Channel Private loop, see Fibre Channel Process login (PRLI), see Fibre Channel INDEX Protocol mapping, see Fibre Channel Public loop device, see Fibre Channel Public loop, see Fibre Channel Quickloop, see Fibre Channel RAID 20–37, 45, 52, 135, 184, 365 cache 36–37 capacity 21 comparison of the RAID levels 34–35 fault-tolerance 24–26, 33, 34 Fibre Channel FC-3 71 hard disk, see Disk subsystems, hard disk hot spare disks 20 level 20 over several disk subsystems 138–139 parity 365 parity blocks 28–33 parity hard disk 28–33 performance 23–25, 32–33, 34 prefetch hit rate 37 priority of data restore 21 RAID (striping) 23–24, 35, 127, 370 RAID 0+1 (mirrored stripes) 24–28 RAID (mirroring) 24, 35, 181–182, 365 RAID 10 (striped mirroring) 28, 35 RAID 10 (striped mirrors) 24–28 RAID 33 RAID (parity) 28–33, 35 RAID (striped parity) 28–33, 35 replacement of defective hard disks 20–22, 33 three-way mirroring 24, 38 volume manager 114 write penalty 30–33 XOR operator 30, 31 RDMA, see Remote Direct Memory Access (RDMA) Realtime data sharing, see Storage sharing Remote Direct Memory Access (RDMA) 92, 101, 108, 120, 365 interoperability 91, 108 iSER, see IP storage, iSER RDMA over TCP 108–109, 365 RNIC 108, 366 SDP, see SDP Virtual Interface Architecture (VIA) 108 Remote mirroring, see disk subsystems Removable media 278–283 INDEX access handle 280 cartridge 280, 355 cartridge label 280 lifecycle, see Removable media management, cartridge recovery 294 sharing, see Storage sharing status 294 tape drive 281 library autoloader 354 client 171 master 171 sharing, see Storage sharing slot 368 tape library 53, 209, 218, 219, 280–283 virtual 171 media changer 281–283 control 282–283 interfaces 282–283 optical media 279 partition 280, 364 robot, see Removable media, media changer side 280, 368 tape 278–279 mount 210, 213, 370 reclamation, see Back-up streaming 210, 213, 369 tape drive sharing, see Storage sharing tape drive, see Removable media, drive tape library sharing, see Storage sharing tape library, see Removable media, library volume 280, 371 Removable media management 277–307 access control 287–289, 291, 292 access synchronization 289 audit trail, see Removable media management, media tracking authentication 287 authorization 287–289 cartridge access authorization 296 life cycle 296 states 296–298 cataloguing 284 common scratch pool 356 drive pool 293 drive sharing, see Storage sharing error statistics 295 397 fault-tolerance 291, 292, 293 grouping 292–293 IEEE 1244 298–306 architecture 299–302 data model 302 drive management protocol (DMP) 300, 302, 306 drive manager 299, 302 library management protocol (LMP) 300, 306 library manager 299, 301 library sharing, see Storage sharing life cycle management 296–298 management features 279–280 management interface 298 media management protocol (MMP) 299, 302–306 media manager 299, 290–291 audit 290 audit trail 280, 295 auditing 295 vaulting 291 monitoring 294 drive status 294 mount request 280 prioritization 293 queuing 287, 289–290 non-privileged clients 302 policy-based 297–298 pooling 292–293 privileged clients 302 problems and requirements 283 reporting 295–296 roles 287–288 application 288 library administrator 288 library operator 288 storage administrator 288 user 288 scratch pool 292–293 scratch tape 280 tape drive sharing, see Storage sharing tape library sharing, see Storage sharing tape quality management 294 task queue 301 trend analysis 295 usage statistics 295 vaulting 298 off-site location 298 398 Removable media management (continued ) policy-based 298 vaults 281 Replication 366 Representation server, see IT architecture, three tier architecture three tier architecture 196 Resource sharing, see Storage sharing RNIC, see Remote direct memory access (RDMA) Robots, see Removable media, media changer Roll forward 366 RS/6000 SP 126 S ID, see Fibre Channel SAFS 366 SAN router 366 SAN, see storage networks 366 SATA 17, 52, 366 Scalability, see Adaptability Scratch pool, see Removable media management 366 SCSI 1, 2, 14, 17, 19, 52–56, 214, 219, 221, 367 3rd-party SCSI copy command 221, 353 arbitration 53 daisy chain 52 enclosure services (SES) 253, 367 Fibre Channel-to-SCSI bridges 85 link extender 55 LUN 53, 176, 362 over Fibre Channel, see Fibre Channel, FCP priority of the target IDs 53 SCSI bus 52 SCSI protocol 52 storage networks with SCSI 54–56 target 370 target ID 53, 176, 370 twin-tailed cabling 54, 371 XCOPY 372 SDP 92, 101, 109, 120, 124, 367 Sequence, see Fibre Channel Sequential access block layer, see SNIA shared storage model Server availability, see Availability cluster, see Cluster consolidation 367 upgrade 4–7 INDEX Server-centric IT architecture, see IT architecture Service subsystem, see SNIA shared storage model SES, see SCSI, enclosure services (SES) Shared disk file systems, see File systems Shared memory 218, 219 Shared storage environment 367 Shared-everything cluster, see Cluster Shared-nothing cluster, see Cluster Shared-null configuration, see Cluster Side, see Removable media Signal encoding 62 Single point of failure 45, 181, 368 Skew 60, 368 Slot, see Removable media, library SMI-S, see CIM Snapshot, see File systems SNIA 368 SNIA shared storage model 309–352 applications 312 block layer 311, 316, 355 extent aggregation layer 339 file/record layer 311, 314, 358 host computer 313 interconnection network 312 logical storage resource 313 physical storage resource 313 sequential access block layer 338 services subsystem 311, 324, 367 storage device 313 storage management functions 313 tape applications 339 tape device 338 tape extent 336 tape extent separator 337 tape format system 339 tape header 337 tape media 338 tape trailer 337 SNMP 259–264, 368 agent 261 architecture 260 community name 263, 356 fabric element MIB 264 Fibre Channel management MIB 264 managed device 260 managed object 260 INDEX management information base (MIB) 260, 363 MIB file 363 MIB hierarchy 260 Network Management System (NMS) 261, 363 object identifier 260 operations 262 security 263 trap 371 trap recipient 264, 371 traps 264 Socket direct protocol (SDP), see SDP Soft zoning, see Fibre Channel SoIP, see IP storage, Storage over IP (SoIP) SRM 369 SSA 17, 52, 56, 369 SSP 369 Storage external 4, 358 flexible assignment 3, 15, 147, 170 internal 4, 360 logical 144 maximum capacity physical 144 replacement of storage devices 147 virtual 144 Storage area network (SAN), see Storage networks Storage consolidation 3, 369 Storage gateway 369 Storage hierarchy, see Back-up Storage management, see Management of storage networks Storage networks 3–7, 15, 165–169, 215, 369 application protocols 167 block-oriented 167 file-oriented 167 definition of terms 167–168 dual SAN, see Availability, dual SAN Fibre Channel SAN 358 iSCSI SAN 361 layers 166 management, see Management of storage networks requirements 56 SAN 56 terminology ‘SAN’ 56 399 transmission techniques 166, 167 transport protocols 166, 167 with SCSI 54–56 Storage over IP, see IP storage, storage over IP (SoIP) Storage pooling, see Storage sharing Storage sharing 169–175 data copying 172–173, 357 data sharing 149, 172–175, 357 drive sharing 286 library partitioning 170 library sharing 170–172, 223, 284, 286 NDMP 234 limitations 140–141, 170 real time data sharing 173–175, 366 storage pooling 15, 169–170 Storage virtualization 134, 144–146, 369 asymmetric 155, 354 data migration 148 data protection 149 definition 144 implementation 145 in storage devices 153–154 in the network 154–155 in the server 153 in-band virtualization, see Storage virtualization, asymmetric metadata controller (MDC) 155, 362 objectives 144 on block level 149–150 on file level 150 out-band virtualization, see Storage virtualization, asymmetric profile-orientation 143, 146 removable media 298 policy-based 146 symmetric 155, 370 tape library 299 virtualization entity 145 Storage-centric IT architecture, see IT architecture Streaming, see Removable media, tape Striping, see RAID, RAID (striping) Support matrix, see Interoperability Switch 370 Switched hub, see Fibre Channel 370 Symmetric storage virtualization, see Storage virtualization System bus, see I/O techniques 400 Tape changing mechanism, see Removable media, media changer Tape drive, see Removable media, drive Tape library, see Removable media, library Tape mount, see Removable media Tape reclamation, see Back-up Tape, see Removable media Target ID, see SCSI TCA, see InfiniBand, target channel adapter (TCA) TCP/IP 52, 119 CPU load 100 socket direct protocol (SDP), see SDP virtual IP address 179 TCP/IP offload engine (TOE) 100, 108, 120, 214, 216, 371 iSCSI offload engine 92 Telnet, see Management of storage networks Three tier architecture, see IT architecture 371 Three-way mirror, see RAID TOE, see TCP/IP offload engine Transaction, see Databases Translated Loop, see Fibre Channel Trap recipient, see SNMP Trap, see SNMP Twin-tailed SCSI cabling, see SCSI ULP, see Fibre Channel Unmanaged hub, see Fibre Channel USB 52 User profile 196 Vault management, see Removable media management, vaulting Vaulting, see Removable media management VI NIC, see Virtual Interface Architecture (VIA), network interface controller (NIC) VI, see Virtual Interface Architecture (VIA), virtual interface (VI) Virtual Interface Architecture (VIA) 52, 106–109, 371 completion queue 107 consumer 107 doorbell 106, 108 Ethernet 106 Fibre Channel 77, 106 InfiniBand 106 interoperability 91 INDEX kernel agent 107 network interface controller (NIC) 107, 371 RDMA, see Remote direct memory access (RDMA) receive queue 108 send queue 108 user agent 107 VI provider 107 virtual interface (VI) 106, 371 work queue pair 108 Virtual IP address 179 Virtual SAN, see Fibre Channel VSAN Virtual shared disk (VSD) 126 Virtualization in the I/O path 135–139 Virtualization, see Storage virtualization Voice networks 168 Voice over IP (VoIP) 98, 169, 371 Volume level 371 Volume manager 113, 136, 371 Volume, see Removable media VSAN, see Fibre Channel VSD 126 WAN 372 WBEM, see CIM Web architecture, see IT architecture Web Flash Archive (Solaris) 228 World Wide Web (WWW) 117, 194 web applications 195–203 web architecture, see IT architecture web server 196 Write penalty, see RAID WWN zoning, see Fibre Channel WWN, see Fibre Channel WWNN, see Fibre Channel WWPN, see Fibre Channel XCOPY, see SCSI Zoning, see Fibre Channel .. .Storage Networks Explained Basics and Application of Fibre Channel SAN, NAS, iSCSI and InfiniBand Ulf Troppens, Rainer Erkens IBM TotalStorage Interoperability Center,... components for Fibre Channel SAN 3.4.5 InterSANs 3.4.6 Interoperability of Fibre Channel SAN 3.5 IP Storage 3.5.1 IP storage standards: iSCSI, iFCP, mFCP, FCIP and iSNS 3.5.2 TCP/IP and Ethernet... Explained Basics and Application of Fibre Channel SAN, NAS, iSCSI and InfiniBand Ulf Troppens, Rainer Erkens IBM TotalStorage Interoperability Center, Mainz, Germany Wolfgang Muller ă IBM Software