Book The Grid Core Technologies
TEAM LinG The Grid The Grid Core Technologies Maozhen Li Brunel University, UK Mark Baker University of Portsmouth, UK Copyright © 2005 John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England Telephone (+44) 1243 779777 Email (for orders and customer service enquiries): cs-books@wiley.co.uk Visit our Home Page on www.wiley.com All Rights Reserved No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1T 4LP, UK, without the permission in writing of the Publisher Requests to the Publisher should be addressed to the Permissions Department, John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England, or emailed to permreq@wiley.co.uk, or faxed to +44 1243 770620 Designations used by companies to distinguish their products are often claimed as trademarks All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners The Publisher is not associated with any product or vendor mentioned in this book This publication is designed to provide accurate and authoritative information in regard to the subject matter covered It is sold on the understanding that the Publisher is not engaged in rendering professional services If professional advice or other expert assistance is required, the services of a competent professional should be sought Other Wiley Editorial Offices John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA Jossey-Bass, 989 Market Street, San Francisco, CA 94103-1741, USA Wiley-VCH Verlag GmbH, Boschstr 12, D-69469 Weinheim, Germany John Wiley & Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australia John Wiley & Sons (Asia) Pte Ltd, Clementi Loop #02-01, Jin Xing Distripark, Singapore 129809 John Wiley & Sons Canada Ltd, 22 Worcester Road, Etobicoke, Ontario, Canada M9W 1L1 Wiley also publishes its books in a variety of electronic formats Some content that appears in print may not be available in electronic books Library of Congress Cataloging in Publication Data Li, Maozhen Core technologies / Maozhen Li, Mark Baker p cm ISBN-13 978-0-470-09417-4 (PB) ISBN-10 0-470-09417-6 (PB) Computational grids (Computer systems) Electronic data processing—Distributed processing I Baker, Mark II Title QA76.9.C58L5 2005 005.3 6—dc22 2005002378 British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library ISBN-13 978-0-470-09417-4 (PB) ISBN-10 0-470-09417-6 (PB) Typeset in 11/13pt Palatino by Integra Software Services Pvt Ltd, Pondicherry, India Printed and bound in Great Britain by Antony Rowe Ltd, Chippenham, Wiltshire This book is printed on acid-free paper responsibly manufactured from sustainable forestry in which at least two trees are planted for each one used for paper production Contents About the Authors xiii Preface xv Acknowledgements xix List of Abbreviations xxi An 1.1 1.2 1.3 1.4 1.5 Introduction to the Grid Introduction Characterization of the Grid Grid-Related Standards Bodies The Architecture of the Grid References Part One System Infrastructure OGSA and WSRF Learning Objectives Chapter Outline 2.1 Introduction 2.2 Traditional Paradigms for Distributed Computing 2.2.1 Socket programming 2.2.2 RPC 2.2.3 Java RMI 2.2.4 DCOM 2.2.5 CORBA 2.2.6 A summary on Java RMI, DCOM and CORBA 2.3 Web Services 2.3.1 SOAP 2.3.2 WSDL 2.3.3 UDDI 2.3.4 WS-Inspection 2.3.5 WS-Inspection and UDDI 2.3.6 Web services implementations 2.3.7 How Web services benefit the Grid 1 11 11 11 12 13 14 15 16 18 19 20 21 23 24 26 27 28 29 33 CONTENTS vi 2.4 OGSA 2.4.1 Service instance semantics 2.4.2 Service data semantics 2.4.3 OGSA portTypes 2.4.4 A further discussion on OGSA 2.5 The Globus Toolkit (GT3) 2.5.1 Host environment 2.5.2 Web services engine 2.5.3 Grid services container 2.5.4 GT3 core services 2.5.5 GT3 base services 2.5.6 The GT3 programming model 2.6 OGSA-DAI 2.6.1 OGSA-DAI portTypes 2.6.2 OGSA-DAI functionality 2.6.3 Services interaction in the OGSA-DAI 2.6.4 OGSA-DAI and DAIS 2.7 WSRF 2.7.1 An introduction to WSRF 2.7.2 WSRF and OGSI/GT3 2.7.3 WSRF and OGSA 2.7.4 A summary of WSRF 2.8 Chapter Summary 2.9 Further Reading and Testing 2.10 Key Points 2.11 References The Semantic Grid and Autonomic Computing Learning Outcomes Chapter Outline 3.1 Introduction 3.2 Metadata and Ontology in the Semantic Web 3.2.1 RDF 3.2.2 Ontology languages 3.2.3 Ontology editors 3.2.4 A summary of Web ontology languages 3.3 Semantic Web Services 3.3.1 DAML-S 3.3.2 OWL-S 3.4 A Layered Structure of the Semantic Grid 3.5 Semantic Grid Activities 3.5.1 Ontology-based Grid resource matching 3.5.2 Semantic workflow registration and discovery in myGrid 3.5.3 Semantic workflow enactment in Geodise 3.5.4 Semantic service annotation and adaptation in ICENI 3.5.5 PortalLab – A Semantic Grid portal toolkit 3.5.6 Data provenance on the Grid 3.5.7 A summary on the Semantic Grid 34 35 37 38 40 40 41 42 42 43 44 50 53 54 56 58 59 60 60 66 69 70 70 72 72 73 77 77 77 78 79 81 83 87 88 88 89 90 91 92 93 94 95 98 99 106 107 CONTENTS vii 3.6 Autonomic Computing 3.6.1 What is autonomic computing? 3.6.2 Features of autonomic computing systems 3.6.3 Autonomic computing projects 3.6.4 A vision of autonomic Grid services 3.7 Chapter Summary 3.8 Further Reading and Testing 3.9 Key Points 3.10 References Part Two Basic Services 108 108 109 110 113 114 115 116 116 121 Grid Security 4.1 Introduction 4.2 A Brief Security Primer 4.3 Cryptography 4.3.1 Introduction 4.3.2 Symmetric cryptosystems 4.3.3 Asymmetric cryptosystems 4.3.4 Digital signatures 4.3.5 Public-key certificate 4.3.6 Certification Authority (CA) 4.3.7 Firewalls 4.4 Grid Security 4.4.1 The Grid Security Infrastructure (GSI) 4.4.2 Authorization modes in GSI 4.5 Putting it all Together 4.5.1 Getting an e-Science certificate 4.5.2 Managing credentials in Globus 4.5.3 Generate a client proxy 4.5.4 Firewall traversal 4.6 Possible Vulnerabilities 4.6.1 Authentication 4.6.2 Proxies 4.6.3 Authorization 4.7 Summary 4.8 Acknowledgements 4.9 Further Reading 4.10 References 123 123 124 127 127 128 129 130 130 132 133 134 134 136 140 140 146 148 148 149 149 149 150 151 151 151 152 Grid Monitoring 5.1 Introduction 5.2 Grid Monitoring Architecture (GMA) 5.2.1 Consumer 5.2.2 The Directory Service 5.2.3 Producers 5.2.4 Monitoring data 153 153 154 155 156 157 159 viii 5.3 Review Criteria 5.3.1 Scalable wide-area monitoring 5.3.2 Resource monitoring 5.3.3 Cross-API monitoring 5.3.4 Homogeneous data presentation 5.3.5 Information searching 5.3.6 Run-time extensibility 5.3.7 Filtering/fusing of data 5.3.8 Open and standard protocols 5.3.9 Security 5.3.10 Software availability and dependencies 5.3.11 Projects that are active and supported; plus licensing 5.4 An Overview of Grid Monitoring Systems 5.4.1 Autopilot 5.4.2 Control and Observation in Distributed Environments (CODE) 5.4.3 GridICE 5.4.4 Grid Portals Information Repository (GPIR) 5.4.5 GridRM 5.4.6 Hawkeye 5.4.7 Java Agents for Monitoring and Management (JAMM) 5.4.8 MapCenter 5.4.9 Monitoring and Discovery Service (MDS3) 5.4.10 Mercury 5.4.11 Network Weather Service 5.4.12 The Relational Grid Monitoring Architecture (R-GMA) 5.4.13 visPerf 5.5 Other Monitoring Systems 5.5.1 Ganglia 5.5.2 GridMon 5.5.3 GRM/PROVE 5.5.4 Nagios 5.5.5 NetLogger 5.5.6 SCALEA-G 5.6 Summary 5.6.1 Resource categories 5.6.2 Native agents 5.6.3 Architecture 5.6.4 Interoperability 5.6.5 Homogeneous data presentation 5.6.6 Intrusiveness of monitoring 5.6.7 Information searching and retrieval 5.7 Chapter Summary 5.8 Further Reading and Testing 5.9 Key Points 5.10 References CONTENTS 161 161 161 161 162 162 162 163 163 163 163 163 164 164 168 172 176 180 185 189 192 196 201 205 209 214 217 217 219 220 221 222 223 225 225 225 226 226 226 227 231 233 236 236 236 CONTENTS Part Three ix Job Management and User Interaction 241 Grid Scheduling and Resource Management Learning Objectives Chapter Outline 6.1 Introduction 6.2 Scheduling Paradigms 6.2.1 Centralized scheduling 6.2.2 Distributed scheduling 6.2.3 Hierarchical scheduling 6.3 How Scheduling Works 6.3.1 Resource discovery 6.3.2 Resource selection 6.3.3 Schedule generation 6.3.4 Job execution 6.4 A Review of Condor, SGE, PBS and LSF 6.4.1 Condor 6.4.2 Sun Grid Engine 6.4.3 The Portable Batch System (PBS) 6.4.4 LSF 6.4.5 A comparison of Condor, SGE, PBS and LSF 6.5 Grid Scheduling with QoS 6.5.1 AppLeS 6.5.2 Scheduling in GrADS 6.5.3 Nimrod/G 6.5.4 Rescheduling 6.5.5 Scheduling with heuristics 6.6 Chapter Summary 6.7 Further Reading and Testing 6.8 Key Points 6.9 References 243 243 243 244 245 245 246 248 248 248 251 251 254 254 254 269 274 279 288 290 291 293 293 295 296 297 298 298 299 Workflow Management for the Grid Learning Outcomes Chapter Outline 7.1 Introduction 7.2 The Workflow Management Coalition 7.2.1 The workflow enactment service 7.2.2 The workflow engine 7.2.3 WfMC interfaces 7.2.4 Other components in the WfMC reference model 7.2.5 A summary of WfMC reference model 7.3 Web Services-Oriented Flow Languages 7.3.1 XLANG 7.3.2 Web services flow language 7.3.3 WSCI 7.3.4 BPEL4WS 7.3.5 BPML 7.3.6 A summary of Web services flow languages 301 301 301 302 303 305 306 308 309 310 310 311 311 313 315 317 318 ... learning about the core technologies that make up the Grid today The material being developed for the companion Web site will supplement the book? ??s content We intend that the book, along with... Organization of the book PREFACE xvii ORGANIZATION OF THE BOOK The organization of the book is shown in Figure P.P.1 We have organized the book into four general parts, which reflect the bottom-up... better than another; rather its purpose is to expose the reader to a wide variety of what we call core technologies so that they can determine which is best for their own use This book is intended