Book The Grid Core Technologies - Preface

Preface Grid technologies and the associated applications are currently of unprecedented interest and importance to a variety of commu- nities. This book aims to outline and describe all of the compo- nents that are currently needed to create a Grid infrastructure that can support a range of wide-area distributed applications. In this book we take a pragmatic approach to presenting the material; we attempt not only to describe a particular component, but also to give practical examples of how that software may be used in context. We also intend to ensure that the companion Web site has extensive material that can be used by not only novices, but experienced practitioners too, to learn or gather technical material that can help in the process of understanding and using various Grid components and tools. PURPOSE AND READERSHIP The purpose of this book is not to convince the reader that one framework, technology or specification is better than another; rather its purpose is to expose the reader to a wide variety of what we call core technologies so that they can determine which is best for their own use. This book is intended for postgraduate students and researchers from various fields who are interested in learning about the core technologies that make up the Grid today. The material being developed for the companion Web site will supplement the book’s content. We intend that the book, along with Web content, will provide sufficient material to allow a complete self-study course of all the components addressed. The book takes a bottom-up approach, addressing lower-level components first, then mid-level frameworks and systems, and then finally higher-level concepts, concluding by outlining a number of xvi PREFACE representative Grid applications that provide examples of how the aforementioned frameworks and components are used in practice. We cover the core technologies currently in Grid environments to a sufficient depth that readers will be prepared to take on research papers and other related literature. In fact, there is often sufficient depth that a reader may use the book as a reference of how to get started with a particular Grid component. The subject material should be accessible to postgraduates and researchers who have a limited knowledge about the Grid, but technically have some knowledge about distributed systems, and experience in programming with C or Java. 2. OGSA and WSRF 3. The Semantic Grid and Autonomic Computing 4. Grid Security 5. Grid Monitoring 6. Grid Scheduling and Resource Management 7. Workflow Management for the Grid 8. Grid Portals 9. Grid Applications – Case Studies System Infrastructure Basic Services Job Management and User Interaction Applications Figure P.1 Organization of the book PREFACE xvii ORGANIZATION OF THE BOOK The organization of the book is shown in Figure P.P.1. We have organized the book into four general parts, which reflect the bottom-up view that we use to address the topics covered. We know that certain topics have been discussed under different parts, but we feel that this should assist the reader label topics more easily and hopefully help them get to grips with the content more easily. The first section, “system infrastructure”, contains the chap- ters that discuss and outline the current architecture, services and instantiations of the Grid. These chapters provide the underpin- ning information that the proceeding chapters build on. The sec- ond section, “basic services”, contains the chapters that describe Grid security and monitoring. Both these chapters explain services that do not actually need to exist to have a Grid environment, but without security and monitoring services it is impossible to have a secure, robust and reliable environment that can be used by higher- level services and applications. The third section we have labelled “Job management and User interaction”. At this level users have potentially direct access to tools and utilities that can change their working environment (in the case of a Portal), or manage and schedule their jobs (in the case of workflow and scheduling sys- tems). Finally, the last section of the book is called “Applications”; here we discuss a number of representative Grid-based applica- tions that highlight the technologies and components discussed in the earlier chapters of the book. Acknowledgements This first edition of our textbook was prepared during mid–late 2004, when the Grid-based technologies were not only at an embry- onic stage, but also in a great state of flux. With any effort, such as writing a book, nothing would really be accomplished in a timely fashion without the aid of a large number of willing helpers and volunteers. The technology landscape that we have been writing about is changing rapidly, so we sought and asked experts in var- ious fields to read through and comment on all parts of the book. We would like to thank the following people for reviewing parts of the book: • Chapter 2 – OGSA and WSRF: Stephen Pickles and Mark McKeown (Manchester Computing, University of Manchester) and Helen Xiang (DSG, University of Portsmouth). • Chapter 3 – The Semantic Grid and Autonomic Computing: Rich Boaks (DSG, University of Portsmouth) and Manish Parashar (Rutgers, The State University of New Jersey, USA). • Chapter 4 – Grid Security: Alistair Mills (Grid Deployment Group, CERN). • Chapter 5 – Grid Monitoring: A special thank you to Garry Smith (DSG, University of Portsmouth), who provided a lot of detailed content for this chapter, and still managed to write and submit his PhD. • Chapter 6 – Grid Scheduling and Resource Management: NG1 – Fritz Ferstl (Sun Microsystems), Condor – Todd Tannen- baum (Condor project, University of Wisconsin, USA), LSF – Songnian Zhou (Platform Computing Inc, Canada), PBS – Bob Henderson (Altair Grid Technologies, USA). • Chapter 7 – Workflow Management for the Grid: Omer Rana (Cardiff University). xx ACKNOWLEDGEMENTS • Chapter 8 – Grid Portals: Rob Allan (Daresbury Laboratory). • Chapter 9 – Grid Applications – Case Studies: Rob Allan (Dares- bury Laboratory). We like to make a special mention of and an acknowledge- ment to Rob Allan (Daresbury Laboratory, UK), who meticulously reviewed the book as a whole and fed back many useful comments about its presentation and content. We would like to say a special thanks to Birgit Gruber, our Wiley editor, who worked closely with us through the production of the book, and generally made the effort involved a pleasant one. COMPANION WEB SITE We have set up a Web site (coregridtechnologies.org) containing companion material to the book that will assist readers and teach- ers. The amount of content will grow with time and eventually include: • Tables and figures from the book in various formats • Slides of the content • Notes highlighting various aspects of the content • Links and references to companion material • Laboratory exercises and solutions • Source code for examples • Potential audio/visual material. Obviously, from the inception of book to its publication and distri- bution, the landscape that we describe will have undulated some more, so the book is a snapshot of the technologies during mid– late 2004. We believe that we can overcome some of the gaps that may appear in the book’s coverage of material by adding the appropriate content to the companion Web site. List of Abbreviations Abbreviation Expanded form Context ACL Access Control List AM Actuator Manager CODE AMUSE Autonomic Management of Ubiquitous Systems for e-Health AppLeS Application Level Scheduler APST AppLeS Parameter Sweep Template AppLeS ASA Autonomic Storage Architecture ASAP Asynchronous Service Access Protocol ASP Active Server Pages Microsoft .Net BLAST Basic Local Alignment Search Tool BPEL4WS Business Process Execution Language for Web Services BPML Business Process Modelling Language CA Certification Authority CCA Common Component Architecture CFD Computational Fluid Dynamics CGI Common Gateway Interface CIM Common Information Model ClassAd Classified Advertisement Condor CMS Compact Muon Solenoid COD Computing On Demand Condor CODE Control and Observation in Distributed Environments xxii LIST OF ABBREVIATIONS CORBA Common Object Request Broker Architecture OMG CPS Certificate Policy Statement CSF Community Scheduler Framework CSR Certificate Signing Request DA Data Analyser GridICE DAG Directed Acyclic Graph Condor DAGMan Directed Acyclic Graph Manager Condor DAIS Database Access and Integration Services DAISGR DAI Service Group Registry OGSA-DAI DAML DARPA Agent Markup Language DAML-S DAML Services DAML DCE Distributed Computing Environment DCOM Distributed Component Object Model Microsoft DCS Data Collector Service GridICE DES Data Encryption Standard DL Description Logic DMTF Distributed Management Task Force DNS Detection and Notification Service GridICE DPML Discovery Process Markup Language Discovery Net DSP Distributed Systems Group GridRM DTD Document Type Definition W3C EC Event Consumer JAMM ECS Element Construction Set EDG European Data Grid EDSO Engineering Design Search and Optimization Geodise EG Event Gateway JAMM EGEE Enabling Grids for E-science in Europe EJB Enterprise JavaBeans J2EE FaCT Fast Classification of Terminologies FIFO First In First Out LIST OF ABBREVIATIONS xxiii GA Genetic Algorithm GAC GPIR Admin Client GPIR GALE Grid Access Language for high-performance computing Environments GAR Grid Archive GT3 GARA GARA GDS Grid Data Service OGSA-DAI GDSF Grid Data Service Factory OGSA-DAI GDS-Perform Grid Data Service Perform OGSA-DAI GDS-Response Grid Data Service Response OGSA-DAI GEMLCA Grid Execution Management for Legacy Code Architecture Geodise Grid Enabled Optimization and DesIgn Search for Engineering GGF Global Grid Forum GIP GPIR Information Provider GPIR GIS Grid Information Services GMA Grid Monitoring Architecture GGF GPDK Grid Portal Development Kit GPIR Grid Portals Information Repository GRAAP-WG Grid Resource Allocation Agreement Protocol Working Group GrADS Grid Application Development Software GRAM Globus Resource Allocation Manager Globus GridFTP Grid File Transfer Protocol Globus GRIM Grid Resource Identity Mapper GT3 GSA Grid System Agent PortalLab GSFL Grid Services Flow Language GSH Grid Service Handle OGSA GSI Grid Security Infrastructure Globus GSR Grid Service Reference OGSA GSSAPI Generic Security Services Application Programming Interface GSSAPI xxiv LIST OF ABBREVIATIONS GT2 Globus Toolkit 2 Globus GT3 Globus Toolkit 3 Globus GT4 Globus Toolkit 4 Globus GUSTO Generic Ultraviolet Sensors Technologies and Observations GWEL Grid Workflow Execution Language GWSDL Grid WSDL GT3 HAT Heterogeneous Application Template AppLeS HM Hawkeye Manager Condor HMA Hawkeye Monitoring Agent Condor HPSS High-Performance Storage System ICENI Imperial College e-Science Networked Infrastructure ICENI IDL Interface Definition Language IPG Information Power Grid IIOP Internet-Inter ORB Protocol CORBA ISAPI Internet Server Application Programming Interface Microsoft .Net J2EE Java 2 Enterprise Edition J2SE Java 2 Standard Edition JAMM Java Agents for Monitoring and Management JAR Java Archive Java Java CoG Java Commodity Grid JAXB Java Architecture for XML Binding J2EE JAXM Java API for XML Messaging J2EE JAXP Java API for XML Processing J2EE JAXR Java API for XML Registries J2EE JAX-RPC Java API for XML-Based RPC J2EE JCE Java Cryptography Extension JCP Java Community Process JCR Java Certificate Request JNDI Java Native Directory Interface JISGA Jini-based Service-Oriented Grid Architecture JRE Java Run time Environment [...]... Measurement Service Microsoft XML Parser Model-View-Controller N1 Grid Engine Networked Application Logger NSF Middleware Initiative Naming Schema Network Weather Service Organization for the Advancement of Structured Information Standards Open Content Syndication Open Grid Computing Environments Open Grid Services Architecture OGSA Data Integration and Access Open Grid Services Infrastructure OGSI Working... GT3 GridICE Microsoft Net GridRM GGF GGF GGF GGF xxvi OLE OMG ONC ORPC OSF OWL OWL-S P2P PASOA PBS PDSR PGT3 PI PIF PII PImR PInR PKI PMA portType PS PUB PSL PSML PVM PWG RBAC RDF RDFS RMI RPC RSA RSL RSS RFT LIST OF ABBREVIATIONS Object Linking and Embedding Object Management Group Open Network Computing Object Remote Procedure Call Open Software Foundation Web Ontology Language OWL Services Peer-to-Peer... MJS MMJFS MPI MS MSXML MVC N1GE NetLogger NMI NS NWS OASIS OCS OGCE OGSA OGSA-DAI OGSI OGSI-WG OIL Java Remote Method Protocol Java Server Page Java Specification Requests LHC Computing Grid Legacy Code Interface Description Lightweight Directory Access Protocol Local Managed Job Factory Service Load Sharing Facility Message Authentication Code Machine Check Architecture Monitoring and Discovery Service... Portlet Wrapper Generator Role-Based Access Control Resource Description Framework RDF Schema Remote Method Invocation Remote Procedure Call Rivest, Shamir and Adleman Resource Specification Language Really Simple Syndication Reliable File Transfer DCOM W3C OWL PortalLab LSF CODE PortalLab PortalLab JAMM WSDL GridICE GridICE PortalLab W3C W3C Java Globus GT3 LIST OF ABBREVIATIONS RUS-WG SA SARS SD SDDF SDE... WfMC WFMS WML WPDL WS Resource Usage Services Working Group Simulated Annealing Severe Acute Respiratory Syndrome Sensor Directory Self-Defining Data Format Service Data Element Sun Grid Engine Semantic Grid Portlet Sensor Manager Simple Network Management Protocol Service-Oriented Architecture Semantic Portlet Matcher Semantic Portlet Register Storage Resource Broker Secure Sockets Layer Simple Workflow... Workflow Language Transport Layer Security User Agent Universal Description, Discovery and Integration User-Hosting Environment Universal Logger Message Uniform Resource Identifier Virtual Organization The Visual Portal Composition Environment World Wide Web Consortium Web Application Repository Web-Based Enterprise Management Williams–Beuren Syndrome Workflow Management Coalition Workflow Management System... Services Description Language Web Services Invocation Framework WS-Inspection Language Web Services Meta Language Web Services Resource Framework Web Services for Remote Portlets External Data Representation eXtensible Markup Language XML Process Definition Language XML Schema Definition eXtensible Stylesheet Language XSL Transformation Yet Another Workflow Language WS W3C Microsoft Net OASIS W3C WfMC W3C... ABBREVIATIONS Object Linking and Embedding Object Management Group Open Network Computing Object Remote Procedure Call Open Software Foundation Web Ontology Language OWL Services Peer-to-Peer Provenance-Aware ServiceOriented Architecture Portable Batch System Portlet Domain Service Repository Platform Globus Toolkit 3.0 Producer Interface Process Interchange Format Portlet Invocation Interface Portlet . complete self-study course of all the components addressed. The book takes a bottom-up approach, addressing lower-level components first, then mid-level frameworks. Organization of the book PREFACE xvii ORGANIZATION OF THE BOOK The organization of the book is shown in Figure P.P.1. We have organized the book into four general