Grid Portals

8 Grid Portals LEARNING OUTCOMES In this chapter, we will study Grid portals, which are Web-based facilities that provide a personalized, single point of access to Grid resources that support the end-user in one or more tasks. From this chapter, you will learn: • What is a Grid portal and what kind of roles will it play in the Grid? • First-generation Grid portals. • Second-generation Grid portals. • The features and limitations of first-generation Grid portals. • The features and benefits of second-generation Grid portals. CHAPTER OUTLINE 8.1 Introduction 8.2 First-Generation Grid Portals 8.3 Second-Generation Grid Portals 8.4 Chapter Summary 8.5 Further Reading and Testing The Grid: Core Technologies Maozhen Li and Mark Baker © 2005 John Wiley & Sons, Ltd 336 GRID PORTALS 8.1 INTRODUCTION The Grid couples geographically dispersed and distributed het- erogeneous resources to provide various services to users. We can consider two main types of Grid users: system developers and end users. System developers are those who build Grid systems using middleware packages such as Globus [1], UNICORE [2] or Condor [3]. The end users are the scientists and engineers who use the Grid to solve their domain-specific problems perhaps via a portal. A Grid portal is a Web-based gateway that provides seamless access to a variety of backend resources. In general, a Grid portal provides end users with a customized view of software and hard- ware resources specific to their particular problem domain. It also provides a single point of access to Grid-based resources that they have been authorized to use. This will allow scientists or engineers to focus on their problem area by making the Grid a transparent extension of their desktop computing environment. Grid portals currently in use include XCAT Science Portal [4], Mississippi Com- putational Web Portal [5], NPACI Hotpage [6], JiPANG [7], The DSG Portal [8], Gateway [9], Grappa [10] and ASC Grid Portal [11]. In this chapter, we will study Grid portals; the technologies they employ and the mechanisms that they use. So far, Grid portal development can be broadly classified into two generations. First- generation Grid portals are tightly coupled with Grid middleware such as Globus, mainly Globus toolkit version 2.x (GT2) written in C. The second generation of Grid portals are those that are starting to emerge and make use of technologies such as portlets to provide more customizable solutions. This chapter is organized as follows. In Section 8.2, we describe technologies involved in the development of first-generation Grid portals. We first present the three-tiered architecture adopted by most portals of this generation. We then introduce some tools that can provide assistance in the construction of these portals. Finally we give a summary on the limitations of first-generation Grid portals. In Section 8.3, we present the state-of-the-art development of second-generation Grid portals. We first introduce the concept of portlets and describe why they are so important for building personalized portals. We then give three portal frameworks that can be used to develop and deploy portlets. We conclude the chapter in Section 8.4 and provide further reading material about portals in Section 8.5. 8.2 FIRST-GENERATION GRID PORTALS 337 8.2 FIRST-GENERATION GRID PORTALS In this section, we will study the first-generation Grid portals from the points of view of architecture, services, implementation tech- niques and integrated tools. Most Grid portals currently in use belong to this category. 8.2.1 A three-tiered architecture The first generation of Grid portals mainly used a three-tier archi- tecture as shown in Figure 8.1. As stated in Gannon et al. [12], they share the following characteristics: • A three-tiered architecture, consisting of an interface tier of a Web browser, a middle tier of Web servers and a third tier of backend services and resources, such as databases, high- performance computers, disk storage and specialized devices. • A user makes a secure connection from their browser to a Web server. • The Web server then obtains a proxy credential from a proxy credential server and uses that to authenticate the user. Figure 8.1 The three-tiered architecture of first-generation Grid portals 338 GRID PORTALS • When the user completes defining the parameters of the task they want to execute, the portal Web server launches an appli- cation manager, which is a process that controls and monitors the actual execution of Grid task(s). • The Web server delegates the user’s proxy credential to the application manager so that it may act on the user’s behalf. In some systems, the application manager publishes an event/ message stream to a persistent event channel archive, which describes the state of an application’s execution and can be moni- tored by the user through their browser. 8.2.2 Grid portal services First-generation Grid portals generally provide the following Grid services. • Authentication: When users access the Grid via a portal, the portal can authenticate users with their usernames and passwords. Once authenticated, a user can request the portal to access Grid resources on the user’s behalf. • Job management: A portal provides users with the ability to manage their job tasks (serial or parallel), i.e. launching their applications via the Web browser in a reliable and secure way, monitoring the status of tasks and pausing or cancelling tasks if necessary. • Data transfer: A portal allows users to upload input data sets required by tasks that are to be executed on remote resources. Similarly the portal allows results sets and other data to be downloaded via a Web browser to a local desktop. • Information services: A portal uses discovery mechanisms to find the resources that are needed and available for a particular task. Information that can be collected about resources includes static and dynamic information such as OS or CPU type, current CPU load, free memory or file space and network status. In addition, other details such as job status and queue information can also be retrieved. 8.2 FIRST-GENERATION GRID PORTALS 339 8.2.3 First-generation Grid portal implementations Most portals of this generation have been implemented with the following technologies: • A dynamic Graphical User Interface (GUI) based on HTML pages, with JSP (Java Server Pages) or JavaScript. Common Gate- way Interface (CGI) and Perl are also used by some portals. CGI is an alternative to JSP for dynamically generating Web contents. • The secure connection from a browser to backend server is via Transport Layer Security (TLS) and Secure HTTP (S-HTTP). • Typically, a Java Servlet or JavaBean on the Web server handles requests from a user and accesses backend resources. • MyProxy [13] and GT2 GSI [14] are used for user authentication. MyProxy provides credential delegation in a secure manner. • GT2 GRAM [15] is used for job submission. • GT2 MDS [16] is used for gathering information on various resources. • GT2 GSIFTP [17] or GT2 GridFTP [18] for data transfer. • The Java CoG [19] provides the access to the corresponding Globus services for Java programs. The first-generation Grid portals mainly use the GT2 to provide Grid services. One main reason for this is that Globus provides a complete package and a standard way for building Grid-enabled services. 8.2.3.1 MyProxy MyProxy is an online credential management system for the Grid. It is used to delegate a user’s proxy credential to Grid portals, which can be authenticated to access Grid resources on the user’s behalf. Storing your Grid credentials in a MyProxy repository allows you to retrieve a proxy credential whenever and wherever you need one. You can also allow trusted servers to renew your proxy credentials using MyProxy, so, for example, long-running 340 GRID PORTALS Figure 8.2 The use of MyProxy with a Grid portal tasks do not fail due to an expired proxy credential. Figure 8.2 shows the steps to securely access the Grid via a Grid portal with MyProxy. 1. Execute myproxy_init command on the computer where your Grid credential is located to delegate a proxy credential on a MyProxy server. The delegated proxy credential normally has a lifetime of one week. The communication between the com- puter and the MyProxy server is securely managed by TLS. You need to supply a username and pass phrase for the identity of your Grid credential. Then you need to supply another different MyProxy pass phrase to secure the delegated proxy credential on the MyProxy server. 2. Log into the Grid portal with the same username and MyProxy pass phrase used for delegating the proxy credential. 3. The portal uses myproxy_get_delegation command to retrieve a delegated proxy credential from the MyProxy server using your username and MyProxy pass phrase. 4. The portal accesses Grid resources with the proxy credential on your behalf. 5. The operation of logging out of the portal will delete your del- egated proxy credential on the portal. If you forget to log off, then the proxy credential will expire at the lifetime specified. The detailed information about credentials and delegation can be found in Chapter 4, Grid Security. 8.2 FIRST-GENERATION GRID PORTALS 341 8.2.3.2 The Java CoG The Java Commodity Grid (CoG) Kit provides access to GT2 ser- vices through Java APIs. The goal of the Java CoG Kit is to provide Grid developers with the advantage to utilize much of the Globus functionality, as well as, access to the numerous additional libraries and frameworks developed by the Java community. Currently GT3 integrates part of Java CoG, e.g. many of the command-line tools in GT3 are implemented with the Java CoG. The Java CoG has been focused on client-side issues. Grid ser- vices that can be accessed by the toolkit include: • An information service compatible with the GT2 MDS imple- mented with Java Native Directory Interface JNDI [20]. • A security infrastructure compatible with the GT2 GSI imple- mented with the iaik security library [21]. • A data transfer mechanism compatible with a subset of the GT2 GridFTP and/or GSIFTP. • Resource management and job submission with the GT2 GRAM Gatekeeper. • Advanced reservation compatible with GT2 GARA [22]. • A MyProxy server managing user credentials. 8.2.4 First-generation Grid portal toolkits In this section, we introduce four representative Grid portal toolkits: GridPort 2.0, GPDK, the Ninf Portal and GridSpeed. These toolkits provide some sort of assistance in constructing the first- generation Grid portals. 8.2.4.1 GridPort 2.0 The GridPort 2.0 (GP2) [23] is a Perl-based Grid portal toolkit. The purpose of GP2 is to facilitate the easy development of application- specific portals. GP2 is a collection of services, scripts and tools, where the services allow developers to connect Web-based inter- faces to backend Grid services. The scripts and tools provide consistent interfaces between the underlying infrastructure, which are based on Grid technologies, such as GT2, and standard Web 342 GRID PORTALS Figure 8.3 The architecture of GP2 technologies, such as CGI. Figure 8.3 shows the architecture of GP2. Its components are described below. Client layer The client layer represents the consumers of Grid portals, typically Web browsers, PDAs or even applications capable of pulling data from a Web server. Clients interact with a GP2 portal via HTML- form elements and use secure HTTP to submit requests. Portal layer The portal layer consists of portal-specific codes. Application por- tals run on standard Web servers and handle client requests and provide responses to those requests. One instance of GP2 can sup- port multiple concurrent application portals, but they must exist on the same Web server where they share the same instance of the GP2 libraries. This allows the application portals to share portal-related user and account data and thereby makes possible a single-login environment. GP2 portals can also share libraries, file space and other services. Portal services layer GP2 and other portal toolkits or libraries reside at the portal ser- vices layer. GP2 performs common services for application portals including the management of session state, portal accounts and Grid information services with GT2 MDS. Grid services layer The Grid services layer consists of those software components and services that are needed to handle user requests to access the Grid. GP2 employs simple, reusable middleware technologies 8.2 FIRST-GENERATION GRID PORTALS 343 e.g. GT2 GRAM for job submission to remote resources; GT2 GSI and MyProxy for security and authentication; GT2 GridFTP and the San Diego Supercomputer Center (SDSC) Storage Resource Broker (SRB) for distributed file collection and management [24, 25]; and Grid Information Services based primarily on proprietary GP2 information provider scripts and the GT2 MDS. GP2 can be used in two ways. The first approach requires that GT2 be installed because GP2 scripts wrap the GT2 command line tools in the form of Perl scripts executed from cgi-bin. GT2 GRAM, GSIFTP, MyProxy are used to access backend Grid ser- vices. The second approach does not require GT2, but relies on the CGI scripts that have been configured to use a primary GP2 Portal as a proxy for accessing GP2 services, such as user authentication, job submission and file transfer. The second approach allows a user to quickly deploy a Web server configured with a set of GP2 CGI scripts to perform generic portal operations. 8.2.4.2 Grid Portal Development Kit (GPDK) GPDK [26] is another Grid portal toolkit that uses Java Server Pages (JSPs) for portal presentation and JavaBeans to access backend Grid resources via GT2. Beans in GPDK are mostly derived from the Java CoG kit. Figure 8.4 shows the architecture of GPDK. Grid service beans in GPDK can be classified as follows. These beans can be used for the implementation of Grid portals. Security The security bean, MyproxyBean, is responsible for obtaining dele- gated credentials from a MyProxy server. The MyproxyBean has a method for setting the username, password and designated lifetime of a delegated credential on the Web server. In addition, it allows delegated credentials to be uploaded securely to the Web server. User profiles User profiles are controlled by three beans: UserLoginBean, User- AdminBean and the UserProfileBean. • The UserLoginBean provides an optional service to authenticate users to a portal. Currently, it only sets a username/password 344 GRID PORTALS Figure 8.4 The GPDK architecture and checks a password file on the Web server to validate user access. • The UserAdminBean provides methods for serializing a UserPro- fileBean and validating a user’s profile. • The UserProfileBean maintains user information including preferences, credential information, submitted job history and computational resources used. The UserProfileBean is generally instantiated with session scope to persist for the duration of the user’s transactions on the portal. Job submission The JobBean contains all the necessary functions used in submitting a job including memory requirements, name of executable code, arguments, number of processors, maximum wall clock or CPU time and the submission queue. A JobBean is passed to a JobSub- missionBean that is responsible for actually launching the job. Two varieties of the JobSubmissionBean currently exist. The GramSub- missionBean submits a job to a GT2 GRAM gatekeeper which can either run the job interactively or submit it to a scheduling system if one exists. The JobInfoBean can be used to retrieve a job-related time-stamped information including the job ID, status and out- puts. The JobHistoryBean uses multiple JobInfo beans to provide a history of information about jobs that have been submitted. The history information can be stored in the user’s profile. [...]... first-generation portals can only provide static Grid services in that they lack a facility to easily expose newly created Grid services to users 350 GRID PORTALS While there are limitations with first-generation Grid portals and portal toolkits, the experiences and lessons learned in developing Grid portals at this stage have paved the way for the development of second-generation Grid portals 8.3 SECOND-GENERATION... building Grid portals It provides a Grid application portal-hosting server that automatically generates and publishes a customized Web interface for accessing the backend Grid services The main aim of GridSpeed is to hide the complexity of the underlying infrastructure from Grid users It allows developers to define and build their Grid application portals on the fly GridSpeed focuses on the generation of portals. .. define a portlet associated with a Grid service to be called a Grid Portlet Figure 8.18 shows how to access a Grid service from a Grid portal via a Grid Portlet The model is that a Grid Portlet interacts with a Grid service provided by Grid middleware such as Globus to access backend resources Since Grid services provided by difference service providers using different Grid middleware technologies can... services Restricted Grid services First-generation Grid portals are tightly coupled with specific Grid middleware technologies such as Globus, which results in restricted portal services It is hard to integrate Grid services provided by different Grid middleware technologies via a portal of this generation Static Grid services A Grid environment is dynamic in nature with more and more Grid services are... standard portlets, portals built from portlets are loosely coupled with Grid middleware technologies Portal frameworks such as Jetspeed, WebSphere Portal and GridSphere have been widely used for building Web portals with portlets They are being integrated with Grid services for constructing Grid portals with Grid Portlets Currently no such framework exists that can provide an IDE in which a Grid portal can... The development of Grid portals with portlets Portlet technology is gaining attention from the Grid community for building second-generation Grid portals to overcome problems encountered in first-generation Grid portal development frameworks and toolkits A portlet in a Grid portal is not just a normal portlet that can be plugged into a portal; it is also associated with a backend Grid service We define... application portal page GRID PORTALS 348 8.2.5 A summary of the four portal tools As shown in Table 8.1, the four toolkits provide various levels of support for portal developers to build Grid portals Apart from GP2, which uses HTML pages for the portal–user interface, the other three toolkits use JSP technology Grid portals can be grouped into two categories: user portals and application portals A user portal... second-generation Grid portals 8.3 SECOND-GENERATION GRID PORTALS In this section, we discuss the development of second-generation Grid portals To overcome the limitations of first-generation portals, portlets have been introduced and promoted for use in building second-generation Grid portals Currently, portlets are receiving increasing attention from both the Grid community and industry In this section we... 8.3 SECOND-GENERATION GRID PORTALS 367 368 GRID PORTALS 8.3.4 A Comparison of Jetspeed, WebSphere Portal and GridSphere IBM WebSphere Portal, Jetspeed and GridSphere are portal frameworks that can be used to build Web portals with portlets While they focus on portlets construction, they differ in their implementations and... needs Portals developed at this stage are not customizable by the users The GridSpeed development team is currently working on the issue 8.2.6 A summary of first-generation Grid portals First-generation Grid portals have been focused on providing basic task-oriented services, such as user authentication, job submission, monitoring and data transfer However, they are typically tightly coupled with Grid . play in the Grid? • First-generation Grid portals. • Second-generation Grid portals. • The features and limitations of first-generation Grid portals. • The. of second-generation Grid portals. CHAPTER OUTLINE 8.1 Introduction 8.2 First-Generation Grid Portals 8.3 Second-Generation Grid Portals 8.4 Chapter Summary