Tài liệu Grid Computing P25 pptx

25 Ninf-G: a GridRPC system on the Globus toolkit Hidemoto Nakada, 1,2 Yoshio Tanaka, 1 Satoshi Matsuoka, 2 and Satoshi Sekiguchi 1 1 National Institute of Advanced Industrial Science and Technology (AIST), Grid Technology Research Center, Tsukuba, Ibaraki, Japan, 2 Tokyo Institute of Technology, Global Scientific Information and Computing Center, Tokyo, Japan 25.1 INTRODUCTION Recent developments in high-speed networking enables collective use of globally dis- tributed computing resources as a huge single problem-solving environment, also known as the Grid. The Grid not only presents a new, more difficult degree of inherent challenges in distributed computing such as heterogeneity, security, and instability, but will also require the constituent software substrates to be seamlessly interoperable across the network. As such, software layers are constructed so that higher-level middleware sits on some common, lower-level software layer, just as it is with single-box computers in which most applications running on top of it share a common operating system and standard libraries. Grid Computing – Making the Global Infrastructure a Reality. Edited by F. Berman, A. Hey and G. Fox  2003 John Wiley & Sons, Ltd ISBN: 0-470-85319-0 626 HIDEMOTO NAKADA ET AL. Currently, the Globus Toolkit [1] serves as the ‘de facto standard’ lower-level substrate for the Grid. Globus provides important Grid features such as authentication, authoriza- tion, secure communication, directory services, and so on that the software above can utilize so as not to replicate the lower-level programming efforts, as well as provide interoperability between the middleware mentioned above. However, the Globus Toolkit alone is insufficient for programming on the Grid. The abstraction level of the Globus, being a lower-level substrate, is terse and primitive in a sense; this is fine for the intended use of the Toolkit, but nevertheless higher-level programming layers would be absolutely necessary for most users. This is analogous to the situation in programming, say, parallel programming on a Linux cluster; one would certainly program with higher-level parallel programming systems such as MPI and OpenMP, rather than by using the raw TCP/IP socket interface of the operating system. Over the years there have been several active research programs in programming mid- dlewares for the Grid environment, including MPICH-G [2] and Nimrod/G [3], Ninf [4], and NetSolve [5]. MPICH-G and MPICH-G2 are MPI systems achieving security and cross-organization MPI interoperability using Globus Toolkit, although as a programming model for the Grid the utility is somewhat less significant compared to tightly coupled environments. Nimrod-G is a high-throughput computing system implemented on Globus toolkit. It automatically invokes existing applications on many computing resources based on deadline scheduling. While the system is quite useful for some applications, the applica- tions area is largely limited to parameter-sweep applications and not general task-parallel programming. Ninf and NetSolve are implementations of the GridRPC [6] programming model, pro- viding simple yet powerful server-client-based framework for programming on the Grid. GridRPC facilitates an easy-to-use set of APIs, allowing easy construction of globally dis- tributed parallel applications without complex learning curves. Both systems have seen successful usages in various Grid application projects. On the other hand, Ninf and NetSolve themselves are mutually interoperable, but they do not interoperate well with other Grid tools on Globus such as GridFTP. The reason is that, because the first versions of both Ninf and NetSolve were essentially developed at the same time as Globus, both systems were built rather independently without fully utilizing the Globus features or taking into account their mutual interoperability. To resolve the situation, we redesigned the whole GridRPC system in collaboration with the NetSolve team at the University of Tennessee, Knoxville. The redesign has been extensive, including the software architecture to fully utilize the Globus features, user-level API specifications that retain the simplicity but generalize call contexts for more flexible Grid-level programming, various changes in the protocols, and so on. The redesign, especially the API specification, has been carefully done so that each group can independently produce respective implementations of GridRPC. Our implementation is Ninf-G, which is in effect a full reimplementation of Ninf on top of Globus. Compared to older implementations, Ninf-G fully utilizes and effectively sits on top of Globus. The result is that applications constructed with Ninf-G can take advantage of any modules implemented for the Globus Toolkit, such as special job managers or GridFTP servers. NINF-G: A GRIDRPC SYSTEM ON THE GLOBUS TOOLKIT 627 For the remainder of this chapter, we discuss the implementation of the Ninf-G system, and demonstrate the usage with a simple parallel programming example with the Ninf-G, along with its performance. In Section 25.2 we briefly introduce the Globus Toolkit; in Section 25.3 we discuss the overall design of GridRPC, and how each feature can be mapped on to lower-level Globus Toolkit features. Section 25.4 outlines the implementa- tion of the Ninf-G, while Sections 25.5 and 25.6 illustrate a typical usage scenario along with its performance evaluation. Section 25.7 will conclude and hint at future directions. 25.2 GLOBUS TOOLKIT The Globus Toolkit is a collection of modules that provides standardized lower-level features for implementing a distributed system on the Grid. Table 25.1 covers the services provided by Globus, in which each service can be used independently when needed. We give brief descriptions of the most relevant modules for GridRPC. GSI : Grid Security Infrastructure (GSI) serves as the common authentication facility underlying all the features of Globus. It enables single sign-on using certificate dele- gation based on PKI. GRAM : Globus Resource Allocation Manager (GRAM) is a ‘secure inetd’ that authenti- cates clients using GSI-based certificates, maps to the local user account, and invokes executable files. MDS : Monitoring and Discovering Service (MDS) is a directory service that provides resource information within the Grid. MDS consists of two layers of Lightweight Direc- tory Access Protocol (LDAP) servers, Grid Index Information Service (GIIS), which manages project-wide information, and Grid Resource Information Service (GRIS), which is responsible for site local information. Globus-I/O: Globus-I/O enables secure communication between Grid peers using GSI, providing standard read/write APIs, plus nonblocking I/O that integrates with the Globus Threads library. GASS : Global Access to Secondary Storage (GASS) provides easy-to-use file transfer facility across Grid peers. GASS can be used in conjunction with GRAM to stage client side files. Table 25.1 Globus services Service Module name Description Security GSI Single sign-on, authentication Information infrastructure MDS/GRIS/GIIS Information service Resource management GRAM/DUROC Computing resource management Data management GASS/Grid FTP Remote data access Communication Nexus/Globus-I/O Communication service Portability Portability library (libc, pthread) 628 HIDEMOTO NAKADA ET AL. 25.3 DESIGN OF NINF-G 25.3.1 GridRPC system GridRPC is a programming model based on client-server remote procedure call (RPC), with features added to allow easy programming and maintenance of code for scientific applications on the Grid. Application programmers write task-parallel client programs using simple and intuitive GridRPC APIs that hide most of the complexities involving Grid programming. As a result, programmers lacking experience on parallel programming, let alone the Grid, could still construct Grid applications effortlessly. At the server side, computation routines are encapsulated into an executable component called the Remote Executable. Client program basically invokes Remote Executables to request computation to be done on the server; asynchronous parallel executions of multiple Remote Executables on different servers result in simple fork-join parallel execution, while nested GridRPC invocations with complex synchronizations resulting in generalized task- parallel computations are also possible if the programmers have control over the Remote Executables on all the constituent nodes. GridRPC system generally consists of the following four components: Client component: Client components are programs that issue requests for GridRPC invo- cations. Each component consists of the user’s main program and the GridRPC library. Server component: Server component invokes Remote Executables as described below, Remote executable: Remote Executables perform the actual computation at the Server. Each component consists of user-supplied server-side compute routine, system gener- ated stub main program, and system supplied communication library. Information service: Information service provides various information for the client com- ponent to invoke and to communicate with the Remote Executable component. 25.3.2 Ninf-G implementation on the Globus toolkit As mentioned earlier, in contrast to previous incarnations of Ninf in which the Globus interface was added as an afterthought, various features of Ninf-G integrates Globus com- ponents directly to implement the GridRPC features. More specifically, Ninf-G employs the following components from the Globus Toolkit as shown in Figure 25.1. GRAM : Serves the role of the server in the old Ninf system. MDS : Publishes interface information and pathname of GridRPC components. Globus-I/O: Client and remote executable communicate with each other using Globus- I/O. GASS : Redirects stdout and stderr of the GridRPC component to the client console. 25.3.3 API of Ninf-G Ninf-G has two categories of APIs. One is the Ninf-compatible ‘legacy’ API, and the other is the new (low-level) GridRPC API that has been subjected to collaborative stan- dardization activities at the Global Grid Forum with the NetSolve team from UTK. Ninf-G NINF-G: A GRIDRPC SYSTEM ON THE GLOBUS TOOLKIT 629 Client Server Client MDS GRAM IDL FILE Numerical library IDL compiler Generate Fork Retrieve 4. Connect back using Globus I/O 3. Invoke executable 2. Interface reply 1. Interface request Remote library executable Interface information LDIF File Figure 25.1 Overview of Ninf-G. Table 25.2 GridRPC API principal functions Function Description int grpc − function − handle − init ( grpc − function − handle − t * handle, char * host − name, int port, char * func − name); Initializes function handle using the provided information. int grpc − call ( grpc − function − handle − t*, .); Performs a blocking RPC with the function handle and arguments. This call blocks until RPC completes. int grpc − call − async ( grpc − function − handle − t*, .); Performs a nonblocking(asynchronous) RPC with the function handle and arguments. Returns sessionID as a future reference to the session. int grpc − wait(int sessionID); Wait for completion of the session specified by the sessionID. int grpc − wait − any ( int * idPtr); Wait for completion of one of the RPCs invoked by grpc − call − async beforehand. serves as one of the reference implementations of the GridRPC API. Principal functions of GridRPC API are shown in Table 25.2. 25.3.4 Server side IDL In order to ‘gridify’ a library, the Ninf library provider describes the interface of the library function using the Ninf IDL to publish his library function, which is only manifested and handled at the server side. The Ninf IDL supports datatypes mainly tailored for 630 HIDEMOTO NAKADA ET AL. Module sample; Define mmul(IN int N, IN double A[N*N], IN double B[N*N], OUT double C[N*N]) Required "mmul_lib.o" Calls "C" mmul(N, A, B, C); Figure 25.2 An example of Ninf IDL file. serving numerical applications: for example, the basic datatypes are largely scalars and their multidimensional arrays. On the other hand, there are special provisions such as support for expressions involving input arguments to compute array sizes, designation of temporary array arguments that need to be allocated on the server side but not transferred, and so on. This allows direct ‘gridifying’ of existing libraries that assumes array arguments to be passed by call-by-reference (thus requiring shared-memory support across nodes via the software), and supplementing the information lacking in the C and Fortran-type systems regarding array sizes, array stride usage, array sections, and so on. As an example, interface description for the matrix multiply is shown in Figure 25.2, in which the access specifiers IN and OUT specify whether the argument is read or written within the Gridified library. Other IN arguments can specify array sizes, strides, and so on with size expressions. In this example, the value of N is referenced to calculate the size of the array arguments A, B, C. In addition to the interface definition of the library function, the IDL description contains the information needed to compile and link the necessary libraries. Ninf-G tools allow the IDL files to be compiled into stub main routines and makefiles, which automates compilation, linkage and registration of Gridified executables. 25.4 NINF-G IMPLEMENTATION We now describe the implementation of the Ninf-G in more detail. 25.4.1 ‘Gridifying’ a library or an application using GridRPC Using Ninf-G, a user needs to merely take a few simple steps to make his application ‘Gridified’ in the following manner on the server side (we note again that no IDL handling is necessary on the client side, as opposed to traditional RPC systems such as CORBA): 1. Describe the interface of a library function or an application with Ninf IDL. 2. Process the Ninf IDL description file, generate stub main routine for the remote exe- cutable and a makefile as described above. 3. Link the stub main routine with the remote library, obtain the ‘Gridified’ executable. NINF-G: A GRIDRPC SYSTEM ON THE GLOBUS TOOLKIT 631 4. Register the ‘Gridified’ executable into the MDS. (Steps 3 and 4 are automated by the makefile). To register information into the MDS, the program that acts as the information provider outputs data complying with the LDAP Data Interchange Format (LDIF); moreover, the program itself is registered within the MDS setup file. Ninf-G places all such relevant information to be registered as a file under a specific directory ($ {GLOBUS LOCATION}/var/gridrpc/ ) readily formatted as a valid LDIF file, and also provides and registers a filter program that performs appropriate filtering as described below. For this purpose, Ninf-G adds the lines as shown in Figure 25.3 into the ‘grid-info-resource- ldif.conf’ file. 25.4.1.1 Generating the LDIF file The Ninf-G IDL compiler also generates interface information source files, which are utilized upon ‘make’ to automatically generate interface information files in XML as well as in LDIF formats. Both embody interface information (in XML), pathname of the remote executable, and the signature of the remote library. Figure 25.4 shows a sample LDIF file generated from an IDL file in Figure 25.2. (Note that the XML-represented interface information is base64 encoded because of its length.) Here, ‘ ROOT DN ’ in the first line will be replaced with the proper root-distinguished name by the information provider filter program described above. 25.4.1.2 Registration into the MDS In order to register the LDIF files generated in the current directory into the MDS, we merely copy them into the $ {GLOBUS LOCATION}/var/gridrpc/ directory. Note that this step is also automated by the makefile. dn: Mds–Software–deployment=GridRPC–Ninf–G, Mds–Host–hn=brain–n.a02.aist.go.jp, \ Mds–Vo–name=local,o=grid objectclass: GlobusTop objectclass: GlobusActiveObject objectclass: GlobusActiveSearch type: exec path: /usr/local/globus/var/gridrpc base: catldif args: –devclassobj –devobjs \ –dn Mds–Host–hn=brain–n.a02.aist.go.jp, Mds–Vo–name=local,o=grid \ –validto–secs 900 –keepto–secs 900 cachetime: 60 timelimit: 20 sizelimit: 10 Figure 25.3 Addition to the grid-info-resource-ldif.conf. 632 HIDEMOTO NAKADA ET AL. dn: GridRPC–Funcname=sample/mmul, Mds–Software–deployment=GridRPC–Ninf–G, _ _ROOT_DN_ _ objectClass: GlobusSoftware objectClass: MdsSoftware objectClass: GridRPCEntry Mds-Software-deployment: GridRPC–Ninf–G GridRPC–Funcname: sample/mmul GridRPC–Module: sample GridRPC–Entry: mmul GridRPC–Path: /home/ninf/tests/sample/_stub_mmul GridRPC–Stub:: PGZ1bmN0aW9uICB2ZXJzaW9uPSIyMjEuMDAwMDAwMDAwIiA+PGZ1bmN0aW9uX2 PSJwZXJmIiBlbnRyeT0icGluZ3BvbmciIC8+IDxhcmcgZGF0YV90eXBlPSJpbnQiIGlvZGVf ··· (The rest is omitted) Figure 25.4 LDIF file for matrix multiply. 25.4.2 Performing GridRPC Now we are ready to make the actual Ninf-G GridRPC call that can be broken down into the steps as shown in Figure 25.1. 1. The client requests interface information and executable pathname from the MDS. 2. MDS sends back the requested information. 3. Client requests the GRAM gatekeeper to invoke the ‘Gridified’ remote executable, passing on the necessary information described below. 4. The remote executable connects back to the client using Globus-I/O for subsequent parameter transfer, and so on. 25.4.2.1 Retrieval of interface information and executable pathname The client retrieves the interface information and executable pathname registered within the MDS using the library signature as a key. The retrieved information is cached in the client program to reduce the MDS retrieval overhead. 25.4.2.2 Invoking the remote executable The client invokes the remote executable (done by Ninf-G via the Globus GRAM), spec- ifying the remote executable path obtained from the MDS and a port address that accepts the callback from the remote executable. Here, the accepting port authenticates its peer using Globus-I/O, preventing malicious third-party attacks; this is because the party that owns the proper Globus proxy certificates derived from the client user certificate can connect to the port. 25.4.2.3 Remote executable callbacks to the client The remote executable obtains the client address and the port from argument list and con- nects back to the client using Globus-I/O. Subsequent remote executable communication with the client will use this port. NINF-G: A GRIDRPC SYSTEM ON THE GLOBUS TOOLKIT 633 25.5 USAGE SCENARIO Here, we show a sample deployment of a distributed parallel application using Ninf- G. As a sample application, consider computing an approximation of the value of pi using a simple Monte Carlo method. 1 The Monte Carlo method generates a huge amount of random numbers, converts them into meaningful input parameters, performs some calculations using the parameters, and statistically processes the results of the calculation to arrive at a meaningful result. Because each computation will largely be independent as a result of independent processing on each randomly generated number, Monte-Carlo is known to be quite suitable for distributed computing. For this example, we assume an environment shown in Figure 25.5. There are four Ninf-G server hosts named Host0, Host1, Host2, and Host3 that run Globus GRAM. A GIIS server runs on HostIS. All the GRIS servers on others hosts register themselves with the GIIS. The Client Program looks up the GIIS server as an MDS server, and performs GridRPC onto the GRAM server on each host. 25.5.1 Setup remote executable To setup the servers, the Monte Carlo calculation is defined as a standard C function, and an IDL description of the function is also defined. Figures 25.6 and 25.7 show the function and its IDL description, respectively. The function receives a seed for random number generation and a number of Monte Carlo trials, and returns the count of how many points have fallen within the circle. 25.5.2 Client program The client program performs GridRPCs onto the servers in parallel using the asynchronous invocation API. The core fragment of the client program is shown in Figure 25.8, demon- strating the relative ease of parallel application construction in Ninf-G. Lookup Ninf-G Client GIIS Register Invoke GRAM GRIS GRAM GRIS GRAM GRIS GRAM GRIS Host0 HostlS Host1 Host2 Host3 Figure 25.5 Usage scenario environment. 1 This is one of the simplest Monte Carlo applications. Define a circle and a square that encompasses the circle. A randomly generated point in the square would fall inside the circle with probability ‘ π 4 ’. In reality, problems are much more complex 634 HIDEMOTO NAKADA ET AL. long pi_trial(int seed, long time){ long 1, counter = 0; srandom(seed); for (1 = 0; 1 < times; 1++){ double x = (double)random() / RAND_MAX; double y = (double)random() / RAND_MAX; if (x * x + y * y < 1.0) counter++; } return counter; } Figure 25.6 Monte Carlo PI trials. Module pi; Define pi_trial (IN int seed, IN long times, OUT long * count) "monte carlo pi computation" Required "pi_trial.o" { long counter; counter = pi_trial(seed, times); *count = counter; } Figure 25.7 IDL for the PI trial function. /* Initialize handles. */ for (i = 0; i < NUM_HOSTS; i++) { if (grpc_function_handle_init(&handles[i], hosts[i], port, "pi/pi_trial") == GRPC_ERROR){ grpc_perror("handle_init"); exit(2); } } for (i = 0; i < NUM_HOSTS; i++){ /* (Parallel non-blocking remote function invocation. */ if ((ids[i] = grpc_call_async(&handles[i], i, times, &count[i])) == GRPC_ERROR){ grpc_perror("pi_trial"); exit(2); } } /* Synchronize on the result return */ if (grpc_wait_all() == GRPC_ERROR){ grpc_perro("wait_all"); exit(2); } Figure 25.8 Client PI program. [...]... a global world-wide computing infrastructure Proc of HPCN ’97 (LNCS-1225), 1997, pp 491–502 5 Casanova, H and Dongarra, J (1996) NetSolve: a network server for solving computational science problems Proceedings of Super Computing ’96, 1996 6 Seymour, K., Nakada, H., Matsuoka, S., Dongarra, J., Lee, C and Casanova, H (2002) GridRPC a remote procedure call API for Grid computing Grid2 002, 2002; submitted... of GridRPC states in the future versions of Ninf-G Ninf-G has just been released as of July 1, 2002, with both Globus 1.1.3 and 2.0 support The Java client is under development and will be released this summer REFERENCES 1 Foster, I and Kesselman, C (1997) Globus: a metacomputing infrastructure toolkit International Journal of Supercomputer Applications, 115–128 2 Foster, I and Karonis, N (1998) A grid- enabled... remote data transfer 25.7 CONCLUSION We described the concepts and the implementation of a Globus-based GridRPC system Ninf-G Thanks to the Globus Toolkit, Ninf-G now provides a secure and interoperable GridRPC framework, compared to its predecessor Modular design of the Globus Toolkit NINF-G: A GRIDRPC SYSTEM ON THE GLOBUS TOOLKIT 637 allowed us to conduct step-by-step deployment of the components,... Foster, I and Karonis, N (1998) A grid- enabled mpi: message passing in heterogeneous distributed computing systems Proc 1998 SC Conference, November, 1998 3 Abramson, D., Foster, I., Giddy, J., Lewis, A., Sosic, R., Sutherst, R and White, N (1997) The Nimrod computational workbench: a case study in desktop metacomputing Proceedings of ACSC ’97, 1997 4 Sato, M., Nakada, H., Sekiguchi, S., Matsuoka, S.,...635 NINF-G: A GRIDRPC SYSTEM ON THE GLOBUS TOOLKIT 25.6 PRELIMINARY EVALUATION To examine the basic performance of Ninf-G, we have preliminarily measured (1) the MDS lookup cost, (2) the GRAM invocation cost, and (3)... 0.5 s if cached Here, we observe that both MDS and GRAM costs are independent of the network latency between the client and the server The costs are not trivial, but we believe it is acceptable on the Grid since MDS query result is cached and reused, and remote executable once invoked can perform computation repeatedly without reinvocation Figure 25.10 shows the data throughput difference between the . Mds-Software-deployment: GridRPC–Ninf–G GridRPC–Funcname: sample/mmul GridRPC–Module: sample GridRPC–Entry: mmul GridRPC–Path: /home/ninf/tests/sample/_stub_mmul GridRPC–Stub::. implementations of the GridRPC [6] programming model, pro- viding simple yet powerful server-client-based framework for programming on the Grid. GridRPC facilitates