Tài liệu High-Performance Parallel Database Processing and Grid Databases- P6 doc

230 Chapter 8 Parallel Universal Qualification—Collection Join Queries Case 1: ARRAYS Hash Table 1 a(250, 75) b(210, 123) f(150, 50, 250) 150(f) 210(b) 250(a) 50(f) 75(a) 123(b) Hash Table 2 250(f) Hash Table 3 Case 2: SETS Hash Table 1 a(250, 75) b(210, 123) f(150, 50, 250) Hash Table 2 250(f) Hash Table 3 Sort 50(f) 75(a) 123(b) 150(f) 210(b) 250(a) a(75, 250) b(123, 210) f(50, 150, 250) Figure 8.7 Multiple hash tables collision will occur between h(150,50,25) and collection f (150,50,250). Collision will occur, however, if collection h is a list. The element 150(h/ will be hashed to hash table 1 and will collide with 150( f /. Subsequently, the element 150(h/ will go to the next available entry in hash table 1, as a result of the collision. Once the multiple hash tables have been built, the probing process begins. The probing process is basically the central part of collection join processing. The prob- ing function for collection-equi join is called a function universal. It recursively Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. 8.4 Parallel Collection-Equi Join Algorithms 231 checks whether a collection exists in the multiple hash table and the elements belong to the same collection. Since this function acts like a universal quantifier where it checks only whether all elements in a collection exist in another collection, it does not guarantee that the two collections are equal. To check for the equality of two collections, it has to check whether collection of class A (collection in the multiple hash tables) has reached the end of collection. This can be done by check- ing whether the size of the two matched collections is the same. Figure 8.8 shows the algorithm for the parallel sort-hash collection-equi join algorithm. Algorithm: Parallel-Sort-Hash-Collection-Equi-Join // step 1 (disjoint partitioning): Partition the objects of both classes based on their first elements (for lists/arrays), or their minimum elements (for sets/bags). // step 2 (local joining) : In each processor, for each partition // a. preprocessing (sorting) // sets/bags only For each collection of class A and class B Sort each collection // b. hash For each object of class A Hash the object into multiple hash tables // c. hash and probe For each object of class B Call universal (1, 1) // element 1,hash table 1 If TRUE AND the collection of class A has reached end of collection Put the matching pair into the result Function universal (element i , hash table j ): Boolean Hash and Probe element i to hash table j If matched // match the element and the object Increment i and j // check for end of collection of the probing class. If end of collection is reached Return TRUE If hash table j exists // check for the hash table result D universal ( i, j ) Else Return FALSE Else Return FALSE Return result Figure 8.8 Parallel sort-hash collection-equi join algorithm Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. 232 Chapter 8 Parallel Universal Qualification—Collection Join Queries 8.4.4 Parallel Hash Collection-Equi Join Algorithm Unlike the parallel sort-hash explained in the previous section, the algorithm described in this section is purely based on hashing only. No sorting is necessary. Hashing collections or multivalues is different from hashing atomic values. If the join attributes are of type list/array, all of the elements of a list can be concatenated and produce a single value. Hashing can then be done at once. However, this method is applicable to lists and arrays only. When the join attributes are of type set or bag, it is necessary to find new ways of hashing collections. To illustrate how hashing collections can be accomplished, let us review how hashing atomic values is normally performed. Assume a hash table is implemented as an array, where each hash table entry points to an entry of the record or object. When collision occurs, a linear linked-list is built for that particular hash table entry. In other words, a hash table is an array of linear linked-lists. Each of the linked-lists is connected only through the hash table entry, which is the entry point of the linked-list. Hash tables for collections are similar, but each node in the linked-list can be connected to another node in the other linked-list, resulting in a “two-dimensional” linked-list. In other words, each collection forms another linked-list for the second dimension. Figure 8.9 shows an illustration of a hash table for collections. For example, when a collection having three elements 3, 1, 6 is hashed, the gray nodes create a circular linked-list. When another collection with three elements 1, 3, 2 is hashed, the white nodes are created. Note that nodes 1 and 3 of this collection collide with those of the previous collection. Suppose another collection having duplicate elements (say elements 5, 1, 5) is hashed; the black nodes are created. Note this time that both elements 5 of the same collection are placed within the same collision linked-list. Based on this method, the result of the hashing is always sorted. When probing, each probed element is tagged. When the last element within a collection is probed and matched, a traversal is performed to check whether the matched nodes form a circular linked-list. If so, it means that a collection is successfully probed and is placed in the query result. Figure 8.10 shows the algorithm for the parallel hash collection-equi join algo- rithm, including the data partitioning and the local join process. 1 2 3 4 5 6 Figure 8.9 Hashing collections/multivalues Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. 8.5 Parallel Collection-Intersect Join Algorithms 233 Algorithm: Parallel-Hash-Collection-Equi-Join // step 1 (data partitioning) Partition the objects of both classes to be joined based on their first elements (for lists/arrays), or their smallest elements (for sets/bags) of the join attribute. // step 2 (local joining): In each processor // a. hash Hash each element of the collection. Collision is handled through the use of linked- list within the same hash table entry. Elements within the same collection are linked in a different dimension using a circular linked-list. // b. probe Probe each element of the collection. Once a matched is not found: Discard current collection, and Start another collection. If the element is found Then Tag the matched node If the element found is the last element in the probing collection Then Perform a traversal If a circle is formed Then Put into the query result Else Discard the current collection Start another collection Repeat until all collections are probed. Figure 8.10 Parallel hash collection-equi join algorithm 8.5 PARALLEL COLLECTION-INTERSECT JOIN ALGORITHMS Parallel algorithms for collection-intersect join queries also exist in three forms, like those of collection-equi join. They are: ž Parallel sort-merge nested-loop algorithm, ž Parallel sort-hash algorithm, and ž Parallel hash algorithm Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. 234 Chapter 8 Parallel Universal Qualification—Collection Join Queries There are two main differences between parallel algorithms for collection- intersect and those for collection-equi. The first difference is that for collection- intersect, the simplest algorithm is a combination of sort-merge and nested-loop, not double-sort-merge. The second difference is that the data partitioning used in parallel collection-intersect join algorithms is non-disjoint data partitioning, not disjoint data partitioning. 8.5.1 Non-Disjoint Data Partitioning Unlike the collection-equi join, for a collection-intersect join, it is not possible to have non-overlap partitions because of the nature of collections, which may be overlapped. Hence, some data needs to be replicated. There are three non-disjoint data partitioning methods available to parallel algorithms for collection-intersect join queries, namely: ž Simple replication, ž Divide and broadcast, and ž Divide and partial broadcast. Simple Replication With a Simple Replication technique, each element in a collection is treated as a single unit and is totally independent of other elements within the same collection. Based on the value of an element in a collection, the object is placed into a partic- ular processor. Depending on the number of elements in a collection, the objects that own the collections may be placed into different processors. When an object has already been placed at a particular processor based on the placement of an element, if another element in the same collection is also to be placed at the same place, no object replication is necessary. Figure 8.11 shows an example of a simple replication technique. The bold printed elements are the elements, which are the basis for the placement of those objects. For example, object a(250, 75) in processor 1 refers to a placement for object a in processor 1 because of the value of element 75 in the collection. And also, object a(250, 75) in processor 3 refers to a copy of object a in processor 3 based on the first element (i.e., element 250). It is clear that object a is replicated to processors 1 and 3. On the other hand, object i (80, 70) is not replicated since both elements will place the object at the same place, that is, processor 1. Divide and Broadcast The divide and broadcast partitioning technique basically divides one class into a number of processors equally and broadcasts the other class to all processors. The performance of this partitioning method will be strongly determined by the size of the class that is to be broadcasted, since this class is replicated on all processors. Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. 8.5 Parallel Collection-Intersect Join Algorithms 235 i(80, 70) Processor 1 Processor 2 Processor 3 Class A (range 0-99) (range 100-199) (range 200-299) c(125, 181) f(150, 50, 250) h(190, 189, 170) a(250, 75) b(210, 123) g(270) r(50, 40) t(50, 60) u(3, 1, 2) w(80, 70) p(123, 210) s(125, 180) v(100, 102, 270) q(237) d(4, 237) f(150,50, 250) e(289, 290) p(123, 210) b(210, 123) f(150, 50, 250) d(4, 237) a(250, 75) Class B v(100, 102, 270) Figure 8.11 Simple replication technique for parallel collection-intersect join There are two scenarios for data partitioning using divide and broadcast. The first scenario is to divide class A and to broadcast class B, whereas the second scenario is the opposite. With three processors, the result of the first scenario is as follows. The division uses a round-robin partitioning method. Processor 1: class A (a; d; g/ and class B (p; q; r; s; t; u;v;w/ Processor 2: class A (b; e; h/ and class B (p; q; r; s; t; u;v;w/ Processor 3: class A (c; f; i/ and class B (p; q; r; s; t; u;v;w/ Each processor is now independent of the others, and a local join operation can then be carried out. The result from processor 1 will be the pair d  q. Processor 2 produces the pair b  p, and processor 3 produces the pairs of c  s; f  r; f  t, and i  w. With the second scenario, the divide and broadcast technique will result in the following data placement. Processor 1: class A (a; b; c; d; e; f; g; h; i/ and class B (p; s;v/. Processor 2: class A (a; b; c; d; e; f; g; h; i/ and class B (q; t;w/. Processor 3: class A (a; b; c; d; e; f; g; h; i/ and class B (r; u/. The join results produced by each processor are as follows. Processor 1 pro- duces b– p and c–s, processor 2 produces d–q; f –t,andi–w, and processor 3 produces f –r. The union of the results from all processors gives the final query result. Both scenarios will produce the same query result. The only difference lies in the partitioning method used in the join algorithm. It is clear from the examples that the division should be on the larger class, whereas the broadcast should be on the smaller class, so that the cost due to the replication will be smaller. Another way to minimize replication is to use a variant of divide and broadcast called “divide and partial broadcast”. The name itself indicates that broadcasting is done partially, instead of completely. Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. 236 Chapter 8 Parallel Universal Qualification—Collection Join Queries Algorithm: Divide and Partial Broadcast // step 1 (divide) 1. Divide class B based on largest element in each collection 2. For each partition of B ( i D 1, 2, , n ) Place partition Bi to processor i // step 2 (partial broadcast) 3. Divide class A based on smallest element in each collection 4. For each partition of A (i D 1, 2, , n ) Broadcast partition Ai to processor i to n Figure 8.12 Divide and partial broadcast algorithm Divide and Partial Broadcast The divide and partial broadcast algorithm (see Fig. 8.12) proceeds in two steps. The first step is a divide step, and the second step is a partial broadcast step. We divide class B and partial broadcast class A. The divide step is explained as follows. Divide class B into n number of par- titions. Each partition of class B is placed in a separate processor (e.g., partition B1 to processor 1, partition B2 to processor 2, etc). Partitions are created based on the largest element of each collection. For example, object p(123, 210), the first object in class B, is partitioned based on element 210, as element 210 is the largest element in the collection. Then, object p is placed on a certain partition, depend- ing on the partition range. For example, if the first partition is ranging from the largest element 0 to 99, the second partition is ranging from 100 to 199, and the third partition is ranging from 200 to 299, then object p is placed in partition B3, and subsequently in processor 3. This is repeated for all objects of class B. The partial broadcast step can be described as follows. First, partition class A based on the smallest element of each collection. Then for each partition Ai where i D 1ton, broadcast partition Ai to processors i to n. This broadcasting tech- nique is said to be partial, since the broadcasting decreases as the partition number increases. For example, partition A1 is basically replicated to all processors, parti- tion A2 is broadcast to processor 2 to n only, and so on. The result of the divide and partial Broadcast of the sample data shown earlier in Figure 8.3 is shown in Figure 8.13. In regard to the load of each partition, the load of the last processor may be the heaviest, as it receives a full copy of A and a portion of B. The load goes down as class A is divided into smaller size (e.g., processor 1). To achieve more load balancing, we can apply the same algorithm to each partition but with a reverse role of A and B;thatis,divide A and partial broadcast B (previously it was divide Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. 8.5 Parallel Collection-Intersect Join Algorithms 237 (range 0-99) (range 100-199) (range 200-299) Partition A1 Partition A2 Partition A3 Class A Partition A1 Objects: a, d, f, i Class B Partition B1 Objects: r, t, u, w Class A Partition A1 Objects: a, d, f, i Class B Partition B2 Object: s Class A Partition A2 Objects: b, c, h Class A Partition A1 Objects: a, d, f, i Class B Partition B3 Objects: p, q, v Class A Partition A2 Objects: b, c, h Class A Partition A3 Objects: e, q Processor 1 : Processor 2 : Processor 3 : DIVIDE e(289, 290) g(270) r(50, 40) u(3, 1, 2) w(80, 70) t(50,60) s(125,180) p(123, 210) v(100, 102,270) q(237) d(4, 237) a(250, 75) f(150, 50, 250) i(80, 70) c(125, 181) h(190, 189, 170) b(210, 123) Class A Class B (range 0-99) (range 100-199) (range 200-299) Partition B1 Partition B2 Partition B3 (Divide based on the largest) PARTIAL BROADCAST (Divide based on the smallest) Figure 8.13 Divide and partial broadcast example B and partial broadcast A). This is then called a “two-way” divide and partial broadcast. Figure 8.14(a and b) shows the results of reverse partitioning of the initial partitioning. Note that from processor 1, class A and class B are divided into three Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. f(150, 50, 250) d(4, 237) i(80, 70) From Processor 1 h(190, 189, 170) a(250, 75) t(50, 60) s(125, 180) v(100, 102, 104) e(289, 290) g(270) r(50, 40) u(3, 1, 2) w(80, 70) p(123, 210) q(237) c(125, 181) i(80, 70) d(4, 237) f(150, 50, 250) h(190, 189, 170) c(125, 181) i(80, 70) d(4, 237) a(250, 75) a(250, 75) f(150, 50, 250) b(210, 123) Partition A11 Partition A12 Partition A13 Partition B11 Partition B12 Partition B13 Partition A21 Partition A22 Partition A23 Partition B21 Partition B22 Partition B23 From Processor 2 From Processor 3 Partition B31 Partition B32 Partition B33 Partition A31 Partition A32 Partition A33 1. DIVIDE b(210, 123) Figure 8.14(a) Two-way divide and partial broadcast (divide) 238 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. From Processor 1 Partition A11 Partition A21 Partition A12 Partition A13 Partition B11 From Processor 2 2. PARTIAL BROADCAST Partition B11 Partition B12 Partition B11 Partition B12 Partition B13 Bucket 11 Bucket 12 Bucket 13 Partition A22 Partition A23 Bucket 21 Bucket 22 Bucket 23 Partition B21 Partition B21 Partition B22 Partition B21 Partition B22 Partition B23 From Processor 3 Partition A31 Partition A32 Partition A33 Bucket 31 Bucket 32 Bucket 33 Partition B31 Partition B31 Partition B32 Partition B31 Partition B32 Partition B33 Figure 8.14(b) Two-way divide and partial broadcast (partial broadcast) 239 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. [...]... access plans Parallelization models High-Performance Parallel Database Processing and Grid Databases, by David Taniar, Clement Leung, Wenny Rahayu, and Sushant Goel Copyright  2008 John Wiley & Sons, Inc 256 9.1 Query Execution Plan 257 Query Scanning and Parsing Internal form of query Parallel Query * Access Plan Formulation Optimization * Scheduling Execution Execution plan Parallel Query * Parallelization... for query processing and optimization The tasks of parallel query optimization can be divided into two major areas, parallel query optimization and parallel query execution Parallel query optimization includes access plan formulation and execution scheduling Access plan formulation is for developing the best sequential query access plan, whereas execution scheduling is for incorporating parallelism... // b probe For each object b(c2) of class S Hash and probe collection c2 into the hash table If there is any match Concatenate object B and the matched object A into query result Figure 8.22 Parallel hash collection-intersect join algorithm ž ž Parallel sort-hash algorithm, and Parallel hash algorithm The main difference between parallel subcollection and collection-intersect algorithms is, in fact,... probed Figure 8.26 Parallel hash sub-collection join algorithm parallel algorithms for relational division using sort-merge, aggregate, and hash-based methods The work in parallel object-oriented query started in the late 1980s and early 1990s The two most early works in parallel object-oriented query processing were written by Khoshafian et al (ICDE 1988), followed by Kim (ICDE 1990) Leung and Taniar (1995)... among these parallelizable basic operations All of the previous chapters deal with parallel query execution, focusing on efficient formulation of parallel algorithms of each query operation This chapter, on the other hand, concentrates on parallel query optimization, and in particular parallel subquery scheduling and dynamic parallel query optimization This chapter starts with a query execution plan to... technique, Ž One-way divide and partial broadcast technique, and Ž Two-way divide and partial broadcast technique b Adopting the two-way divide and partial broadcast technique, show the results of the collection-intersect join query using the parallel sort-merge nested-loop algorithm 8.5 Parallel subcollection join query exercises: 8.9 Exercises 255 a Adopting the two-way divide and partial broadcast technique... replication, divide and broadcast, and divide and partial broadcast These are non-disjoint data partitioning The local join process uses sort-merge nested-loop, sort-hash, or pure hash When sort-merge nested-loop is used, the simple replication data partitioning is not applicable Parallel Subcollection Join Algorithms Data partitioning methods available are divide and broadcast and divide and partial broadcast... objects into query result Figure 8.21 Parallel sort-hash collection-intersect join algorithm 8.5.4 Parallel Hash Collection-Intersect Join Algorithm Like the parallel sort-hash algorithm, parallel hash may use any of the three non-disjoint data partitioning available for parallel collection-intersect join, such as simple replication, divide and broadcast, or divide and partial broadcast The local join... Parallelization Models * Parallel Algorithms Execution Figure 9.1 Query Result optimization and parallel algorithms contain the basic form of parallelism for basic query operations Complex queries are normally decomposed into multiple basic operations, and for each basic operation an appropriate parallelism algorithm is applied Execution scheduling deals with managing execution plans among these parallelizable... Khoshafian et al (ICDE 1988), followed by Kim (ICDE 1990) Leung and Taniar (1995) described various parallelism models, including intra- and interclass parallelism The research group of Sampaio and Smith et al published various papers on parallel algebra for object databases (1999), experimental results (2001), and their system called Polar (2000) 254 A B C D E F G H I J K L M N O Chapter 8 Table R 7, 202, . collection-intersect join algorithm ž Parallel sort-hash algorithm, and ž Parallel hash algorithm The main difference between parallel subcollection and collection-intersect algorithms. available to parallel algorithms for collection-intersect join queries, namely: ž Simple replication, ž Divide and broadcast, and ž Divide and partial broadcast. Simple