EFFICIENT DELEGATION ALGORITHMS FOR OUTSOURCING COMPUTATIONS ON MASSIVE DATA STREAMS

EFFICIENT DELEGATION ALGORITHMS FOR OUTSOURCING COMPUTATIONS ON MASSIVE DATA STREAMS VED PRAKASH NATIONAL UNIVERSITY OF SINGAPORE 2015 EFFICIENT DELEGATION ALGORITHMS FOR OUTSOURCING COMPUTATIONS ON MASSIVE DATA STREAMS VED PRAKASH (B.Sc.(Hons), NUS) A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY CENTRE FOR QUANTUM TECHNOLOGIES NATIONAL UNIVERSITY OF SINGAPORE 2015 Declaration I hereby declare that the thesis is my original work and it has been written by me in its entirety. I have duly acknowledged all the sources of information which have been used in the thesis. This thesis has also not been submitted for any degree in any university previously. VED PRAKASH July 20, 2015 i Acknowledgements I would like to express my sincere appreciation to my advisor Hart- mut Klauck, who has been a remarkable mentor. He has been extremely encouraging and that tide me through this trying journey. The relentless support has also allowed me to grow as a research scientist. His meticu- lous supervision has provided me much guidance on both my research and career path fronts. I would like to express my gratitude for his thorough reviews throughout the course of the preparation of this thesis. I would also like to thank my thesis committee members, Rahul Jain and Frank Stephan for their provision and guidance in the foundational years of my PhD studies. I would like to thank the Centre for Quantum Technologies (CQT) for giving me the opportunity to be able to receive this education under extremely privileged circumstances. This dissertation would not have been possible without the funding from CQT. Following, I would like to thank all of my friends who have been cheering me on via various channels, for they were the sources of motivation for me to strive towards my goal. Words cannot express how grateful I am to my parents for all of the sacrifices they have made. Most imperatively, I would like to express my utmost appreciation to my beloved wife, Ong Phyllis, who spent sleepless nights with me while I penned down my ideas. She has also always been my support in times when there was no one to answer my queries. ii Table of Contents Declaration i Acknowledgements ii Summary vi List of Figures viii List of Symbols ix 1 Introduction 1 1.1 Structure of this Thesis and Contributions Made . . . . . . 3 2 Data Streaming and Communication Complexity 8 2.1 The Data Stream Models . . . . . . . . . . . . . . . . . . . . 8 2.2 Communication Complexity . . . . . . . . . . . . . . . . . . 11 2.3 Frequency Moments . . . . . . . . . . . . . . . . . . . . . . . 16 2.4 Other Problems in the Streaming Model . . . . . . . . . . . 19 3 Constant Round Interactions in Data Streams and Merlin- Arthur Classes 21 3.1 The Annotation Model . . . . . . . . . . . . . . . . . . . . . 22 3.1.1 Basic Annotation Protocols . . . . . . . . . . . . . . 24 3.2 Frequency Moments Revisited in the Annotation Model . . . 26 3.2.1 Protocols for Frequency Moments . . . . . . . . . . . 27 iii 3.3 Merlin-Arthur Communication Models . . . . . . . . . . . . 32 3.3.1 Online Merlin-Arthur Communication Models . . . . 36 3.3.2 Communication Complexity Classes . . . . . . . . . . 42 3.3.3 Lower Bounds for the Annotation Model . . . . . . . 43 3.3.4 A Lower Bound for OMA k . . . . . . . . . . . . . . . 50 3.4 Merlin-Arthur and IP Streaming Model . . . . . . . . . . . . 53 3.5 Related Results . . . . . . . . . . . . . . . . . . . . . . . . . 58 4 Interactive Streaming Model 61 4.1 Generic Protocol for NC . . . . . . . . . . . . . . . . . . . . 61 4.2 An Online Merlin-Arthur Protocol for PSPACE cc . . . . . . 64 4.3 Practical Interactive Protocols . . . . . . . . . . . . . . . . . 70 5 An Improved Interactive Streaming Algorithm for F 0 76 5.1 Overview of Our Techniques . . . . . . . . . . . . . . . . . . 76 5.2 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 77 5.3 Comparison of Our Results . . . . . . . . . . . . . . . . . . 88 6 A New Model for Verifying Computations on Data Streams 89 6.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 6.2 The New Model . . . . . . . . . . . . . . . . . . . . . . . . 91 6.3 Algorithms In Our New Model . . . . . . . . . . . . . . . . . 93 6.3.1 Median . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.3.2 Longest Increasing Subsequence . . . . . . . . . . . . 100 6.3.3 FULL RANK . . . . . . . . . . . . . . . . . . . . . . 113 6.4 A Lower Bound on the Number of Rounds . . . . . . . . . . 128 7 Conclusion and Open Problems 135 7.1 Open Problems . . . . . . . . . . . . . . . . . . . . . . . . . 139 iv Bibliography 142 Appendix 157 A.1 Schwartz-Zippel Lemma . . . . . . . . . . . . . . . . . . . . 157 A.2 Coding Theory . . . . . . . . . . . . . . . . . . . . . . . . . 157 A.3 Interactive Proof Systems . . . . . . . . . . . . . . . . . . . 158 v Summary In numerous real world applications, one needs to store almost the whole data set in order to compute certain functions of the data, where we require the answer to be exact or even approximate in some cases. This thesis will examine a model for data streaming algorithms where we engage the ser- vices of external third parties to do difficult computations for the client. The main motivating application of this is cloud computing, where we not only require the cloud to store the massive data set, but execute compu- tations on the data set and communicate the results to the client as well. The client should be able to verify the correctness of the result within his computational restrictions. We will discuss algorithms to achieve this in different streaming models, depending on the interaction between the client and the external third party who is also called the prover. The communication complexity model augmented with a prover is a very important tool used to analyze the theoretical properties of the data streaming model with a prover. We use this to give an improved lower bound for approximating the frequency moments in the annotation stream- ing model, where there is a single help message from the prover after the stream has ended. We also investigate a restricted version of this model and show lower bounds in this restricted model. We will use our lower bounds to study the theoretical properties of the streaming model with a prover, where the prover and the client are allowed to interact. We give an improvement of previous work in [30] which requires  O( √ n) communication between the prover and the client to compute the number vi of distinct elements exactly using O(log m) messages, where n is the length of the stream and m is the size of the universe. Our algorithm gives an exponential improvement on the total communication needed while main- taining the same number of messages exchanged. We also investigate a new streaming model that only bounds the com- munication overhead, i.e., the amount of communication sent from the prover to the client per symbol of the data stream. This streaming model is different from previous models defined in [20,21,28–30,56,102]. We will design algorithms for four different streaming problems in this new model. For one of these streaming problems (perfect matching problem), there is no known efficient interactive streaming algorithm in the previous mod- els [29,30,52]. We will analyse the limitations of this new model. We show that the verification phase with a large number of communication rounds between the prover and the client after the stream has ended is unavoidable for certain problems in a restricted streaming model where the messages from the client to the prover are just some of his random bits. vii List of Figures Figure 2.2.1 Protocol tree. . . . . . . . . . . . . . . . . . . . . 13 Figure 3.3.1 MA communication protocol. . . . . . . . . . . . 35 Figure 3.3.2 AM communication protocol. . . . . . . . . . . . 36 Figure 3.3.3 OMA 2 communication protocol. . . . . . . . . . . 40 Figure 3.3.4 OIP 2 communication protocol. . . . . . . . . . . 40 Figure 3.3.5  OIP 2 communication protocol. . . . . . . . . . . 41 Figure 3.3.6 OAM communication protocol. . . . . . . . . . . 42 Figure 6.3.1 Descending chains that do not cross. . . . . . . . 103 Figure 6.3.2 Descending chains that cross. . . . . . . . . . . . 104 viii [...]... reasons for using the services of third parties to execute computations for the verifier One obvious reason would be that the verifier 2 does not have the resources (mainly due to space constraints) to execute the computations by himself If one generates massive data only once in a while, it is more practical to rent some hundreds of computers for a few hours and get the third party to do the necessary computations. .. for online communication complexity classes 1 Data streaming algorithms are designed to process massive data sets arriving one at a time in an online fashion, i.e with small time overhead The space used by these algorithms should be minimal Due to the enormous amount of data being generated in this century, designing efficient streaming algorithms and models to handle these huge data sets are important... the annotation model for verifying computations on data streams, which was first introduced in [21] In this model, the prover provides an annotation/proof to the verifier after the data stream has ended The proof is processed by the verifier in a streaming fashion and the verifier is allowed to use randomness to 3 process the proof stream As a warmup, we give simple annotation protocols based on fingerprinting... 1}n and i ∈ [n], Bob forms a n-bit string y which is zero on all positions except the i-th position where it is one They run the one-way 15 Disj protocol on inputs (x, y) If the output is disjoint, Bob concludes that xi = 0 Otherwise, he concludes that xi = 1 The Disj function is the generic co-NP complete problem in communication complexity [8] Even if multiple rounds of communication are allowed between... known lower bounds on data streams, we will see how the Merlin-Arthur communication complexity model can be used to give further insight on the prover-verifier streaming model 21 3.1 The Annotation Model In this section we define the model of streaming computations with a helper/prover In the annotation model we consider two parties, the prover, and the verifier who wish to compute a function f (σ) Both parties... work of Alon, Matias and Szegedy [6] and look at the limitations of this model 2.1 The Data Stream Models The input stream is denoted by σ = a1 , · · · , an , where the ai ’s are sometimes referred to as symbols in this thesis The data stream defines a function A : [N ] → R The data elements in the stream arrive in an online fashion, and the system has no control over the order in which the data streams. .. objective of data streaming algorithms is to process a massive data set arriving one item at a time in an online fashion, i.e., with small time overhead, while at the same time minimizing 8 the workspace used by the algorithm In this thesis, we use the unit cost RAM model to measure the update time per symbol seen on the stream In this model, each field operation1 takes unit time These algorithms are only allowed... one-way communication complexity model is important for the study of streaming algorithms for the purpose of proving space lower bounds In this model, there is a single message from Alice to Bob and Bob has to output the answer based on Alice’s message One-way communication complexity was first introduced by Yao [107] and this subtopic of communication complexity was taken up in greater consideration... before giving annotation protocols for the exact computations of the frequency moments F2 , F0 and F∞ The main purpose for doing so is to illustrate that we can obtain sublinear annotation protocols for problems which require linear space in the standard streaming model, which is the model without the prover We introduce the Merlin-Arthur communication complexity model to address lower bounds for data. .. simpler and more efficient as they are not based on interactive proof techniques which have an additional verification phase after the stream ends In previous works [29, 30], the main conversations take place after the stream has ended and during this verification phase, the prover has to perform exponentially more operations than the verifier This additional verification phase is not present in our protocols . EFFICIENT DELEGATION ALGORITHMS FOR OUTSOURCING COMPUTATIONS ON MASSIVE DATA STREAMS VED PRAKASH NATIONAL UNIVERSITY OF SINGAPORE 2015 EFFICIENT DELEGATION ALGORITHMS FOR OUTSOURCING COMPUTATIONS. Section 4.2, we show that “IP=PSPACE” holds for online communication complexity classes. 1 Data streaming algorithms are designed to process massive data sets arriving one at a time in an online. study the power and limitations of algorithms in different models for delegating computations on data streams to third parties. 1.1 Structure of this Thesis and Contribu- tions Made • In Chapter 2,