Graduate Texts in Mathematics 45 Editorial Board F W Gehring P R Halmos Managing Editor c C Moore M Loeve Probability Theory I 4th Edition Springer-Verlag New York Heidelberg Berlin M Loeve Departments of Mathematics and Statistics University of California at Berkeley Berkeley, California 94720 Editorial Board P R Halmos F W Gehring c C Moore Managing Editor University of California Department of Mathematics Santa Barbara, California 93106 University of Michigan Department of Mathematics Ann Arbor, Michigan 48104 University of California at Berkeley Department of Mathematics Berkeley, California 94720 AMS Subject Classifications 28-01, 60A05, 60Bxx, 60E05, 60Fxx Library of Congress Cataloging in Publication Data Loeve, Michel, 1907Probability theory (Graduate texts in mathematics; 45) Bibliography p Includes index Probabilities I Title II Series QA273.L63 1977 519.2 76-28332 All rights reserved No part of this book may be translated or reproduced in any form without written permission from Springer-Verlag © 1963 by M Loeve © 1977 by Springer-Verlag Inc Softcover reprint of the hardcover 4th edition 1977 Origipally published in the University Series in Higher Mathematics (D Van Nostrand Company); edited by M H Stone, L Nirenberg, and S S Chern ISBN 978-1-4684-9466-2 ISBN 978-1-4684-9464-8 (eBook) DOI 10.1007/978-1-4684-9464-8 To LINE and To the students and teachers of the School in the Camp de Draney PREFACE TO THE FOURTH EDITION This fourth edition contains several additions The main ones concern three closely related topics: Brownian motion, functional limit distributions, and random walks Besides the power and ingenuity of their methods and the depth and beauty of their results, their importance is fast growing in Analysis as well as in theoretical and applied Probability These additions increased the book to an unwieldy size and it had to be split into two volumes About half of the first volume is devoted to an elementary introduction, then to mathematical foundations and basic probability concepts and tools The second half is devoted to a detailed study of Independence which played and continues to playa central role both by itself and as a catalyst The main additions consist of a section on convergence of probabilities on metric spaces and a chapter whose first section on domains of attraction completes the study of the Central limit problem, while the second one is devoted to random walks About a third of the second volume is devoted to conditioning and properties of sequences of various types of dependence The other two thirds are devoted to random functions; the last Part on Elements of random analysis is more sophisticated The main addition consists of a chapter on Brownian motion and limit distributions It is strongly recommended that the reader begin with less involved portions In particular, the starred ones ought to be left out until they are needed or unless the reader is especially interested in them I take this opportunity to thank Mrs Rubalcava for her beautiful typing of all the edi tions since the inception of the book I also wish to thank the editors of Springer-Verlag, New York, for their patience and care M.L 'January, 1977 Berkeley, California PREFACE TO THE THIRD EDITION This book is intended as a text for graduate students and as a reference for workers in Probability and Statistics The prerequisite is honest calculus The material covered in Parts Two to Five inclusive requires about three to four semesters of graduate study The introductory part may serve as a text for an undergraduate course in elementary probability theory The Foundations are presented in: the Introductory Part on the background of the concepts and problems, treated without advanced mathematical tools; Part One on the Notions of Measure Theory that every probabilist and statistician requires; Part Two on General Concepts and Tools of Probability Theory Random sequences whose general properties are given in the Foundations are studied in: Part Three on Independence devoted essentially to sums of independent random variables and their limit properties; Part Four on Dependence devoted to the operation of conditioning and limit properties of sums of dependent random variables The last section introduces random functions of second order Random functions and processes are discussed in: Part Five on Elements of random analysis devoted to the basic concepts of random analysis and to the martingale, decomposable, and Markov types of random functions Since the primary purpose of the book is didactic, methods are emphasized and the book is subdivided into: unstarred portions, independent of the remainder; starred portions, which are more involved or more abstract; complements and details, including illustrations and applications of the material in the text, which consist of propositions with fre- PREFACE TO THE THIRD EDITION quent hints; most of these propositions can be found in the articles and books referred to in the BiBliography Also, for teaching and reference purposes, it has proved useful to name most of the results Numerous historical remarks about results, methods, and the evolution of various fields are an intrinsic part of the text The purpose is purely didactic: to attract attention to the basic contributions while introducing the ideas explored Books and memoirs of authors whose contributions are referred to and discussed are cited in the Bibliography, which parallels the text in that it is organized by parts and, within parts, by chapters Thus the interested student can pursue his study in the original literature This work owes much to the reactions of the students on whom it has been tried year after year However, the book is definitely more concise than the lectures, and the reader will have to be armed permanently with patience, pen, and calculus Besides, in mathematics, as in any form of poetry, the reader has to be a poet in posse This third edition differs from the second (1960) in a number of places Modifications vary all the way from a prefix ("sub" martingale in lieu of "semi"-martingale) to an entire subsection (§36.2) To preserve pagination, some additions to the text proper (especially 9, p 656) had to be put in the Complements and Details It is hoped that moreover most of the errors have been eliminated and that readers will be kind enough to inform the author of those which remain I take this opportunity to thank those whose comments and criticisms led to corrections and improvements: for the first edition, E Barankin, S Bochner, E Parzen, and H Robbins; for the second edition, Y S Chow, R Cogburn, J L Doob, J Feldman, B Jamison, J Karush, P A Meyer, J W Pratt, B A Sevastianov, J W Woll; for the third edition, S Dharmadhikari, J Fabius, D Freedman, A Maitra, U V Prokhorov My warm thanks go to Cogburn, whose constant help throughout the preparation of the second edition has been invaluable This edition has been prepared with the partial support of the Office of Naval Research and of the National Science Foundation M.L April, 1962 Berkeley, California CONTENTS OF VOLUME I GRADUATE TEXTS IN MATHEMATICS VOL 45 INTRODUCTORY PART: ELEMENTARY PROBABILITY THEORY SECTION I PAGE 3 INTUITIVE BACKGROUND Events Random events and trials Random variables II AXIOMS; INDEPENDENCE AND THE BERNOULLI CASE III Axioms of the fini te case Simple random variables Independence Bernoulli case Axioms for the countable case Elementary random variables Need for nonelementary random variables DEPENDENCE AND CHAINS *5 *6 *7 8 11 12 15 17 22 24 24 Conditional probabilities Asymptotically Bernoullian case Recurrence Chain dependence Types of states and asymptotic behavior Motion of the system Stationary chains 36 39 COMPLEMENTS AND DETAILS 42 25 26 28 30 PART ONE: NOTIONS OF MEASURE THEORY CHAPTER I: SETS, SPACES, AND MEASURES SETS, CLASSES, AND FUNCTIONS 1.1 Definitions and notations 1.2 Differences, unions, and intersections 1.3 Sequences and limits 1.4 Indicators of sets Xl 55 55 56 57 59 CONTENTS OF VOLUME I Xli SECTION PAGE 1.5 Fields and u-fields 1.6 Monotone classes *1 Product sets *1.8 Functions and inverse functions *1.9 Measurable spaces and functions *2 TOPOLOGICAL SPACES *2.1 *2.2 *2.3 *2.4 Topologies and limits Limit points and compact spaces Countability and metric spaces Linearity and normed spaces ADDITIVE SET FUNCTIONS 3.1 Additivity and continuity 3.2 Decomposition of additive set functions *4 CONSTRUCTION OF MEASURES ON u-FIELDS *4.1 Extension of measures *4.2 Product probabilities *4.3 Consistent probabilities on Borel fields *4.4 Lebesgue-Stieltjes measures and distribution functions COMPLEMENTS AND DETAILS 59 60 61 62 64 65 66 69 72 78 83 83 87 88 88 91 93 96 100 CHAPTER II: MEASURABLE FUNCTIONS AND INTEGRATION MEASURABLE FUNCTIONS 5.1 5.2 5.3 MEASURE AND CONVERGENCES 6.1 6.2 6.3 Definitions and general properties Convergence almost everywhere Convergence in measure INTEGRATION 7.1 7.2 Numbers Numerical functions Measurable functions Integrals Convergence theorems 111 111 114 116 118 119 125 Indefinite integrals and Lebesgue decomposition Product measures and iterated integrals Iterated integrals and infinite product spaces 130 130 135 137 COMPLEMENTS AND DETAILS 139 INDEFINITE INTEGRALS; ITERATED INTEGRALS 8.1 8.2 *8.3 103 103 105 107 INDEX Abel theorem, 400 Addition property, 10 Additive set function, 83 continuity, 85 continuity theorem, 85 countably,83 decomposition, 87 decomposition theorem, 88 extension, 88 extension theorem, 88 fini te, 83, 111 finitely, 83 restriction, 88 u-additive, 83, 111 u-finite, 83, 111 Adherence, 66 Adherent point, 66 Alexandrov, 190, 409 Allard,44 Almost everywhere, 112 convergence, 114 mutual convergence, 114 Almost sure(ly), 148 convergence, 152, 248, 260 mutual convergence, 153 stabili ty, 244, 249, 274, 260 stability criterion, 264 Almost uniform convergence, 140 Andersen and Jessen, 92, 408 Andersen, 368, 379, 404, 411 equivalence, 377, 391 equivalence lemma, 378 Andre, Desire, 47 Arcsine law, 379, 404 Asymptotic behaviour theorem, 399 Asymptotic (Cont.) passage theorem, 36 uniform negligibility, 302 Atom, 100 Attraction domain of, 360 partial, 403 stability and-criteria, 364 standard,402 Axioms of the countable case, 16 of the finite case, Baire category theorem, 75 functions, 109 Banach,407 space, 81 Base countable, 72 of a cylinder, 62 Baxter, 369, 390, 411 Bawly, 302, 410 Bernoulli, 407 case, 12, 244, 280 extended, 26 law of large numbers, 14,244, 282 Berry, 294,410 Billingsley, 196, 409 Bienayme, 408 equality, 12,246 Blackwell, 369, 411 Bochner, 408, 409 theorem, 220 Boltzmann, 42, 43 Bolzano-Weierstrass property, 70 413 414 INDEX Borel(ian), 107,407,410 Can telli lemma, 240 cylinder, 93 field, 93, 104 function, 111, 156 functions theorem, 156 line, 93, 107 sets, 93, 104 space, 93, 107 strong law of large numbers, 18, 19, 26, 244 zero-one criterion, 240 Bose-Einstein statistics, 43, 44 Bounded functional, 80 Liapounov theorem, 213, 282 set, 74 totally, 75 variances, 303 variances limit theorem, 305 Bourbaki, 407 Breiman, 409 Brey, 409 Brunk, 271, 410 Cantelli, 20, 240,409,410 Cantor theorem, 74 Caratheodory extension theorem, 88 Category first, 75 second,75 theorem, 75 Cauchy mutual convergence criterion, 74, 104, 114 Centering, 244 at expectations, 244 at medians, 256 function, 350 Central convergence criterion, 323, 326 inequalities, 316 limit problem, 302 limit theorem, 321, 322 statistical theorem, 20 Chain, 29 constant, 29 elemen tary, 29 stationary, 39 Chained classes, 28 events, 28 random variables, 29 Change of variable lemma, 190 Characteristic function(s), 198 composition theorem, 226 continuity theorem, 204, 224 convergence criterion, 204 and dichotomy, 386 extension theorem, 224 general properties, 207 infinitely decomposable, 306 in tegral, 202 inversion formula, 199 triangular, 386 self-decomposable, 334 stable, 338, 363 uniform convergence theorem, 204 Chung, 407, 409, 410 and Fuchs, 368, 383, 411 and Ornstein, 383, 411 Class closed under, 59 lower, 272 monotone, 60 of sets, 55 upper, 272 Classical degenerate convergence criterion, 290 limit problem, 286 normal convergence criterion, 292 Closed class of states, 36 model, 22 set, 66 Closure theorem infinitely decomposablelaws,309 INDEX Combinatorial lemma, 378 method, 47 Compact locally, 71 set, 69 space, 69 Compactification, 71 Compactness properties, 70 relative, 195 relative, criterion, 195 Compactness theorem for distribution functions, 181 metric spaces, 76 separated spaces, 70 Comparison convergences theorem, 117 lemma(s), 303, 320 Complement, 4, 56 Complete convergence, 180 convergence criterion, 204 measure, 91 metric space, 74 Completeness theorem, L r- , 163 Completion of a metric space, 77 of