Graduate Texts in Mathematics 45 Editorial Board F W Gehring P.R Halmos Managing Editor C C Moore M Loeve Probability Theory I 4th Edition Springer Science+Business Media, LLC M Loeve Departments of Mathematics and Statistics University of California at Berkeley Berkeley, California 94720 Editorial Board P.R Halmos F W Gehring C C Moore Managing Editor University of California Department of Mathematics Santa Barbara, California 93106 University of Michigan Department of Mathematics Ann Arbor, Michigan 48104 University of California at Berkeley Department of Mathematics Berkeley, California 94720 AMS Subject Classifications 28-01, 60A05, 60Bxx, 60E05, 60Fxx Library of Congress Cataloging in Publication Data Loeve, Michel, 1907Probability theory (Graduate texts in mathematics; 45) Bibliography p Includes index Probabilities I Title II Series QA273.L63 1977 519.2 76-28332 All rights reserved No part of this book may be translated or reproduced in any form without written permission from Springer-Verlag © 1963 by M Loeve © 1977 by Springer Science+Business Media New York Originally published by Springer-Verlag New York in 1977 Origipally published in the University Series in Higher Mathematics (D Van Nostrand Company); edited by M H Stone, L Nirenberg, and S S Chern ISBN 978-0-387-90127-5 ISBN 978-1-4757-6288-4 (eBook) DOl 10.1007/978-1-4757-6288-4 To LINE and To the students and teachers of the School in the Camp de Draney PREFACE TO THE FOURTH EDITION This fourth edition contains several additions The main ones concern three closely related topics: Brownian motion, functional limit distributions, and random walks Besides the power and ingenuity of their methods and the depth and beauty of their results, their importance is fast growing in Analysis as well as in theoretical and applied Probability These additions increased the book to an unwieldy size and it had to be split into two volumes About half of the first volume is devoted to an elementary introduction, then to mathematical foundations and basic probability concepts and tools The second half is devoted to a detailed study of Independence which played and continues to play a central role both by itself and as a catalyst The main additions consist of a section on convergence of probabilities on metric spaces and a chapter whose first section on domains of attraction completes the study of the Central limit problem, while the second one is devoted to random walks About a third of the second volume is devoted to conditioning and properties of sequences of various types of dependence The other two thirds are devoted to random functions; the last Part on Elements of random analysis is more sophisticated The main addition consists of a chapter on Brownian motion and limit distributions It is strongly recommended that the reader begin with less involved portions In particular, the starred ones ought to be left out until they are needed or unless the reader is especially interested in them I take this opportunity to thank Mrs Rubalcava for her beautiful typing of all the editions since the inception of the book I also wish to thank the editors of Springer-Verlag, New York, for their patience and care M.L January, 1977 Berkeley, California PREFACE TO THE THIRD EDITION This book is intended as a text for graduate students and as a reference for· workers in Probability and Statistics The prerequisite is honest calculus The material covered in Parts Two to Five inclusive requires about three to four semesters of graduate study The introductory part may serve as a text for an undergraduate course in elementary probability theory The Foundations are presented in: the Introductory Part on the background of the concepts and problems, treated without advanced mathematical tools; Part One on the Notions of Measure Theory that every probabilist and statistician requires; Part Two on General Concepts and Tools of Probability Theory Random sequences whose general properties are given in the Foundations are studied in: Part Three on Independence devoted essentially to sums of independent random variables and their limit properties; Part Four on Dependence devoted to the operation of conditioning and limit properties of sums of dependent random variables The last section introduces random functions of second order Random functions and processes are discussed in: Part Five on Elements of random analysis devoted to the basic concepts of random analysis and to the martingale, decomposable, and Markov types of random functions Since the primary purpose of the book is didactic, methods are emphasized and the book is subdivided into: unstarred portions, independent of the remainder; starred portions, which are more involved or more abstract; complements and details, including illustrations and applications of the material in the text, which consist of propositions with fre- PREFACE TO THE THIRD EDITION quent hints; most of these propositions can be found in the articles and books referred to in the Bibliography Also, for teaching and reference purposes, it has proved useful to name most of the results Numerous historical remarks about results, methods, and the evolution of various fields are an intrinsic part of the text The purpose is purely didactic: to attract attention to the basic contributions while introducing the ideas explored Books and memoirs of authors whose contributions are referred to and discussed are cited in the Bibliography, which parallels the text in that it is organized by parts and, within parts, by chapters Thus the interested student can pursue his study in the original literature This work owes much to the reactions of the students on whom it has been tried year after year However, the book is definitely more concise than the lectures, and the reader will have to be armed permanently with patience, pen, and calculus Besides, in mathematics, as in any form of poetry, the reader has to be a poet in posse This third edition differs from the second (1960) in a number of places Modifications vary all the way from a prefix ("sub" martingale in lieu of "semi"-martingale) to an entire subsection (§36.2) To preserve pagination, some additions to the text proper (especially 9, p 656) had to be put in the Complements and Details It is hoped that moreover most of the errors have been eliminated and that readers will be kind enough to inform the author of those which remain I take this opportunity to thank those whose comments and criticisms led to corrections and improvements: for the first edition, E Barankin, S Bochner, E Parzen, and H Robbins; for the second edition, Y S Chow, R Cogburn, J L Doob, J Feldman, B Jamison, J Karush, P A Meyer, J W Pratt, B A Sevastianov, J W Wall; for the third edition, S Dharmadhikari, J Fabius, D Freedman, A Maitra, U V Prokhorov My warm thanks go to Cogburn, whose constant help throughout the preparation of the second edition has been invaluable This edition has been prepared with the partial support of the Office of Naval Research and of the National Science Foundation M.L April, 1962 Berkeley, California CONTENTS OF VOLUME I GRADUATE TEXTS IN MATHEMATICS VOL 45 INTRODUCTORY PART: ELEMENTARY PROBABILITY THEORY SECTION PAGE 3 I INTUITIVE BACKGROUND Events Random events and trials Random variables II AxiOMS; INDEPENDENCE AND THE BERNOULLI CASE Axioms of the finite case Simple random variables Independence Bernoulli case Axioms for the countable case Elementary random variables Need for nonelementary random variables Ill DEPENDENCE AND CHAINS *5 *6 *7 Conditional probabilities Asymptotically Bernoullian case Recurrence Chain dependence Types of states and asymptotic behavior Motion of the system Stationary chains CoMPLEMENTS AND DETAILS 8 11 12 15 17 22 24 24 25 26 28 30 36 39 42 PART ONE: NOTIONS OF MEASURE THEORY CHAPTER I: SET~ SPACES, AND MEASURES SETs, CLAssEs, AND FuNcTIONS • 1.1 1.2 1.3 1.4 Definitions and notations Differences, unions, and intersections Sequences and limits Indicators of sets xi 55 55 56 57 59 CONTENTS OF VOLUME I Xll PAGE SECTION Fields and u-fields Monotone classes Product sets Functions and inverse functions Measurable spaces and functions 59 60 61 62 64 *2 ToPOLOGICAL SPACES *2.1 Topologies and limits *2.2 Limit points and compact spaces *2.3 Countability and metric spaces *2.4 Linearity and normed spaces 65 1.5 1.6 *1.7 *1.8 *1.9 ADDITIVE SET FuNcTIONS 3.1 Additivity and continuity 3.2 Decomposition of additive set functions *4 CoNSTRUCTION OF MEASURES ON u-FIELDS *4.1 Extension of measures *4.2 Product probabilities *4.3 Consistent probabilities on Borel fields *4.4 Lebesgue-Stieltjes measures and distribution functions CoMPLEMENTS AND DETAILS 66 69 72 78 83 83 87 88 88 91 93 96 100 CHAPTER II: MEASURABLE FUNCTIONS AND INTEGRATION MEASURABLE FuNCTIONS 5.1 Numbers 5.2 Numerical functions 5.3 Measurable functions 103 103 105 107 MEASURE AND CONVERGENCES 6.1 Definitions and general properties 6.2 Convergence almost everywhere 6.3 Convergence in measure 111 Ill 114 116 INTEGRATION 7.1 Integrals 7.2 Convergence theorems 118 119 125 INDEFINITE INTEGRALS; ITERATED INTEGRALS 8.1 Indefinite integrals and Lebesgue decomposition 8.2 Product measures and iterated integrals *8.3 Iterated integrals and infinite product spaces 130 130 135 137 CoMPLEMENTS AND DETAILS 139 INDEX Abel theorem, 400 Addition property, 10 Additive set function, 83 continuity, 85 continuity theorem, 85 countably, 83 decomposition, 87 decomposition theorem, 88 extension, 88 extension theorem, 88 finite, 83, 111 finitely, 83 restriction, 88 u-additive, 83, 111 u-fini te, 83, 111 Adherence, 66 Adherent point, 66 Alexandrov, 190, 409 Allard, 44 Almost everywhere, 112 convergence, 114 mutual convergence, 114 Almost sure(ly), 148 convergence, 152, 248, 260 mutual convergence, 153 stability, 244, 249, 274, 260 stability criterion, 264 Almost uniform convergence, 140 Andersen and Jessen, 92, 408 Asymptotic (Cont.) passage theorem, 36 uniform negligibility, 302 Atom, 100 Attraction domain of, 360 partial, 403 stability and-criteria, 364 standard, 402 Axioms of the countable case, 16 of the finite case, Baire category theorem, 75 functions, 109 Banach, 407 space, 81 Base countable, 72 of a cylinder, 62 Baxter, 369, 390, 411 Bawly, 302, 410 Bernoulli, 407 case, 12, 244, 280 extended, 26 law of large numbers, 14, 244, 282 Berry, 294, 410 Billingsley, 196, 409 Bienayme, 408 equality, 12, 246 Blackwell, 369, 411 Bochner, 408, 409 theorem, 220 Boltzmann, 42, 43 Bolzano-Weierstrass property, 70 Ande~en,368,379,40~411 equivalence, 377, 391 equivalence lemma, 378 Andre, Desire, 47 Arcsine law, 379, 404 Asymptotic behaviour theorem, 399 413 414 INDEX Borel(ian), 107, 407, 410 Can telli lemma, 240 cylinder, 93 field, 93, 104 function, 111, 156 functions theorem, 156 line, 93, 107 sets, 93, 104 space, 93, 107 strong law of large numbers, 18, 19, 26, 244 zero-one criterion, 240 Bose-Einstein statistics, 43, 44 Bounded functional, 80 Liapounov theorem, 213, 282 set, 74 totally, 75 variances, 303 variances limit theorem, 305 Bourbaki, 407 Breiman, 409 Brey, 409 Brunk, 271, 410 Cantelli, 20, 240, 409, 410 Cantor theorem, 74 Caratheodory extension theorem, 88 Category first, 75 second, 75 theorem, 75 Cauchy mutual convergence criterion, 74, 104, 114 Centering, 244 at expectations, 244 at medians, 256 function, 350 Central convergence criterion, 323, 326 inequalities, 316 limit problem, 302 limit theorem, 321, 322 statistical theorem, 20 Chain, 29 constant, 29 elementary, 29 stationary, 39 Chained classes, 28 events, 28 random variables, 29 Change of variable lemma, 190 Characteristic function(s), 198 composition theorem, 226 continuity theorem, 204, 224 convergence criterion, 204 and dichotomy, 386 extension theorem, 224 general properties, 207 infinitely decomposable, 306 integral, 202 inversion formula, 199 triangular, 386 self-decomposable, 334 stable, 338, 363 uniform convergence theorem, 204 Chung,407,409,410 and Fuchs, 368, 383, 411 and Ornstein, 383, 411 Class closed under, 59 lower, 272 monotone, 60 of sets, 55 upper, 272 Classical degenerate convergence criterion, 290 limit problem, 286 normal convergence criterion, 292 Closed class of states, 36 model, 22 set, 66 Closure theorem infinitely decomposablelaws,309 INDEX Combinatorial lemma, 378 method, 47 Compact locally, 71 set, 69 space, 69 Compactificati on, 71 Compactness properties, 70 relative, 195 relative, criterion, 195 Compactness theorem for distribution functions, 181 metric spaces, 76 separated spaces, 70 Comparison convergences theorem, 117 lemma(s), 303, 320 Complement, 4, 56 Complete convergence, 180 convergence criterion, 204 measure, 91 metric space, 74 Completeness theorem, Lr- , 163 Completion of a metric space, 77 of u-field, 91 Complex random variable, 154 Composition, 204, 283 and decomposition theorem, 283 lemma, 227 theorem, 206 Compound(s) Poisson, 347 theorem, 237 u-field, 237 Conditional expectation, 24 probability, 24 regular probability, 138 Consistency theorem, 94 Consistent, 93 Constant chain, 29 415 Continuity additive set functions-theo rem, 85 characteristic functions theorem, 204, 224 F- , interval, 187 P- , set, 189 Continuous Ffunction, 67 functional, 80 Pset function, 85 Convergence almost everywhere, 114 almost sure, 153 almost uniform, 140 complete, 180 essential, 262 in quadratic mean, 260 in rth mean, 159 of sequences of sets, 58 of types, 216 uniform, 114 weak, 180 Convergence criterion(ria) almost everywhere, 116 central, 323, 327 complete, 204 degenerate, 290, 329 iid, 346 iid central, 348 normal, 292, 328 Poisson, 308, 329 pr.'s on metric space, 190 weak, 203 Convergence theorem(s) comparison, 117 dominated, 126 Fatou-Lebesgu e, 12 Lr- , 165 moments, 186 monotone, 125 of types, 216 uniform, 204 416 INDEX Convex function, 161 Correspondence lemma, 344 theorem, 97 Countable(ly), 16 base, 72 class, 57 set, 57 set operations, 57 valued, 64, 106 Covering open, 69 rule, 16 c.-inequality, 157 Cramer, 271, 408, 409 Cylinder Borel, 62 product, 62 Daniell, 94, 408 Decomposable infinitely, 308 self- , 334 Decomposition theorem(s) chains, 37 composition and, theorem, 283 degenerate type, 283 distribution functions, 178,200 Hahn, 87 Lebesgue, 131 normal type, 283 Poisson type, 283 Degenerate characteristic function, 215 convergence criterion, 290, 329 distribution function, 215 law, 215 random time, 376 random variable, 215 random walk, 370 type, 215 Dense set, 72 nowhere, 75 Denumerable, 16 class, 57 Denumerable (Cont.) set, 57 set operations, 57 valued, 64, 106 Diameter of a set, 74 Dichotomy, 380 criterion, 382 Difference equations, 48 proper, 56 of sets, 56 Direction(ted), 67 set, 68 Dirichlet, 187 Disjoint class, 57 events, sets, 57 Distance of points, 73 points and sets, 78 sets, 77 Distribution, 168, 172, 175 empirical distribution function, 20 function, 20, 96, 169, 177 invariant, 39 Ld-, 370 probability, 168 Doblin, 30, 302, 354, 403, 410, 411 Domain, 62, 108 of attraction, 360 partial, 401 standard, 402 Dominated convergence theorem, 126 Doubrovsky, 408 Duality rule, 57 Dugue, 409 Dunford, 408 Egorov theorem, 141 Einstein, 43, 44 Elementary chain, 29 INDEX Elementary (Cont.) function, 64, 107 probability field, 16 random variable, 17, 152 Empirical distribution function, 20 Empty set, 4, 54 Equivalence Andersen, 377, 393 Andersen, lemma, 378 class, 154 convergence, 245 lemma, laws, 290 lemma, series, 245 tail, 245 theorem(s), 263, 379 Equivalent distribution functions, 96 functions, 114 random variables, 154 states, 36 Erdos, 34 Esseen, 294, 410 Essential convergence, 262 divergence, 262 Euler, 47 Events, 3, 8, 151 disjoint, elementary, 151 exchangeable, 45, 21 impossible, 4, 151 independent, 11, 73, 235 null, 152 random, 5, sure, 4, 151 tail, 241 Everreturn state, 36 Exchangeable events, 45, 373 random variables, 373 Expectation centering at, 244 criterion, 384 indefinite, 153 417 Expectation (Cont.) of a random function, 156 of a random sequence, 154 of a random variable, 10, 17, 153, 154 Exponential bounds, 266 identities, 388, 396 Extended Bernoulli law of large numbers, 26 Borel line, 93, 107 Borel space, 93, 107 Borel strong law of large numbers, 26 central convergence criterion, 326 central limit theorem, 322 convergence criterion, 306 Helly-Bray lemma, 183 identities, 395 Extension, 88 of characteristic functions, 225 of linear functionals, 81 of measures, 88 Factorization(s) extreme, 396 sample space, 392, 396 unique, theorem, 389 Feller, 34, 292, 302, 353, 354, 369, 371, 383, 407, 409-411 Fermi-Dirac statistics, 42, 43 Field(s), 59 Borel, 93, 104 compound, 156 Lebesgue, 129 of outcomes, probability, product, 61, 62 u-, 59 Finite intersection property, 70 interval lemma, 45 Finetti, de, 302, 411 418 Finitely-valued, 64, 106 First category, 75 limit theorems, 282 Fortet, 409 Frechet, 187, 408 Fubini theorem, 136 Function(s) additive set, 83 Baire, 111 centering, 350 characteristic, 199, 202 continuous, 67 convex, 161 countably-valued, 64, 106 denumerably-valued, 64, 106 distribution, 20, 96, 169, 177 domain of, 62, 107 elementary, 64, 107 equivalent, 114 F-continuous, 187 finite, 105 finitely-valued, 64, 106 of function, 64, 106 inverse, 63, 106 measurable, 65, 107 non-negative definite, 219 numerical, 105 P-continuous, 187 positive part of, 105 random, 152, 156 range of, 63 range space, 62, 105 simple, 64, 107 tail, 241 Functional, 80 bounded, 80 continuous, 80 linear, 80 normed, 80 Gambler's ruin, 48 Geometric probabilities, 49 Glivenko, 408 -Cantelli, 21 INDEX Gnedenko, 302, 354, 407, 409, 411 Gumbel, 45 Hadamard, 30 Hahn and Rosenthal, 408 Hahn decomposition theorem, 87 Halmos, 196, 408 Hausdorff, 408 space, 68 Heine-Borel property, 70 Helly, 409 Helly-Bray lemma, 182 extended, 183 generalized, 187 Helly-Bray theorem, 184 Herglotz lemma, 220 Hewitt-Savage, 374, 411 zero-one law, 374 Hilbert space, 80 Hitting time, 377 lemma, 374 Holder inequality, 158 Huygens principle, 28 Identification property, 73 Image, Inverse of a class, 63, 106 of a set, 63, 106 Impossible event, 4, 110 Improper integral, 130 Increments inequality, 208 Indecomposable class of states, 36 Indefinite expectation, 154 integral, 130 Independent classes, 11, 235 events, 11, 235 random functions, 237 random variables, 11, 237 random vectors, 237 u-fields, 236 trials, INDEX Indicator(s), 9, 59 method of, 44 Induced partition, 64, 106 probability space, 168, 171 u-field, 64 topology, 66 Inequality( ties) basic, 159 central, 316 Cr, 157 Holder, 158 integral, 208 Kolmogorov, 25, 247, 275 Levy, 259 Liapounov, 177 Schwarz, 158 symmetrization, 259 Tchebichev, 11, 160 truncation, 209 weak symmetrization, 258 Inferior limit, 58 Infimum, 56, 103 Infinite decomposability, 308 numbers, 103 Infinitely often, 241 Integrable, 119 uniformly, 164 Integral characteristic function, 202 inequality, 208 representation theorem, 166 Integral(s) Daniell, 146 Darboux-Young, 144 definitions, 119 elementary properties, 120 improper, 130 iterated, theorem, 137 Kolmogorov, 145 Lebesgue, 129, 143 Lebesgue-Stiel tj es, 128 Riemann, 129 Riemann-Stieltjes, 129 419 Integration by parts lemma, 358 Interior, 66 point, 66 Intermediate value theorem, 102 Intersection(s), 4, 56 finite-property, 70 Interval(s), 61, 62, 104 finite-lemma, 397 Invariance theorem, 39 Invariant distribution, 39 Inverse function, 63, 106 image, 63, 106 Inversion formula, 199 Iterated logarithm, law of, 219 regular conditional probability theorem, 138 Kac, 407, 410 Katz, 411 Karamata, 354 main theorem, 356 Kawata, 210, 409 Kelley, 408 Kemperman, 369, 393, 395, 410 Khintchine, 28, 302, 410 measure, 343 representation, 343 Kolmogorov, 30, 94, 302,407, 408, 410 approach, 145 inequalities, 25, 247, 275 strong law of large numbers, 251 three series criterion, 249 zero-one law, 241 Kronecker lemma, 250 Lambert, 46 Laplace, 22, 281, 286, 287, 407 Law of large numbers Bernoulli, 14, 26, 244, 282 Borel, strong, 18, 19, 26, 244 classical, 290 Kolmogorov, strong, 251 420 INDEX Law(s), 174 degenerate, 215, 281 equivalence lemma, 290 infinitely decomposable, 308 normal, 213, 281 of the iterated logarithm, 219 Poisson, 282 probability, 174, 214 self-decomposable, 334 stable, 326, 363 types of, 215 universal, 403 zero-one, 241, 374 Lebesgue, 408 approach, 143 decomposition theorem, 131 field, 129 integral, 129 measure, 128 sets, 129 Lebesgue-Stieltjes field, 128 integral, 128 measure, 128 Le Cam, 193, 409 Levy, P., 199, 301, 302, 408, 410 continuity theorem, 204 inequalities, 259 function(s), 361 measure, 343 representation, 343 Liapounov, 411 inequality, 174 theorem, 213, 287, 289 Limit of a directed set, 68 along a direction, 68 inferior, 58 superior, 58 Limit of a sequence of functions, 113 laws, 214 numbers, 104 sets, 58 Limit problem central, 302 classical, 286 Lindeberg, 292, 411 Line Borel, 93, 107 extended real, 104 real, 93, 103 Linear closure, 79 functional, 80 space, 70 Linearly ordered, 67 Liouville theorem, 369 Lomnicki, 409 Lower class, 272 variation, 87 £,completeness theorem, 163 convergence theorem, 164 spaces, 162 Lusin theorem, 140 Marcinkiewitz, 225, 254, 302, 409 Markov, 407 chain, 28 dependence, 28 inequality, 160 Lukacz, 408 Matrices, method of, 48 Matrix, transition probability, 29 Mean rth mean, 159 Measurable function, 107 sets, 60, 64, 107 space, 60, 64, 107 Measure, 84, 112 convergence in, 116 Khin tchine, 343 Lebesgue, 129 Lebesgue-Stieltjes, 128 Levy, 343 normed, 91, 151 INDEX Measure (Cont.) outer, 88 outer extension of, 89 product, 136 signed, 87 space, 112 Median, 256 centering at, 256 Metric compactness theorem, 76 linear space, 79 space, 73 topology, 73 Minimal class over, 60 Minkowski inequality, 158 Moment(s) convergence problem, 187 convergence theorem, 186 kth, 157, 186 lemma, 254 rth absolute, 157, 186 Monotone class, 60 convergence theorem, 125 sequences of sets, 58 Montmort, 46 JL -measurable, 88 Multiplication lemma, 238 property, 11 rule, 24 theorem, 238 Negligibility, uniform asymptotic, 302, 314 Neighborhood, 66 Neyman, 407 Nikodym, 133, 408 Nonhereditary systems, 28 Nonrecurrent state, 31 No-return state, 31 Norm of a functional, 79 Hilbert, 80 of a mapping, 79 421 Normal approximation theorem, 300 convergence criterion, 307 decomposition theorem, 283 law, 213, 281 type, 283 Normalized distribution function, 199 Normed functional, 80 linear space, 79 measure, 91, 151 sums, 331 Nowhere dense, 75 Null set, 91, 112 state, 32 Numerical function, 105 Open covering, 69 set, 66 Ordering, partial, 67 Orthogonal random variables, 246 Outcome(s), of an experiment, field of, Outer extension, 89 measure, 88 Owen, 411 Parseval relation, 386 Parzen, 407 Petrov, 410 Physical statistics, 42 Planck, 44 Poincare recurrence theorem, 28 Poisson compound, 347 convergence criterion, 229, 329 decomposition theorem, 283 law, 282 theorem, 15 type, 283 422 INDEX Pollaczec, 396, 394, 400, 411 -Spitzer identity, 393, 400 Pollard, 34 Polya, 368, 409 Port, 369, 393, 412 Positive part, 105 state, 32 Positivity criterion, 33 Possible state(s), 370 value(s), 370 values theorem, 371 Probability, 5, 8, 16, 91, 151, 152 condi tiona!, 6, 24 convergence in, 153 convergence on metric spaces, 189, 190 distribution, 168 field, law, 214 product-theorem, 92 rule, total, 24 stability in, 244 sub, 187 transition, 29 Probability space, 91, 151, 152 induced, 168 product, 92 transition, 29 product, 92 sample, 168 Product cylinder, 62 field, 61, 62 measurable space, 61, 62, 137 measure, 136 probability, 92 probability theorem, 242 scalar, 80 set, 61 u-field, 61, 62 space, 61, 62 Prohorov, 190, 193, 264, 409 Quadratic mean convergence in, 260 Radon-Nikodym theorem, 133 extension, 134 Raikov, 283, 411 Random event, 5, function, 152, 156 sequence, 152, 155 time, 375 time identities, 390 time translations, 376 trial, variable, 6, 9, 17, 152 vector, 152, 155 walk, 47, 378, 379 Range, 63 space, 62, 105 Ranked random variables, 350 sums, 405 Ray, 369, 395, 412 Real line, 93 line, extended, 93, 107 number, 93 number extended, 93 space, 93 Recurrence, 380 criterion, 32 theorem(s), 27, 384 Recurrent state(s), 31, 380 walk, 28 Regular variation, 354 criterion, 354 Relative compactness, 190 theorem, 195 Representation theorem, 313 integral, 166 Restriction, 88 Return criterion, 32 state, 31 Riemann integral, 129 INDEX Riemann-Stieltjes integral, 129 Riesz, F., 222, rth absolute moment, 157, 186 mean, 159 Ruin, gambler's, 48 Saks,408 Savage,374 Scalar product, 80 Scheffe, 408 Schwarz inequality, 158 Section, 61, 62, 135 Self-decomposable(bili ty), 334 criterion, 335 Separable space, 68 Separation theorem, 68 Sequence(s) convergence equivalent, 245 random, 152, 155 tail of, 241 tail equivalent, 245 Series criterion three, 249 two, 263 Set function additive, 83 continuous, 85 countably additive, 83 finite, 82, 111 finitely, 83 u-additive, 83, 111 u-finite, 83, 11 Set(s) Borel, 93, 104 bounded, 74 closed, 66 compact, 69 dense, 72 directed, 68 empty, 4, 54 Lebesgue, 129 measurable, 60, 64, 107 null, 91, 112 open, 66 423 Set(s) (Cont.) product, 61 su bdirected, 69 totally bounded, 75 Shohat, 187 u-additive, 83, 111 u-field(s), 59 compound, 156, 235 independent, 236 induced, 64 product, 61, 62 tail, 241 Signed measure, 87 Simple function, 64, 107 random variable, 6, 152 Snell, 66 Space adjoint, 81 Banach, 79 Borel, 93, 107 compact, 69 complete, 74 Hausdorff, 68 Hilbert, 80 induced probability, 168 linear, 79 measurable, 60, 64, 107 measure, 112 metric, 73 metric linear, 79 normal, 78 normed linear, 79 probability, 91, 151, 152 product, 61, 62 product measurable, 61, 62, 137 product measure, 136 product probability, 91 range, 62, 105 sample probability, 168 separated, 68 of sets, 55 topological, 66 Sphere, 73 424 Spitzer, 369, 393, 394, 404, 410, 412 basic identity, 396 basic theorem, 401 Stability almost sure, 244 almost sure criterion, 264 and attraction criterion, 364 in probability, 244, 246 Stable characteristic function, 338 law, 338, 363 State(s) closed class of, 36 equivalent, 36 everreturn, 36 indecomposable class of, 36 nonrecurrent, 31, 380 no return, 31 nu\1,32 period of, 33 positive, 32 possible, 370 recurrent, 31, 380 return, 31 transient, 380 Stationary chain, 39 Steinhaus, 409 Stiel tj es, 128, 129 Stochastic variable, 174 Stochastically independent, 11 Strong law of large numbers Borel, 18, 19, 26, 244 Kolmogorov, 241 Structure corollary, 348 theorem, 310 Subspace linear, 79 topological, 66 Sum of sets, 4, 51 Superior limit, 58 Supremum, 56, 103 Sure almost, 151 INDEX Sure (Cont.) event, 4, 151 Symmetrization, 257 inequalities, 259 inequalities, weak, 257 Tail equivalence, 245 event, 241 function, 241 of a sequence, 241 u-field, 241 Tchebichev, 409 inequality, 11, 160 theorem, 287 Tight(ness), 194 lemma, 194 and relative compactness, 195 theorem, 194 Three alternatives, 375 alternatives criteria, 399 series criterion, 249 Toeplitz lemma, 250 Topological space, 66 subspace, 66 Topology, 66 metric, 73 reduced, 66 Total(ly) bounded set, 75 probability rule, 24 variation, 87 Transition probability, 29 Trial(s) deterministic, identical, 5, independent, 5, random, repeated, 5, Triangle property, 73 Triangular characteristic function, 386 probability density, 386 INDEX Truncation, 245 inequality, 209 Tulcea, 138, 408 Tucker, 410 Two-series criterion, 251 Type(s), 215 convergence of, 216 degenerate, 215, 282 normal, 282 Poisson, 282 Ugakawa, 102 Uniform asymptotic negligibility, 302, 314 continuity, 77 convergence, 114 convergence theorem, 204 Union, 4, 56 Upper class, 272 variation, 87 Urysohn, 78 Uspensky, 407 Value(s), possible, 370 theorem, 371 Variable, random, 69, 17, 152 Variance, 12, 244 425 Variances, bounded, 302 limit theorem, 305 Variation lower, 87 regular, 354 slow, 354 total, 87 of truncated moments, 359 upper, 87 Vector, random, 152, 155 Wald's relation, 377, 397 Weak compactness theorem, 181 convergence, 180 convergence, to a pr., 190 symmetrization inequalities, 257 convergence, to a pr., 190 Weierstrass theorem, Wendel, 412 Zero-one criterion, Borel, 24 law, Kolmogorov, 241 law, Hewitt-Savage, 374 Zygmund, 409 Graduate Texts in Mathematics Soft and hard cover editions are available for each volume up to Vol 14, hard cover only from Vol 15 TAKEUTI/ZARING Introduction to Axiomatic Set Theory vii, 250 pages 1971 OxTOBY Measure and Category viii, 95 pages 1971 ScHAEFFER Topological Vector Spaces xi, 294 pages 1971 HILTON/STAMMBACH A Course in Homological Algebra ix, 338 pages 1971 (Hard cover edition only) MACLANE Categories for the Working Mathematician ix, 262 pages 1972 HUGHES/PIPER Projective Planes xii, 291 pages 1973 SERRE A Course in Arithmetic x, 115 pages 1973 TAKEUTI/ZARING Axiomatic Set Theory viii, 238 pages 1973 HUMPHREYS Introduction to Lie Algebras and Representation Theory xiv, 169 pages 1972 10 11 COHEN A course in Simple Homotopy Theory xii, 114 pages 1973 CONWAY Functions of One Complex Variable 2nd corrected reprint xiii, 313 pages 1975 (Hard cover edition only.) 12 13 BEALS Advanced Mathematical Analysis xi, 230 pages 1973 ANDERSON/FULLER Rings and Categories of Modules ix, 339 pages 1974 14 GoLUBITSKY/GVILLEMIN Stable Mappings and Their Singularities x, 211 pages 1974 15 BERBERIAN Lectures in Functional Analysis and Operator Theory x, 356 pages 1974 16 WINTER The Structure of Fields xiii, 205 pages 1974 17 RoSENBLATT Random Processes 2nd ed x, 228 pages 1974 18 HALMOS Measure Theory xi, 304 pages 1974 19 HALMOS A Hilbert Space Problem Book xvii, 365 pages 1974 20 HusEMOLLER Fibre Bundles 2nd ed xvi, 344 pages 1975 21 HUMPHREYS Linear Algebraic Groups xiv 272 pages 1975 22 BARNES/MAcK An Algebraic Introduction to Mathematical Logic x, 137 pages 1975 23 24 GREUB Linear Algebra 4th ed xvii, 451 pages 1975 HoLMES Geometric Functional Analysis and Its Applications x, 246 pages 1975 25 HEWITT/STROMBERG Real and Abstract Analysis 3rd printing viii, 476 pages 1975 26 MANES Algebraic Theories x, 356 pages 1976 27 KELLEY General Topology xiv, 298 pages 1975 28 ZARISKI/SAMUEL Commutative Algebra I xi, 329 pages 1975 29 ZARISKI/SAMUEL Commutative Algebra II x, 414 pages 1976 30 JACOBSON Lectures in Abstract Algebra I: Basic Concepts xii, 205 pages 1976 31 JACOBSON Lectures in Abstract Algebra II: Linear Algebra xii, 280 pages 1975 32 JACOBSON Lectures in Abstract Algebra III: Theory of Fields and Galois Theory ix, 324 pages 1976 33 HIRSCH Differential Topology x, 222 pages 1976 34 SPITZER Principles of Random Walk 2nd ed xiii, 408 pages 1976 35 WERMER Banach Algebras and Several Complex Variabl~s 2nd ed xiv, 162 pages 1976 36 KELLEY/NAMIOKA Linear Topological Spaces xv, 256 pages 1976 37 MoNK Mathematical Logic x, 531 pages 1976 38 GRAUERT/FRITZSCHE Several Complex Variables viii, 207 pages 1976 39 ARVESON An Invitation to C*-Algebras x, 106 pages 1976 40 KEMENY/SNELL/KNAPP Denumerable Markov Chains 2nd ed xii, 484 pages 1976 41 APOSTOL Modular Functions and Dirichlet Series in Number Theory x, 198 pages 1976 42 SERRE Linear Representations of Finite Groups approx 176 pages 1976 43 GILLMAN I J ERISON Rings of Continuous Functions xiii, 300 pages 1976 44 KENDIG Elementary Algebraic Geometry approx 304 pages 1977 45 LOEVE Probability Theory 4th ed Vol xvii, 425 pages 1977 46 LOEVE Probability Theory 4th ed Vol In preparation 1977 ... Spaces Lr 15 1 15 1 15 5 15 6 16 2 10 PROBABILITY DISTRIBUTIONS 10 .1 Distributions and distribution functions 10 .2 The essential feature of pr theory 16 8 16 8 17 2 CoMPLEMENTs AND DETAILS 17 4 CHAPTER... t m= n+l Qm, we have The expected return time r can be written and the central relation can be written Pn = n n L (qm-1L QmPn -m = m= l m= l qm)Pn -m, so that n L qmPn -m m=O = n -1 L m= O qmPll -1 -m. .. FUNCTIONS 11 DISTRIBUTION FuNCTIONS 11 .1 Decomposition 11 .2 Convergence of d.f.'s 11 .3 Convergence of sequences of integrals *11 .4 Further extension and convergence of moments *11 .5 Discussion *12