convergence of probability measures by patrick billingsley 1968

Convergence of Probability Measures Patrick Billingsley Departments of Statistics and Mathematics The University of Chicago JOHN WILEY & SONS, New York • Chichester • Brisbane • Toronto Copyright © 1968 by John Wiley & Sons, Inc . All rights reserved . Reproduction or translation of any part of this work beyond that permitted by Sections 107 or 108 of the 1976 United States Copyright Act without the permission of the copyright owner is unlawful . Requests for permission or further information . should be addressed to the Permissions Department, John Wiley & Sons, Inc . Library of Congress Catalog Card Number : 68-23922 S BN 471 07242 7 Printed in the United States of America 20 19 18 17 16 15 14 13 TO MY MOTHER Preface Asymptotic distribution theorems in probability and statistics have from the beginning depended on the classical theory of weak convergence of distribu- tion functions in Euclidean space-convergence, that is, at continuity points of the limit function . The past several decades have seen the creation and extensive application of a more inclusive theory of weak convergence of probability measures on metric spaces . There are many asymptotic results that can be formulated within the classical theory but require for their proofs this more general theory, which thus does not merely study itself . This book is about weak-convergence methods in metric spaces, with applications sufficient to show their power and utility . The Introduction motivates the definitions and indicates how the theory will yield solutions to problems arising outside it . Chapter 1 sets out the basic general theorems, which are then specialized in Chapter 2 to the space of continuous functions on the unit interval and in Chapter 3 to the space of functions with discontinuities of the first kind . The results of the first three chapters are used in Chapter 4 to derive a variety of limit theorems for dependent sequences of random variables . Although standard measure-theoretic probability and metric-space topol- ogy are assumed, no general (nonmetric) topology is used, and the few results required from functional analysis are proved in the text or in an appendix . Mastering the impulse to hoard the examples and applications till the last, thereby obliging the reader to persevere to the end, I have instead spread them evenly through the book to illustrate the theory as it emerges in stages . Chicago, March 1968- ~ Patrick Billingsley vii Acknowledgements My thanks go to Soren Johansen, Samuel Karlin, David Kendall, Ronald Pyke, and Flemming Topsoe, who read large parts of the manuscript ; the book owes much to their detailed suggestions, and I am very grateful . I should also like to thank Mary Woolridge for her typing, cheerful, swift, and error-free . The writing of this book was supported in part by the Statistics Branch, Office of Naval Research, and in part by Research Grant No . 8026 from the Division of Mathematical, Physical, and Engineering Sciences of the National Science Foundation . vm ~~~~~~~~ ~~~~~~~~~~~~ ~ ~~~~~~~~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ ~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ ~ ~~~~~~~~~~~~~~~~~~~~~~~ ~~ ~~~~~~~~~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~ ~ ~ ~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~ ~~~~~~~ ~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ ~ ~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ ~ ~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~ ~~~~~~~~~~ ~ ~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~ ~~ ~ ~~~~~~~~ ~ ~ ~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~ ~~~~~~~~~~ ~~~ ~~~ ~~~~~~~ ~~~~~ ~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~ ~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~ ~~ ~~~~~~~~~ ~ ~~~~~~~~~~~ ~ ~~ ~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~ ~~~ ~~~~~~ ~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~ ~~ ~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~ ~~ ~ ~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~ ~~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~ ~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~ ~~ ~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~ ~~~~~~ ~~~ ~~~~~~~~~~~~~~~~ ~~ ~~~~~~~ ~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~ ~~~~~~~~~~~ ~~ ~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~ ~ ~~~~~~~~~ ~ ~ ~~~ ~~ ~ ~~~~~~~~~~~~~~~ ~~ ~~~ ~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~ ~~~~~~ ~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~ ~~~~~~~~ ~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~ ~~ ~ ~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~ ~ ~~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~ ~~ ~ ~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~ ~ ~ ~~~~~~~~~~~~~~~~~~~ ~ ~~~ ~~ ~ ~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~ ~ ~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~ ~~~~~~~~~~~~~~~~~~~~~ ~~ ~ ~~~~~~~~~~~~ ~~~ ~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~ ~ ~~~~~~~~~~~~~ ~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~ ~~~ ~~~~~~~~ ~~~~~~~~~~~ ~ ~~~~~~~~~~ ~ ~~~ ~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~ ~~~~~~~~~~~~ ~ ~ ~~~~ ~~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~ ~ ~~~~~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~ ~~~~~~~~~~~ ~~~~ ~~~~~~~~~ ~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~ ~~~ ~~~~~ ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [...]... fact under ies the circ e of ideas centering on the notion of weak convergence ; a though we have defined weak convergence by requiring the convergence of the integra s of functions in C(S), in the next section we sha characterize it in terms of the convergence of the measures of certain sets Tightness The fo owing notion of tightness proves important both in the theory of weak convergence and in its... the concept of weak convergence of distribution functions is tied to the real line (or to Euclidean space, at any rate), the concept of weak convergence of probability measures can be formulated for the general metric space, which is the real reason for preferring the latter concept Let S be an arbitrary metric space, let / be the class of Borel subsets of S (v° is the a-field generated by the open... defined by (1) X (4) Fn(x) = and (5) F(x) = 0 if x F, and this time the proviso does come into play : (3) fails at x = 0 For a better understanding of this notion of weak convergence, which underlies a large class of limit theorems in probability theory, consider the probability measures Pn and P generated by arbitrary distribution functions Fn and F These probability. .. probabi ity measures on (S, ) These five conditions are equiva ent : t If we topo ogize the space Z(S) of a probabi ity measures on (S, 90) by taking as the genera basic neighborhood of P the set of Q such that I f fZ dP - $ fz dQI < E for i = 1, , k, where E is positive and the fZ ie in C(S), then weak convergence is convergence in this topo ogy The topo ogica structure of Z(S), which wi be of no direct... the proof of Theorem 2 1 Other Criteria It is sometimes convenient to prove weak convergence by showing that P,JA) * P(A) for some specia c ass of sets A THEOREM 2.2 Let GI be a subc ass of 9P such that (i) ° ' is c osed under the formation offinite intersections and (ii) each open set in S is a finite or countab e union of e ements of QI If P,a (A) > P(A) for every A in G', then P,a =:> P Proof If... infinity Just as in the case of the random element of C defined by (13), we can then go on to derive the limiting distributions of sup Jn(F,,,(t, (o) - t) = sup Yn (t, w) o (iii) H (iv) I Of course, (i) (ii) is trivia (v) Proof of (ii) > (iii) Suppose (ii) ho ds and that F is c osed Suppose 6 > 0 For sma enough e, G = {x : p(x, F) < e} satisfies P(G) < P(F) + S, t Each subset of S mentioned is assumed to ie in Y Properties of Weak Convergence 13 since the sets of this form decrease . Convergence of Probability Measures Patrick Billingsley Departments of Statistics and Mathematics The University of Chicago JOHN WILEY & SONS, New York • Chichester • Brisbane • Toronto Copyright. notion of weak convergence ; athough we have defined weak convergence by requir- ing the convergence of the integras of functions in C(S), in the next section we sha characterize it in terms of. characterize it in terms of the convergence of the measures of certain sets . Tightness The foowing notion of tightness proves important both in the theory of weak convergence and in its appications .