probability theory with economic applications

Chapter B P r o b a b ility T h e o r y v ia the L e bes q u e Integral The prim ary objective of this c hapter is to in troduce the basic prob ab ility model from the measure theoretic point of view. Consequently, we first start with discussing the idea behind the form al notion of a prob ability space, and provide a fairly introduc- tory discussion of finite meas ure theory. A good part of this discussion is likely to be new for the economics studen t, so our pace is quite leisurely. In particular, w e discussalgebrasandσ-algebras in detail, pay due atte ntion t o B ore l σ-algebras, and prove s ever al elem enta ry properties of pr ob ab ility mea su re s. Mor eover, we outline the constructions of some useful probabilit y spaces, including those that are induced by di stribution functions. As usual, these constructions are ac hiev ed by inv oking the fundam ental extension theorem of Carathéodory. We omit the proof of the existence part of this theorem, but prove its uniqueness part as an application o f the Sierpinski Class Lemm a. We then introduce the notion of a random variable, and discuss the notion of Borel meas ura bility at some length. The high poin t of the c h ap ter is the in troduction of the Lebesgue integration the- ory within the context of finite measure spaces. In fact, we almost exclusiv ely w ork with probab ility measures, so the Lebesgue integral for the present exposition is none other than the so-called expectation functional. O ur treatmen t is again leisurely. In particular, we introduce the fundamen tal convergence theorems for the Lebesgue in- tegral b y means of a step-by-step approac h. For instance, th e Monotone Conv ergence Theorem is given in four different formulations. First, we pro ve it fo r a sequence of nonnegative random variab les the point wise limit of which is real-valued. Th en w e drop the nonnegativity assumption from the statement of the theorem, and then reintroduce it but this time work with sequences that con verge almost surely to an extended real-valued function. O ur fourth form ulation states the result in full gener- ality. We also study other important properties of the expectation functional, suc h as its linearity, th e c h ange of variables form ula , and Jensen’s Inequality. The c h ap te r concludes with a brief introduction to the normed linear space of in tegrable random variables, and other related spaces. There is, of course, no shortage of truly excellent textbooks on probability theory. In particular, the classic treatmen ts of Billingsley (1986), Durrett (1991), Shiryaev (1996) and Chung (2001) ha ve a scope far more comprehensive than ours. The proofs thatweomithereandinthefollowingchapterscanberecoveredfromanyoneofthese books. A more recent reference, which the presen t author finds most commendable, is Frist edt and Gr ay (1997). 1 1 Event Spaces The most fund amental notion of probability theory is tha t of a proba bility measure. Rough ly speaking, a probab ility measure tells us the likelihood of observing any conceiva ble event in an experimen t the outcome of whic h is uncertain. To formally introduce t his c oncept, howev er, we need to model t he e lusive term “conceivable event” in this description — hence the next subsection. 1.1 σ-Algebras Dhilqlwlrq. Given any nonempty set X,letA and Σ be nonempty subsets of 2 X . The class A is called an algeb r a on X if (i) X\A ∈ A for all A ∈ A;and (ii) A ∪ B ∈ A for all A, B ∈ A. The collection Σ is called a σ-algebra on X if it satisfies (i) and (iii)  ∞ A i ∈ Σ whenever A i ∈ Σ for each i =1,2, Any element of Σ is called a Σ-measurable set in X. If Σ is a σ-algebra on X, we refer to t he pair (X, Σ) as a measur ab le spa ce . In words, an algebra on X is a nonempty collection of subsets of X that is closed under complementation and ta kin g pairwise (and thus finite) unions. It is rea dily verified that both ∅ and X belong to any algebra A on X, andthatanalgebrais closed under taking pairwise (and thus finite) intersections. (To prove the first claim, obse rv e that, since A is nonempt y, there exists an A ⊆ X in A, and hence X\A belongs to X. Thus X = A ∪ (X\A) ∈ A.) Moreover, a collection Σ of subsets of X is a σ-algebra,ifitisanalgebraandisclosedundertakingcountable unions. By the de Mo rg an Law, this also implies that Σ is closed under taking countable (finite or infinite) intersections: If C is a nonempty coun table subset of Σ, then  C ∈ Σ. It is useful to note that there is no difference between an algebra and a σ-algebra when the ground set X under consideration is finite. 1 Before c on s ide rin g some exam p le s, let us provide a quick in te r p retation of the form al mode l at hand. Given a nonempty set X and a σ-algebra Σ on X,wethink of X as the set of all possible outcomes that may result in an experim ent, the so- called sample sp ace, and view a ny member of Σ (and only suc h a subset of X) as an “ev ent” that may take place in the experiment. To illustrate, consider the experim ent of rolling an ordinary die once. It is natural to tak e X := {1, . , 6} as the s ample space of this experimen t. But what is a n “event” h ere? The answer depends on the actual scenario that one wishes to model. If it is possible to discern 1 These are relatively easy claims, but it is probably a good idea to warm up by proving them. In particular, how do you kno w that a σ-algebra is actually an algebra? 2 the differences between all sub sets of X, then we would take 2 X as the σ-algebra of the m odel, thereby deeming an y subset of X as a conceivabl e ev ent (e.g. {1, 2, 3} w ould be the ev en t that “a num ber strictly less than 4 comes up”). O n the other hand, the situatio n we w ish to m odel ma y c all for a different t ype of an ev ent space. For exa mple, if w e wan t to model the beliefs of a person who will be told after the experiment only whetherornot1hascomeup,{1, 2, 3} w ould n ot really be deemed as a conceivable event. (If the outcome is 2, one would lik e to sa y that {1, 2, 3} has occurred, but given her informational limitation, our individual has no wa y of concluding this.) Ind e ed , this person may have an assessm ent only of the likelihood of 1 coming up in the experiment, so a non trivial “event” for her is either “1 comes up” or “1 does n’t come up.” Consequ e ntly, to model the beliefs of this individual, it mak es more sen se to ch oose a σ-algebra lik e {∅,X,{1}, {2, , 6}}. An “even t” in this model w ould then be one of the four members of this particular collection of sets. In practice, then, there is some latitude in choosing a p articular class Σ of events to endow a sample space X with. Ho wev er, we cannot do this in a completely arbitrary way . If A is an event, then w e need to be able to talk about this event not occurring, that is, to deem the set X\A also as an event. This is guaranteed b y condition (i) above. Similarly, we wish to be able to ta lk about at l e ast one of c ountably many events occurring, and this is the rationa le beh ind con d ition (iii) above. In addition, conditions (i) and (iii) force us to view “countably many events occurring simultaneously” as an e vent as well. To giv e an exam ple, consider the experiment of rolling an ordinary die arbitrarily many tim es. Clearly, we would take X = N ∞ as the s ample space o f t his experiment. Suppose next t hat we would lik e to be able talk about the situa tion that in the ith roll of t he die, num ber 2 com es up. Then we w ould choose a σ-algebra that wou ld certain ly contain all sets of the form A i := {(ω m ) ∈ N ∞ : ω i =2}. This σ-alg ebra must contain m any other types of subsets of X. For instance, the situ ation that “in neither the first nor the second roll 2 turns up” must formally be an event, because {(ω m ) ∈ N ∞ : ω 1 , ω 2 =2} equals (X\A 1 ) ∩ (X\A 2 ). Similarly, since each A i is deemed as an “event,” a σ-algebra main tains that  ∞ A i (“2 comes up at least once through the rolls”) and  ∞ A i (“eac h roll results in 2 coming up”) are considered as “events” in our model. In short, g iven a σ-algebra Σ on X, the int uit i ve concept of an “event” is formalized as any Σ-measurab le set. That is, and ma rk this, we sa y tha t A is an even t if and only if A ∈ Σ, and for this reason a σ-algebra on X is often referred to as an even t space on X. One ma y define man y different ev ent spaces on a given sample space, so what an “ev ent” really is depends on the model one c hooses to work with. E{dpsoh 1. [1] 2 X and {∅,X} are σ-algebras on an y nonempt y set X. The collection 2 X corresponds to the finest event space allo w ing each subset of X to be deemed as an “ev ent.” 2 By con trast, {∅,X} is the coarsest possible even t space that allows one to perceive of only two ty pes of event s, “nothing happens” and “something happens.” 2 I have already told you that certain subsets of X may not be deemed as “e vents” for an o bserver 3 [2] Let X := {a, b, c, d}. None of the collections {∅}, {X}, {∅,X,{a}} and {∅,X,{a}, {b, c, d}, {b}, {a, c, d}} qualify as an algebra on X. On the other hand, each of the col- lections {∅,X}, {∅,X,{a}, {b, c, d}} and {{∅,X,{a}, {b, c, d}, {b}, {a, c, d}, {a, b}, {c, d}} is an algebra on X. [3] If X is finite and A is an algebra on X, then A is a σ-algebra. So, as noted earlier, the distinction between the notions of an algebra and a σ-algebra disappear inthecaseoffinite sample spaces. [4] Let us agree to call an interval right-semiclosed if i t has t he form (a, b] with −∞ ≤ a ≤ b<∞, or of the form (a, ∞) with −∞ ≤ a. The class of all righ t semic lose d in te rvals is obviously not an algebra on R. But the set A of all finite unions of right-semic los ed inter vals — called the algebra induced b y righ t- semiclosed intervals —isanalgebraonR.Infact,A is the smallest algebra that contains all right-sem iclosed intervals . I t is not a σ-algebra. (Proofs?) [5] A := {S ⊆ N :min{|S| , |N\S|} < ∞} is an algebra on N but it is not a σ-algebra. Indeed, {i} ∈ A for each odd i ∈ N, but {1, 3, } /∈ A.  Exercise 1. Let X be a metric space, and let A 1 be the class that consists of all open subsets of X, A 2 the class of all closed subsets of X, and A 3 := A 1 ∪ A 2 . Determine if any of these classes is an algebra or a σ-algebra. Exercise 2. Let X beanynonemptyset,andΩ aclassofσ-algebras on X. (a) Show that  Ω is a σ-algebra on X. (b) Give an e xample to sho w that  Ω need not be an algebra even if Ω is finite. Exercise 3. Define A :=  A ⊆ N :( 1 n |A ∩ {1, , n}|) is convergent  . (Note. For any A ∈ A,thenumberlim 1 n |A ∩ {1, , n}| is called the asymptotic density of A.) True or false: A is an algebra but not a σ-algebra. ∗ Exercise 4. Show that a σ-algebra cannot be countably infinite. In practice it is not uncommon that we have a pretty good idea about the kinds of sets we wish to consid er as ev ents, but we have difficulty in terms o f finding a “good” σ-algebra for t he problem because the collection of s ets we ha ve at h and does not constitute a σ-algebra. T he resolution is usually to extend the collection with limited information, so 2 X may not always be the relev ant event space to endow X with. (I will talk about this issue at greater length when studying the notion of conditional probability in Chapter F.) Apart from this, there are also technical reasons for why one cannot alway s view 2 X as a useful event space. Roughly speaking, when X is an infinite set, 2 X may be “too large” of a set for one to be able to assign probability numbers to each element of 2 X in a nontrivial way. (More on this in Section 3.5.) 4 of sets which we are interested in to a σ-algebra in a minimal wa y. (We consider a minimal extension because we wish to depart from our “in teresting” sets as little as possib le. Other wis e taking 2 X as the event space would trivially solve t he problem of extension.) This idea leads us to the follo wing fundamen tal concept. Dhilqlwlrq. Let X be a nonempt y set and A a nonempt y subclass of 2 X .The sma lle st σ-algebra on X that contain s A (in the sense t hat this σ-algebra is included in any other σ-algebra that contains A ) is called the σ-algebra gene rated by A, and is denoted as σ(A). For example, if X := {a, b, c}, then σ({∅})=σ({X})={∅,X}, σ({∅,X,{a}})= {∅,X,{a}, {b, c}}, and σ({∅,X,{a}, {b}})=2 X . Of course, we ha v e Σ = σ(Σ) for an y σ-algebra Σ on any non emp ty set. Does any nonempt y class of sets generate a σ-algebra? The answer does not follow readily from the definition above, because i t is not se lf-evid ent i f we can always find a s mallest σ-algeb ra that extend s any given nonempty class of se ts. Our first proposition, howev e r, sh ows that we can actually do this, so t h ere is r eally no e xis te nce problem regar ding genera ted σ-alge bras. 3 Pursrvlwlrq 1. Let X be a nonempty set and A a nonempty subclass of 2 X .There exists a unique smallest σ-algebra that includes A,soσ(A)iswell-defined. We ha v e σ(A)=  {Σ : Σ is a σ-algebr a and A ⊆ Σ} . Exercise 5. H Prove Proposition 1. Exercise 6. Does the σ-algebra generated by the algebra of Example 1.[4] include all open sets in R? Exercise 7. H Compute σ(A), where A := {S ⊆ R :min{|S| , |R\S|} < ∞}. 1.2 Borel σ-alge br a s Let X be an y metric space, and let O X stand for the set of all open sets in X. The mem bers of O X are of obvious importance, but unfortunately O X need not ev en be an algebra. In metric spaces, then, it is natural to consider the σ-algeb r a generated by O X . This σ-algebra is called the Borel σ-algebra on X, and its members are referred to as Borel sets (or in probabilistic jargon, Borel e vents). Throughout this text, we denote the Borel σ-algebra on a m etric space X by B(X). By definition, therefore, w e hav e B(X)=σ(O X ). 3 As you will soon painfully find out, however, the explicit characterization of a generated σ- algebra can be a seriously elusive problem. Just to get a feeling for the difficulties that one ma y encoun ter in this regard, try t o “compute” the σ-algebra σ ({{a} : a ∈ Q}) on R. 5 Notation. We write B[a, b] for B([a, b]), and B(a, b] for B((a, b]),where−∞ <a< b<∞. E{dpsoh 2. By definition, B(R)=σ(O R ), but one does no t actually need all open sets in R for generat ing B(R). For instance, what if we used instead the class of all open in tervals, call it A 1 , as a pr imitive collection and attemp t to find σ(A 1 )? This w ould lead us exactly to the σ-algeb ra σ(O R )! To see this, observe first that σ(O R ) is ob viou s ly a σ-algebra that contains A 1 so that we clearly have σ(A 1 ) ⊆ σ(O R ). (Recall the definition of σ(A 1 )!) To establish the converse containment, remember that every open set in R can be written as the union of countably m an y open intervals. (Right?) Thus, w e ha ve O R ⊆ σ(A 1 ). (Why exactly?) But then, since σ(O R ) is the sma lle st σ-algebra th at con tain s O R ,andσ(A 1 ) is of cou rs e a σ-algebra, w e must have σ(O R ) ⊆ σ(A 1 ). So, w e conclude: σ(O R )=σ(A 1 ). In fact, there are all sorts of other w ays of generating the Borel σ-algebra on R. For instance, consider the follo wing classes: A 2 := th e set of all closed intervals A 3 := th e set of all closed sets in R A 4 := th e set of all intervals of the form (a, b] A 5 := th e set of all intervals of the form (−∞,a] A 6 := th e set of all intervals of the form (−∞,a). It is easy to show that all of these collection s genera te the same σ-algebra: B(R):=σ(O R )=σ(A 1 )=···= σ(A 6 ). (1) We h ave already sho wed that σ(O R )=σ(A 1 ). On the other hand, f or any closed interval [a, b], we have [a, b]=  ∞  a − 1 i ,b+ 1 i  ∈ σ(O R ), so w e ha v e A 2 ⊆ σ(O R ) so that σ(A 2 ) ⊆ σ(O R ). Conversely, for any open interval (a, b), we have (a, b)=  ∞  a + 1 i ,b− 1 i  ∈ σ(A 2 ). So A 1 ⊆ σ(A 2 ), and it follo ws that σ(A 1 ) ⊆ σ(A 2 ). The rest of the claims in (1) can be pro ved sim ilarly.  This example shows that different collections of sets migh t well generate the same σ-algebra. In fact, it is generally true that the Borel σ-algebra on a metric space is also generated by the class of all closed subsets of this space. That is, for any metric space X, B(X):=σ({O ⊆ X : O is open})=σ({S ⊆ X : S is closed}). (Verify!) T h e fo llowin g exerc ises play on this them e a bit more. Exercise 8. Show that there is a countable subset A of 2 R such that σ(A)=B(R). 6 Exercise 9. For any n ∈ N, let A 1 := {X n J i : J i is a bounded open interval,i=1, ,n} , A 2 := {X n J i : J i is a bounded right-closed in terval,i=1, , n} , A 3 := {X n J i : J i is a bounded closed interval,i=1, , n} . Show that we have B(R n )=σ(A 1 )=σ(A 2 )=σ(A 3 ). Exercise 10. Prove: If X is a separable metric space, then B(X)=σ({N ε,X (x): x ∈ X and ε > 0}). ∗ Exercise 11. For any m ∈ N, and (t i ,B i ) ∈ [0, 1] ×B[0, 1],i=1, , m, define A(t 1 , , t m ,B 1 , , B m ):={f ∈ C[0, 1] : f(t i ) ∈ B i ,i=1, , m}, and A := {A(t 1 , , t m ,B 1 , ,B m ):m ∈ N and (t i ,B i ) ∈ [0, 1]×B[0, 1],i =1, , m}. Prove that σ(A)=B(C[0, 1]). The fact that there is ofte n no wa y of giving an explicit description of a generated σ-algebra is a source of discom f ort. Ne verth eless , one c an usually say quite a b it about σ(A) ev en without ha ving a specific formula th at tells us h ow its members are deriv ed from those of A. Indeed, in all of the examples (exercises) considered abo ve, w e (you) have “ computed” σ(A) by using the definition of the “generated σ-algebra” directly. The fo llowing ex ercise provides anoth er illustration of this. Exercise 12. H Let X be a metric space, and Y ametricsubspaceofX. Prov e that B(Y )={B ∩ Y : B ∈ B(X)}. Th e obse r vation noted in the previous exe r cis e is quite useful. For instance , it implies th at the knowledge of B(R) is sufficien t to d escribe the c lass of all Borel subsets of [0, 1];wehaveB[0, 1] = {B ∩ [0, 1] : B ∈ B(R)}. Similarly, B(R n + )= {B ∩ R n + : B ∈ B(R n )}. We conclude with a less immediate corollary. Exercise 13. H For any S ∈ B[0, 1] and α ∈ R, show that (S + α) ∩ [0, 1] ∈ B[0, 1]. 2 Probabilit y S paces We are now ready to introduce the con cep t of pro ba bility measure. 4 4 The ori gins of probability theory goes back to the famous exchange between Blaise Pascal and Pierre Fermat t hat started in 1654. While Pascal and Ferma t were mostly concerned with gambling 7 Dhilqlwlrq. Let (X, Σ) be a measurable space. A function p : Σ → R is said to be σ-additive if p  ∞  i=1 A i  = ∞  i=1 p(A i ) for a ny (A m ) ∈ Σ ∞ with A i ∩ A j = ∅ for each i = j. Any σ-additive function p : Σ → R + with p(∅)=0is called a measure on Σ (or on X if Σ is clear from the context), and we refer to the list (X, Σ,p) as a measure space.Ifp(X) < ∞, then p is called a finite measure, and the list (X, Σ,p) is referred to as a finite measure space. In particular, if p(X)=1holds, then p is said to be a probability measure , andinthiscase,(X, Σ,p) is called a pro bability spa ce. Dhilqlwlrq. Gi ven a metric space X,anymeasurep on B(X) is called a Borel measure on X, andinthiscase(X, B(X),p) is referred to as a Borel space.If,in addition, p is a probability measure, then (X, B(X),p) is called a Borel probabilit y space. Notation. Throughout this text, the set of all Borel probabilit y measures on a metric space X is denoted a s P(X). We think of a probability mea sure p as a function that assigns to each event (that is, to each member of the σ-algeb ra that p is defined on) a number between 0 and 1. This n umber corresponds to the likelihood of the occurrence of that event. The map p is σ-additive in the sense that it is additiv e with respect to countably many pairwise disjoint ev ents. This additivity property, which is the heart and soul of measure theory, entails several other useful properties for probability measures. For instance, it implies that any proba b ility measure is finitely additive,thatis, p  m  i=1 A i  = m  i=1 p(A i ) for any finite class {A 1 , , A m } of pairw ise disjoin t events. (To verify this, use σ- additivity an d the fact that {A 1 , , A m , ∅, ∅, } is a c ollection of pairwise disjoint ev ents.) In turn, this implies that, for an y even t s A and B with A ⊆ B, we have p(B\A)=p(B)− p(A),becausep(B)=p(A ∪ (B\A)) = p(A)+p(B\A). Some other useful properties of probability measures are giv en next. t ype problems, the importance and applicability of the general topic was shortly understood, a nd the subject was dev eloped by many mathematicians, including Jakob Bernoulli, Abraham de Moivre, and Pierre Laplace. Despite the host of work that took place in the 18th and 19th centuries, howev er, a universally agreed definition of “probabilit y” did not appear until 1933. A t this date Andrei Kolmogorov introduced the (axiomatic) definition that we are about to present, and set the theory on rigorous grounds, m uch the same w ay Euclid has given an axiomatic basis for planar geometry. 8 Exercise 14. Let (X, Σ,p) be a probability space, m ∈ N, and let A, B, A i ∈ Σ, i =1, , m. Pro ve: (a)If A ⊆ B, then p(A) ≤ p(B), (b) p(X\A)=1− p(A), (c) p(  m A i ) ≤  m p(A i ), (d) (Bonferroni’s Inequality) p(  m A i ) ≥  m p(A i ) − (m − 1). Warnin g. One is often tempted to conclude from Exercise 14.(a) that an y subset of an event of proba bility zero occurs with probability zero. Th ere is a catc h here . How do yo u know that this subset is assigned a probability at all? For instan ce, let X := {a, b, c}, Σ := {∅,X,{a, b }, {c}} and let p be th e probability measure on Σ that satisfies p({c})=1. Here, while p({a, b})=0, it is not true that p({a})=0since p is not even d efined at {a}. This probab ility space maintains that {a} is not an event. Note. Those probability spaces for which any subset of an event o f probability zero is an event (and hence occurs with probability zero) are called complete.Withthe excep tion o f a few (optional) rem arks, however, this notion w ill not play an im portant role in the present exposition. Exercise 15. (The Exclusion-Inclusion Formula) Let (X, Σ,p) be a probability space, m ∈ N, and A 1 , , A m ∈ Σ. Where N t := {(i 1 , ,i t ) ∈ {1, , m} t : i 1 < ···<i t },t=1, , m, show that p  m  i=1 A i  =  i∈N 1 p(A i ) −  (i,j)∈N 2 p(A i ∩ A j )+  (i,j,k)∈N 3 p(A i ∩ A j ∩ A k ) − ···+(−1) m−1 p  m  i=1 A i  . The follo w ing are simple but surprisingly useful observations. Pursrvlwlrq 2. Let (X, Σ,p) be a probability space, and let (A m ) ∈ Σ ∞ . If A 1 ⊆ A 2 ⊆ ···(in which case w e say that (A m ) is an increasin g seq uence),then lim p(A m )=p  ∞  i=1 A i  . On the other hand, if A 1 ⊇ A 2 ⊇ ··· (in which case we say that (A m ) is a de creasing sequence),then lim p(A m )=p  ∞  i=1 A i  . Proof. Let (A m ) ∈ Σ ∞ be an increasing sequence. Set B 1 := A 1 and B i := A i \A i−1 ,i=2, , and note that B i ∈ Σ for each i and  ∞ A i =  ∞ B i . But 9 B i ∩ B j = ∅ for any i = j, so, by σ-additivit y, p  ∞  i=1 A i  = p  ∞  i=1 B i  = ∞  i=1 p(B i ) = lim m→∞ m  i=1 p(B i ) = lim m→∞ p  m  i=1 B i  = lim m→∞ p (A m ) . The proof of the second claim is left as an exercise.  As an immed iate application of this result and Exercise 14.(c), w e obtain a ba sic inequality of probability theory: Boole’s Inequalit y. For any prob ability space (X, Σ,p), p  ∞  i=1 A i  ≤ ∞  i=1 p(A i ) for any (A m ) ∈ Σ ∞ . Proof. Use Proposition 2 and Exercise 14.(c).  Exercise 16. H Let (X, Σ,p) be a probability space. Show that if (A m ) ∈ Σ ∞ satisfies p(A i ∩ A j )=0for every i = j, then p  ∞  i=1 A i  = ∞  i=1 p(A i ). Exercise 17. Given an y probability space (X, Σ,p),showthat p  ∞  i=1 A i  − p  ∞  i=1 B i  ≤ ∞  i=1 (p(A i ) − p(B i )) for all (A m ), (B m ) ∈ Σ ∞ with B m ⊆ A m ,m=1, 2, Exercise 18. H Let (X, B(X),p) be a Borel probabilit y space, and let O X and C X denote the class of all open and closed subsets of X, respectively. (a)Provethat sup{p(T ) ∈ C X : T ⊆ S} = p(S)=inf{p(O) ∈ O X : S ⊆ O} for any S ∈ B(X). (b)Showthat,ifX is σ-compact, that is, it can be written as a union of countably many compact subsets of itself, then p(S)=sup{p(K):Kis a compact subset of Xwith K ⊆ S} for any S ∈ B(X). (Note. Suc h a Borel probability measure is said to be regular.) 10 [...]... c, d} and A := {{a, b}, {b, c}} so that σ(A) = 2X Now let p be the probability measure on 2X that assigns probability 1 to the outcomes b and d, and let q be the probability measure that assigns 2 probability 1 to the outcomes a and c Clearly, p and q are probability measures on 2 2X with p = q on A but p = q in general (Compare with Proposition 4.) What goes wrong here is that the Sierpinski Class... induces a probability measure pf on a σ-algebra Σ on X as follows: pf (S) := f (ω) ω∈ supp(f )∩S for any S ∈ Σ (Note If supp(f ) ∩ S = ∅, then pf (S) = 0.) For obvious reasons, such a probability measure is called simple We denote the set of all simple probability measures by 12 Ps (X), and say that (X, Σ, pf ) is a simple probability space It is easily seen that any probability space (X, 2X , p) with |X|... in the model at hand Compute the probability of each of these events Exercise 23 Let (X, Σ, q) be a probability space, and S a subset of X with S ∈ Σ / (a) Show that σ(Σ ∪ {S}) = {(S ∩ A) ∪ ((X\S) ∩ B) : A, B ∈ Σ} (b) By using Carathéodory’s Extension Theorem, show that there is a probability measure p on σ(Σ ∪ {S}) such that p(A) = q(A) for each A ∈ Σ (c) Do part (b) without using Carathéodory’s Extension... definition, a Borel probability measure is simple iff it has finite support (Note Such measures are referred to as simple lotteries in decision theory. ) Exercise 20 Let X be a nonempty countable set and p : 2X → R Prove: (a) (X, 2X , p) is a probability space iff there exists an f ∈ [0, 1]X such that p(S) = f (ω) (b) If (X, 2X , p) is a probability space such that there exist an ε > 0 and f ∈ [ε, 1]X with p(S)... bit silly to say that x is defined on a probability space (X, Σ, p) — x is fully identified by the measurable space (X, Σ) However, in probability theory, one is foremost interested in the distribution of a random variable, and that surely depends on the probability measure that one uses on (X, Σ) For this reason, probabilists often talk of “a random variable x on a probability space (X, Σ, p), ” and henceforth,... following: One can always define a probability measure on the reals by means of a distribution function Interestingly, the converse of this is also true That is to say, any Borel probability measure p on R arises this way Indeed, for any such p, the map t → p((−∞, t]) on R is a distribution function This means that, on R, a Borel probability measure can actually be identified with a distribution function... For instance, if X := (a, b] with −∞ < a < b < ∞, and F ∈ [0, 1]X is an increasing and right-continuous function with F (a+) = 0 and F (b) = 1, then we can define the Lebesgue-Stieltjes probability measure induced by F on (a, b] by using precisely the approach developed in Example 5 It is not difficult to show that this measure is the restriction of the Lebesgue-Stieltjes probability measure induced by... subset B of an -null event Then ([0, 1], L[0, 1], ∗ ) is a complete probability space.14 This space — called the Lebesgue probability space — extends ([0, 1], B[0, 1], ) in the sense that B[0, 1] ⊆ L[0, 1] and ∗ |B[0,1] = Moreover, it is the smallest such extension in the sense that if ([0, 1], Σ, μ) is any complete probability space with B[0, 1] ⊆ Σ and μ|B[0,1] = , then L[0, 1] ⊆ Σ Curiously, L[0,... statement Thomas (1985) provides an alternative proof that derives from basic graph theory Note Lebesgue nonmeasurable sets cannot be found by the finite constructive method Loosely said, Solovay (1970) have shown that the existence of such a set in [0, 1] cannot be proved (within the axiomatic system of standard set theory) without invoking the Axiom of Choice (If you’re interested in these sort of things,... motivation behind the term “continuity of a probability measure.” 5 11 Exercise 19 Let A := {S ⊆ N : min{|S| , |N\S|} < ∞}, and define p : A → [0, 1] as p(S) := 1, |N\S| < ∞ 0, |S| < ∞ Show that p is finitely additive, but not σ -additive Let us conclude this section with a brief summary At this point, you should be somewhat comfortable with the notion of probability space (X, Σ, p) In such a space, . an event. Note. Those probability spaces for which any subset of an event o f probability zero is an event (and hence occurs with probability zero) are called complete.Withthe excep tion o f. collection with limited information, so 2 X may not always be the relev ant event space to endow X with. (I will talk about this issue at greater length when studying the notion of conditional probability. measure. 4 4 The ori gins of probability theory goes back to the famous exchange between Blaise Pascal and Pierre Fermat t hat started in 1654. While Pascal and Ferma t were mostly concerned with gambling 7 Dhilqlwlrq.