We consider two subsets E and F ; the intersection of E and F , denoted by E ∩ F , is the set of all the elements that are both in E and in F E and F represent subsets of events, then the events in E ∩ F occur only if both E and F occur E ∩ F is equivalent to a logical and
ECE 307 – Techniques for Engineering Decisions Basic Probability Review George Gross Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved OUTLINE Definitions Axioms on probability Conditional probability Independence of events Probability distributions and densities discrete continuous © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved SAMPLE SPACE Consider an experiment with uncertain outcomes but with the entire set of all possible outcomes known The sample space S is the set of all possible outcomes; an outcome is an element of S © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved SAMPLE SPACE Examples of sample spaces flipping a coin: S = { H,T } tossing a die: S = {1, 2, 3, 4, 5, 6} flipping two coins:S = {( H , H ), ( H , T ), (T , H ), (T , T )} tossing two dice: S = {( i, j ) : i, j = 1, , 6} hours of life of a device: S = { x : ≤ x < ∞ } © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved SUBSETS We say a set E is a subset of a set F if E is contained in F and we write E ⊂ F or F ⊃ E If E and F are sets of events, then E ⊂ F implies that each event in E is also an event in F Theorem E ⊂ F and F ⊂ E ⇔ E =F © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved SUBSETS F ⊂E F E S © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved EVENTS An event E is an element or a subset of the sample space S Some examples of events are: flipping a coin: E = { H } , F = {T } tossing a die: E = {2,4,6} is the event that the die lands on an even number © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved EVENTS flipping two coins: E = {( H, H ) , ( H,T )} is the event of the outcome T on the second coin tossing two dice: E = {(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)} is the event of sum of the two tosses is hours of life of a device: E = {5 < x ≤ 10} is the event that the life of a device is greater than and at most 10 hours © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved UNION OF SUBSETS We consider two subsets E and F ; the union of E and F denoted by E ∪ F is the set of all the elements that are either in E or in F or in both E and F E and F represent subsets of events, the E ∪F occurs only if either E or F or both occur E ∪ F is equivalent to a logical or © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved UNION OF SUBSETS Examples: E = {2,4,6} ,F = {1,2,3} ⇒ E ∪ F = {1,2,3,4,6} E = { H } , F = {T } ⇒ E ∪F = { H ,T } ≡ S E = set of outcomes of tossing two dice with sum being an even number F = set of outcomes of tossing two dice with sum being an odd number ⇒ E ∪F =S © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved 10 APPLICATION EXAMPLE A company is willing to sell a product G: different levels of product sold result in different net profits and have different probabilities: level of sales probability net profits [M $] high 0.38 medium 0.12 low 0.50 The standard deviation and variance of the net profits X for the product are computed as © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved 59 EXAMPLE E {X} = n ∑ x P { X = x } = ( 0.38) + ( 0.12) + ( 0.50) i i i =1 = 3.52 M$ var { X } = n ∑ ⎡⎣ x i − E { X }⎤⎦ P { X = x i } i =1 = 0.38 ( − 3.52 ) + 0.12 ( − 3.52 ) + 0.5 ( − 3.52 ) 2 = 13.8496 ( M$)2 σ X = var { X } = 13.8496 = 3.72 M$ © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved 60 EXAMPLE Consider the following probabilities: P {Y = 10 | X = 2} = 0.9 P { X = 2} = 0.3 P {Y = 20 | X = 2} = 0.1 P { X = 4} = 0.7 P {Y = 10 | X = 4} = 0.25 P {Y = 20 | X = 4} = 0.75 and compute the covariance and correlation between X and Y © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved 61 EXAMPLE y 20 10 © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved x 62 EXAMPLE Using the definition of conditional probability: P { X = , Y = 10} = P {Y = 10 X = 2} P { X = 2} = ( 0.9 ) ( 0.3 ) = 0.27 P { X = , Y = 20} = P {Y = 20 X = 2} P { X = 2} = ( 0.1 ) ( 0.3 ) = 0.03 P { X = , Y = 10} = P {Y = 10 X = 4} P { X = 4} = ( 0.25 ) ( 0.7 ) = 0.175 P { X = , Y = 20} = P {Y = 20 X = 4} P { X = 4} = ( 0.75 ) ( 0.7 ) = 0.525 © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved 63 EXAMPLE P {Y = 10} = P {Y = 10 X = 2} P { X = 2} + P {Y = 10 X = 4} P { X = 4} = 0.27 + 0.175 = 0.445 P {Y = 20} = − ( 0.445 ) = 0.555 E { X } = ( 0.3 ) + ( 0.7 ) = 3.4 σX = 2 0.3 ( -1.4) + 0.7 0.6 = 0.917 ( ) ( ) E {Y } = ( 0.445 ) 10 + ( 0.555 ) 20 = 5.55 σY = ( 0.445 ) ( - 5.55) + ( 0.555 ) 4.45 © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved = 4.970 64 EXAMPLE x i - E { X } y j - E {Y } ⎡⎣ x i - E { X }⎤⎦ ⋅ P X,Y ⎡ y j - E {Y }⎤ ⎣ ⎦ { xi yj 10 -1.4 -5.55 7.77 0.27 20 -1.4 4.45 -6.23 0.03 10 0.6 -5.55 -3.33 0.175 20 0.6 4.45 2.67 0.525 © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved x i ,y i } 65 EXAMPLE cov { X , Y } = ( 0.27 ) 7.77 + ( 0.03)( - 6.23) + ( 0.525) 2.67 = 2.73 ρ XY = cov { X , Y } σ X σY = 2.73 ( 0.917 )( 4.970) © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved = 0.60 66 CONTINUOUS PROBABILITY DISTRIBUTIONS The specification of continuous probability distribution of a continuous r.v X may be expressed either in terms of a a probability density function ( p.d.f.) f X ( ⋅ ) f X ( x ) dx ≈ P { x < X ≤ x + dx} or, a cumulative distribution function ( c.d.f.) FX ( ⋅ ) which expresses the probability that the value of X is less or equal to a given value x F X ( x ) = P { X ≤ x} = x ∫−∞ f X (ξ ) dξ © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved 67 EXPECTED VALUE, VARIANCE, STANDARD DEVIATION The expected value μ X is given by E { X } = ∫ −∞ ξ f X (ξ ) dξ +∞ The variance var { X } of X is defined by var { X } = ∫ −∞ ⎡⎣ξ − E { X }⎤⎦ f X (ξ ) dξ +∞ The standard deviation σ X of X is σ X = var { X } © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved 68 THE COVARIANCE AND THE CORRELATION The covariance cov { X,Y } of the two continuous r.v.s X and Y +∞ ∫ −∞ ( x − E { X }) ( y − E {Y }) f (ξ ,η ) dξ dη f X,Y (⋅, ⋅) is the joint density function of X cov { X,Y } = ∫ −∞ where +∞ X,Y and Y The correlation coefficient ρ X,Y is computed by ρ X,Y cov { X, Y } = σ Xσ Y © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved 69 APPLICATION We wish to guess the age A of a movie star based on the following data: we are sure that she is older than 29 and not older than 65 we assume the probability that she is between 40 and 50 is 0.8 and P { A > 50} = 0.15 we also estimate that P { A ≤ 40} = 0.05 and P { A ≤ 44} = P { A > 44} © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved 70 APPLICATION We construct the table of cumulative probability P { A ≤ 29} = 0.00 P { A ≤ 40} = 0.05 P { A ≤ 44} = 0.50 P { A ≤ 50} = 0.85 P { A ≤ 65} = 1.00 © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved 71 APPLICATION P { A ≤ x} 1.00 0.75 0.5 0.25 10 20 30 40 50 50 60 years x © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved 72 APPLICATION fA ( x) { } P 40 < age ≤ 50 ~ x 10 20 30 40 50 60 70 years x © 2006 – 2009 George Gross, University of Illinois at Urbana-Champaign, All Rights Reserved 73