Điện tử viễn thông lect06 1 khotailieu

6 Stochastic processes (2) lect06.ppt S-38.1145 - Introduction to Teletraffic Theory – Spring 2006 Stochastic processes (2) Contents • • Markov processes Birth-death processes Stochastic processes (2) Markov process • Consider a continuous-time and discrete-state stochastic process X(t) – with state space S = {0,1,…,N} or S = {0,1, } • Definition: The process X(t) is a Markov process if P{ X (t n +1 ) = xn +1 | X (t1 ) = x1, K, X (tn ) = xn } = P{ X (t n +1 ) = xn +1 | X (tn ) = xn } for all n, t1< … < tn+1 and x1,…, xn +1 • This is called the Markov property – Given the current state, the future of the process does not depend on its past (that is, how the process has evolved to the current state) – As regards the future of the process, the current state contains all the required information Stochastic processes (2) Example • Process X(t) with independent increments is always a Markov process: X (tn ) = X (tn −1 ) + ( X (tn ) − X (t n −1 )) • Consequence: Poisson process A(t) is a Markov process: – according to Definition 3, the increments of a Poisson process are independent Stochastic processes (2) Time-homogeneity • Definition: Markov process X(t) is time-homogeneous if P{ X (t + ∆ ) = y | X (t ) = x} = P{ X (∆ ) = y | X (0) = x} for all t, ∆ ≥ and x, y ∈ S – In other words, probabilities P{X(t + ∆) = y | X(t) = x} are independent of t Stochastic processes (2) State transition rates • Consider a time-homogeneous Markov process X(t) • The state transition rates qij, where i, j ∈ S, are defined as follows: qij := lim 1h P{ X ( h) = j | X (0) = i} h↓0 • • The initial distribution P{X(0) = i}, i ∈ S, and the state transition rates qij together determine the state probabilities P{X(t) = i}, i ∈ S, by the Kolmogorov equations Note that on this course we will consider only time-homogeneous Markov processes 6 Stochastic processes (2) Exponential holding times • • • Assume that a Markov process is in state i During a short time interval (t, t+h] , the conditional probability that there is a transition from state i to state j is qijh + o(h) (independently of the other time intervals) Let qi denote the total transition rate out of state i, that is: qi := ∑ qij j ≠i • • • Then, during a short time interval (t, t+h] , the conditional probability that there is a transition from state i to any other state is qih + o(h) (independently of the other time intervals) This is clearly a memoryless property Thus, the holding time in (any) state i is exponentially distributed with intensity qi Stochastic processes (2) State transition probabilities • Let Ti denote the holding time in state i and Tij denote the (potential) holding time in state i that ends to a transition to state j Ti ∼ Exp(qi ), Tij ∼ Exp(qij ) • Ti can be seen as the minimum of independent and exponentially distributed holding times Tij (see lecture 5, slide 44) Ti = Tij j ≠i • Let then pij denote the conditional probability that, when in state i, there is a transition from state i to state j (the state transition probabilities); pij = P{Ti = Tij } = qij qi Stochastic processes (2) State transition diagram • A time-homogeneous Markov process can be represented by a state transition diagram, which is a directed graph where – nodes correspond to states and – one-way links correspond to potential state transitions link from state i to state j ⇔ qij > • Example: Markov process with three states, S = {0,1,2} − + 0   Q =  − +  + + −   q20 q21 q12 q01 Stochastic processes (2) Irreducibility • Definition: There is a path from state i to state j (i → j) if there is a directed path from state i to state j in the state transition diagram – In this case, starting from state i, the process visits state j with positive probability (sometimes in the future) • Definition: States i and j communicate (i ↔ j) if i → j and j → i • Definition: Markov process is irreducible if all states i ∈ S communicate with each other – Example: The Markov process presented in the previous slide is irreducible 10 Stochastic processes (2) Global balance equations and equilibrium distributions • • Consider an irreducible Markov process X(t), with state transition rates qij Definition: Let π = (πi | πi ≥ 0, i ∈ S) be a distribution defined on the state space S, that is: ∑i∈S π i = (N) It is the equilibrium distribution of the process if the following global balance equations (GBE) are satisfied for each i ∈ S: ∑ j ≠i π i qij = ∑ j ≠i π j q ji (GBE) – It is possible that no equilibrium distribution exists, but if the state space is finite, a unique equilibrium distribution does exist – By choosing the equilibrium distribution (if it exists) as the initial distribution, the Markov process X(t) becomes stationary (with stationary distribution π) 11 Stochastic processes (2) Example −  Q = 0 − 1 µ  0  1 −  µ π + π1 + π = 1 (N) π ⋅1 = π ⋅1 π ⋅1 = π ⋅1 + π ⋅ µ (GBE) π ⋅ (1 + µ ) = π ⋅1 ⇒ 1+ µ π = 3+1µ , π = 3+ µ , π = 3+1µ 12 Stochastic processes (2) Local balance equations • • • Consider still an irreducible Markov process X(t).with state transition rates qij Proposition: Let π = (πi | πi ≥ 0, i ∈ S) be a distribution defined on the state space S, that is: ∑i∈S π i = (N) If the following local balance equations (LBE) are satisfied for each i,j ∈ S: • π i qij = π j q ji (LBE) then π is the equilibrium distribution of the process • Proof: (GBE) follows from (LBE) by summing over all j ≠ i • In this case the Markov process X(t) is called reversible (looking stochastically the same in either direction of time) 13 Stochastic processes (2) Contents • • Markov processes Birth-death processes 14 Stochastic processes (2) Birth-death process • Consider a continuous-time and discrete-state Markov process X(t) – with state space S = {0,1,…,N} or S = {0,1, } • Definition: The process X(t) is a birth-death process (BD) if state transitions are possible only between neighbouring states, that is: | i − j |> • ⇒ qij = In this case, we denote µi := qi,i −1 ≥ λi := qi ,i +1 ≥ – In particular, we define µ0 = and λN = (if N < ∞) 15 Stochastic processes (2) Irreducibility • Proposition: A birth-death process is irreducible if and only if λi > for all i ∈ S\{N} and µi > for all i ∈ S\{0} • State transition diagram of an infinite-state irreducible BD process: à1 λ1 µ2 λ2 µ3 State transition diagram of a finite-state irreducible BD process: λ0 µ1 λ1 λN−2 µ2 µN−1 N-1 λN−1 µN N 16 Stochastic processes (2) Equilibrium distribution (1) • Consider an irreducible birth-death process X(t) • We aim is to derive the equilibrium distribution π = (πi | i ∈ S) (if it exists) Local balance equations (LBE): • π i λi = π i +1à i +1 Thus we get the following recursive formula: π i +1 = • (LBE) λi π µ i +1 i ⇒ i λ j −1 πi = π0 ∏ µ j =1 j Normalizing condition (N): i λ j −1 π = π ∑ i ∑ ∏ µj i∈S i∈S j =1 =1 (N) 17 17 Stochastic processes (2) Equilibrium distribution (2) • • Thus, the equilibrium distribution exists if and only if i λ j −1 ∑ ∏ µj i∈S j =1 Finite state space: The sum above is always finite, and the equilibrium distribution is i λ j −1 πi = π0 , j =1 j < N i λ j −1   π = 1 + ∑ ∏ µ   i =1 j =1 j    Infinite state space: If the sum above is finite, the equilibrium distribution is i λ j −1 πi = π0 ∏ µ , j =1 j ∞ i λ j −1   π = 1 + ∑ ∏ µ   i =1 j =1 j    −1 −1 18 18 Stochastic processes (2) Example −  Q = µ 0  λ 0  − λ µ −  λ µ λ µ π i λ = π i +1µ ⇒ π i +1 = ρπ i ( ρ := λ / µ ) (LBE) ⇒ π i = π 0ρi π + π + π = π (1 + ρ + ρ ) = ⇒ πi = (N) ρi 1+ ρ + ρ 19 19 Stochastic processes (2) Pure birth process • • Definition: A birth-death process is a pure birth process if µi = for all i ∈ S State transition diagram of an infinite-state pure birth process: • • λ1 λ2 State transition diagram of a finite-state pure birth process: • λ0 λ0 λ1 λN−2 N-1 λN−1 N Example: Poisson process is a pure birth process (with constant birth rate λi = λ for all i ∈ S = {0,1,…}) Note: Pure birth process is never irreducible (nor stationary)! 20 Stochastic processes (2) THE END 21