1074 PROBABILITY THEORY For the integral (20.4.1.10) to exist, it is necessary and sufficient that the following limit exist: lim λ→0 n k=1 n l=1 B ξξ (s k , s l )(t k – t k–1 )(t l – t l–1 ), where λ =max k (t k – t k–1 ). In particular, the integral (20.4.1.10) exists if the following repeated integral exists: b a b a B ξξ (s, t) dt ds. 20.4.2. Models of Stochastic Processes 20.4.2-1. Stationary stochastic process. A stochastic process ξ(t)issaidtobestationary if its probability characteristics remain the same in the course of time, i.e., are invariant under time shifts t → t + a, ξ(t) → ξ(t + a) for any given a (real or integer for a stochastic process with continuous or discrete time, respectively). For a stationary process, the mean value (the expectation) E{ξ(t)} = E{ξ(0)} = m is a constant, and the correlation function is determined by the relation E{ ξ(t)ξ(t + τ)} = B ξξ (τ), (20.4.2.1) where ξ(t)is the function conjugate to a function ξ(t). The correlation function is positive definite: n k=1 n j=1 c k c j B ξξ (t k – t j )=E n k=1 c k ξ(t k ) ≥ 0. In this case, the following relations hold: B ξξ (τ)=B ξξ (–τ), B ξζ (τ)=B ζξ (–τ) |B ξξ (τ)| ≤ B ξξ (0), |B ξζ (τ)| 2 ≤ B ξξ (0)B ζζ (0), (20.4.2.2) where B ξξ (s, t)andB ζξ (s, t) are the function conjugate to a function B ξξ (s, t)andB ζξ (s, t), respectively. Stochastic processes for which E{ξ(t)} and E{ ξ(t)ξ(t + τ)} are independent of t are called stationary stochastic processes in the wide sense. Stochastic processes, all of whose characteristics remain the same in the course of time, are called stationary stochastic processes in the narrow sense. K HINCHIN’S THEOREM. The correlation function B ξξ (τ) of a stationary stochastic process with continuous time can always be represented in the form B ξξ (τ)= ∞ –∞ e iτλ dF (λ), (20.4.2.3) where F (λ) is a monotone nondecreasing function, i is the imaginary unit, and i 2 =–1 . 20.4. STOCHASTIC PROCESSES 1075 If B ξξ (τ) decreases sufficiently rapidly as |τ | →∞(as happens most often in applications provided that ξ(t) is understood as the difference ξ(t)–E{ξ(t)}, i.e., it is assumed that E{ξ(t)} = 0), then the integral on the right-hand side in (20.4.2.3) becomes the Fourier integral B ξξ (τ)= ∞ –∞ e iτλ f(λ) dλ,(20.4.2.4) where f(λ)=F λ (λ) is a monotone nondecreasing function. The function F(λ) is called the spectral function of the stationary stochastic process, and the function f (λ) is called its spectral density. The process ξ(t) itself admits the spectral resolution ξ(t)= ∞ –∞ e itλ dZ (λ), (20.4.2.5) where Z(λ) is a random function with uncorrelated increments (i.e., a function such that E dZ (λ 1 ) dZ (λ 2 ) = 0 for λ 1 ≠ λ 2 ) satisfying the condition |E{dZ (λ)}| 2 = dF (λ)andthe integral is understood as the mean-square limit of the corresponding sequence of integral sums. 20.4.2-2. Markov processes. A stochastic process ξ(t) is called a Markov process if for two arbitrary times t 0 and t 1 , t 0 < t 1 , the conditional distribution of ξ(t 1 ) given all values of ξ(t)fort ≤ t 0 depends only on ξ(t 0 ). This property is called the Markov property or the absence of aftereffect. The probability of a transition from state i to state j in time t is called the transition probability p ij (t)(t ≥ 0). The transition probability satisfies the relation p ij (t)=P [ξ(t)=j|ξ(0)=i]. (20.4.2.6) Suppose that the initial probability distribution p 0 i = P ξ(0)=i , i = 0, 1, 2, is given. In this case, the joint probability distribution of the random variables ξ(t 1 ), , ξ(t n )forany0 = t 0 < t 1 < ··· < t n is given by P ξ(t 1 )=j 1 , , ξ(t n )=j n = i p 0 i p ij 1 (t 1 –t 0 )p ij 2 (t 2 –t 1 ) p j n–1 j n (t n –t n–1 ); (20.4.2.7) and, in particular, the probability that the system at time t > 0 is in state j is p j (t)= i p 0 i p ij (t), j = 0, 1, 2, The dependence of the transition probabilities p ij (t) on time t ≥ 0 is given by the formula p ij (s + t)= k p ik (s)p kj (t), i, j = 0, 1, 2, (20.4.2.8) Suppose that λ ij =[p ij (t)] t t=0 , j = 0, 1, 2, The parameters λ ij satisfy the condition λ ii = lim h→0 p ii (h)–1 h =– i≠j λ ij , λ ij = lim h→0 p ij (h) h ≥ 0 (i ≠ j). (20.4.2.9) 1076 PROBABILITY THEORY THEOREM. Under condition (20.4.2.9), the transition probabilities satisfy the system of differential equations [p ij (t)] t = k λ ik p kj (t), i, j = 0, 1, 2, (20.4.2.10) The system of differential equations (20.4.2.10) is called the system of backward Kol- mogorov equations. T HEOREM. The transition probabilities p ij (t) satisfy the system of differential equations [p ij (t)] t = k λ kj p ik (t), i, j = 0, 1, 2, (20.4.2.11) The system of differential equations (20.4.2.11) is called the system of forward Kol- mogorov equations. 20.4.2-3. Poisson processes. For a flow of events, let Λ(t) be the expectation of the number of events on the interval [0, t). The number of events in the half-open interval [a, b) is a Poisson random variable with parameter Λ(b)–Λ(a). The probability structure of a Poisson process is completely determined by the function Λ(t). The Poisson process is a stochastic process ξ(t), t ≥ 0, with independent increments having the Poisson distribution; i.e., P [ξ(t)–ξ(s)=k]= [Λ(t)–Λ(s)] k k! e Λ(t)–Λ(s) for all 0 ≤ s ≤ t, k = 0, 1, 2, ,andt ≥ 0. A Poisson point process is a stochastic process for which the numbers of points (counting multiplicities) in any disjoint measurable sets of the phase space are independent random variables with the Poisson distribution. In queueing theory, it is often assumed that the incoming traffic is a Poisson point process. The simplest point process is defined as the Poisson point process characterized by the following three properties: 1. Stationarity. 2. Memorylessness. 3. Orderliness. Stationarity means that, for any finite group of disjoint time intervals, the probability that a given number of events occurs on each of these time intervals depends only on these numbers and on the duration of the time intervals, but is independent of any shift of all time intervals by the same value. In particular, the probability that k event occurs on the time interval from τ to τ + t is independent of τ and is a function only of the variables k and t. Memorylessness means that the probability of the occurrence of k events on the time interval from τ to τ +t is independent of how many times and how the events occurred earlier. This means that the conditional probability of the occurrence of events on the time interval from τ toτ +t underany possible assumptions concerning the occurrence of the events before time τ coincides with the unconditional probability. In particular, memorylessness means that the occurrences of any number of events on disjoint time intervals are independent. Orderliness expresses the requirement that the occurrence of two or more events on a small time interval is practically impossible. 20.4. STOCHASTIC PROCESSES 1077 20.4.2-4. Birth–death processes. Suppose that a system can be in one of the states E 0 , E 1 , E 2 , , and the set of these states is finite or countable. In the course of time, the states of the system vary; on a time interval of length h, the system passes from the state E n to the state E n+1 with probability λ n h + o(h) and to the state E n–1 with probability υ n h + o(h). The probability to stay at the same state E n on a time interval of length h is equal to 1 – λ n h – υ n h + o(h). It is assumed that the constants λ n and υ n depend only on n and are independent of t and of how the system arrived at this state. The stochastic process described above is called a birth–death process. If the relation υ n = 0 holds for any n ≥ 1, then the process is called a pure birth process. If the relation λ n = 0 holds for any n ≥ 1, then the process is called the death process. Let p k (t) be the probability that the system is in the state E k at time t. Then the birth–death process is described by the system of differential equations [p 0 (t)] t =–λ 0 p 0 (t)+υ 1 p 1 (t), [p k (t)] t =–(λ k + υ k )p k (t)+λ k–1 p k–1 (t)+υ k+1 p k+1 (t), k ≥ 1. (20.4.2.12) Example 1. Consider the system consisting of the states E 0 and E 1 . The system of differential equations for the probabilities p 0 (t)andp 1 (t)hastheform [p 0 (t)] t =–λp 0 (t)+υp 1 (t), [p 1 (t)] t = λp 0 (t)–υp 1 (t). The solution of the system of equations with the initial conditions p 0 (0)=1, p 1 (0)=0 has the form [p 0 (t)] t = υ υ + λ 1 + υ λ e –(υ+λ)t , [p 1 (t)] t = λ υ + λ 1 – υ λ e –(υ+λ)t . FELLER THEOREM. For the solution p k (t) of the pure birth equations to satisfy the relation ∞ k=0 p k (t)=1,( 20 .4.2.13) for all t , it is necessary and sufficient that the following series diverge: ∞ k=0 1 λ k .( 20 .4.2.14) 1078 PROBABILITY THEORY In the case of a pure birth process, the system of equations (20.4.2.11) can be solved by simple successive integration, because the differential equations have the form of simple recursive relations. In the general case, it is already impossible to find the function p k (t) successively. The relation ∞ k=0 p k (t)=1 holds for all t if the series ∞ k=1 k i=1 υ i λ i (20.4.2.15) diverges. If, in addition, the series ∞ k=1 k i=1 λ i – 1 υ i (20.4.2.16) converges, then there exist limits p k = lim t→∞ p k (t)(20.4.2.17) for all t. If relation (20.4.2.17) holds, then system (20.4.2.12) becomes – λ 0 p 0 + υ 1 p 1 = 0, –(λ k + υ k )p k + λ k–1 p k–1 + υ k+1 p k+1 = 0, k ≥ 1. (20.4.2.18) The solutions of system (20.4.2.18) have the form p k = λ k–1 υ k p k–1 = k i=1 λ i – 1 υ i p 0 .(20.4.2.19) The constant p 0 is determined by the normalization condition ∞ k=0 p k (t)=1: p 0 = 1 + ∞ k=1 k i=1 λ i – 1 υ i .(20.4.2.20) Example 2. Servicing with queue. A Poisson flow of jobs with parameter λ arrives at n identical servers. A server serves a job in random time with the probability distribution H(x)=1 – e –υx . If there is at least one free server when a job arrives, then servicing starts immediately. But if all servers are occupied, then the new jobs enter a queue. The conditions of the problem satisfy the assumptions of the theory of birth–death processes. In this problem, λ k = λ for any k, υ k = kυ for k ≤ n,andυ k = nυ for k ≥ n. By formulas (20.4.2.19) and (20.4.2.20), we have p k = ⎧ ⎪ ⎨ ⎪ ⎩ ρ k k! p 0 for k ≤ n, ρ k n! n k–n p 0 for k ≥ n, where ρ = λ/υ. REFERENCES FOR CHAPTER 20 1079 The constant p 0 is determined by the relation p 0 = n k=0 ρ k k! + ρ n n! ∞ k=n+1 ρ n k–n –1 . If ρ < n,then p 0 = 1 + n k=1 ρ k k! + ρ n+1 n!(n – ρ)! –1 . But if ρ ≥ n, the series in the parentheses is divergent and p k = 0 for all k; i.e., in this case the queue to be served increases in time without bound. Example 3. Maintenance of machines by a team of workers. A team of l workers maintains n identical machines. The machines fail independently; the probability of a failure in the time interval (t, t + h) is equal to λh + o(h). The probability that a machine will be repaired on the interval (t, t + h) is equal to υh + o(h). Each worker can repair only one machine; each machine can be repaired only by one worker. Find the probability of the event that a given number of machines is out of operation at a given time. Let E k be the event that exactly k machines are out of operation at a given time. Obviously, the system can be only in the states E 0 , , E n . We deal with a birth–death process such that λ k = (n – k)λ for 0 ≤ k < n, 0 for k = n, υ k = kυ for 1 ≤ k < l, lυ for l ≤ k ≤ n. By formulas (20.4.2.19) and (20.4.2.20), we have p k = ⎧ ⎪ ⎨ ⎪ ⎩ n! k!(n – k)! ρ k p 0 for 1 ≤ k ≤ l, n! l n–k l!(n – k)! ρ k p 0 for l ≤ k ≤ n, where ρ = λ/υ. The constant p 0 is determined by the relation p 0 = l k=0 n! k!(n – k)! ρ k + n k=l+1 n! l n–k l!(n – k)! ρ k –1 . References for Chapter 20 Bain,L.J.andEngelhardt,M.,Introduction to Probability and Mathematical Statistics, 2nd Edition (Duxbury Classic), Duxbury Press, Boston, 2000. Bean, M. A., Probability: The Science of Uncertainty with Applications to Investments, Insurance, and Engi- neering, Brooks Cole, Stamford, 2000. Bertsekas, D. P. and Tsitsiklis, J. N., Introduction to Probability, Athena Scientific, Belmont, Massachusetts, 2002. Beyer, W. H. (Editor), CRC Standard Probability and Statistics Tables and Formulae, CRC Press, Boca Raton, 1990. Burlington, R. S. and May, D., Handbook of Probability and StatisticsWith Tables, 2nd Edition, McGraw-Hill, New York, 1970. Chung, K. L., A Course in Probability Theory Revised, 2nd Edition, Academic Press, Boston, 2000. DeGroot, M. H. and Schervish, M. J., Probability and Statistics, 3rd Edition, Addison Wesley, Boston, 2001. Devore, J. L., Probability and Statistics for Engineering and the Sciences (with CD-ROM and InfoTrac), 6th Edition, Duxbury Press, Boston, 2003. Freund, J. E., Introduction to Probability, Rep. Edition (Dover Books on Advanced Mathematics), Dover Publications, New York, 1993. Garcia, L., Probability and Random Processes for Electrical Engineering: Student Solutions Manual, 2nd Edition, Addison Wesley, Boston, 1993. Gnedenko, B. V. and Khinchin, A. Ya., An Elementary Introduction to the Theory of Probability, 5th Edition, Dover Publications, New York, 1962. Ghahramani, S., Fundamentals of Probability, with Stochastic Processes, 3rd Edition, Prentice Hall, Engle- wood Cliffs, New Jersey, 2004. Goldberg, S., Probability: An Introduction, Dover Publications, New York, 1987. 1080 PROBABILITY THEORY Grimmett, G. R. and Stirzaker, D. R., Probability and Random Processes, 3rd Edition, Oxford University Press, Oxford, 2001. Hines, W. W., Montgomery, D. C., Goldsman, D. M., and Borror, C. M., Probability and Statistics in Engineering, 4th Edition, Wiley, New York, 2003. Hsu, H., Schaum’s Outline of Probability, Random Variables, and Random Processes, McGraw-Hill, New York, 1996. Kokoska, S. and Zwillinger, D. (Editors), CRC Standard Probability and Statistics Tables and Formulae, Student Edition, Chapman & Hall/CRC, Boca Raton, 2000. Lange, K., Applied Probability, Springer, New York, 2004. Ledermann, W., Probability (Handbook of Applicable Mathematics), Wiley, New York, 1981. Lipschutz, S., Schaum’s Outline of Probability, 2nd Edition, McGraw-Hill, New York, 2000. Lipschutz, S. and Schiller, J., Schaum’s Outline of Introduction to Probability and Statistics, McGraw-Hill, New York, 1998. Mendenhall,W.,Beaver,R.J.,andBeaver,B.M.,Introduction to Probability and Statistics (with CD-ROM), 12th Edition, Duxbury Press, Boston, 2005. Milton,J.S.andArnold,J.C.,Introduction to Probability and Statistics: Principles and Applications for Engineering and the Computing Sciences, 2nd Edition, McGraw-Hill, New York, 2002. Montgomery, D. C. and Runger, G. C., Applied Statistics and Probability for Engineers, Student Solutions Manual, 4th Edition, Wiley, New York, 2006. Pfeiffer, P. E., Concepts of Probability Theory, 2nd Rev. Edition, Dover Publications, New York, 1978. Pitman, J., Probability, Springer, New York, 1993. Ross, S. M., Applied Probability Models with Optimization Applications (Dover Books on Mathematics), Rep. Edition, Dover Publications, New York, 1992. Ross, S. M., Introduction to Probability Models, 8th Edition, Academic Press, Boston, 2002. Ross, S. M., A First Course in Probability, 7th Edition, Prentice Hall, Englewood Cliffs, New Jersey, 2005. Rozanov, Y. A., Probability Theory: A Concise Course, Dover Publications, New York, 1977. Scheaffer, R. L. and McClave, J. T., Probability and Statistics for Engineers, 4th Edition (Statistics), Duxbury Press, Boston, 1994. Seely, J. A., Probability and Statistics for Engineering and Science, 6th Edition, Brooks Cole, Stamford, 2003. Shiryaev, A. N., Probability, 2nd Edition (Graduate Texts in Mathematics), Springer, New York, 1996. Ventsel, H., Th ´ eorie des Probabilit ´ es, Mir Publishers, Moscow, 1987. Ventsel, H. and Ovtcharov, L., Probl ´ emes Appliqu ´ es de la Th ´ eorie des Probabilit ´ es, Mir Publishers, Moscow, 1988. Walpole, R. E., Myers, R. H., Myers, S. L., and Ye, K., Probability & Statistics for Engineers & Scientists, 8th Edition, Prentice Hall, Englewood Cliffs, New Jersey, 2006. Yates, R. D. and Goodman, D.J., Probability and Stochastic Processes: A Friendly Introduction for Electrical and Computer Engineers, 2nd Edition, Wiley, New York, 2004. Zwillinger, D. (Editor), CRC Standard Mathematical Tables and Formulae, 31st Edition, Chapman & Hall/CRC, Boca Raton, 2001. . conditions of the problem satisfy the assumptions of the theory of birth–death processes. In this problem, λ k = λ for any k, υ k = kυ for k ≤ n ,and k = nυ for k ≥ n. By formulas (20.4.2.19) and (20.4.2.20),. Outline of Probability, Random Variables, and Random Processes, McGraw-Hill, New York, 1996. Kokoska, S. and Zwillinger, D. (Editors), CRC Standard Probability and Statistics Tables and Formulae, Student. CRC Standard Probability and Statistics Tables and Formulae, CRC Press, Boca Raton, 1990. Burlington, R. S. and May, D., Handbook of Probability and StatisticsWith Tables, 2nd Edition, McGraw-Hill, New