ScanGate document
JOURNAL OF SCIENCE, Nat Sci., t.XII, n°2 - 1996 OPTIMAL CONTROL PROBLEM FOR DIFFUSION PROCESSES WITH LONG RUN AVERAGE COST PER UNIT TIMES Nguyen Huu Du Faculty of Mathematics, Mechanics and Informatics College of Natural Sciences - VNU INTRODUCTION n this paper, we study the control problem for dynamical systems whose state X; at time t cribed by a stochastic differential equation of the form AX, = b(X,,U,) dt + o( Xe) dWe { Xo =ze Rt (1.1) : (W;,) is a wiener process defined on a stochastic basis (1, %, P), satisfying the standard tions and 6b,o are given function \s usually the control process (¿) ìs supposed to be progressively measurable with respect filtration %, with value in a metric compact space A For any admissible control (U;) we jer the solution X{/(z) of (1.1) associated with the control (U,) staring from z at t= We Jer the average cost of (U;) per unit time J(2,0) = limsup Bf T¬oo h(Xu tị) ae biềNtaswdnudlie Visa JŸs; Đang tiedia: tu Chis problem has been dealt with in various ways In |7] admissible control are defined as taps from R¢ into A satisfying the Lipschitz condition and under his assumption the author hown that there exists a Lipschitz feedback control Kushner in [6] used relaxed control to problems for “wideband noise driven” with are “close” to a diffusion In this article we use rethods dealt with by [5] for discrete cases to develop these results Under the elliptic and iess hypotheses we show that there exists an optimal markovian measure Hence using the ion theorem we can prove the existence of optimal markovian control process NOTATIONS AND HYPOTHESES Hypotheses vet A be a metric compact space, called the actions space and let ao: R4+dx d matrices b: RÂx A Rt h:RxAơR 35 (2.1) k Throughout this paper we suppose that the following hypotheses are satisfied: Hypothesis 1: o,6,h continuous by are continuous functions in (z,a) and h ts bounded ; o locally Lig We consider the generator associated with the coefficients o and of the equation (1) a =1 3ƒ of L* f(z) = 22508, a., + Db(e.a) where a;; is the tj**- element of the matrix ga*(z) and 6(z) = (by, 62, ,64)? ;a € A operator L defined on the space of twice continuous differential function with bounded deriva „ say C3(R4) Hypothesis 2: The operator L* is uniformly elliptic go*(z)>al That this : there is a a > such tha for any re Rt 2.2 Admissible control We define admissible controls as the case of finite horizon then X{ is the solution of (1) associated with U if and only if Let U be an admissible co Q(4,0) = 11%) - 1%) - f " 16 4(Xa) dh is a P- martingale after s for any f € C?(R*) Therefore we have the following definition Definition An admissible control is a term U° = (9, %, P, X:,U,v) such that: (i) (0, %, P) is a probability space with the filtration 7, (ii) U; is a progressively measurable process with values in A (ii) (X¿) is a continuous process with values in R4 such that OW 4,0) = $12) ~ 11%) — f° 1°10)dh + is a P- martingale for any f € C?(R*) (iv) Xo has the distribution v The set of admissible controls starting from v at t = is denoted by Ao Let U9 € A consider the payoff 1_ 77 Jo(v, U°) = limsup zi h{(X,, U)dt T¬—œ T7 where P(®) = ƒ ®dP Our aim is to minimise Jo(v, U) over class A°(v) 2.8 Relaxed controls We write for V the space of generalised actions consisting of random measures on Rt x the form dt.q(t,da) where dt is the Lebesgue measure on R* and q(t, da) is a probability 36 any t © R* With the vague topology , V is a metric compact space This is the same as the se topology on R* x A (see {3}) We write, from now on, f(t.) for f, f(t,a) qe(da) for any measurable f The space V ndowed with its Borel o— field V which is also the smallest o— field such that the maps » Sr+ Sa £(s,4) 9(ds,da) are measurable for any measurable function f continuous in a with tpact support We also introduce the filtration (V,) where V; is the o— algebra generated by measures {1io.4)¢: € V} The details of these definitions can be refereed in [1] Gnition A relaxed control is a term U = (9, %, P, X:,9,v) such that (i) (Q, %, P) is a probability space with the filtration % satisfying the general hypotheses (ii) ¢ is a V— valued stochastic process , #,— adapted (iii) (X-) is a continuous process with values in R4 such that GÁ1,U) = /4X)=70e)= [` ƒ T*ƒ(Xu)a(h, da) dh (2.5) P- martingale for any f € C?(R*) {iv) The distribution of Xo is v The set of relaxed control with the initial distribution v is denoted by A(v) For any U € A(v) consider the payoff 1_ J(u,U) = limsup T-00 | 77 h(t,a) g(t, da) ds J() = im{J(,U) :U € A(v)}¡ J* = inf J(v) write J(z) for J(v) whenever v is Diract mass at z The pair (v*,U*) where U* € v* is said be minimum if J(v*,U*) = J*, The term (1,%, P, X:,q) is called optimal if for any € Rt relaxed control U = (1, %, P, X:,q, x) satisfies the relation J(z) = J(2,0) she following we can see that the set of admissible controls Ao(v) is complete in the sense that =inf inf{J(v, U) :Ữ€Á(v)}= inf inf {Jo(v, U°) : U% © Ay} Rules of Controls Similar a in [1], we formulate the problem on the canonical space XY consisting of continuous ctions from R*+ into R* endowed with the topology of uniform convergence on every bounded val Let X, be the natural right - continuous filtration and (X;) be the canonical process ined on XY We put Y=Xxv; T=fT\uev >t finition A control rule ( more briefly : a rule ) is a probability measure R on X such that system (X, X;,.R, Xt, q, v) is a relaxed control 37 The set of rules starting from v is denoted by R(v) In term of control rules, optimal rela control can be interpreted as a family of rules {Rz € R(x) : z € R“ such that J(z, R„) = J(z) any z € R4, We are now able to define markovian controls by two ways : either we consider them i term U = (1%, Pe, Xe, 9,2) such that q = dt x q(X;, da) or as a family of rules {Rz : € such that the term (, Xz, X¢, Re, z) is a family of homogeneous markovian process We will s! that two these definitions are equivalent Theorem The family of rules {Rz : x € R*} is markovian iff there exists a valued - meas map q from Rt into P(A) and a relazed control (1,%, P, X¢,dtq(Xz,da)) such that Rz is the of the couple (X;,dt x q(t,da)) under P given Xo =z Proof The sufficient condition is obvious The necessary one is proved by a similar way as Theo 6.7 in [1] and we not reproduce it here In order to show the existence of optimal rules we follow the argument dealt with by Kur in [5] for the discrete case First, we prove that there exists an optimal pair (v*, R*) such thai is stationary distribution of X, under R After that by virtue of recurrent property of (X,) we show that this rule is an optimal markovian control Let U € A(v) be a relaxed control we put id 3#(D) = ạP [ 1p(X)dt DeB(R®), Hypothesis T>0 For any v, the family t#():T>0,U€ A)) is tight ( Hypothesis will be true if we consider reflected diffusion processes in a bounded domai R? (see (7]) or if the following hypothesis 3’ is satisfied Set a a 82) = 32 su(8)5;¡ W4)= giai [š 325) +3 km s)z =1 and ®†(r)= iy sup |zl=v2r (2); Vt(r) = i=l sup sup W°{z) |z|=v2 +4 It is easy to check that @† > is locally Lipschitz continuous and W* is measurable (see [2] 374) Hence there exists the minimal diffusion process & generated by the operator Lt =o (G+@ ()-F14 Hypothesis 8°: The family {o§(.); T > 0} where ¿ [_ P(&€B),T>o 2f(B)= T 38 be ndeed, let (1, %,P, Xe,g) be an arbitrary relaxed control We put H(t) =f stipeyy ds then vagy to verify that naximum where A and A respectively is the minimum of smallest < H(t) eyde Yee tt f tj 1ix(u)Is>e}H"()du 4‡ 1(|x(w)J>>e) đÈ ypothesis 8, {+ (:),T > 0} is tìght xem (se (5) Lemma 9.1 ) For any initial distribution v and for any R € R(v), there ¡a probabtlity measure u on R# x Á such that J Hevo)u(as,40) < 3,8) i (2.7) : L* f(2)u(dz,da) = (2.8) ROJA ny f € Ch f Let v and R € R(v) arbitrary We put if? 1p(Xe,a)dt ; D € B(R4 x A) aRR (D) = 7*/ {78 : T > 0} is tight then there exists a sequence (T,) — oo such that 7B) SF ol) th is a continuous function then T J h(z, a)u(dz, da) = him, | (2, a)oB, (dz, da) < limeup +R Ỉ - h(X44&) we he other hand, for any f € C? we have CW, 0) = 11%) ~ 106) - f 011%) ah P- martingale then by the law of large numbers it follows that 1% 0= lim E L4(Xea)dt = f 1°/(e)a(dz,da) 39 so we get (2.8) > Lemma If {un} is a sequence of probability measures satisfying Condition (2.8) then {un} tight Proof Since A is compact then the sequence of projections of {u,} on A is automatically ¢ compact Suppose that n(dz,da) = pp(dz)v,(x,da) and b,(z) = f b(z,a) va(z, da) Let X;, be the solution of the equation 4Xp(t) = ba (Xn(t))dt + o(Xn(t)) dW (i X,(0) =0€ Rt Then U" = (0,4,P, Xn(t), wn(Xn,da), ) is an relaxed control and U" € A(5o) On the ot hand, from (2.3) and (2.8) it follows that yn is the unique invariant measure of Equation (2.9) ¡ pn(D) = im f T 1p(Xa(t)) dt for any set D good enough By Hypothesis 3, the sequence {7p"(.) = AP Sy 1p(Xn(t)) đt 0;n > 0} is tỉght then it follows the tightness of {u,} The proof is complete > Lemma 8: there exists a probability measure v* and a markovsan control rule R* such that J(v*, R*) = J* tie, the pair (v*, R*) is optimal Proof Let {Up, Rn} be a sequence minimising J(-,-) This means that jim, J(ạ, Ra) = J° By Lemma 2, for any n there exists a probability measure on ## x A., namely ¿a(:), such tha Í HĂe) (4,46) < (vn, Ra) From Lamma 2, the sequence {jn} is tight Then there is a probability measure ¿ on R¢ x A: @ sequence (ne) — oo such that Bn SẼ“ k Hence, we have la and a)u(dz, da) = lim ng—eo [rs a)n, (dz, da) < liminf J(vp,,Rn,) = J* ngằ—>e [ 212) n(d2,4a) = Suppose that u(dz,da) = v(dz) - q(x, da) We put t(z)= Í b(ø,a)a(z.42) 40 the equation dX, = 6"(X;) dt + ø(X,)dW, (2.10) a unique stationary solution with the stationary distribution v(dz) It is easy to check that vair (v, R, is optimal where Ry is the law of (X.,dt-q(X_,da) on Y The result follows Since the diffusion matrix a(z) = o(z)o*(z) is non-degenerate then the invariant measure v is lutely continuous with respect to the Lebesgue measure on R4 and support v = R? (see [AK]) the limit Ji ‡P |, lan L - na T s for any z € R* as ( see [7] Th 6.1) This means that 9(z, da) is a markovian optimal rol Denoting A(z)f = Da; wk and < 6°, Vf >= D A(z, a) gL, we now consider the equa- A(z)®+ < 6%, VO > +h%(z) = J* (2.11) equation has a generalised solution ® since o is non degenerate Let a(z) = {a € A: A(z) ®+ < b%(2), V® > +h(z, 0) < J* 2.10) s(z) # @ and g(z,s(z)) # for any z € R4 Hence there is a measurable selection R4 — A such that 86 hố, wey 32 +h(z,u"(z)) +A(z, u*(z)) < J* from this result we can follow that the set A° is dense in £ as we have mentioned The proof y so we obmit it here aowledgement Author which to thank the Referee’ for paying attentions to this article 41 REFERENCES N.E Karoui, N.H Du, and M J Pique Compactification method in the control of degenera N Ikeda and Watanabe, Stochastic differential equations and diffusion processes Holland, Amsterdam, 1981 J Jacod and J Memin Sur un type de la convergence en loi et la convergence en probabili Seminaire de Strasbourg 15, Lecture notes in Math 851 H.B Krylov Control of diffusion processes Nauka, Moscow, 1977 (in Russian) M Kurano The existence of minimum pair of state and policy for markovian decision proc diffusions Stochastics, Vol 20 (1987), 129- 169 : Nort under the hypothesis of Doblin S.I.A.M Journal of Optim Control, Vol 27 (1989), 296- 3( H.J Kushner Optimality conditions for the average cost per unit time problem with diffusi model, 8.1.A.M Journal Control and Optimization, Vol 16, No (1978), 33-346 A Leisarowits Controlled diffusion processes on infinite horison with the overtaking criteric TẠP App Math and Optm., Vol 17 (1988), 61-78 CHÍ KHOA HỌC ĐHQGHN, KHTN, t.XII, n°2, 1996 ĐIỀU KHIỂN TỐI ƯU CÁC QUÁ TRÌNH KHUẾCH TÁN VỚI GIÁ TRUNG BÌNH THEO THỜI GIAN Nguyễn Hữu Dư Trường ĐH Khoa học tự nhiên - DHQG Hà Nội Bài báo đề cập đến tốn điều khiển tối ưu q trình khuếch tán với giá trung bì theo thời gian Dưới giả thiết tính compăắc tương đối lớp điều khiển không ø biến hệ số khuếch tán q trình tín hiệu, nhờ phương pháp tương tự [5], tồn điều khiển tối ưu Markov Bài báo mở rộng phương pháp từ rời rạc lên trường hợp liên tục 42 ... TỐI ƯU CÁC QUÁ TRÌNH KHUẾCH TÁN VỚI GIÁ TRUNG BÌNH THEO THỜI GIAN Nguyễn Hữu Dư Trường ĐH Khoa học tự nhiên - DHQG Hà Nội Bài báo đề cập đến tốn điều khiển tối ưu q trình khuếch tán với giá trung... Dưới giả thiết tính compăắc tương đối lớp điều khiển không ø biến hệ số khuếch tán q trình tín hiệu, nhờ phương pháp tương tự [5], tồn điều khiển tối ưu Markov Bài báo mở rộng phương pháp từ rời... Leisarowits Controlled diffusion processes on infinite horison with the overtaking criteric TẠP App Math and Optm., Vol 17 (1988), 61-78 CHÍ KHOA HỌC ĐHQGHN, KHTN, t.XII, n°2, 1996 ĐIỀU KHIỂN TỐI ƯU CÁC