Markov processes and the Kolmogorov equations

Chapter 16 Markov processes and the Kolmogorov equations 16.1 Stochastic Differential Equations Consider the stochastic differential equation: dX t= at; X t dt +  t; X t dB t: (SDE) Here at; x and  t; x are given functions, usually assumed to be continuous in t; x and Lips- chitz continuous in x ,i.e., there is a constant L such that jat; x , at; y j Ljx,yj; jt; x , t; y j Ljx,yj for all t; x; y . Let t 0 ;x be given. A solution to (SDE) with the initial condition t 0 ;x is a process fX tg tt 0 satisfying X t 0 =x; X t= Xt 0 + t Z t 0 as; X s ds + t Z t 0 s; X s dB s; t  t 0 The solution process fX tg tt 0 will be adapted to the filtration fF tg t0 generated by the Brow- nian motion. If you know the path of the Brownian motion up to time t , then you can evaluate X t . Example 16.1 (Drifted Brownian motion) Let a be a constant and  =1 ,so dX t=adt+dB t: If t 0 ;x is given and we start with the initial condition X t 0 =x; 177 178 then X t=x+at,t 0 +Bt,Bt 0 ; t  t 0 : To compute the differential w.r.t. t , treat t 0 and B t 0  as constants: dX t=adt+dB t: Example 16.2 (Geometric Brownian motion) Let r and  be constants. Consider dX t=rX t dt + X t dB t: Given the initial condition X t 0 =x; the solution is X t=xexp   B t , B t 0  + r , 1 2  2 t , t 0   : Again, to compute the differential w.r.t. t , treat t 0 and B t 0  as constants: dX t=r, 1 2  2 Xtdt + Xt dB t+ 1 2  2 Xt dt = rX t dt + X t dB t: 16.2 Markov Property Let 0  t 0 t 1 be given and let hy  be a function. Denote by IE t 0 ;x hX t 1  the expectation of hX t 1  , given that X t 0 =x .Nowlet  2 IR be given, and start with initial condition X 0 = : We have the Markov property IE 0;  hX t 1      F t 0   = IE t 0 ;X t 0  hX t 1 : In other words, if you observe the path of the driving Brownian motion from time 0 to time t 0 ,and based on this information, you want to estimate hX t 1  , the only relevant information is the value of X t 0  . You imagine starting the SDE at time t 0 at value X t 0  , and compute the expected value of hX t 1  . CHAPTER 16. Markov processes and the Kolmogorov equations 179 16.3 Transition density Denote by pt 0 ;t 1 ; x; y  the density (in the y variable) of X t 1  , conditioned on X t 0 =x .Inotherwords, IE t 0 ;x hX t 1  = Z IR hy pt 0 ;t 1 ; x; y  dy : The Markov property says that for 0  t 0  t 1 and for every  , IE 0;  hX t 1      F t 0   = Z IR hy pt 0 ;t 1 ; Xt 0 ;y dy : Example 16.3 (Drifted Brownian motion) Consider the SDE dX t=adt+dB t: Conditioned on X t 0 =x , the random variable X t 1  is normal with mean x + at 1 , t 0  and variance t 1 , t 0  , i.e., pt 0 ;t 1 ; x; y= 1 p 2t 1 ,t 0  exp  , y , x + at 1 , t 0  2 2t 1 , t 0   : Note that p depends on t 0 and t 1 only through their difference t 1 , t 0 . This is always the case when at; x and  t; x don’t depend on t . Example 16.4 (Geometric Brownian motion) Recall that the solution to the SDE dX t=rX t dt + X t dB t; with initial condition X t 0 =x , is Geometric Brownian motion: X t 1 =xexp  B t 1  , B t 0  + r , 1 2  2 t 1 , t 0   : The random variable B t 1  , B t 0  has density IP fB t 1  , B t 0  2 dbg = 1 p 2t 1 , t 0  exp  , b 2 2t 1 , t 0   db; and we are making the change of variable y = x exp  b +r, 1 2  2 t 1 , t 0   or equivalently, b = 1  h log y x , r , 1 2  2 t 1 , t 0  i : The derivative is dy db = y; or equivalently, db = dy y : 180 Therefore, pt 0 ;t 1 ;x; y dy = IP fX t 1  2 dyg = 1 y p 2t 1 , t 0  exp  , 1 2t 1 , t 0  2 h log y x , r , 1 2  2 t 1 , t 0  i 2  dy: Using the transition density and a fair amount of calculus, one can compute the expected payoff from a European call: IE t;x X T  , K  + = Z 1 0 y , K  + pt; T ; x; y dy = e rT ,t xN  1  p T , t h log x K + rT , t+ 1 2  2 T ,t i  ,KN  1  p T , t h log x K + rT , t , 1 2  2 T , t i  where N = 1 p 2 Z  ,1 e , 1 2 x 2 dx = 1 p 2 Z 1 , e , 1 2 x 2 dx: Therefore, IE 0;  e ,rT ,t X T  , K  +     F t  = e ,rT ,t IE t;X t X T  , K  + = X tN  1  p T , t  log X t K + rT , t+ 1 2  2 T ,t  , e ,rT ,t KN  1  p T,t  log X t K + rT , t , 1 2  2 T , t  16.4 The Kolmogorov Backward Equation Consider dX t= at; X t dt +  t; X t dB t; and let pt 0 ;t 1 ;x; y  be the transition density. Then the Kolmogorov Backward Equation is: , @ @t 0 pt 0 ;t 1 ; x; y =at 0 ;x @ @x pt 0 ;t 1 ; x; y + 1 2  2 t 0 ;x @ 2 @x 2 pt 0 ;t 1 ; x; y : (KBE) The variables t 0 and x in KBE are called the backward variables. In the case that a and  are functions of x alone, pt 0 ;t 1 ; x; y  depends on t 0 and t 1 only through their difference  = t 1 , t 0 . We then write p ; x; y  rather than pt 0 ;t 1 ; x; y  ,and KBE becomes @ @ p ; x; y =ax @ @x p ; x; y + 1 2  2 x @ 2 @x 2 p ; x; y : (KBE’) CHAPTER 16. Markov processes and the Kolmogorov equations 181 Example 16.5 (Drifted Brownian motion) dX t=adt+dBt p ; x; y= 1 p 2 exp  , y , x + a  2 2  : @ @ p = p  =  @ @ 1 p 2  exp  , y , x , a  2 2  ,  @ @ y , x , a  2 2  1 p 2 exp  , y , x , a  2 2  =  , 1 2 + ay , x , a   + y , x , a  2 2  p: @ @x p = p x = y , x , a  p: @ 2 @x 2 p = p xx =  @ @x y , x , a   p + y , x , a  p x = , 1  p + y , x , a  2  2 p: Therefore, ap x + 1 2 p xx =  ay , x , a   , 1 2 + y , x , a  2 2 2  p = p  : This is the Kolmogorov backward equation. Example 16.6 (Geometric Brownian motion) dX t=rX t dt + X t dB t: p ; x; y= 1 y p 2 exp  , 1 2 2 h log y x , r , 1 2  2  i 2  : It is true but very tedious to verify that p satisfies the KBE p  = rxp x + 1 2  2 x 2 p xx : 16.5 Connection between stochastic calculus and KBE Consider dX t=aXt dt +  X t dB t: (5.1) Let hy  be a function, and define v t; x= IE t;x hX T ; 182 where 0  t  T .Then v t; x= Z hypT ,t; x; y  dy ; v t t; x= , Z hyp  T ,t; x; y  dy ; v x t; x= Z hyp x T ,t; x; y  dy ; v xx t; x= Z hyp xx T , t; x; y  dy : Therefore, the Kolmogorov backward equation implies v t t; x+ axv x t; x+ 1 2  2 xv xx t; x= Z hy h ,p  T ,t;x; y +axp x T ,t;x; y + 1 2  2 xp xx T , t; x; y  i dy =0 Let 0; be an initial condition for the SDE (5.1). We simplify notation by writing IE rather than IE 0; . Theorem 5.50 Starting at X 0 =  , the process v t; X t satisfies the martingale property: IE  v t; X t     F s  = v s; X s; 0  s  t  T: Proof: According to the Markov property, IE  hX T      F t  = IE t;X t hX T  = v t; X t; so IE v t; X tjFs = IE  IE  hX T      F t      F s  = IE  hX T      F s  = IE s;X s hX T  (Markov property) = v s; X s: Itˆo’s formula implies dv t; X t = v t dt + v x dX + 1 2 v xx dX dX = v t dt + av x dt + v x dB + 1 2  2 v xx dt: CHAPTER 16. Markov processes and the Kolmogorov equations 183 In integral form, we have v t; X t = v 0;X0 + Z t 0 h v t u; X u + aX uv x u; X u + 1 2  2 X uv xx u; X u i du + Z t 0  X uv x u; X u dB u: We know that v t; X t is a martingale, so the integral R t 0 h v t + av x + 1 2  2 v xx i du must be zero for all t . This implies that the integrand is zero; hence v t + av x + 1 2  2 v xx =0: Thus by two different arguments, one based on the Kolmogorov backward equation, and the other basedonItˆo’s formula, we have come to the same conclusion. Theorem 5.51 (Feynman-Kac) Define v t; x= IE t;x hX T ; 0  t  T; where dX t=aXt dt +  X t dB t: Then v t t; x+axv x t; x+ 1 2  2 xv xx t; x=0 (FK) and v T; x= hx: The Black-Scholes equation is a special case of this theorem, as we show in the next section. Remark 16.1 (Derivation of KBE) We plunked down the Kolmogorov backward equation with- out any justification. In fact, one can use Itˆo’s formula to prove the Feynman-Kac Theorem, and use the Feynman-Kac Theorem to derive the Kolmogorov backward equation. 16.6 Black-Scholes Consider the SDE dS t= rS t dt + S t dB t: With initial condition S t=x; the solution is S u= xexp n B u , B t + r , 1 2  2 u , t o ; u  t: 184 Define v t; x= IE t;x hS T  = IEh  x exp n B T  , B t+r, 1 2  2 T , t o ; where h is a function to be specified later. Recall the Independence Lemma:If G is a  -field, X is G -measurable, and Y is independent of G , then IE  hX; Y      G  =  X ; where  x= IEhx; Y : With geometric Brownian motion, for 0  t  T ,wehave S t=S0 exp n Bt+r, 1 2  2 t o ; ST= S0 exp n BT +r, 1 2  2 T o = St |z  F t -measurable exp n  B T  , B t + r , 1 2  2 T , t o | z  independentof F t We thus have S T = XY; where X = S t Y = exp n B T  , B t + r , 1 2  2 T , t o : Now IEhxY =vt; x: The independence lemma implies IE  hS T      F t  = IE hXY jF t = v t; X  = v t; S t: CHAPTER 16. Markov processes and the Kolmogorov equations 185 We have shown that v t; S t = IE  hS T      F t  ; 0  t  T: Note that the random variable hS T  whose conditional expectation is being computed does not depend on t . Because of this, the tower property implies that v t; S t; 0  t  T , is a martingale: For 0  s  t  T , IE  v t; S t     F s  = IE  IE  hS T      F t      F s  = IE  hS T      F s  = v s; S s: This is a special case of Theorem 5.51. Because v t; S t is a martingale, the sum of the dt terms in dv t; S t must be 0. By Itˆo’s formula, dv t; S t = h v t t; S t dt + rS tv x t; S t + 1 2  2 S 2 tv xx t; S t i dt + Stv x t; S t dB t: This leads us to the equation v t t; x+rxv x t; x+ 1 2  2 x 2 v xx t; x= 0; 0 tT; x0: This is a special case of Theorem 5.51 (Feynman-Kac). Along with the above partial differential equation, we have the terminal condition v T; x= hx; x  0: Furthermore, if S t= 0 for some t 2 0;T ,thenalso S T =0 . This gives us the boundary condition v t; 0 = h0; 0  t  T: Finally, we shall eventually see that the value at time t of a contingent claim paying hS T  is ut; x= e ,rT,t IE t;x hS T  = e ,rT ,t v t; x at time t if S t=x . Therefore, v t; x= e rT,t ut; x; v t t; x= ,re rT ,t ut; x+e rT,t u t t; x; v x t; x= e rT,t u x t; x; v xx t; x= e rT,t u xx t; x: 186 Plugging these formulas into the partial differential equation for v and cancelling the e rT ,t ap- pearing in every term, we obtain the Black-Scholes partial differential equation: ,rut; x+u t t; x+rxu x t; x+ 1 2  2 x 2 u xx t; x=0; 0 tT; x0: (BS) Compare this with the earlier derivation of the Black-Scholes PDE in Section 15.6. In terms of the transition density pt; T ; x; y = 1 y p 2T , t exp  , 1 2T , t 2  log y x , r , 1 2  2 T , t  2  for geometric Brownian motion (See Example 16.4), we have the “stochastic representation” ut; x=e ,rT,t IE t;x hS T  (SR) = e ,rT ,t Z 1 0 hy pt; T ; x; y  dy : In the case of a call, hy =y,K + and ut; x= xN  1  p T ,t  log x K + rT , t+ 1 2  2 T ,t  , e ,rT ,t KN  1  p T,t  log x K + rT , t , 1 2  2 T , t  Even if hy  is some other function (e.g., hy = K,y + , a put), ut; x is still given by and satisfies the Black-Scholes PDE (BS) derived above. 16.7 Black-Scholes with price-dependent volatility dS t= rS t dt +  S t dB t; v t; x= e ,rT,t IE t;x S T  , K  + : The Feynman-Kac Theorem now implies that ,rvt; x+ v t t; x+rxv x t; x+ 1 2  2 xv xx t; x=0; 0 tT; x0: v also satisfies the terminal condition v T; x= x,K + ; x  0; [...]... based on the Kolmogorov backward equation, and the other based on Itˆ ’s formula, we have come to the same conclusion o Theorem 5.51 (Feynman-Kac) Define vt; x = IE t;xhX T ; where 0  t  T; dX t = aX t dt + X t dBt: Then vtt; x + axvxt; x + 1 2 xvxxt; x = 0 2 and (FK) v T; x = hx: The Black-Scholes equation is a special case of this theorem, as we show in the next section... x = e,rT ,t IE t;xS T  , K + : The Feynman-Kac Theorem now implies that ,rvt; x + vtt; x + rxvxt; x + 1 2xvxxt; x = 0; 2 v also satisfies the terminal condition vT; x = x , K +; x  0; 0  t T; x 0: CHAPTER 16 Markov processes and the Kolmogorov equations and the boundary condition vt; 0 = 0; 187 0  t  T: An example of such a process is the following from J.C Cox, Notes on options... section Remark 16.1 (Derivation of KBE) We plunked down the Kolmogorov backward equation without any justification In fact, one can use Itˆ ’s formula to prove the Feynman-Kac Theorem, and use o the Feynman-Kac Theorem to derive the Kolmogorov backward equation 16.6 Black-Scholes Consider the SDE dS t = rS t dt + S t dB t: With initial condition the solution is S t = x; n o S u = x exp B u... Now IEhxY  = vt; x: The independence lemma implies   IE hS T  F t = IE hXY jF t = v t; X  = v t; S t: CHAPTER 16 Markov processes and the Kolmogorov equations We have shown that   vt; S t = IE hS T  F t ; 185 0  t  T: Note that the random variable hS T  whose conditional expectation is being computed does not depend on t Because of this, the tower property implies... avx dt + vx dB + 2 2 vxx dt: (Markov property) CHAPTER 16 Markov processes and the Kolmogorov equations 183 In integral form, we have v t; X t = v0; X 0 Z th i + vt u; X u + aX uvxu; X u + 1 2 X uvxxu; X u du 2 + Z0t 0 X uvxu; X u dB u: Rt We know that v t; X t is a martingale, so the integral 0 for all t This implies that the integrand is zero; hence h i vt +... erT ,tuxxt; x: 186 Plugging these formulas into the partial differential equation for v and cancelling the erT ,t appearing in every term, we obtain the Black-Scholes partial differential equation: ,rut; x + utt; x + rxuxt; x + 1 2x2uxxt; x = 0; 2 0  t T; x  0: (BS) Compare this with the earlier derivation of the Black-Scholes PDE in Section 15.6 In terms of the transition density   y... y  + 1 2xpxxT , t; x; y  dy = 0 2 Let 0;   be an initial condition for the SDE (5.1) We simplify notation by writing IE rather than IE 0; Theorem 5.50 Starting at X 0 =  , the process v t; X t satisfies the martingale property:   IE v t; X t F s = v s; X s; 0  s  t  T: Proof: According to the Markov property,   IE hX T  F t = IE t;X thX T  = v t; X t;... of Theorem 5.51 Because v t; S t is a martingale, the sum of the formula, dt terms in dv t; S t must be 0 h By Itˆ ’s o i 1 dv t; S t = vtt; S t dt + rS tvxt; S t + 2 2 S 2tvxxt; S t dt + S tvxt; S t dB t: This leads us to the equation vtt; x + rxvxt; x + 1 2x2vxxt; x = 0; 2 0  t T; x  0: This is a special case of Theorem 5.51 (Feynman-Kac) Along with the. .. have the “stochastic representation” ut; x = e,rT ,tIE t;xhS T  Z1 = e,rT ,t hy pt; T ; x; y dy: (SR) 0 In the case of a call, and hy  = y , K +  x + rT , t + 1 2 T , t log K ut; x = x N 2 T ,t 1  x  ,rT ,t K N p 1 2 T , t ,e log K + rT , t , 2 T ,t Even if hy  is some other function (e.g., hy  = K , y + , a put), ut; x is still given by and p1 satisfies the. .. differential equation, we have the terminal condition vT; x = hx; x  0: Furthermore, if S t = 0 for some t 2 0; T , then also S T  = 0 condition vt; 0 = h0; This gives us the boundary 0  t  T: Finally, we shall eventually see that the value at time t of a contingent claim paying hS T  is ut; x = e,rT ,t IE t;xhS T  = e,rT ,t v t; x at time t if S t = x Therefore, vt; x = erT . CHAPTER 16. Markov processes and the Kolmogorov equations 187 and the boundary condition v t; 0 = 0; 0  t  T: An example of such a process is the following. formula to prove the Feynman-Kac Theorem, and use the Feynman-Kac Theorem to derive the Kolmogorov backward equation. 16.6 Black-Scholes Consider the SDE dS