As indicated in Sect.3.6 for the infinite horizon case, the transversality condition must be changed to
tlim→∞e−ρtλ(t) = 0. (5.65) Furthermore, this condition may no longer be a necessary condition; see Sect.3.6. It is a sufficient condition for optimality however, in conjunc- tion with the other sufficiency conditions stated in Theorem 2.1.
5.2. Optimal Financing Model 181 As demonstrated in Example 3.7, a common method of solving an infinite horizon problem is to take the limit as T → ∞ of the finite horizon solution and then prove that the limiting solution obtained solves the infinite horizon problem. The proof is important because the limit of the solution may or may not solve the infinite horizon problem. The proof is usually based on the sufficiency conditions of Theorem 2.1, modified slightly as indicated above for the infinite horizon case.
We now analyze the infinite horizon case following the above proce- dure. We start with Case A.
Case A:g≤r.
Let us first consider the case ρ > g and examine the solution in forward time obtained in (5.44)–(5.48) as T goes to infinity. Clearly (5.45) and (5.46) disappear, and (5.44) and (5.48) can be written as
u∗(t) = 0, v∗(t) =g/r, x∗(t) =x0egt, t≥0, (5.66) λ(t) = 1−g/r
ρ−g = ¯λ, t≥0. (5.67) Clearly λ(t) satisfies (5.65). Furthermore,
W2(t) =rλ−1 = r−ρ
ρ−g >0, t≥0,
which implies that the firm is in Subcase A3 for t ≥ 0. The maximum principle holds, and (5.66) and (5.67) represent an optimal solution for the infinite horizon problem. Note that the assumption ρ > g together with our overall assumption that ρ < r givesg < r so that 1−v∗ > 0, which means a constant fraction of earnings is being paid as dividends.
Note that the value of the adjoint variable ¯λ in this case is a con- stant and its form is reminiscent of Gordon’s classic formula; see Gordon (1962). In the control theory framework, the value of ¯λ represents the marginal worth per additional unit of earnings. Obviously, a unit in- crease in earnings will mean an increase of 1−v∗ or 1−g/r units in dividends. This, of course, should be capitalized at a rate equal to the discount rate less the growth rate (i.e.,ρ−g),which is precisely Gordon’s formula.
Forρ≤g, it is clear from (5.48) thatλ(t) does not satisfy (5.65). A moment’s reflection shows that for ρ≤g, the objective function can be made infinite. For example, any control policy with earnings growing at
182 5. Applications to Finance rateq, ρ≤q≤g,coupled with a partial dividend payout, i.e., aconstant v such that 0< v <1,gives an infinite value for the objective function.
That is, withu∗ = 0, v∗=q/r <1,we have J =
∞
0 e−ρt(1−u∗−v∗)x∗dt= ∞
0 e−ρt(1−q/r)x0eqt=∞. Since there are many policies which give an infinite value to the objective function, the choice among them may be decided on subjective grounds. We will briefly discuss only the constant (over time) optimal policies. If g < r, then the rate of growth q may be chosen in the closed interval [ρ, g]; if g = r, then q may be chosen in the half-open interval [ρ, r). In either case, the choice of a low rate of growth (i.e., a high proportional dividend payout) would mean a higher dividend rate (in dollars per unit time) early in time, but a lower dividend rate later in time because of the slower growth rate. Similarly the choice of high growth rate means the opposite in terms of dividend payments in dollars per unit time.
To conclude, we note that forρ≤g in Case A, the limiting solution of the finite case is an optimal solution for the infinite horizon problem in the sense that the objective function becomes infinite. However, this will not be the situation in Case B; see also Remark 5.7.
Case B:g > r.
The limit of the finite horizon optimal solution is to grow at the maximum allowable growth rate with
u= g−r
rc and v= 1
all the way. Sinceτ1 disappears in the limit, the stockholders will never collect dividends. The firm has become an infinite sink for investment.
In fact, the limiting solution is a pessimal solution because the value of the objective function associated with it is zero. From the point of view of optimal control theory, this can be explained as before in Case A whenρ≤g.In Case B, we haveg > rso that (sincer > ρthroughout the chapter) we have ρ < g. For this, as noted in Remark 5.5, λ(τ) increases without bound as τ increases and, therefore, (5.64) does not have a solution.
As in Case A withρ < g, any control policy with earnings growing at rate q ∈ [ρ, g] coupled with a constant v, 0 < v < 1, has an infinite value for the objective function.
5.2. Optimal Financing Model 183 In summary, we note that the only nondegenerate case in the infinite horizon problem is when ρ > g. In this case, which occurs only in Case A, the policy of maximum allowable growth is optimal. On the other hand, when ρ≤g,whether in Case A or B, the infinite horizon problem has nonunique policies with infinite values for the objective function.
Before solving a numerical example, we will make an interesting re- mark concerning Case B.
Remark 5.7 Let (u∗T, vT∗) denote the optimal control for the finite horizon problem in Case B. Let (u∗∞, v∞∗ ) denote any optimal con- trol for the infinite horizon problem in Case B. We already know that J(u∗∞, v∞∗ ) =∞.Define an infinite horizon control (u∞, v∞) by extend- ing (u∗T, vT∗) as follows:
(u∞, v∞) = lim
T→∞(u∗T, vT∗).
We now note that for our model in Case B, we have
Tlim→∞J(u∗T, v∗T) =∞ andJ( lim
T→∞(u∗T, v∗T)) =J(u∞, v∞) = 0. (5.68) Obviously (u∞, v∞) is not an optimal control for the infinite horizon problem. Since the two terms in (5.68) are not equal, we can say in tech- nical terms thatJ(u, v),regarded as a mapping, is not aclosedmapping.
However, if we introduce a salvage value Bx(T), B > 0, for the finite horizon problem, then the new objective function,
JB(u, v) =
⎧⎪
⎨
⎪⎩ T
0 e−ρt(1−u−v)xdt+Bx(T)e−ρT, ifT <∞, ∞
0 e−ρt(1−u−v)xdt+ limT→∞{Bx(T)e−ρT}, if T =∞,
is a closed mapping in the sense that
Tlim→∞JB(u∗T, v∗T) =∞ andJB( lim
T→∞(u∗T, vT∗)) =JB(u∞, v∞) =∞ for the modified model.
Example 5.1 We will now assign numbers to the various parameters in the optimal financing problem in order to compute the optimal solution.
Let
x0 = 1000/month, T = 60 months, r = 0.15, ρ = 0.10, g = 0.05, c = 0.98.
184 5. Applications to Finance Solution Sinceg≤r, the problem belongs to Case A. We compute
τ1= 1
ρln[r/(r−ρ)] = 10 ln 3≈11 months.
The optimal controls for the problem are
u∗= 0, v∗=g/r= 1/3, t∈[0,49), u∗= 0, v∗= 0, t∈[49,60], and the optimal state trajectory is
x(t) =
⎧⎪
⎨
⎪⎩
1000e0.05t, t∈[0,49), 1000e2.45, t∈[49,60].
The value of the objective function is J∗ =
49
0 e−0.1t(1−1/3)(1000)e0.05tdt+ 60
49 1000e2.45ãe−0.1tdt
= 12,578.75.
Note that the infinite horizon problem is well defined in this case, since g < ρand g < r.The optimal controls are
u∗ = 0, v∗=g/r= 1/3, and
J = ∞
0 e−0.1t(2/3)(1000)e0.05tdt= 2000/0.15 = 13,3331 3. In Exercise5.14, you are asked to extend the optimal financing model to allow for debt financing. Exercise5.15requires you to reformulate the optimal financing model (5.21) with decisions expressed in dollars per unit of time rather than in terms relative tox.Exercise5.16extends the model to allow the rate of return on the assets to decrease as the assets grow.
Exercises for Chapter 5 185 Exercises for Chapter 5
E 5.1 Find the optimal policies for the simple cash balance model (Sects.5.1.1 and 5.1.2) with x0 = 2, y0 = 2, U1 = U2 = 5, T = 1, α= 0.01,and the following specifications for the interest rates:
(a) r1(t) = 1/2, r2(t) = 1/3.
(b) r1(t) =t/2, r2(t) = 1/3.
(c) Sketch the optimal policy in (b) in the (t, λ2/λ1) space, like in Fig.5.2.
E 5.2 Formulate the extension of the model in Sect.5.1.1 when over- draft and short selling are disallowed in the following two cases: (a) α = 0 and (b) α >0. State the maximum principle (4.29) as it applies to these cases.
Hint: Adjoin the control constraints to the Hamiltonian in form- ing the Lagrangian. For (b), write u=u1−u2 as in (5.10).
E 5.3 It is possible to guess the optimal solution for Exercise 5.2when α= 0, T = 10, x0= 0, y0= 3,
r1(t) =
⎧⎪
⎨
⎪⎩
0 for 0≤t <5, 0.3 for 5≤t≤10, r2(t) = 0.1 for 0≤t≤10,
and U1 = U2 = ∞ (allowing for impulse controls). Show that the optimum policy remains the same for each α∈[0,1−1/e].
Hint: Use an elementary compound interest argument.
E 5.4 Do the following for Exercise 5.3 withU1 =U2 = 1, so that the control constraints are −1≤u≤1.
(a) Give reasons why the solution shown in Fig.5.7is optimal.
(b) Compute f(t∗) in terms oft∗. (c) ComputeJ in terms of t∗.
(d) Find t∗ that maximizesJ by settingdJ/dt∗ = 0.
Hint: Because this is a long and tedious calculus problem, you may wish to use Mathematicaor MAPLE to solve this problem.
186 5. Applications to Finance
Figure 5.7: Solution for Exercise 5.4
E 5.5 For the solution found in Exercise 5.4, show by using the maxi- mum principle (4.29) that the adjoint trajectories are:
λ1(t) =
⎧⎪
⎨
⎪⎩
λ1(0) =e1.5, 0≤t≤5, λ1(5)e−0.3(t−5) =e3−0.3t, 5≤t≤10, and
λ2(t) =
⎧⎪
⎨
⎪⎩
λ2(0)e−0.1t∗ =e1.5+0.1(t∗−t), 0≤t≤f(t∗)≈6.52,
23 +13e3−0.3t, f(t∗)< t≤10, where t∗≈1.97.Sketches of these functions are shown in Fig.5.8.
E 5.6 Argue that as the lower and upper bounds on u go to −∞ and +∞ in Exercise5.4, respectively,t∗ goes to 0 andf(t∗) goes to 5.Show that this solution is consistent with the guess in Exercise 5.3. Finally, find the corresponding impulse solution and show that it satisfies the maximum principle as applied in Exercise 5.2.
E 5.7 Discuss the optimal equity financing model of Sect.5.2.1 when c = 1. Show that only one control variable is needed. Then solve the problem.
Exercises for Chapter 5 187
Figure 5.8: Adjoint trajectories for Exercise5.5
E 5.8 What happens in the optimal equity financing model whenr < ρ?
Guess the optimal solution (without actually solving it).
E 5.9 In Sect.5.2.3, we obtained the optimal solution in Case B. Express the corresponding control, state, and adjoint trajectories in forward time.
E 5.10 Letg= 0.12 in Example5.1. Re-solve the finite horizon problem with this new value of g. Also, for the infinite horizon problem, state a policy which yields an infinite value for the objective function.
E 5.11 Reformulate and solve the simple cash balance problem of Sects.5.1.1and 5.1.2, if the earnings on bonds are paid in cash.
E 5.12 Add a salvage value function e−ρTBx(T),
where B ≥ 0,to the objective function in the problem (5.21) and ana- lyze the modified problem due to Sethi (1978b). Show how the solution changes as B varies from 0 to 1/rc.
E 5.13 Suppose we extend the model of Exercise 5.12 to include debt.
For this let z denote the total debt at time t and w ≥ 0 denote the
188 5. Applications to Finance amount of debt issued expressed as a proportion of current earnings.
Then the state equation for z is
˙
z=wx, y(0) =y0.
How would you modify the objective function, the state equation for x, and the growth constraint (5.19)? Assumei to be the constant interest rate on debt, andi < r.
E 5.14 Remove the assumption of an arbitrary upper bound g on the growth rate in the financing model of Sect.5.2.1by introducing a convex cost associated with the growth rate. With r re-interpreted now as the gross rate of return, obtain the net increase in rate of earnings by the rate of increase in gross earnings less the cost associated with the growth rate.
Also assumec= 1 as in Exercise5.7. Formulate the resulting model and apply the maximum principle to find the form of the optimal policy. You may assume the cost function to be quadratic in the growth rate to get an explicit form for the solution.
E 5.15 Reformulate the optimal financing model (5.21) with y(t) as the state variable, U(t) as the new equity financing rate in dollars per unit of time, andV(t) as the retained earnings in dollars per unit of time.
Hint: This formulation has mixed constraints requiring the La- grangian formulation of the maximum principle (3.42) introduced in Chap.3. Note further that it can be converted into the form (5.21) by setting U =ux, V =vx,and x=ry.
E 5.16 In Exercise 5.15, we assume a constant rate of returnr on the assets so that the total earnings rate at time tisry(t) dollars per unit of time. Extend this formulation to allow for a decreasing marginal rate of return as the assets grow. More specifically, replace ryby an increasing, strictly concave functionR(y)>0 withR(0) =randR(¯y) =ρfor some
¯
y > y0 > 0. Obtain the optimal solution in the case when r > g > ρ, 0 < c <1, T sufficiently large, and y0 < y1 <y,¯ wherey1 is defined by the relation R(y1)/y1 =g.See Perrakis (1976).
E 5.17 Find the form of the optimal policy for the following model due to Davis and Elzinga (1971):
maxu,v
J =
T
0 e−ρt(1−v)Erdt+P(T)e−ρT
Exercises for Chapter 5 189 subject to
P˙ =k[rE(1−v)−ρP], P(0) =P0, E˙ =rE[v+u(c−E/P)], E(0) =E0, and the control constraints
u≥0, v≥0, cu+v≤g/r.
Here P denotes the price of a stock, E denotes equity per stock and k >0 is a constant. Also, assumer > ρ > gand 1/c < r/ρ <1/c+ (ck+ 1)g/(ρck). This example requires the use of the generalized Legendre- Clebsch condition (D.69) in Appendix D.8.
Chapter 6
Applications to Production and Inventory
Applications of optimization methods to production and inventory prob- lems date back at least to the classical EOQ (Economic Order Quantity) model or the lot size formula of Harris (1913). The EOQ is essentially a static model in the sense that the demand is constant and only a sta- tionary solution is sought. A dynamic version of the lot size model was analyzed by Wagner and Whitin (1958). The solution methodology used there was dynamic programming.
An important dynamic production planning model was developed by Holt et al. (1960). In their model, referred to as the HMMS model, they considered both production costs and inventory holding costs over time.
They used calculus of variations techniques to solve the continuous-time version of their model. In Sect.6.1, a model of Thompson and Sethi (1980), similar to the HMMS model, is formulated and completely solved using optimal control theory. The turnpike solution is also obtained when the horizon is infinite.
In Sect.6.2, we introduce the wheat trading model of Ijiri and Thompson (1970), in which a wheat speculator must buy and sell wheat in an optimal way in order to take advantage of changes in the price of wheat over time. In Sects.6.2.1–6.2.3, we solve the model when the short- selling of wheat is allowed. In Sect.6.2.4, we follow Norstr¨om (1978) to solve a simple example that disallows short-selling.
©Springer Nature Switzerland AG 2019 S. P. Sethi,Optimal Control Theory,
https://doi.org/10.1007/978-3-319-98237-3 6
191
192 6. Applications to Production and Inventory In Sect.6.3, we introduce a warehousing constraint, i.e., an upper bound on the amount of wheat that can be stored, in the wheat trading model. In addition to being realistic, the introduction of the warehousing constraint helps us to illustrate the concepts of decision and forecast horizons by means of examples. This section is expository in nature, but theoretical developments of these ideas are available in the literature.