For the maximum principle (8.53) we assumed that Hk and g were con- cave in uk so that the set of admissible controls was convex. These are fairly strong assumptions which will now be relaxed and a general max- imum principle stated. The proof can be found in Canon et al. (1970).
Other references on discrete maximum principles are Halkin (1966) and Holtzman (1966). The problem to be solved is:
max
J =
T−1 k=0
F(xk, uk, k)
(8.67) subject to
xk =f(xk, uk, k), x0 given
uk∈Ωk, k = 0,1, . . . ,(T −1). (8.68) Assumptions required are:
(i) F(xk, uk, k) and f(xk, uk, k) are continuously differentiable in xk for everyuk and k.
(ii) The sets {−F(x,Ωk, k), f(x,Ωk, k)} are b-directionally convex for every xand k, whereb= (−1,0, . . . ,0).That is, given v and w in Ωk and 0≤λ≤1 , there exists u(λ)∈Ωk such that
F(x, u(λ), k)≥λF(x, v, k) + (1−λ)F(x, w, k) and
f(x, u(λ), k) =λf(x, v, k) + (1−λ)f(x, w, k)
for every x and k. It should be noted that convexity implies b- directional convexity, but not the converse.
(iii) Ωk satisfies the Kuhn-Tucker constraint qualification.
With these assumptions replacing the assumptions of Theorem 8.3, and since there is no salvage value term in (8.67) meaning that S(xT, T) ≡ 0, the maximum principle (8.53) with λT = 0 holds with control constraint set g(uk, k) ≥bk replaced by uk ∈ Ω. When the sal- vage function S(xT, T) is not identically zero, the objective function in
Exercises for Chapter 8 277 (8.67) is replaced by the Bolza form objective function (8.42). In Ex- ercise 8.20, you are asked to convert the problem defined by (8.42) and (8.68) to its Lagrange form, and then obtain the corresponding assump- tions on the salvage value functionS(xT, T) for the results of this section to apply. For a fixed-end-point problem, i.e., when xT is also given in (8.68), the more general maximum principle holds withλT a constant to be determined. Exercise8.17is an example of a fixed-end-point problem.
Finally, when there are lags in the system dynamics, i.e., when the state of the system in a period depends not only on the state and the control in the previous period, but also on the values of these variables in prior periods, it is easy to adapt the discrete maximum principle to deal with such systems; see Burdet and Sethi (1976). Exercise 8.22 presents an advertising model containing lags in its sales-advertising dynamics.
Some concluding remarks on the applications of discrete-time optimal control problems are appropriate. Real-life examples that can be mod- eled as such problems include the following: payments of principal and interest on loans; harvesting of crops; production planning for monthly demands; etc. Such problems would require efficient computational pro- cedures for their solution. Some references dealing with computational methods for discrete optimal control problems are Murray and Yakowitz (1984), Dunn and Bertsekas (1989), Pantoja and Mayne (1991), Wright (1993), and Dohrmann and Robinett (1999). Another reason that makes the discrete optimal control theory important arises from the fact that computers are being used increasingly in the control of dynamic systems.
Finally, Pepyne and Cassandras (1999) have explored an optimal con- trol approach to treat discrete event dynamic systems (DEDS). They also apply the approach to a transportation problem, modeled as a polling system.
Exercises for Chapter 8
E 8.1 Determine the critical points of the following functions:
(a) h(y, z) =−5y2−z2+ 10y+ 6z+ 27, (b) h(y, z) = 5y2−yz+z2−10y−18z+ 17.
E 8.2 Let h be twice differentiable with its Hessian matrix defined to be H =hxx.Let ¯x be a critical point, i.e., a solution of hx = 0. LetHj
be the jth principal minor, i.e., thej×j submatrix found in the first j
278 8. The Maximum Principal: Discrete Time rows and the first j columns of H. Let |Hj| be the determinant of Hj. Then, y0 is a local maximum ofh if
H1<0, |H2|>0, |H3|<0, . . . ,(−1)n|Hn|= (−1)n|H|>0 evaluated at ¯x, and ¯xis a local minimum of h if
H1>0, |H2|>0, |H3|>0, . . . ,|Hn|=|H|>0
evaluated at ¯x. Apply these conditions to Exercise 8.1 to identify local minima and maxima of the functions in (a) and (b).
E 8.3 Find the optimal speed in cases (a) and (b) below:
(a) During times of an energy crisis, it is important to economize on fuel consumption. Assume that when travelingxmile/hour in high gear, a truck burns fuel at the rate of
1 500
2500 x +x
gallons/mile.
If fuel costs 50 cents per gallon, find the speed that will minimize the cost of fuel for a 1000 mile trip. Check the second-order con- dition.
(b) When the government imposed this optimal speed in 1974, truck drivers became so angry that they staged blockades on several free- ways around the country. To explain the reason for these blockades, we found that a crucial figure was the hourly wage of the truckers, estimated at $3.90 per hour at that time. Recompute a speed that will minimize the total cost of fuel and the driver’s wages for the same trip. You do not need to check for the second-order condition.
E 8.4 Use (8.5)–(8.7) to derive Eq. (8.8).
E 8.5 Verify Eq. (8.8) in Example8.1by determiningh∗(a) and expand- ing the function h∗(10 +) in a Taylor series around the value 10.
E 8.6 Maximize h(x) = (1/3)x3−6x2+ 32x+ 5 subject to each of the following constraints:
(a) x≤6 (b) x≤20.
Exercises for Chapter 8 279 E 8.7 Rework Example8.4by replacing (2,2) with each of the following points:
(a) (0,−1) (b) (1/2,1/2).
E 8.8 Add the equality constraint 2x=yto the problem in Example8.4 and solve it.
E 8.9 Solve the problem:
⎧⎪
⎪⎪
⎪⎪
⎨
⎪⎪
⎪⎪
⎪⎩
maxh(x, y) subject to
x2≤(2−y)3, y≥0,
for (a) h(x, y) =x+y, (b)h(x, y) =x+ 2y, and (c)h(x, y) =x+ 3y.
Comment on the solution in each of the cases (a), (b), and (c).
E 8.10 Constraint Qualification. Show that the feasible region in two dimensions, determined by the constraints (1−x)3−y ≥0, x≥0,and y≥0,does not satisfy the constraint qualification (8.36) at the boundary point (1,0). Also sketch the feasible region to see the presence of a cusp at point (1,0).
E 8.11 Constraint Qualification. Show that the feasible region in two dimensions, determined by the constraintsx2+y2≤1, x≥0,andy≥0, satisfies the constraint qualification (8.36) at the boundary point (1,0).
Also sketch the feasible region to contrast it with that in Exercise 8.10.
E 8.12 Solve graphically the problem of minimizing x subject to the constraints
1−x≥0, y ≥0, x3−y≥0.
Show that the constraints do not satisfy the constraint qualification (8.36) at the optimal point.
E 8.13 Rewrite the maximum principle (8.53) for the special case of the linear Mayer form problem obtained when F ≡0 and S(xT, T) =cxT, where cis ann-component row vector of constants.
280 8. The Maximum Principal: Discrete Time E 8.14 Show that the necessary conditions for uk to be an optimal so- lution for (8.52) are given by (8.50) and (8.51).
E 8.15 Prove Theorem8.3.
E 8.16 Formulate and solve a discrete-time version of the cash balance model of Sect.5.1.1.
E 8.17 Minimum Fuel Problem. Consider the problem:
⎧⎪
⎪⎪
⎪⎪
⎪⎪
⎪⎪
⎨
⎪⎪
⎪⎪
⎪⎪
⎪⎪
⎪⎩ min
!
J =T−1 k=0 |uk|"
subject to
xk=Axk+buk, x0 and xT given uk∈[−1,1], k= 0,1, . . . , T−1,
whereAis a given matrix. Obtain the expression for the adjoint variable and the form of the optimal control.
E 8.18 Current-Value Formulation. Obtain the current-value formula- tion of the discrete maximum principle. Assume that r is the discount rate, i.e., 1/(1 +r) is the discount factor.
E 8.19 Convert the Bolza form problem (8.42)–(8.44) to the equiva- lent linear Mayer form; see Sect.2.1.4 for a similar conversion in the continuous-time case.
E 8.20 Convert the problem defined by (8.42) and (8.68) to its La- grange form. Then, obtain the assumptions on the salvage value function S(xT, T) so that the results of Sect.8.3apply. Under these assumptions, state the maximum principle for the Bolza form problem defined by (8.42) and (8.68).
E 8.21 Use Excel to solve the production planning problem given by (8.62) and (8.63) with I0 = 1, Pˆ = 30, Iˆ= 15, h=c= 1, T = 8,and Sk = k3 −12k2 + 32k+ 30, k = 0,1,2, . . . ,(T −1). This is a discrete time version of Example6.1so that you can compare your solution with Fig.6.1.
Exercises for Chapter 8 281 E 8.22 An Advertising Model(Burdet and Sethi 1976). Let xk denote the sale and uk, k= 1,2, . . . , T −1,denote the amount of advertising in periodk.Formulate the sales-advertising dynamics as
xk =−δxk+r k
l=0
fkl(xl, ul), x0 given,
where δ and r are decay and response constants, respectively, and fkl(xl, ul) is a nonnegative function that decreases withxl and increases with ul.In the special case when
fkl(xl, ul) =γlkul, γlk >0,
obtain optimal advertising amounts to maximize the total discounted profit given by
T−1 k=1
(πxk−uk)(1 +ρ)−k,
where, as in Sect.7.2.1,π denotes per unit sales revenue, ρ denotes the discount rate, and the inequalities 0 ≤ uk ≤ Qk represent the restric- tions on the advertising amount uk.For the continuous-time version of problems with lags, see Hartl and Sethi (1984b).
Chapter 9
Maintenance and Replacement
The problem of simultaneously determining the lifetime of an asset or an activity along with its management during that lifetime is an important problem in practice. The most typical example is the problem of opti- mal maintenance and replacement of a machine; see Rapp (1974) and Pierskalla and Voelker (1976). Other examples occur in forest manage- ment, such as in N¨aslund (1969), Clark (1976), and Heaps (1984), and in advertising copy management, such as in Pekelman and Sethi (1978).
The first major work dealing with machine replacement problems ap- peared in 1949 as a MAPI (Machinery and Applied Products Institute) study by Terborgh (1949). For the most part, this study was confined to those problems where the optimization was carried out only with respect to the replacement lives of the machines under consideration. Boiteux (1955) and Mass´e (1962) extended the single machine replacement prob- lem to include the optimal timing of a partial replacement of the machine before its actual retirement. N¨aslund (1966) was the first to solve a gen- eralized version of the Boiteux problem by using the maximum principle.
He considered optimal preventive maintenance applied continuously over the entire period instead of a single optimal partial replacement before the machine is retired. Thompson (1968) presented a modification of N¨aslund’s model which is described in the following section.
©Springer Nature Switzerland AG 2019 S. P. Sethi,Optimal Control Theory,
https://doi.org/10.1007/978-3-319-98237-3 9
283
284 9. Maintenance and Replacement