Consider a single machine whose resale value gradually declines over time. Its output is assumed to be proportional to its resale value. By applying preventive maintenance, it is possible to slow down the rate of decline of the resale value. The control problem consists of simultane- ously determining the optimal rate of preventive maintenance and the sale date of the machine. Clearly this is an optimal control problem with unspecified terminal time; see Sect.3.1and Example 3.6.
9.1.1 The Model
In order to define Thompson’s model, we use the following notation:
T = the sale date of the machine to be determined, ρ = the constant discount rate,
x(t) = the resale value of the machine in dollars at timet; let x(0) =x0,
u(t) = the preventive maintenance rate at time t (mainte- nance here means money spent over and above the minimum required for necessary repairs),
g(t) = the maintenance effectiveness function at timet(mea- sured in dollars added to the resale value per dollar spent on preventive maintenance),
d(t) = the obsolescence function at timet(measured in terms of dollars subtracted fromx at timet),
π = the constant production rate in dollars per unit time per unit resale value; assumeπ > ρ or else it does not pay to produce.
It is assumed that g(t) is a nonincreasing function of time and d(t) is a nondecreasing function of time, and that for all t
u(t)∈Ω = [0, U], (9.1)
where U is a positive constant.
The present value of the machine is the sum of two terms, the dis- counted income (production minus maintenance) stream during its life plus the discounted resale value atT:
J = T
0 [πx(t)−u(t)]e−ρtdt+x(T)e−ρT. (9.2)
9.1. A Simple Maintenance and Replacement Model 285 The state variable x is affected by the obsolescence factor, the amount of preventive maintenance, and the maintenance effectiveness function.
Thus,
˙
x(t) =−d(t) +g(t)u(t), x(0) =x0. (9.3) In the interests of realism we assume that
−d(t) +g(t)U ≤0, t≥0. (9.4) The assumption implies that preventive maintenance is not so effective as to enhance the resale value of the machine over its previous values;
rather, it can at most slow down the decline of the resale value, even when preventive maintenance is performed at the maximum rate U. A modification of (9.3) is given in Arora and Lele (1970). See also Hartl (1983b).
The optimal control problem is to maximize (9.2) subject to (9.1) and (9.3).
9.1.2 Solution by the Maximum Principle
This problem is similar to Model Type (a) of Table 3.3 with the free- end-point condition as in Row 1 of Table 3.1. Therefore, we follow the steps for solution by the maximum principle stated in Chap.3.
The standard Hamiltonian as formulated in Sect.2.2is
H = (πxưu)eưρt+λ(ưd+gu), (9.5) where the adjoint variable λsatisfies
λ˙ =−πe−ρt, λ(T) =e−ρT. (9.6) Since T is unspecified, the required additional terminal condition (3.15) for this problem is
−ρe−ρTx(T) =−H, (9.7)
which must hold on the optimal path at timeT.
The adjoint variable λ can be easily obtained by integrating (9.6), i.e.,
λ(t) =e−ρT + T
t
πe−ρτdτ =e−ρT +π
ρ[e−ρt−e−ρT]. (9.8) The interpretation of λ(t) is as follows. It gives, in present value terms, the marginal profit per dollar of gain in resale value at time t.
286 9. Maintenance and Replacement The first term represents the present value of one dollar of additional salvage value at T brought about by one dollar of additional resale value at the current time t. The second term represents the present value of incremental production fromttoT brought about by the extra produc- tivity of the machine due to the additional one dollar of resale value at time t.
Since the Hamiltonian is linear in the control variableu,the optimal control for a problem with any fixed T is bang-bang as in Model Type (a) in Table 3.3. Thus,
u∗(t) = bang
0, U;{e−ρT +π
ρ(e−ρt−e−ρT)}g(t)−e−ρt
. (9.9) To interpret this optimal policy, we see that the term
{e−ρT +π
ρ(e−ρt−e−ρT)}g(t)
is the present value of the marginal return from increasing the preventive maintenance by one dollar at timet.The last terme−ρtin the argument of the bang function is the present value of that one dollar spent for pre- ventive maintenance at timet.Thus, in words, the optimal policy means the following: if the marginal return of one dollar of additional preven- tive maintenance is more than one dollar, then perform the maximum possible preventive maintenance, otherwise do not perform any at all.
To find how the optimal control switches, we need to examine the switching function in (9.9). Rewriting it as
e−ρt
πg(t) ρ −(π
ρ −1)eρ(t−T)g(t)−1
(9.10) and taking the derivative of the bracketed terms with respect to t, we can conclude that the expression inside the square brackets in (9.10) is monotonically decreasing with timeton account of the assumptions that π/ρ >1 and thatg(t) is nonincreasing witht(see Exercise9.1). It follows that there will not be a singular control for any finite interval of time.
Furthermore, sincee−ρt >0 for allt, we can conclude that the switching function can only go from positive to negative and not vice versa. Thus, the optimal control will be eitherU,or zero, or U followed by zero. The switching time ts is obtained as follows: equate (9.10) to zero and solve fort. If the solution is negative, let ts= 0,and if the solution is greater
9.1. A Simple Maintenance and Replacement Model 287 thanT, letts=T, otherwise settsequal to the solution. It is clear that the optimal control in (9.9) can now be rewritten as
u∗(t) =
⎧⎪
⎨
⎪⎩
U t≤ts, 0 t > ts.
(9.11)
Note that all of the above calculations were made on the assumption that T was fixed, i.e., without imposing condition (9.7). On an optimal path, this condition, which uses (9.5), (9.7), and (9.8), can be restated as
−ρe−ρT∗x∗(T∗) = −{πx∗(T∗)−u∗(T∗)}e−ρT∗
−e−ρT∗{−d(T∗) +g(T∗)u(T∗)}.
(9.12)
This means that when u∗(T∗) = 0 (i.e., ts< T∗),we have x∗(T∗) = d(T∗)
π−ρ, (9.13)
and when u∗(T∗) =U (i.e., ts=T∗),we have x∗(T∗) = d(T∗)−[g(T∗)−1]U
π−ρ . (9.14)
Since d(t) is nondecreasing, g(t) is nonincreasing, and x(t) is non- increasing, Eq. (9.13) or Eq. (9.14), whichever the case may be, has a solution for T∗.
9.1.3 A Numerical Example
It is instructive to work an example of this model in which specific values are assumed for the various functions. Examples that illustrate other kinds of qualitatively different behavior are left as Exercises 9.3–9.5.
Suppose U = 1, x(0) = 100, d(t) = 2, π = 0.1, ρ = 0.05, and g(t) = 2/(1 +t)1/2.Then (9.3) specializes to
˙
x(t) =−2 + 2u(t)
√1 +t, x(0) = 100. (9.15) First, we write the condition on ts by equating (9.10) to 0, which gives
π−(π−ρ)e−ρ(T−ts)= ρ
g(ts). (9.16)
288 9. Maintenance and Replacement In doing so, we have assumed that the solution of (9.16) lies in the open interval (0, T).As we will indicate later, special care needs to be exercised if this is not the case.
Substituting the data in (9.16) we have
0.1−0.05e−0.05(T−ts)= 0.025(1 +ts)1/2, which simplifies to
(1 +ts)1/2= 4−2e−0.05(T−ts). (9.17) Then, integrating (9.15), we find
x(t) =−2t+ 4(1 +t)1/2+ 96, ift≤ts, and hence
x(t) = −2ts+ 4(1 +ts)1/2+ 96−2(t−ts)
= 4(1 +ts)1/2+ 96−2t, ift > ts.
Since we have assumed 0< ts < T, we substitutex(T) into (9.13), and obtain
4(1 +ts)1/2+ 96−2T = 2/0.05 = 40, which simplifies to
T = 2(1 +ts)1/2+ 28. (9.18) We must solve (9.17) and (9.18) simultaneously. Substituting (9.18) into (9.17), we find thatts must be a zero of the function
h(ts) = (1 +ts)1/2−4 + 2e−[2(1+ts)1/2−ts+28]/20. (9.19) A simple binary search program was written to solve this equation, which obtained the valuets = 10.6.Substitution of this into (9.18) yields T = 34.8.Since this satisfies our supposition that 0< ts< T,we can conclude our computations. Thus, if we let the unit of time be 1 month, then the optimal solution is to perform preventive maintenance at the maximum rate during the first 10.6 months, and thereafter not at all. The sale date is at 34.8 months after purchase. Figure 9.1gives the functionsx(t) and u(t) for this optimal maintenance and sale date policy.
If, on the other hand, the solution of (9.17) and (9.18) did not satisfy our supposition, we would need to follow the procedure outlined earlier in the section. This would result ints = 0 orts=T. Ifts= 0,we would obtain T from (9.18), and conclude u∗(t) = 0, 0≤t≤T. Alternatively, if ts =T, we would need to substitute x(T) into (9.14) to obtain T. In this case the optimal control would be u∗(t) =U, 0≤t≤T.
9.1. A Simple Maintenance and Replacement Model 289
Figure 9.1: Optimal maintenance and machine resale value 9.1.4 An Extension
The pure bang-bang result in the model developed above is a result of the linearity in the problem. The result can be enriched as in Sethi (1973b) by generalizing the resale value equation (9.3) as follows:
˙
x(t) =−d(t) +g(u(t), t), (9.20) where g is nondecreasing and concave in u. For this section, we will assume the sale date T to be fixed for simplicity and g to be strictly concave inu,i.e.,gu ≥0 andguu<0 for allt. Also,gt≤0, gut≤0,and g(0, t) = 0; see Exercise 9.7for an example of the function g(u, t).
The standard Hamiltonian is
H = (πxưu)eưρt+λ[ưd+g(u, t)], (9.21) whereλis given in (9.8). To maximize the Hamiltonian, we differentiate it with respect to u and equate the result to zero. Thus,
Hu=−e−ρt+λgu = 0. (9.22) If we letu0(t) denote the solution of (9.22), thenu0(t) maximizes the Hamiltonian (9.21) because of the concavity ofg in u.Thus, for a fixed T, the optimal control is
u∗(t) = sat[0, U;u0(t)]. (9.23)
290 9. Maintenance and Replacement To determine the direction of change in u∗(t), we obtain ˙u0(t). For this, we use (9.22) and the valueλ(t) from (9.8) to obtain
gu = e−ρt
λ(t) = 1
π
ρ −(πρ−1)eρ(t−T). (9.24) Since π > ρ, the denominator on the right-hand side of (9.24) is mono- tonically decreasing with time. Therefore, the right-hand side of (9.24) is increasing with time. Taking the time derivative of (9.24), we have
gut+guuu˙0 = ρ2(π−ρ)eρ(t−T)
[π−(π−ρ)eρ(t−T)]2 >0.
But gut≤0 and guu <0,it is therefore obvious that ˙u0(t)<0.In order now to sketch the optimal control u∗(t) specified in (9.23), let us define 0≤t1 ≤t2 ≤T such thatu0(t)≥U fort≤t1 and u0(t)≤0 for t≥t2. Then, we can rewrite the sat function in (9.23) as
u∗(t) =
⎧⎪
⎪⎪
⎪⎪
⎨
⎪⎪
⎪⎪
⎪⎩
U fort∈[0, t1], u0(t) fort∈(t1, t2),
0 fort∈[t2, T].
(9.25)
In (9.25), it is possible to have t1 = 0 and/ort2=T.In Fig.9.2we have sketched a case whent1 >0 and t2< T.
Note that while u0(t) in Fig.9.2 is decreasing over time, the way it will decrease will depend on the nature of the function g. Indeed, the shape of u0(t), while always decreasing, can be quite general. In particular, you will see in Exercise 9.7that the shape ofu0(t) is concave and, furthermore, u0(t)>0, t≥0,sot2=T in that case.