Adaptive Control System Part 9 potx

Appendix Proof of Lemma 3.1 First, let us prove (7.34). (@) Since xP tÀ1 f nÀ1 ; FFF; tÀ1 gS 0 tÀ1 ,  t P S 0 tÀ1 implies that P tÀ1  t  0. Thus, we have  t P S 0 tÀ1 A   t P tÀ1  t    t 0  0 A:1 (A) Assume that   t P tÀ1  t  0. Then, the fact that P tÀ1  t is a linear combination of the vectors  nÀ1 ; FFF; t implies that P tÀ1  t P S 0 t , that is, there exist constants c nÀ1 ; FFF; c t such that P tÀ1  t   t inÀ1 c i  i A:2 Hence, using the de®nition (3.23), we can infer P tÀ1  t À c t  t   tÀ1 inÀ1 c i  i P S 0 tÀ1 A:3 Since  t P S 0 tÀ1  S 0 t , we can decompose the vector  t into  t  v Æ  v c Æ A:4 where v Æ denotes the component of  t which belongs to S 0 tÀ1 and v c Æ P S 0 t is the component of  t which is orthogonal to S 0 tÀ1 . Then (A.3) can be reorganized as P tÀ1  t À c t v c Æ  c t v Æ   tÀ1 inÀ1 c i  i P S 0 tÀ1 A:5 We know that P tÀ1  t cf nÀ1 ; FFF; tÀ1 gS 0 tÀ1 . Hence, we conclude that P tÀ1  t À c t v c Æ cS 0 tÀ1 . Combining this with the fact that P tÀ1  t À c tÀn2 v c Æ P S 0 tÀ1 (from (A.5)) yields P tÀ1  t À c t v c Æ  0 that is P tÀ1  t  c t v c Æ A:7 Multiplying both sides of equation (A.7) by   t , we obtain   t P tÀ1  t  c t   t v c Æ A:8 In view of the decomposition (A.4), we can simplify (A.8) and obtain   t P tÀ1  t  c t   t v c Æ  c t v Æ  v c Æ   v c Æ  c t kv c Æ k 2 A:9 Adaptive Control Systems 181 On the other hand, from the assumption we have   t P tÀ1  t  0. Thus, equation (A.9) implies c t kv c Æ k 2  0 A:10 from which we can conclude that c tÀn2 v c Æ  0 A:11 Substituting (A.11) into (A.7) gives P tÀ1  t  0, that is  t PxP tÀ1 S 0 tÀ1 A:12 So, we have shown that   t P tÀ1  t  0 A  t P S 0 tÀ1 A:13 To prove (7.35), we ®rst use (7.34):  t P S 0 tÀ1 A   t P tÀ1  t  0 A:14 But, when   t P tÀ1  t  0 the update laws (7.32)±(7.33) yield   t1    t and P t  P tÀ1 . Finally, the proof of (7.36) is as follows: We know that ~  t cf nÀ1 ; FFF; tÀ1 g. Thus, v P S 0 tÀ1 implies that ~   t v  0, which can be rewritten as    t v    v. On the other hand, v P S 0 tÀ1 implies that v P S 0 tlÀ1 , since S 0 tÀ1  S 0 tlÀ1 . Hence, we also conclude that ~   tl v  0, that is,    tl v    v. Combining these, we obtain    t v    v     tl v ; Vl  0; 1; 2; FFF A:15 References [1] Krstic  , M., Kanellakopoulos, I. and Kokotovic  , P. V. (1995). Nonlinear and Adaptive Control Design, Wiley-Interscience, NY. [2] Marino, R. and Tomei, P. (1995). Nonlinear Control Design: Geometric, Adaptive and Robust, Prentice-Hall, London. [3] Chen, F C. and Khalil, H. K. (1995). `Adaptive Control of a Class of Nonlinear Discrete-time Systems Using Neural Networks', IEEE Transactions on Automatic Control, Vol. 40, 791±801. [4] Song, Y. and Grizzle, J. W. (1993). `Adaptive Output-feedback Control of a Class of Discrete-time Nonlinear Systems', Proceedings of the 1993 American Control Conference, San Francisco, CA, 1359±1364. [5] Yeh, P C. and Kokotovic  , P. V. (1995). `Adaptive Control of a Class of Nonlinear Discrete-time Systems', International Journal of Control, Vol. 62, 302±324. 182 Active identi®cation for control of discrete-time uncertain nonlinear systems [6] Yeh, P C. and Kokotovic  , P. V. (1995). `Adaptive Output-feedback Design for a Class of Nonlinear discrete-time Systems', IEEE Transactions on Automatic Control, vol. 40, 1663±1668. [7] Goodwin, G. C. and Sin, K. S. (1984). Adaptive Filtering, Prediction and Control, Prentice-Hall, Englewood Clis, NJ. [8] Chen, H. F. and Guo, L. (1991). Identi®cation and Stochastic Adaptive Control, Birkha È user, Boston, MA. [9] Guo, L. and Wei, C. (1996). `Global Stability/Instability of LS-based Discrete-time Adaptive Nonlinear Control', Preprints of the 13th IFAC World Congress, San Francisco, CA, July, Vol. K, 277±282. [10] Guo, L. (1997). `On Critical Stability of Discrete-time Adaptive Nonlinear Control', IEEE Transactions on Automatic Control, Vol. 42, 1488±1499. [11] Kanellakopoulos, I. (1994). `A Discrete-time Adaptive Nonlinear System', IEEE Transactions on Automatic Control, Vol. 39, 2362±2364. [12] A Ê stro È m, K. J. and Wittenmark, B. (1984). Computer Controlled Systems, Prentice- Hall, Englewood Clis, NJ. [13] Nes Ï ic  , D. and Mareels, I. M. Y. (1998). `Dead Beat Controllability of Polynomial Systems: Symbolic Computation Approaches', IEEE Transactions on Automatic Control, Vol. 43, 162±175. [14] Zhao, J. and Kanellakopoulos, I. (1997). `Adaptive Control of Discrete-time Strict- feedback Nonlinear Systems', Proceedings of the 1997 American Control Conference, Albuquerque, NM, June, 828±832. [15] Zhao, J. and Kanellakopoulos, I. (1997). `Adaptive Control of Discrete-time Output-feedback Nonlinear Systems', Proceedings of the 36th Conference on Decision and Control, San Diego, CA, December, 4326±4331. [16] Feldbaum, A. A. (1965). Optimal Control Systems, Academic Press, NY. Adaptive Control Systems 183 8 Optimal adaptive tracking for nonlinear systems M. Krstic  and Z H. Li Abstract We pose and solve an `inverse optimal' adaptive tracking problem for non- linear systems with unknown parameters. A controller is said to be inverse optimal when it minimizes a meaningful cost functional that incorporates integral penalty on the tracking error state and the control, as well as a terminal penalty on the parameter estimation error. The basis of our method is an adaptive tracking control Lyapunov function (atclf) whose existence guarantees the solvability of the inverse optimal problem. The controllers designed in this chapter are not of certainty equivalence type. Even in the linear case they would not be a result of solving a Riccati equation for a given value of the parameter estimate. Our abandoning of the CE approach is motivated by the fact that, in general, this approach does not lead to optimality of the controller with respect to the overall plant-estimator system, even though both the estimator and the controller may be optimal as separate entities. Our controllers, instead, compensate for the eect of parameter adaptation transients in order to achieve optimality of the overall system. We combine inverse optimality with backstepping to design a new class of adaptive controllers for strict-feedback systems. These controllers solve a problem left open in the previous adaptive backstepping designs ± getting transient performance bounds that include an estimate of control eort, which is the ®rst such result in the adaptive control literature. 8.1 Introduction Because of the burden that the Hamilton±Jacobi±Bellman (HJB) pde's impose on the problem of optimal control of nonlinear systems, the eorts made over the last few years in control of nonlinear systems with uncertainties (adaptive and robust, see, e.g., Krstic  et al., 1995; Marino and Tomei, 1995; and the references therein) have been focused on achieving stability rather than optimality. Recently, Freeman and Kokotovic  (1996a, b) revived the interest in the optimal control problem by showing that the solvability of the (robust) stabilization problem implies the solvability of the (robust) inverse optimal control problem. Further extensive results on inverse optimal nonlinear stabil- ization were presented by Sepulchre et al. (1997). The dierence between the direct and the inverse optimal control problems is that the former seeks a controller that minimizes a given cost, while the latter is concerned with ®nding a controller that minimizes some `meaningful' cost. In the inverse optimal approach, a controller is designed by using a control Lyapunov function (clf) obtained from solving the stabilization problem. The clf employed in the inverse optimal design is, in fact, a solution to the HJB pde with a meaningful cost. In this chapter we formulate and solve the inverse optimal adaptive tracking problem for nonlinear systems. We focus on the tracking rather than the (set- point) regulation problem because, even when a bound on the parametric uncertainty is known, tracking cannot (in general) be achieved using robust techniques ± adaptation is necessary to achieve tracking. The cost functional in our inverse optimal problem includes integral penalty on both the tracking error state and control, as well as a penalty on the terminal value of the parameter estimation error. To solve the inverse optimal adaptive tracking problem we expand upon the concept of adaptive control Lyapunov functions (aclf) introduced in our earlier paper (Krstic  and Kokotovic  , 1995) and used to solve the adaptive stabilization problem. Previous eorts to design adaptive `linear-quadratic' controllers (see, e.g., Ioannou and Sun, 1995) were based on the certainty equivalence principle: a parameter estimate computed on the basis of a gradient or least-squares update law is substituted into a control law based on a Riccati equation solved for that value of the parameter estimate. Even though both the estimator and the controller independently possess optimality properties, when combined, they fail to exhibit optimality (and even stability becomes dicult to prove) because the controller `ignores' the time-varying eect of adaptation. In contrast, the Lyapunov-based approach presented in this chapter results in controllers that compensate for the eect of adaptation. A special class of systems for which we constructively solve the inverse optimal adaptive tracking problem in this chapter are the parametric strict- feedback systems, a representative member of a broader class of systems dealt with in Krstic  et al. (1995), which includes feedback linearizable systems and, in particular, linear systems. A number of adaptive designs for parametric strict- feedback systems are available, however, none of them is optimal. In this Adaptive Control Systems 185 chapter we present a new design which is optimal with respect to a meaningful cost. We also improve upon the existing transient performance results. The transient performance results achieved with the tuning functions design in Krstic  et al. (1995), even though the strongest such results in the adaptive control literature, still provide only performance estimates on the tracking error but not on control eort (the control is allowed to be large to achieve good tracking performance). The inverse optimal design in this chapter solves the open problem of incorporating control eort in the performance bounds. The optimal adaptive control problem posed here is not entirely dissimilar from the problem posed in the award-winning paper of Didinsky and Bas° ar (1994) and solved using their cost-to-come method. The dierence is twofold: (a) our approach does not require the inclusion of a noise term in the plant model in order to be able to design a parameter estimator, (b) while Didinsky and Bas° ar (1994) only go as far as to derive a Hamilton±Jacobi±Isaacs equation whose solution would yield an optimal controller, we actually solve our HJB equation and obtain inverse optimal controllers for strict-feedback systems. A nice marriage of the work of Didinsky and Bas° ar (1994) and the backstepping design in Krstic  et al. (1995) was brought out in the paper by Pan and Bas° ar (1996) who solved an adaptive disturbance attenuation problem for strict-feedback systems. Their cost, however, does not impose a penalty on control eort. This chapter is organized as follows. In Section 8.2, we pose the adaptive tracking problem (without optimality). The solution to this problem is given in Sections 8.3 and 8.4 which generalize the results of Krstic  and Kokotovic  (1995). Then in Section 8.5 we pose and solve the inverse optimal problem for general nonlinear systems assuming the existence of an adaptive tracking Lyapunov function (atclf). A constructive method for designing atclf's based on backstepping is presented in Section 8.6, and then used to solve the inverse optimal adaptive tracking problem for strict-feedback systems in Section 8.7. A summary of the transient performance analysis is given in Section 8.8. 8.2 Problem statement: adaptive tracking We consider the problem of global tracking for systems of the form  x  f xFx  gxu y  hx 8:1 where x  R n , u  R, the mappings f x, Fx, gx and hx are smooth, and  is a constant unknown parameter vector which can take any value in R p .To make tracking possible in the presence of an unknown parameter, we make the following key assumption. 186 Optimal adaptive tracking for nonlinear systems (A1) For a given smooth function y r t, there exist functions t; and  r t; such that dt; dt  f t;  Ft;  gt; r t; y r tht;; t  0;   R p 8:2 Note that this implies that @ @ h  t;0; t  0;   R p 8:3 For this reason, we can replace the objective of tracking the signal y r th  t; by the objective of tracking y r th t;  t, where  t is a time function Ð an estimate of  customary in adaptive control. Consider the signal x r tt;  t which is governed by  x r  @t;   @t  @t;   @       f x r Fx r     gx r  r t;   @t;   @      8:4 We de®ne the tracking error e  x  x r  x  t;   and compute its derivative:  e  f xf x r gxgx r  r t;    Fx  Fx r     @t;   @       gxu   r t;    ~ f  ~ F  F r ~   @ @       g ~ u 8:5 where ~       and ~ f t; e;   X f xf x r gxgx r  r t;   ~ Ft; e;   X FxFx r  F r t;   X Fx r  ~ u X u   r t;   8:6 (With a slight abuse of notation, we will write gx also as gt; e;  .) The global tracking problem is then transformed into the problem of global stabilization of the error system (8.5). This problem is, in general, not solvable with static feedback. This is obvious in the scalar case n  p  1 where, even in the case y r tx r t0, a control law u  x independent of  would have the impossible task to satisfy x f xFx  gxx < 0 for all x  0 and all   R. Therefore, we seek dynamic feedback controllers to stabilize system (8.5) for all . Adaptive Control Systems 187 De®nition 2.1 The adaptive tracking problem for system (8.1) is solvable if (A1) is satis®ed and there exist a function ~ t; e;   smooth on R  R n 0  R p with ~ t; 0;  0, a smooth function t; e;  ,anda positive de®nite symmetric p  p matrix À, such that the dynamic controller ~ u  ~ t; e;  8:7     Àt; e;  8:8 guarantees that the equilibrium e  0; ~   0 of the system (8.5) is globally stable and et0ast for any value of the unknown parameter   R p . 8.3 Adaptive tracking and atclf's Our approach is to replace the problem of adaptive stabilization of (8.5) by a problem of nonadaptive stabilization of a modi®ed system. This allows us to study adaptive stabilization in the Sontag±Artstein framework of control Lyapunov functions (clf) (Sontag, 1983; Artstein, 1983; Sontag, 1989). De®nition 3.1 A smooth function V a X R   R n  R p  R  , positive de®nite, decrescent, and proper (radially unbounded) in e (uniformly in t) for each ,is called an adaptive tracking control Lyapunov function (atclf) for (8.1) (or alternatively, an adaptive control Lyapunov function (aclf) for (8.5)), if (A1) is satis®ed and there exists a positive de®nite symmetric matrix À  R pp such that for each   R p , V a t; e; is a clf for the modi®ed nonadaptive system  e  ~ f  ~ F  FÀ  @V a @    @ @ À  @V a @e F    g ~ u 8:9 that is, V a satis®es inf ~ u IR & @V a @t  @V a @e ~ f  ~ F  FÀ  @V a @    @ @ À  @V a @e F    g ~ u !' < 0 8:10 In the sequel we will show that in order to achieve adaptive stabilization of (8.5) it is necessary and sucient to achieve nonadaptive stabilization of (8.9). Noting that for ~ t0 the system (8.5) reduces to the nonadaptive system  e  ~ f  ~ F  g ~ u 8:11 we see that the modi®cation in (8.9) is FÀ  @V a @    @ @ À  @V a @e F   8:12 188 Optimal adaptive tracking for nonlinear systems Since these terms are present only when À is nonzero, the role of these terms is to account for the eect of adaptation. Since V a t; e; has a minimum at e  0 for all t and , the modi®cation terms vanish at the e  0, so e  0isan equilibrium of (8.9). We now show how to design an adaptive controller (8.7)±(8.8) when an atclf is known. Theorem 3.1 The following two statements are equivalent: (1) There exists a triple  ~ ; V a ; À such that ~ t; e; globally uniformly asymptotically stabilizes (8.9) at e  0 for each   R p with respect to the Lyapunov function V a t; e;. (2) There exists an atclf V a t; e; for (8.1). Moreover, if an atclf V a t; e; exists, then the adaptive tracking problem for (8.1) is solvable. Proof 1  2 Obvious because 1 implies that there exists a continuous function W X R   R n  R p  R  , positive de®nite in e (uniformly in t) for each , such that @V a @t  @V a @e ~ f  ~ F  FÀ  @V a @    @ @ À  @V a @e F    g ~  ! Wt; e; 8:13 Thus V a t; e; is a clf for (8.9) for each   R p , and therefore it is an atclf for (8.1). 2  1 The proof of this part is based on Sontag's formula (Sontag, 1989). We assume that V a is an atclf for (8.1), that is, a clf for (8.9). Sontag's formula applied to (8.9) gives a control law smooth on R  R n 0  R p : ~ t; e;  @V a @t  @V a @e " f   @V a @t  @V a @e " f  2   @V a @e g  4 s @V a @e g ; @V a @e gt; e;0 0; @V a @e gt; e;0 V b b b b b b b ` b b b b b b b X 8:14 where " f  ~ f  ~ F  FÀ  @V a @    @ @ À  @V a @e F   8:15 With the choice (8.14), inequality (8.13) is satis®ed with the continuous function Adaptive Control Systems 189 Wt; e;  @V a @t t; e; @V a @e " f t; e;  2  @V a @e gt; e;  4 s 8:16 which is positive de®nite in e (uniformly in t) for each , because (8.10) implies that @V a @e gt; e;0  @V a @t t; e; @V a @e " f t; e; < 0; e  0 ; t  0 8:17 We note that the control law ~ t; e;, smooth away from e  0, will be also continuous at e  0 if and only if the atclf V a satis®es the following property, called the small control property (Sontag, 1989): for each   R p and for any ">0 there is a >0 such that, if e  0 satis®es e , then there is some ~ u with  ~ u" such that @V a @t  @V a @e ~ f  ~ F  FÀ  @V a @    @ @ À  @V a @e F    g ~ u ! < 0 8:18 for all t  0. Assuming the existence of an atclf we now show that the adaptive tracking problem for (8.1) is solvable. Since 2  1, there exists a triple  ~ ; V a ; À and a function W such that (8.13) is satis®ed. Consider the Lyapunov function candidate Vt; e;  V a t; e;   1 2      À 1    8:19 With the help of (8.13), the derivative of V along the solutions of (8.5), (8.7), (8.8), is  V  @V a @t  @V a @e ~ f  ~ F  F r ~   @ @   Àt; e;  g ~ t; e;   !  @V a @   Àt; e;   ~   t; e;    @V a @t  @V a @e ~ f  ~ F    g ~ t; e;   !  @V a @e F ~   @V a @e @ @   À  @V a @   À  ~    Wt; e;   @V a @   À  @V a @e F    @V a @   À  @V a @e @ @   À  @V a @e F    @V a @e @ @   À  ~    @V a @e F    ~    8:20 Choosing t; e;    @V a @e Ft; e;     8:21 190 Optimal adaptive tracking for nonlinear systems [...]... actual control is u ˆ u ‡ r1 where r1 ˆ @t @yr @ 2 '…yr †„ • r …t† À y …yr …t††2 @y2 r  A repeated application of Lemma 4.1 (generalized as in Krstic et al., 199 5 (page •  138, to the case where  ˆ u ‡ …x; †„ ) recovers our earlier result (Krstic et al., 199 2) The  Corollary 4.1 [Krstic et al., 199 2] The adaptive quadratic tracking problem for the following system is solvable Adaptive Control. .. derivative of (8. 19) along the solutions of (8.5), (8.7), (8.8) is given by (8.22) Corollary 3.1 The adaptive quadratic tracking problem for the system (8.1) is solvable if and only if there exists an atclf Va …t; e; † Adaptive Control Systems 193 Proof The `if' part is contained in the proof of Theorem 3.1 where the ” Lyapunov function V…t; e; † is in the form (8. 19) To prove the `only if' part, we start... …8:48† 198 Optimal adaptive tracking for nonlinear systems (1) The nonadaptive control law ~ ~ ~ u ˆ à …t; e; † ˆ …t; e; †; !2 minimizes the cost functional  I À Á Ja ˆ l…t; e; † ‡ r…t; e; †~2 dt; u …8: 49 V P Rp 0 along the solutions of the nonadaptive system (8.47), where !  2 Á @Va @Va À " À1 @Va ~ ‡ f ‡ g ‡ … À 2†r g l…t; e; † ˆ À2 @t @e @e …8:50† …8:51† (2) The inverse optimal adaptive. .. 8.4 Adaptive backstepping With Theorem 3.1, the problem of adaptive stabilization is reduced to the problem of ®nding an atclf This problem is solved recursively via backstepping Lemma 4.1 If the adaptive quadratic tracking problem for the system • x ˆ f …x† ‡ F…x† ‡ g…x†u y ˆ h…x† …8:32† is solvable with a g1 control law, then the adaptive quadratic tracking problem for the augmented system 194 Optimal... an update law  ˆ À…t; e; † with ~ (8.21) The control law …t; e; † is stabilizing for the modi®ed system (8 .9) but may not be stabilizing for the original system (8.5) However, as the proof of ” ~ Theorem 3.1 shows, its certainty equivalence form …t; e; † is an adaptive globally stabilizing control law for the original system (8.5) The modi®ed system `anticipates' parameter estimation transients,.. .Adaptive Control Systems we get • V ” ÀW…t; e; † 191 …8:22† ~ Thus the equilibrium e ˆ 0;  ˆ 0 of (8.5), (8.7), (8.8) is globally stable, and by LaSalle's theorem, e…t† 3 0 as t 3 I By De®nition 2.1, the adaptive tracking problem for (8.1) is solvable The adaptive controller constructed in the proof of Theorem 3.1 consists of a • ” ” ” ~ ~ control law u ˆ …t; e; † given... tracking error e ˆ x À xr , we get the error system 192 Optimal adaptive tracking for nonlinear systems ~ • ~ e1 ˆ e2 ‡ '„  ‡ '„  r • ~ e2 ˆ u À @2 • ”  ” @ …8:26† ~ ~ where ' ˆ '…x1 † À '…xr1 †, 'r ˆ '…xr1 † ˆ '…yr †, u ˆ u À r The modi®ed nonadaptive error system is   @Va „ •1 ˆ e2 ‡ '„  ‡ '„ À ~ e @ …8:27†  „ @2 @Va „ • ~ À ' e2 ˆ u À @ @e1 The control law ~ ~ u ˆ …t; e; †   ! ~ ~... @ …8:40† Adding (8. 39) and (8.40), with (8.13) and (8.37) we get (8.38).This proves by ~ Theorem 3.1 that V1 …t; e; ; † is an atclf for system (8.33), or, an aclf for (8.34), and by Corollary 3.1 the adaptive quadratic tracking problem for this system is solvable The new tuning function is determined by the new atclf V1 and given by 196 Optimal adaptive tracking for nonlinear systems „   !„ !„... system 194 Optimal adaptive tracking for nonlinear systems • x ˆ f …x† ‡ F…x† ‡ g…x† • ˆu is also solvable …8:33† y ˆ h…x† Proof Since the adaptive quadratic tracking problem for the system (8.32) is solvable, by Corollary 3.1 there exists an atclf Va …t; e; † for (8.32), and by ~ ~ Theorem 3.1 it satis®es (8.13) with a control law …t; e; † De®ne  ˆ ”  À r …t; † and consider the system ” ~ @... Control Systems • xi ˆ xi‡1 ‡ 'i …x1 ; F F F ; xi †„ ; 197 i ˆ 1; F F F ; n À 1 • xn ˆ u ‡ 'n …x1 ; F F F ; xn †„  …8:45† y ˆ x1 8.5 Inverse optimal adaptive tracking While in the previous sections our objective was only to achieve adaptive tracking, in this section our objective is to achieve its optimality in a certain sense De®nition 5.1 The inverse optimal adaptive tracking problem for system . ( 199 7). `On Critical Stability of Discrete-time Adaptive Nonlinear Control& apos;, IEEE Transactions on Automatic Control, Vol. 42, 1488±1 499 . [11] Kanellakopoulos, I. ( 199 4). `A Discrete-time Adaptive. Grizzle, J. W. ( 199 3). `Adaptive Output-feedback Control of a Class of Discrete-time Nonlinear Systems', Proceedings of the 199 3 American Control Conference, San Francisco, CA, 13 59 1364. [5]. and Kokotovic  , P. V. ( 199 5). Nonlinear and Adaptive Control Design, Wiley-Interscience, NY. [2] Marino, R. and Tomei, P. ( 199 5). Nonlinear Control Design: Geometric, Adaptive and Robust, Prentice-Hall,