Iterative learning control designs for autonomous driving applications

Iterative Learning Control Designs For Autonomous Driving Applications Submitted by: Nguyen Van Lanh Master Thesis Master of Control System Engineering HAN University of Applied Science, the Netherlands Academic supervisor: Richarch Kaandorp Company supervisor: Dr Son Tong Siemens Industry Software, Belgium Siements Industry Software NV 8th April 2018 Acknowledgement I would like to thank my dear teacher and academic supervisor, Richarch Kaandorp, who gave me a professional supervision with invaluable lessons Richard is a warm-hearted person and he is always willing to answer my questions and discuss with me in his classes In order to finish my thesis, he gave me useful and insightful comments helping me to follow the right track In addition, I would love to express my gratitude to my company supervisor, Dr Son Tong He was always happy to help me solve confusions and led me to obtaining the final results of the thesis Besides, Son is an open-minded person who has been good to me Without his encouragement, I could not finish this final work in my master study Furthermore, I am grateful to Thai Nguyen University of Technology (TNUT) in Vietnam, where I studied and have been working for Additionally, I would like to acknowledge Prof Cuong Duy Nguyen at TNUT in Vietnam He always supports and motivates me in my academic research I also would like to thank Vietnamese Government for sponsoring my Master study at HAN University of Applied Science, Netherlands in form of Project 599 scholarship Finally, I wish to thank my family Your love and belief have brought me up and gone further Thank you! Leuven, March 2018 Lanh Nguyen I Abstract In this master thesis, iterative learning control (ILC) is introduced to deal with the problem of designing the most optimal control signal in autonomous driving applications that require tracking a fixed reference trajectory By exploiting data/information from the previous iterations, the learning control algorithm can obtain better tracking control performance for the next iteration, and hence outperforms conventional control approaches such as feedback control In addition, the control design is based on optimization, where kinematic and dynamic constraints of the vehicle, such as acceleration and steering, are taken into account The learning algorithms can also be used in combination with other traditional control techniques, for example, the conventional feedback control is designed in the first iteration, then learning control is applied to improve performance in the subsequent iterations In this thesis, we use RoFaLT, a nonlinear optimization-based learning control tool, to implement the ILC controllers Finally, the learning control designs are simulated in a co-simulation fashion of LMS Amesim and Prescan software in two different scenarios: autonomous valet parking and racing car The results show the advantages of ILC controllers in improving tracking performance while guaranteeing system constraints Keywords: Advanced Driver Assistance Systems (ADAS), Iterative Learning Control (ILC), Optimal Control II Contents Nomenclature IV List of Figures VII List of Tables VIII Introduction 1.1 Siemens Industry Software NV 1.2 Autonomous Driving 1.3 Goal of this thesis 1.4 Simulation with LMS Imagine.Lab Amesim 1.5 Demostration with PreScan Vehicle dynamics 2.1 Vehicle model 2.2 Tire model 2.3 Slip-free bicycle model 2.4 Valiation 10 1 1 Feedback controller 14 Iterative Learning Control (ILC) 4.1 Overview of ILC controller 4.1.1 PD-type design 4.1.2 Plant Inversion Methods 4.1.3 Quadratically Optimal Design (Q-ILC) 4.1.4 Current-Iterative Learning Control 4.2 RoFaLT tool 4.2.1 Model correction step 4.2.2 Control step 17 17 18 18 19 19 20 21 22 Autonomous Applications and Simulation Results 24 5.1 Application 1: Valet Parking 24 5.2 Application 2: Racing 30 Conclusion and recommendation 41 6.1 Conclusion 41 6.2 Recommendation 41 References 43 III Nomenclature Symbols X Y x y ϕ v δ a g κ α ω F L l I C1 C2 Cr0 Cr2 B C D Global X coordinate Global Y coordinate Local x coordinate Local y coordinate Global orientation Longitudinal velocity Steering angle Commanded acceleration Acceleration of gravity Path curvature Slip angle Yaw rate Force Length of wheelbase Length of wheelbase Moment of inertia Geometrical [l/L] Geometrical [1/L] Zero order friction parameter Second order friction parameter Stiffness factor Shape factor Peak factor [m] [m] [m] [m] [rad] [m/s] [rad] [m/s2 ] [m/s2 ] [−] [rad] [rad/s] [N ] [m] [m] [kg.m2 ] [−] [m−1 ] [−] [−] [−] [−] [−] Subscripts f r l r front wheel rear wheel left right x y z y IV x axis y axis z axis nominal Abbreviations SISW ADAS CG ODE LTI PID ILC MIMO NLP MPC Siemens Industry Software NV Advanced Driver Assistance System Center of Gravity Ordinary Differential Equation Linear time-invariant Proportional - Integral - Derivative Iterative Learning Control Multiple-Input Multiple Output Nonlinear Programming Model Predictive Controller V List of Figures 1.1 1.2 1.3 Vehicle dynamics basic [7] AMEsim vehicle dynamics simulations of the vehicle chassis [7] A scenario in Prescan 2.1 2.2 2.3 2.4 2.5 2.6 Model Coordinate System [2] Bicycle model [2] Geometry for bicycle model Simulink model of Validation Validate results when the vehicle goes Validate results when the vehicle goes 3.1 3.2 3.3 Block diagram of feedback controller 14 Step response of PID controller 15 Result of feedback controller 16 4.1 4.2 4.3 Block diagram of basic iterative learning control 18 Current Iteration ILC architecture 20 Schematic overview of the considered two-step learning algorithms [9] 21 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11 5.12 5.13 5.14 5.15 5.16 5.17 5.18 5.19 5.20 5.21 Parking trajectory Block diagram of feed-forward controller Results of ILC, forward path Results of ILC, backward path Result of last iteration, forward path Result of last iteration, backward path Correction terms and control signals of each iteration, Correction terms and control signals of each iteration, Result of ILC, full parking trajectory Screenshot of demos Racing trajectory Block diagram of ILC-Feedback controller MPC setting MPC setting MPC setting Result of ILC after each iteration in normal weather Result of last iteration, normal weather condition Correction terms and control signals Result of ILC in adverse weather Result of last iteration, adverse weather Tire slip in normal weather VI forward backward 11 12 12 forward path backward path 24 25 26 26 27 27 28 28 29 30 31 31 33 33 34 35 35 36 37 37 38 5.22 5.23 5.24 5.25 Tire slip in adverse weather Control signal, normal weather Control signal, adverse weather Screenshot of videos in Prescan VII 38 39 39 40 List of Tables 2.1 : Parameter of bicycle model 13 3.1 : Parameter of bicycle model 15 5.1 : Parameter of bicycle model 32 VIII Chapter Introduction 1.1 Siemens Industry Software NV Siemens Industry Software NV (SISW) is a business unit of Siemens Digital Factory Division The company provides software, services and systems in the areas of managing the product lifecycle and management of industrial operations SISW works collaboratively with clients to offer industrial software solutions that help companies worldwide to achieve a sustainable competitive advantage by realizing, making real their important innovations 1.2 Autonomous Driving The autonomous driving technologies and systems has been developing dramatically recently in both research and commercial products, for example, autonomous parking technologies function in several cars, i.e Ford, Toyota Lexus, Audi A6, BMW i3 [19] Audi’s self-driving car performed an autonomous journey from San Francisco to Las Vegas on January, 2015 [20] and Waymo’s vehicles have been tested on public roads without human in the driver seat from November, 2017 [21] These intelligent driving assist systems bring various benefits to the society, for example, removing difficulty and stress, reducing the amount of time and traffic disruption, as well as preventing minor dents and scratches by less-skilled drivers 1.3 Goal of this thesis In this thesis, we aim to generate a feedforward control signal using ILC algorithms that realize the motion of the vehicle along a given path while taking into account system dynamics and constraints The reference trajectory of the vehicle is predetermined The requirements of the control system are: smallest tracking error while acceleration and steering control signals are smooth and be constrained ILC has several advantages over a well-designed feedback and feedforward controller [1] A feedback controller has to react to inputs and disturbance, therefore, always has a lag in transient tracking This lag can be eliminated by feedforward controller, but only for known or measurable signals ILC is anticipatory and can compensate for exogenous signals, such as repeating disturbances by learning from Figure 5.9: Result of ILC, full parking trajectory After getting results for both forward path and backward path, only control signals generated from the last iteration are used to control the car following the full parking trajectory Figure 5.9 shows that the vehicle follows the predetermined trajectory accurately Compared to a combination of Figure 5.5 and Figure 5.6, position error in Figure 5.9 is slightly bigger when the vehicle goes backward This is due to the difference between the ending position of car when it goes forward and the initial reference position of the backward path This difference, however, is acceptable since tracking position error is still smaller than 0.05 [m] and meets the requirement conditions Demonstration on Prescan and Amesim Figure 5.10 shows a screenshot of videos on Prescan and AMEsim animation to demonstrate the performance of the designed controller in the parking scenario In this figure, the upper photo is captured from Prescan and the lower ones are obtained from AMEsim animation 29 Figure 5.10: Screenshot of demos 5.2 Application 2: Racing This section presents another application of ILC for a racing car optimal control problem ILC approach is undoubtedly a potential candidate for racing control as the car aims to track the same or similar path lap-to-lap, and it is desirable to improve racing time performance Here we assume that the reference racing trajectory is predetermined, as shown in 5.11 The ILC algorithm calculate the optimal input signal for the next lap using information obtained from the previous lap The challenging of racing application is high speed running could lead to undesirable tire-road slip Slip on wheels could be considered as an unknown disturbance which affects adversely to the tracking performance of the racing car Since ILC cannot deal with unanticipated and nonrepeating disturbance, feedback controller is proposed to be used in combination with ILC A schematic diagram of this method is shown in Figure 5.12 30 Figure 5.11: Racing trajectory Figure 5.12: Block diagram of ILC-Feedback controller As presented in section 4.2, it is necessary to have an initial data for the first ILC iteration; therefore, another controller is used in the first lap In the field of autonomous vehicle control, MPC has become an attractive method for trajectory tracking problems [2], [3], [5], [11] Compared to other feedback control techniques, the main advantages of MPC are capabilities to handle MIMO system and constraints The main idea of MPC is presented in [3] and [11], in which the error is described with respect to the nominal trajectory, called error state as below:    ex cos ϕn sin ϕn     e  − sin ϕn cos ϕn xe =  y  =  eϕ   0 ev 31 0    X − Xn      Y − Yn  =   , 0  ϕ − ϕn  v −   Xn    Yn  where xn (s) =   is the nominal trajectory  ϕn  Then, the tracking control problem will be equivalent to driving the error state to zero The MPC optimization problem is formulated as follows At time instant t, future states are predicted over a finite prediction horizon (N) Based on these predictions, optimal future control inputs at time instants t + k with (k = 1, 2, , N ) over the horizon are calculated Only the control input at time instant t + is applied to the system, after which the problem is shifted to the next instant and new optimization problem is solved using updated measurements In the case of tracking problem, optimal means following the reference trajectory as close as possible, while limiting the control effort, given the state dynamics and the state and input constraints The MPC implementation is followed from [3]: Figure 5.13, Figure 5.14 and Figure 5.15 show the simulation results of the designed MPC controller with different values of weighting matrices Q and R, where   qx 0    qy 0  Q= ;  0 qϕ  0 qv R= rx rδ Table 5.1: : Parameter of bicycle model Setting Setting Setting N 11 11 11 qx 1 qy 1 qϕ 0.1 30 30 qv 0.1 10 100 0.01 0.1 rδ 0.01 0.1 0.1 Note that Qk and Rk are weighted matrices in the MPC cost function By changing them, we can prioritize the weights of cost terms In setting 1, qx and qy are set much higher than qϕ , qv and , rb Consequently, the priority in minimizing position error is higher than in minimizing velocity, yaw angle and control effort Figure 5.13 shows the results of setting 1, in which position error is very small, less than [m] However, there is a strong oscillation in steering input, which is an undesired control signal 32 Figure 5.13: MPC setting In setting 2, the value of qϕ , qv and , rb are increased, in which qϕ is higher than qv So, the priority in minimizing position error is reduced, leading to higher position error, but smoother steering control input, as can be seen in Figure 5.14 This input, however, still illustrates a weak oscillation Figure 5.14: MPC setting 33 In setting 3, we continue to increase the value of qv and Therefore, compared to longitudinal control, lateral control is less priority This leads to higher position error, as shown in figure 5.15; in contrast, the steering control signal is significantly smoother As a result, this setting will be selected as the initial data for the ILC implementation Figure 5.15: MPC setting ILC is used in subsequent laps to improve tracking performance of the racing car Figure 5.16 demonstrates simulation results of ILC performance using RoFaLT In this figure, ILC position error is remarkably smaller than that of using MPC, except at the beginning from t = to 16 [s] This is because we added constrain in rate of change of control signal with respect to time in the correction step of ILC This added constraint allows to get a smoother control signal following our design specifications The figure illustrates that norm of position error using the designed MPC controller is approximately 6.9 [m] After iterations using ILC, it is reduced to nearly 5.5 [m] and converged at this value next iterations 34 Figure 5.16: Result of ILC after each iteration in normal weather To have a closer look at the advantages of the ILC design, the simulation results of the last iteration are shown in Figure 5.17 Compare to MPC results (in Figure 5.15), we can see that the ILC position error is considerable smaller, while the steering control signal is also smoother In both figures, measured velocity and measured yaw angle are almost the same as their references Figure 5.17: Result of last iteration, normal weather condition 35 Correction terms and ILC control signals after each iterations are shown in Figure 5.18, showing how models are updated iteratively Figure 5.18: Correction terms and control signals To analyze further the advantage of ILC in combination with feedback controller, an unknown disturbance is added to the system by changing the friction coefficient between the tire and the road in the AMEsim model In normal weather, its default value equals to In adverse weather, such as snowy or rainy days, this value is set to about 0.6 Using longitudinal, side and turn slips and camber angle sensors, slip on wheels are measured Figure 5.19 shows the result of online ILC in adverse weather This figure illustrates that the controller improves tracking performance substantially after each iteration Similar to running in a normal condition, the car still follows the given trajectory accurately, while tracking error decreases and converses quickly after several laps The performance of last iteration is shown in Figure 5.20 The long slip also can be seen in this figure where measured velocity is over reference velocity at slip times, leading to a big velocity error This error leads to a dramatically decrease in acceleration input, cause by feedback controller is combined with ILC in control algorithm 36 Figure 5.19: Result of ILC in adverse weather Figure 5.20: Result of last iteration, adverse weather The simulation results of slip are shown in Figure 5.21 when the racing car runs in normal weather and in Figure 5.22 when the racing car runs in adverse weather In Figure 5.21, both turn slip and long slip are very small, less than 5.10−3 [degree] and 0.03 [m−1 ] correspondingly In this figure, side slip is significant, up to [null] This slip, however, is almost unchanged after each iteration and has similar shape 37 as steering input It means that side slip occurs when the car turns In contrasts, Figure 5.22 shows a very high value in long slip when the car goes at high speed, up to 40 [m−1 ] at time t = 65 [s] and t = 110 [s] when the speed of the car is 40 [m/s] At these times, side slip oscillates but decreases after each iteration when the ILC is used Figure 5.21: Tire slip in normal weather Figure 5.22: Tire slip in adverse weather 38 The role of feedback controller in the combination with ILC is demonstrated in Figure 5.23 and Figure 5.24, in which control signal is shown in detail for both normal and adverse weather conditions In these figures, the feedback control signal plays a substantial part in the total control signal giving to the car Especially, the acceleration feedback control signal is more dominant than that of the ILC when the vehicle slips at high speed in adverse weather, as shown in Figure 5.24 Figure 5.23: Control signal, normal weather Figure 5.24: Control signal, adverse weather 39 In summary, compared to MPC controller, tracking error is reduced significantly after using online ILC Furthermore, using a combination of ILC and feedback controller, the system is stable even in the presence of unknown disturbance when the car runs at a high speed in adverse weather conditions Demonstration on Prescan and Amesim A screenshot of the videos in Prescan as shown in Figure 5.25 illustrates the performance of the designed controller in racing application The upper image is a view from the top of the racing map enabling us to determine the current position of the car while the lower ones are views from behind the car in different weather conditions Figure 5.25: Screenshot of videos in Prescan 40 Chapter Conclusion and recommendation 6.1 Conclusion The main objective of this study is applying iterative learning control design methodologies for autonomous driving applications The designed ILC implementation using RoFaLT toolbox has shown to improve tracking control performance than two conventional non-learning controllers PID and MPC in the autonomous valet parking and racing car applications, respectively It is seen that by learning from previous iterations, the optimal control input can be satisfactory obtained Moreover, it is worth noting that it is necessary to combine ILC with a feedback control in order to deal with uncertainties 6.2 Recommendation Future research will focus on applying and validating the discussed ILC designs in experimental setups such as industrial robotics and miniature race car 41 Bibliography [1] Douglas A Bristow, Marina Tharayil and Andrew G Alleyne, ”A survey of Iterative learning control, a learning method for high-performance tracking control”, IEEE control systems magazine, June 2006 [2] G Fontana, ”Autonomous Driving of Miniature Race Cars”, Master thesis, Milan, 2014 [3] Robin Verschueren, ”Design and implementation of a time-optimal controller for model race cars”, Master thesis, KU Leuven, Leuven, Belgium, 2014 [4] P Spengler and C Gammeter, ”Modeling of 1:43 scale race cars”, MA thesis ETH Zrich, 2010 (cit on p 20) [5] Stijn De Bruyne, ”Model-based control of mechatronic systems”, PhD dissertation, Leuven University, Belgium, 2016 [6] Edwin Tazelar, Carolien Stroomer, Bram Veenhuizen, ”An Introduction to modelling dynamic”, textbook, HAN University of Applied Science, the Netherlands [7] LMS Amesim demos [8] N Amann, D.H Owens, and E Rogers, ”Iterative learning control for discretetime systems with exponential rate of convergence”, IEE Proc.: Control Theory Applicat., vol 143, no 2, pp 217224, 1996 [9] Armin Steinhauser, Tong Duy Son, Erik Hostens and Jan Swevers, ”RoFaLT: An Optimization-based learning control tool for nonlinear systems”, The 15th International Workshop on Advanced Motion Control (AMC2018), Tokyo, Mar 2018 [10] Nitin R Kapania, ”Trajectory planning and control for an autonomous vehicle”, PhD dissertation, Stanford University, USA, 2016 [11] Lukas Wunderli, ”MPC based Trajectory Tracking for 1:43 scale Race Cars”, Masters thesis ETH Zurich, 2011 [12] H Pacejka, ”Tire and vehicle dynamics”, Elsevier, 2005 [13] https://en.wikipedia.org/wiki/Automatic parking [14] Tong Duy Son, ”Robust Multi-objective Iterative Learning Control”, PhD dissertation, Leuven University, Belgium, 2016 42 [15] Benjamas Panomruttanarug, ”Application of iterative learning control in tracking a Dubins path parallel parking”, International Journal of Automotive Technology, Vol 18, No 6, pp 10991107, 2017 [16] Siemens Industry Software PLM https://www.plm.automation.siemens.com/en/products/lms/imaginelab/amesim/ [17] TASS International http: //https://tass.plm.automation.siemens.com/prescan [18] Smitha A.B, Sachin C.N Shetty, Bianca Baby, ”PID Controller Tuning and Its Case Study”, International Journal for Research in Applied Science Engineering Technology (IJRASET), Volume Issue VI, June 2016, ISSN: 2321-9653 [19] https://www.thecarconnection.com/news/1067819 2012-family-cars-with -self-parking-technology [20] http://www.dailymail.co.uk/sciencetech/article-2897662/The-stress -free-road-trip-Audi-sends-self-driving-car-550-mile-journey-SanFrancisco-Las-Vegas.html [21] https://www.theverge.com/2017/11/7/16615290/waymo-self-drivingsafety-driver-chandler-autonomous 43 ... Alleyne, ”A survey of Iterative learning control, a learning method for high-performance tracking control? ??, IEEE control systems magazine, June 2006 [2] G Fontana, ? ?Autonomous Driving of Miniature... applying iterative learning control design methodologies for autonomous driving applications The designed ILC implementation using RoFaLT toolbox has shown to improve tracking control performance... data/information from the previous iterations, the learning control algorithm can obtain better tracking control performance for the next iteration, and hence outperforms conventional control