On learning based parameter calibration and ramp metering of freeway traffic systems

On Learning based Parameter Calibration and Ramp Metering of Freeway Traffic Systems BY XINJIE ZHAO A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING NATIONAL UNIVERSITY OF SINGAPORE 2013 Acknowledgments I Acknowledgments I would like to express my deepest appreciation to Prof Jian-Xin Xu for his generous support, excellent guidance, and encouragement His erudite academic knowledge, professional research experience and precise insights have always been valuable sources and excellent examples for me I owe an immense debt of gratitude to him for having given me the consistent and generous help about learning and research Without his kindest help, this thesis and many others would have been impossible Thanks also go to Electrical & Computer Engineering Department in National University of Singapore, for the financial support during my pursuit of a PhD I would like to thank Dr Dipti Srinivasan at National University of Singapore, Dr Balaji Parasumanna and Dr Vishal Sharma who provided me kind encouragement and constructive suggestions for my research I am also grateful to all my friends in Energy Management & Microgrid Laboratory, the National University of Singapore Their kind assistance and friendship have made my life in Singapore exciting and colorful Most importantly, I would thank my family members for their support, understanding, patience and love during the past several years In particular, I would like to thank my wife, Xiao Zhou, for every wonderful moment that she had spent accompanying and supporting me This thesis, thereupon, is dedicated to her and our beloved daughter, Ai-Xiao Zhao Contents Acknowledgments I Nomenclature VIII Summary X List of Figures XII List of Tables 1 Introduction 1.1 Modeling and Control of Freeway Traffic 1.2 Literature Review 1.2.1 Freeway Traffic Flow Modeling 1.2.2 Parameter Calibration 1.2.3 Freeway Traffic Control 12 1.2.4 Recent Advances and Important Issues 20 1.3 Focus of the Research and Main Objective 23 1.4 Outline of Dissertation 27 1.5 Main Contributions 31 II Contents III Revisit on SPSA, FLC, METANET and ALINEA 33 2.1 The SPSA Algorithm 33 2.2 The FLC Algorithm 35 2.3 The METANET Model 37 2.4 The ALINEA Algorithm 40 Hybrid Iterative Parameter Calibration Algorithm for Macroscopic Freeway Modeling 42 3.1 Introduction 42 3.2 Problem Formulation 45 3.3 Hybrid Iterative Calibration Approach 47 3.3.1 Simultaneous Perturbation Based Gradient Estimation 47 3.3.2 The Hybrid Algorithm 48 3.3.3 Convergence Analysis 49 3.4 51 3.4.1 Description of Data 51 3.4.2 Simulation Setup 53 3.4.3 Results And Discussion 53 3.4.4 3.5 Illustrative Examples Further Investigation 58 Conclusion Optimal Freeway Local Ramp Metering Using FLC and PSO 60 63 4.1 Introduction 63 4.2 Problem Statement 68 4.2.1 68 Freeway Model Contents 4.2.2 IV 69 Motivations 69 Design of FLC Algorithm 71 4.3.3 PSO Based Parameter Tuning 73 4.3.4 Fitness Function 75 Numerical Experiments 76 4.4.1 Simulation Setup 76 4.4.2 4.5 FLC Based Local Ramp Metering 4.3.2 4.4 69 4.3.1 4.3 Objective Results and Discussions 77 Conclusion 82 FLC based Adaptive Freeway Local Ramp Metering with SPSA based Parameter Learning 83 5.1 Introduction 83 5.2 Problem Formulation 88 5.2.1 Freeway Model 88 5.2.2 Objective 89 FLC based Ramp Metering Algorithm 90 5.3.1 Input Membership Functions 90 5.3.2 Fuzzy Rule Base 91 5.3.3 Inference and Defuzzification 92 5.4 SPSA based Parameter Learning 93 5.5 Illustrative Examples 95 5.5.1 Simulation Setup 95 5.5.2 Results and Discussion 96 5.3 Contents 5.5.3 5.6 V Further Discussion 100 Conclusion 103 A Novel and Efficient Local Coordinative Freeway Ramp Metering Strategy with SPSA based Parameter Learning 106 6.1 Introduction 106 6.2 Problem Formulation 110 6.2.1 6.2.2 6.3 The Problem 110 Objective 111 The Local Coordinative Ramp Metering Strategy 112 6.3.1 6.3.2 Input Membership Functions 115 6.3.3 Fuzzy Rule Base 116 6.3.4 Inference and Defuzzification 117 6.3.5 Objective 118 6.3.6 6.4 System Structure 112 SPSA Based Parameter Learning 118 Numerical Example 121 6.4.1 6.4.2 6.5 Simulation Setup 121 Results and Discussions 123 Further Discussions on Equity Issues 126 6.5.1 6.5.2 6.6 Case Studies 126 Results and Discussions 128 Conclusions 134 Contents VI Networked Freeway Ramp Metering Using Macroscopic Traffic Scheduling and SPSA based Parameter Learning 137 7.1 Introduction 137 7.2 Problem Formulation 142 7.2.1 7.2.2 7.3 METANET Model Revisit 142 ALINEA based Ramp Metering 144 Optimal Macroscopic Freeway Traffic Scheduling with Parameter Learning 145 7.3.1 7.3.2 The Macroscopic Traffic Scheduling Strategy 146 7.3.3 Cost function 146 7.3.4 Objective 148 7.3.5 7.4 Networked Freeway Ramp Metering Problem 145 SPSA Based Parameter Learning 148 Illustrative Example 150 7.4.1 7.4.2 Case Studies 152 7.4.3 Results and Discussion 153 7.4.4 7.5 Simulation Setup 150 Further Discussion 160 Conclusions 163 Conclusions 165 8.1 Summary of Results 165 8.2 Suggestions for Future Work 170 Bibliography 171 Appendix A: Proof Details 184 Contents VII Appendix B: Publication List 187 Nomenclature Symbol Meaning or Operation ∀ for all ∃ there exists ∈ in the set ∞ infinity ˆ θ the estimate of θ T a time interval (second) k index of time interval β constant feedback gain o measured occupancy at the downstream location of the merging area o∗ desired mainstream occupancy x vector of traffic states rmax upper bound of ramp metering flow rmin lower bound of ramp metering flow ρ(k) mainstream density in the mainstream merging area at time step k (veh/lane/km) ρd (k) desired density in the mainstream merging area at time step k (veh/lane/km) ω(k) number of queueing vehicle on the on-ramp link at time step k (veh) v(k) mainstream traffic speed at time step k (km/hour) r(k) flow rate of merging traffic on the on-ramp link (veh/hour) i index of particle VIII Nomenclature Symbol j IX Meaning or Operation index of generation Vij velocity of particle i at generation j ηj weighting factor at generation j ηmax initial inertia weight ηmin final inertia weight c1 , c2 weighting factor rand() random number uniformly distributed between and θj i p θi parameters found by particle i at generation j personal best solution with the best fitness found so far for the particle i θg i global best solution of particle i found so far among a 3-member neighborhood with best fitness Gen ALINEA maximum generation number Asservissement LIN´aire d’Entr´e Autorouti`re e e e DV U Driver Vehicle Unit FLC Fuzzy Logic Control ILC Iterative Learning Control ITS Intelligent Transportation Systems LCRM Local Coordinative Ramp Metering METANET MTS NR PARAMICS PATH SA A macroscopic freeway traffic flow model Macroscopic Traffic Scheduling Newton Raphson PARAllel MICroscopic traffic Simulator Partners for Advanced Transportation TecHnology Stochastic Approximation SPSA Simultaneous Perturbation Stochastic Approximation TTS Total Time Spent WTTS Weighted Total Time Spent Bibliography 174 [23] M Cremer and M Papageorgiou, “Parameter identification for a traffic flow model,” Automatica, vol 17, no 6, pp 837–843, 1981 [24] M J D Powell, “On search directions for minimization algorithms,” Mathematical Programming, vol 4, pp 193–201, 1973 [25] K I M Mckinnon and K I M Mckinnon, “Convergence of the nelder-mead simplex method to a non-stationary point,” SIAM J Optim, Tech Rep., 1996 [26] A Kotsialos, M Papageorgiou, M Mangeas, and H Haj-Salem, “Coordinated and integrated control of motorway networks via non-linear optimal control,” Transportation Research Part C: Emerging Technologies, vol 10, no 1, pp 65 – 84, 2002 [27] M Papageorgiou and A Kotsialos, “Freeway ramp metering: an overview,” Trans Intell Transport Sys., vol 3, no 4, pp 271–281, dec 2002 [28] I Papamichail, A Kotsialos, I Margonis, and M Papageorgiou, “Coordinated ramp metering for freeway networks - a model-predictive hierarchical control approach,” Transportation Research Part C: Emerging Technologies, vol 18, no 3, pp 311–331, 2010 [29] D P Masher, D W Ross, P J Wong, P L Z P L Tuan, and S Peracek, Guidelines for design and operation of ramp control systems, ser NCHRP Research Results Digest Transportation Research Board, Mar 1976 [30] Probabilistic nature of breakdown at freeway merge junctions, vol 1484, 1995 [31] Defining highway capacity as a function of the breakdown probability, vol 1776, 2001 Bibliography 175 [32] M J Cassidy and J Rudjanakanoknad, “Increasing the capacity of an isolated merge by metering its on-ramp,” Transportation Research Part B: Methodological, vol 39, no 10, pp 896 – 913, 2005 [33] P M., H.-S H., and B J M., “Alinea: A local feedback control law for on-ramp metering,” Transportation Research Record, no 1320, pp 58 – 64, 1991 [34] E Smaragdis, E Papageorgiou, and E Kosmatopoulos, “A flow-maximizing adaptive local ramp metering strategy,” Transportation Research, Part B (Methodological), vol 38B, no 3, pp 251 – 70, Mar 2004 [35] E Smaragdis and M Papageorgiou, “A series of new local ramp metering strategies,” Transportation Research Record: Journal of the Transportation Research Board, vol 1856, pp 74 – 86, 2003 [36] I Papamichail, M Papageorgiou, V Vong, and J Gaffney, “Heuristic rampmetering coordination strategy implemented at monash freeway, australia,” Transportation Research Record, vol 2178, pp 10 – 20, Oct, 2010 [37] A Hegyi, B De Schutter, and H Hellendoorn, “Model predictive control for optimal coordination of ramp metering and variable speed limits,” Transportation Research Part C (Emerging Technologies), vol 13, no 3, pp 185 – 209, June 2005 [38] A Ghods, L Fu, and A Rahimi-Kian, “An efficient optimization approach to realtime coordinated and integrated freeway traffic control,” IEEE Trans Intell Transp Syst (USA), vol 11, no 4, pp 873 – 84, 2010 Bibliography 176 [39] D Zhao, X Bai, F.-Y Wang, J Xu, and W Yu, “Dhp method for ramp metering of freeway traffic,” IEEE Transactions on Intelligent Transportation Systems, vol 12, no 4, pp 990 – 999, 2011 [40] W Powell, Approximate Dynamic Programming: Solving the Curses of Dimensionality, ser Wiley Series in Probability and Statistics Wiley-Interscience, 2007 [41] A Kotsialos, M Papageorgiou, and F Middelham, “Local and optimal coordinated ramp metering for freeway networks,” Journal of Intelligent Transportation Systems: Technology, Planning, and Operations, vol 9, no 4, pp 187–203, 2005 [42] M Papageorgiou, J Blosseville, and H Haj-Salem, “Modelling and real-time control of traffic flow on the southern part of boulevard peripherique in paris: Part II: coordinated on-ramp metering,” Transportation Research Part A: General, vol 24, no 5, pp 361–370, 1990 [43] P Zheng and M McDonald, “Estimation of travel time using fuzzy clustering method,” IET Intelligent Transport Systems, vol 3, no 1, pp 77 – 86, 2009 [44] L Li, W.-H Lin, and H Liu, “Type-2 fuzzy logic approach for short-term traffic forecasting,” IEE Proceedings - Intelligent Transport Systems, vol 153, no 1, pp 33–40, 2006 [45] D Srinivasan, S Sanyal, and V Sharma, “Freeway incident detection using hybrid fuzzy neural network,” IET Intell Transp Syst (UK), vol 1, no 4, pp 249 – 59, 2007 Bibliography 177 [46] P Balaji, X German, and D Srinivasan, “Urban traffic signal control using reinforcement learning agents,” IET Intelligent Transport Systems, vol 4, no 3, pp 177–188, 2010 [47] I Arel, C Liu, T Urbanik, and A Kohls, “Reinforcement learning-based multiagent system for network traffic signal control,” IET Intelligent Transport Systems, vol 4, no 2, pp 128–135, 2010 [48] S Vulkanovic and O Ernhofer, “Field evaluation of the fuzzy logic based ramp metering algorithm accezz,” in IFAC Proceedings Volumes (IFAC-PapersOnline), vol 11, no PART 1, 2006, pp 119 – 123 [49] A Ghods, A Kian, and M Tabibi, “Adaptive freeway ramp metering and variable speed limit control: a genetic-fuzzy approach,” IEEE Intelligent Transport Syst Mag (USA), vol 1, no 1, pp 27 – 36, Spring 2009 [50] J Xu, X Zhao, and D Srinivasan, “Fuzzy logic controller for freeway ramp metering with particle swarm optimization and PARAMICS simulation,” in 2010 IEEE International Conference on Fuzzy Systems, 18-23 July 2010, ser 2010 IEEE International Conference on Fuzzy Systems Piscataway, NJ, USA: IEEE, 2010, p pp [51] X Chen and A Tian, “Research on fuzzy on-ramp metering and simulation in urban expressway,” in Proceedings 2010 International Conference on Optoelectronics and Image Processing (ICOIP 2010), vol 2, 2010, pp 221 – [52] T Jiang and X Liang, “Fuzzy self-adaptive pid controller for freeway ramp metering,” in 2009 International Conference on Measuring Technology and Mechatronics Automation (ICMTMA), vol 2, 2009, pp 570 – Bibliography 178 [53] B K., K H., and M A D., “Neuro-fuzzy approach for ramp metering,” in IEE Conference Publication, no 472, 2000, pp 101 – 105 [54] T Cynthia, M Deirdre, and J Les, “Fuzzy ramp metering: Design overview and simulation results,” Transportation Research Record, no 1634, pp 10 – 18, 1998 [55] B K and K H., “An evolutionary fuzzy system for coordinated and traffic responsive ramp metering,” in Proceedings of the Hawaii International Conference on System Sciences, 2001, pp 80 – [56] P V.C., A F., P J., F F.C., and X W.L., “Integrated fuzzy signal and rampmetering at a diamond interchange,” Journal of Advanced Transportation, 2011 [57] H Wang, Y Chen, and Z Shi, “A hierarchy control strategy for coordinated ramp metering,” in Proceedings of the 2003 IEEE International Conference on Intelligent Transportation Systems (Cat No.03TH8708), vol 2, 2003, pp 1070 – [58] L Xin-rong, L Zhi-yong, and M Zong-yuan, “Nonlinear feedback ramp controller in freeway based on fuzzy logic,” Control Theory and Applications, vol 23, no 1, pp 99 – 102, Feb 2006 [59] K Eil, N Sreemannarayan, L Rich, and A James, “Comparative analysis of operational algorithms for coordinated ramp metering,” in Transportation Research Record, no 1748, 2001, pp 144 – 152 [60] Z Yan and C Yang-Zhou, “New method for controlling freeway on-ramp based on fuzzy logic,” Journal of System Simulation, vol 17, no 10, pp 2486 – 9, 2005/10/ Bibliography 179 [61] X Liang and Z Li, “Freeway ramp control based on genetic pi and fuzzy logic,” in 2008 Pacific-Asia Workshop on Computational Intelligence and Industrial Application PACIIA 2008, 2008, pp 382 – [62] Z Hou, J.-X Xu, and H Zhong, “Freeway traffic control using iterative learning control-based ramp metering and speed signaling,” Vehicular Technology, IEEE Transactions on, vol 56, no 2, pp 466 –477, march 2007 [63] Z Hou, X Xu, J Yan, J.-X Xu, and G Xiong, “A complementary modularized ramp metering approach based on iterative learning control and alinea,” IEEE Transactions on Intelligent Transportation Systems, vol 12, no 4, pp 1305 – 18, Dec 2011 [64] Z Hou, H Zhong, and J Xu, “An iterative learning approach for local ramp metering,” in Proceedings of the 2004 IEEE International Conference on Control Applications, vol 2, 2004, pp 989 – 94 [65] J Yan and Z Hou, “Iterative learning control based freeway ramp metering with iteration-varying parameter,” International Journal of Innovative Computing, Information and Control, vol 6, no 3A, pp 875 – 84, Mar 2010 [66] X Zhao, D Srinivasan, and J Xu, “Iterative learning control of freeway flow via ramp metering and simulation on paramics,” in TENCON 2009 2009 IEEE Region 10 Conference, 2009, pp pp – [67] J Spall, “Multivariate stochastic approximation using a simultaneous perturbation gradient approximation,” Automatic Control, IEEE Transactions on, vol 37, no 3, pp 332–341, 1992 Bibliography 180 [68] H Robbins and S Monro, “A stochastic approximation method,” Annals of Mathematical Statistics, vol 22, no 3, pp 400–407, 1951 [69] J Kiefer and J Wolfowitz, “Stochastic estimation of the maximum of a regression function,” Annals of Mathematical Statistics, vol 23, no 3, pp 462–466, 1952 [70] L Zadeh, “Fuzzy sets,” Information and Control, vol 8, no 3, p 338353, 1965 [71] M Papageorgiou, J Blosseville, and H Hadj-Salem, “Modelling and real-time control of traffic flow on the southern part of boulevard peripherique in paris: Part i: Modelling,” Transportation Research Part A: General, vol 24, no 5, pp 345–359, 1990 [72] ——, “Macroscopic modelling of traffic flow on the boulevard peripherique in paris,” Transportation Research Part B: Methodological, vol 23, no 1, pp 29–47, 1989 [73] Z M HOU Zhong-Sheng, JIN Sheng-Tai, “Iterative learning identification method for the macroscopic traffic flow model,” Acta Automatica Sinica, vol 34, no 1, pp 64–71, 2008 [74] T Hirokami, Y Maeda, and H Tsukada, “Parameter estimation using simultaneous perturbation stochastic approximation,” Transactions of the Institute of Electrical Engineers of Japan, Part C, vol 124-C, no 11, pp 2241 – 8, 2004 [75] J C Spall, “Overview of the simultaneous perturbation method for efficient optimization,” American Society of Civil Engineers - Task Committee Reports, pp 141 – 154, 1999 Bibliography 181 [76] L Jung-Beom, “Calibration of traffic simulation models using simultaneous perturbation stochastic approximation (spsa) method extended through bayesian sampling methodology,” Ph.D dissertation, The State University of New Jersey, 2008 [77] C T Kelley, Iterative Methods for Optimization Philadephia, PA: Society for Industrial and Applied Mathematics, 1999 [78] J A Wattleworth and D S Berry, “Peak-period analysis and control of a freeway system,” Highway Research Record, pp – 25, 1965 [79] S C Schwartz and H H Tan, “Integrated control of freeway entrance ramps by threshold regulation,” in Proceedings of the 1977 IEEE Conference on Decision and Control, vol I, 1977, pp 984 – [80] D P Masher, D W Ross, P J Wong, P L Tuan, P L Zeidler, and S Peracek, “Guidelines for design and operating of ramp control systems,” Standford Research Institute, SRI, Menid Park, CA, Technical Report, SRI Project 3340 NCHRP 3-22, 1975 [81] G Gomes and R Horowitz, “Optimal freeway ramp metering using the asymmetric cell transmission model,” Transportation Research Part C (Emerging Technologies), vol 14, no 4, pp 244 – 62, Aug 2006 [82] J Castro, “Fuzzy logic controllers are universal approximators,” IEEE Transactions on Systems, Man and Cybernetics, vol 25, no 4, pp 629 – 635, 1995 [83] Fuzzy C Development System Users Manual, Togai Infra Logic, Inc., Irvine, CA, 1992 Bibliography 182 [84] I C Trelea, “The particle swarm optimization algorithm: convergence analysis and parameter selection,” Information Processing Letters, vol 85, no 6, pp 317–325, 2003 [85] F Van Den Bergh, “An analysis of particle swarm optimizers,” Ph.D dissertation, University of Pretoria, Pretoria, South Africa, South Africa, 2002 [86] M Clerc and J Kennedy, “The particle swarm - explosion, stability, and convergence in a multidimensional complex space,” Trans Evol Comp, vol 6, no 1, pp 58–73, Mar 2002 [87] J Kennedy and R C Eberhart, Swarm intelligence San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2001 [88] T Xu, L Sun, Z Peng, and Y Hao, “Integrated route guidance and ramp metering consistent with drivers’ en-route diversion behaviour,” IET Intelligent Transport Systems, vol 5, no 4, pp 267–276, 2011 [89] A Ghods, A Kian, and M Tabibi, “Adaptive freeway ramp metering and variable speed limit control: a genetic-fuzzy approach,” Intelligent Transportation Systems Magazine, IEEE, vol 1, no 1, pp 27 –36, spring 2009 [90] G D B Cameron and G I D Duncan, “Paramics - parallel microscopic simulation of road traffic,” The Journal of Supercomputing, vol 10, no 1, pp 25–53, 1996 [91] S An, J Cui, and L Li, “Agent-based approach to model commuter behaviour’s day-to-day dynamics under pre-trip information,” IET Intelligent Transport Systems, vol 5, no 1, pp 70–79, 2011 Bibliography 183 [92] J Martinez, K Nakano, and K Higuchi, “Parameter estimation using a modified version of spsa algorithm applied to state space models,” Transactions of the Institute of Electrical Engineers of Japan, Part D, vol 129, no 12, pp 1187 – 94, 2009 [93] M PAPAGEORGIOU and R MAYR, “Optimal decomposition methods applied to motorway traffic control,” International Journal of Control, vol 35, no 2, pp 269–280, 1982 [94] Development and evaluation of a dynamic metering control model, Chania, Greece, 1997 [95] H Zhang and W Recker, “On optimal freeway ramp control policies for congested traffic corridors,” Transportation Research Part B, p 417436, 1999 [96] L Zhang and D Levinson, “Optimal freeway ramp control without origindestination information,” Transportation Research Part B, pp 869–877, 2004 [97] J R Scariza, “Evaluation of coordinated and local ramp metering algorithm using microscopic traffic simulation,” Master’s thesis, Massachusetts Institute of Technology Dept of Civil and Environmental Engineering, 2003 Appendix A: Proof Details A.1 Proof of Theorem Proof At each iteration, by the mean value theorem and Taylor’s expansion, we have ¯ dLi = J θ i1 dθ i , (8.1) ¯ δLi = J θ i2 δθ i , (8.2) and ¯ ˆ ˆ ¯ where θ i1 denotes a point on the line segment of θ i and θ i + dθ i , θ i2 denotes a point on ˆ the line segment of θ ∗ and θ i ¯ ¯ Let Ji1 and Ji2 denote J θ i1 and J θ i2 respectively The evolution of cost functions during the parameter calibration process is expressed as: Li+1 = Li + dLi = Li + Ji1 dθ i ˆ = Li − Ji1 Ji−1 Li (8.3) Using L∗ to subtract both sides of the above relationship, we have ˆ ˆ δLi+1 = (I − Ji1 Ji−1 )δLi + Ji1 Ji−1 L∗ (8.4) ˆ ˆ Let ρ1 be the upper bound of I − Ji1 Ji−1 , and ǫ1 be the upper bound of Ji1 Ji−1 L∗ , 184 Appendix A: Proof Details 185 apply norm on both sides of the relationship (8.4), we have ˆ ˆ δLi+1 = (I − Ji1 Ji−1 )δLi + Ji1 Ji−1 L∗ ˆ ≤ I − Ji1 Ji−1 ˆ δLi + Ji1 Ji−1 L∗ ≤ ρ1 δLi + ǫ1 i ≤ ρi+1 L0 + ǫ1 ρ1 j j=0 = ρi+1 L0 1 − ρ1 i+1 + ǫ1 − ρ1 (8.5) ǫ1 , − ρ1 (8.6) hence if ρ < 1, we have lim δLi = i→∞ which means θ i converges to a bounded neighborhood of θ ∗ Since ǫ1 depends on , the radius of the bounded neighborhood can be reduced by adopting a smaller updating gain By proper gradient estimation method, e.g simultaneous perturbation algorithm, ˆ we are able to obtain estimated Jacobian matrix such that Ji1 ≃ J θ i By adopting ¯ ˆ a small enough updating rate which makes θ i1 and θ i close enough to each other, we ˆ obtain I − Ji1 Ji−1 ≤ ρ1 < 1, satisfying the convergence criteria for δLi Since ˆ ˆ ˆ θ i+1 = θ i −ai Ji−1 Li , (8.7) using θ ∗ to subtract both sides of the above relationship, we have ˆ ˆ ˆ θ ∗ − θ i+1 = θ ∗ − θ i +ai Ji−1 Li (8.8) Appendix A: Proof Details 186 hence, we have ˆ δθ i+1 = δθ i + Ji−1 (Li − L∗ + L∗ ), ˆ ˆ = δθ i − Ji−1 δLi + Ji−1 L∗ ˆ ˆ = δθ i − Ji−1 Ji2 δθ i + Ji−1 L∗ ˆ ˆ = (I − Ji−1 Ji2 )δθ i + Ji−1 L∗ (8.9) Apply norm on both sides of the above relationship, we have ˆ ˆ δθ i+1 = (I − Ji−1 Ji2 )δθ i + Ji−1 L∗ ˆ ˆ ≤ (I − Ji−1 Ji2 )δθ i + Ji−1 L∗ ˆ ˆ = (I − Ji−1 Ji2 ) δθ i + Ji−1 L∗ (8.10) ˆ Let ρ2 denote the upper bound of I − Ji−1 Ji2 , ǫ2 denote the upper bound of ˆ Ji−1 L∗ , we have δθ i+1 ≤ ρ2 δθ i + ǫ2 i ≤ ρ2 i+1 δθ + ǫ2 ρ2 j j=0 = ρ2 i+1 δθ + ǫ2 − ρ2 i+1 − ρ2 (8.11) From the above relationship, it is clear that if ρ2 < 1, we get lim δθ i = i→∞ ǫ2 , − ρ2 (8.12) indicating that θ i converges to a bounded neighborhood of θ ∗ Similar to the convergence of δLi , the radius of this neighborhood can also be reduced by using a small updating gain Appendix B: Published/Submitted Papers Refereed Journal Articles: [1] Xu, J.; Zhao, X.; Srinivasan, D., “On optimal freeway local ramp metering using fuzzy logic control with particle swarm optimisation,” Intelligent Transport Systems, IET , vol.7, no.1, pp.95,104, March 2013 [2] Xinjie Zhao, Jian-Xin Xu, and Dipti Srinivasan, “A Novel and Efficient Local Coordinative Freeway Ramp Metering Strategy with SPSA based Parameter Learning”, Minor Revision (Revision Submitted), Intelligent Transport Systems, IET [3] Jian-Xin Xu, Xinjie Zhao and Dipti Srinivasan, “A Hybrid Iterative Parameter Estimation Approach for Macroscopic Freeway Traffic Modeling”, In Revision, Transactions on Intelligent Transportation Systems, IEEE [4] Xinjie Zhao, Jian-Xin Xu, and Dipti Srinivasan, “FLC based Adaptive Freeway Local Ramp Metering with SPSA based Parameter Learning”, Submitted, Journal of Intelligent Transportation Systems, JITS [5] Xinjie Zhao, Jian-Xin Xu, and Dipti Srinivasan, “Networked Freeway Ramp Metering Using Macroscopic Traffic Scheduling and SPSA based Parameter Learning”, Submitted, 187 Appendix B 188 Systems Journal, IEEE International Conference Articles: [1] Jianxin Xu, Xinjie Zhao, and Dipti Srinivasan, “Flow based local ramp metering using iterative learning and PARAMICS platform”, in 2009 IEEE International Conference on Control and Automation, Piscataway, NJ, USA, 2009 [2] Xinjie Zhao, Dipti Srinivasan, and Jianxin Xu, “Iterative learning control of freeway flow via ramp metering and simulation on PARAMICS”, TENCON 2009 - 2009 IEEE Region 10 Conference, Singapore, 2009 [3] Jianxin Xu, Xinjie Zhao, and Dipti, Srinivasan, “Fuzzy logic controller for freeway ramp metering with particle swarm optimization and PARAMICS simulation”, Fuzzy Systems (FUZZ), 2010 IEEE International Conference on, Barcelona, Spain [4] Xinjie Zhao, Jianxin Xu, and Dipti Srinivasan, “Iterative learning control of freeway flow via ramp metering and simulation on PARAMICS”, SSCI 2013 - 2013 IEEE Symposium Series on Computational Intelligence, Singapore ... local and networked freeway systems, and most of them utilized freeway traffic flow models on design and analysis of control systems These control techniques, ranging from conventional feedback control... commonly used methods Ramp metering has received much attention from researchers due to its good efficiency in dealing with freeway congestion and improving freeway traffic conditions Freeway ramp metering. .. based parameter calibration for macroscopic traffic flow modeling and design of learning control strategies for local and coordinated freeway ramp metering A hybrid iterative parameter calibration