Chaotic Systems Part 2 pot

Chaotic Systems 14 Fig. 10. Time series of the observed value (network targets) and the predicted value (network outputs) for the 5-min traffic volume. 4.2.2 10-min traffic volume The network inputs and targets are the 14-dimensional delay coordinates: x(i), x(i-10), x(i- 20 ),…, x(i-130), and x(i+1), respectively. Similarly, by using Bayesian regularization, the effective number of parameters is first found to be 108, as shown in Fig. 11; therefore, the appropriate number of neurons in the hidden layer is 7 (one half of the number of elements in the input vector). Replace the number of neurons in the hidden layer with 7 and train the network again. The training process stops at 11 epochs because the validation error has increased for 5 iterations. Fig. 12 shows the scatter plot for the training set with correlation coefficient ρ=0.93874. Simulate the trained network with the prediction set. Fig. 13 shows the scatter plot for the prediction set with the correlation coefficient ρ=0.91976. Time series of the observed value (network targets) and the predicted value (network outputs) are shown in Fig. 14. If the strategy “early stopping” is disregarded and 100 epochs is chosen for the training process, the performance of the network improves for the training set, but gets worse for the validation and prediction sets. If the number of neurons in the hidden layer is increased to 14 and 28, the performance of the network for the training set tends to improve, but does not have the tendency to improve for the validation and prediction sets, as listed in Table 4. No. of Neurons Data 7 14 28 Training Set 0.93874 0.95814 0.96486 Validation Set 0.92477 0.87930 0.88337 Prediction Set 0.91976 0.90587 0.91352 Table 4. Correlation coefficients for training, validation and prediction data sets with the number of neurons in the hidden layer increasing (10-min traffic volume). Short-Term Chaotic Time Series Forecast 15 Fig. 11. The convergence process to find effective number of parameters used by the network for the 10-min traffic volume. Fig. 12. The scatter plot of the network outputs and targets for the training set of the 10-min traffic volume. Chaotic Systems 16 Fig. 13. The scatter plot of the network outputs and targets for the prediction set of the 10- min traffic volume. Fig. 14. Time series of the observed value (network targets) and the predicted value (network outputs) for the 10-min traffic volume. Short-Term Chaotic Time Series Forecast 17 4.2.3 15-min traffic volume The network inputs and targets are the 14-dimensional delay coordinates: x(i), x(i-5), x(i- 10 ),…, x(i-65), and x(i+1), respectively. In a similar way, the effective number of parameters is found to be 88 from the results of Bayesian regularization, as shown in Fig. 15. Instead of using 6 neurons obtained by Eq. (11), 7 neurons (one half of the number of elements in the input vector), are used in the hidden layer for consistence. Replace the number of neurons in the hidden layer with 7 and train the network again. The training process stops at 11 epochs because the validation error has increased for 5 iterations. Fig. 16 shows the scatter plot for the training set with correlation coefficient ρ=0.95113. Simulate the trained network with the prediction set. Fig. 17 shows the scatter plot for the prediction set with the correlation coefficient ρ=0.93333. Time series of the observed value (network targets) and the predicted value (network outputs) are shown in Fig. 18. If the strategy “early stopping” is disregarded and 100 epochs is chosen for the training process, the performance of the network gets better for the training set, but gets worse for the validation and prediction sets. If the number of neurons in the hidden layer is increased to 14 and 28, the performance of the network for the training set tends to improve, but does not have the tendency to significantly improve for the validation and prediction sets, as listed in Table 5. No. of Neurons Data 7 14 28 Training Set 0.95113 0.96970 0.97013 Validation Set 0.88594 0.93893 0.92177 Prediction Set 0.93333 0.94151 0.94915 Table 5. Correlation coefficients for training, validation and prediction data sets with the number of neurons in the hidden layer increasing (15-min traffic volume). Fig. 15. The convergence process to find effective number of parameters used by the network for the 15-min traffic volume. Chaotic Systems 18 Fig. 16. The scatter plot of the network outputs and targets for the training set of the 15-min traffic volume. Fig. 17. The scatter plot of the network outputs and targets for the prediction set of the 15- min traffic volume. Short-Term Chaotic Time Series Forecast 19 Fig. 18. Time series of the observed value (network targets) and the predicted value (network outputs) for the 15-min traffic volume. 4.3 The multiple linear regression Data collected for the first nine days are used to build the prediction model, and data collected for the tenth day to test the prediction model. To forecast the near future behavior of a trajectory in the reconstructed 14-dimensional state space with time delay τ= 20, the number of 200 nearest states of the trajectory, after a few trials, is found appropriate for building the multiple linear regression model. Figs. 19-21 show time series of the predicted and observed volume for 5-min, 10-min, and 15-min intervals whose correlation coefficients ρ’s are 0.850, 0.932 and 0.951, respectively. All forecasts are all one time interval ahead of occurrence, i.e., 5-min, 10-min and 15-min ahead of time. These three figures indicate that the larger the time interval, the better the performance of the prediction mode. To study the effects of the number K of the nearest states on the performance of the prediction model, a number of K’s are tested for different time intervals. Figs. 22-24 show the limiting behavior of the correlation coefficient ρ for the three time intervals. These three figures reveal that the larger the number K, the better the performance of the prediction mode, but after a certain number, the correlation coefficient ρ does not increase significantly. 5. Conclusions Numerical experiments have shown the effectiveness of the techniques introduced in this chapter to predict short-term chaotic time series. The dimension of the chaotic attractor in the delay plot increases with the dimension of the reconstructed state space and finally reaches an asymptote, which is fractal. A number of time delays have been tried to find the limiting dimension of the chaotic attractor, and the results are almost identical, which indicates the choice of time delay is not decisive, when the state space of the chaotic time series is being reconstructed. The effective number of neurons in the hidden layer of neural networks can be derived with the aid of the Bayesian regularization instead of using the trial and error. Chaotic Systems 20 00:00 03:00 06:00 09:00 12:00 15:00 18:00 21:00 00:00 Time (hr) 0 40 80 120 Traffic Volume (Veh.) ρ= 0.850 Observation Prediction Fig. 19. Time series of the predicted and observed 5-min traffic volumes. 00:00 03:00 06:00 09:00 12:00 15:00 18:00 21:00 00:00 Time (hr) 0 50 100 150 200 250 Traffic Volume (Veh.) ρ=0.932 Observation Prediction Fig. 20. Time series of the predicted and observed 10-min traffic volumes. Short-Term Chaotic Time Series Forecast 21 00:00 03:00 06:00 09:00 12:00 15:00 18:00 21:00 00:00 Time (hr) 0 100 200 300 400 Traf fic Volume (Veh.) ρ= 0.951 Observation Prediction Fig. 21. Time series of the predicted and observed 15-min traffic volumes. 0 200 400 600 800 1000 K 0.7 0.8 0.9 1 ρ Fig. 22. The limiting behavior of the correlation coefficient ρ with K increasing for the 5-min traffic volume. Chaotic Systems 22 0 200 400 600 K 0.7 0.8 0.9 1 ρ Fig. 23. The limiting behavior of the correlation coefficient ρ with K increasing for the 10- min traffic volume. 0 100 200 300 400 500 K 0.7 0.8 0.9 1 ρ Fig. 24. The limiting behavior of correlation coefficient ρ with K increasing for the 15-min traffic volume. Short-Term Chaotic Time Series Forecast 23 Using neurons in the hidden layer more than the number decided by the Bayesian regularization can indeed improve the performance of neural networks for the training set, but does not necessarily better the performance for the validation and prediction sets. Although disregarding the strategy “early stopping” can improve the network performance for the training set, it causes worse performance for the validation and prediction sets. Increasing the number of nearest states to fit the multiple linear regression forecast model can indeed enhance the performance of the prediction, but after the nearest states reach a certain number, the performance does not improve significantly. Numerical results from these two forecast models also show that the multiple linear regression is superior to neural networks, as far as the prediction accuracy is concerned. In addition, the longer the traffic volume scales are, the better the prediction of the traffic flow becomes. 6. References Addison, P. S. and Low, D. J. (1996). Order and Chaos in the Dynamics of Vehicle Platoons, Traffic Engineering and Control, July/August, pp. 456-459, ISSN 0041-0683. Albano, A. M., Passamante, A., Hediger, T. and Farrell, M. E. (1992). Using Neural Nets to Look for Chaos, Physica D, Vol. 58, pp. 1-9, ISSN 0167-2789. Alligood, K. T., Sauer, T. D., and Yorke, J. A. (1997). Chaos: An Introduction to Dynamical Systems , Springer-Verlag, ISBN 3-540-78036-x, New York. Aquirre, L. A. and Billings, S. A. (1994). Validating Identified Nonlinear Models with Chaotic Dynamics, International Journal of Bifurcation and Chaos in Applied Sciences and Engineering , Vol.4, No. 1, pp. 109-125, ISSN 0218-1274. Argoul, F., Arnedo, A., and Richetti, P. (1987). Experimental Evidence for Homoclinic Chaos in Belousov-Ehabotinski Reaction, Physics Letters, Section A, Vol. 120, No. 6, pp.269- 275, ISSN 0375-9601. Bakker, R., Schouten, J. C., Takens, F. and van den Bleek, C. M. (1996). Neural Network Model to Control an Experimental Chaotic Pendulum, Physical Review E, 54A, pp. 3545-3552, ISSN 1539-3755. Deco, G. and Schurmann, B. (1994). Neural Learning of Chaotic System Behavior, IEICE Transactions Fundamentals , Vol. E77-A, No. 11, pp.1840-1845, ISSN 0916-8508. Demuth, H., Beale, M., and Hagan, M. (2010). Neural Network Toolbox User’s Guide, The MathWorks, Inc., ISBN 0-9717321-0-8, Natick, Massachusetts. Dendrinos, D. S. (1994). Traffic-Flow Dynamics: A Search for Chaos, Chaos, Solitons, & Fractals , Vol. 4, No. 4, pp. 605-617, ISSN 0960-0779. Disbro, J. E. and Frame, M. (1989). Traffic Flow Theory and Chaotic Behavior, Transportation Research Record 1225, pp. 109-115. ISSN: 0361-1981 Farmer, J. D. and Sidorowich, J. J. (1987). Predicting Chaotic Time Series, Physical Review Letters , Vol. 59, pp. 845-848, ISSN 0031-9007. Fu, H., Xu, J. and Xu, L. (2005). Traffic Chaos and Its Prediction Based on a Nonlinear Car- Following Model, Journal of Control Theory and Applications, Vol. 3, No. 3, pp. 302- 307, ISSN 1672-6340. Gazis, D. C., Herman, R., and Rothery, R. W. (1961). Nonlinear Follow-The-Leader Models of Traffic Flow, Operations Research, Vol. 9, No. 4, pp. 545-567, ISSN 0030-364X. Glass, L., Guevau, X., and Shrier, A. (1983). Bifurcation and Chaos in Periodically Stimulated Cardiac Oscillator, Physica 7D, pp. 89-101, ISSN 0167-2789. [...]... 0.0103 0. 028 3 Rössler attractor h =2 h=6 h=7 0.0117 0.1889 0.3107 0.0144 0 .29 01 0.4890 0.0184 0 .21 14 0. 321 2 0. 024 0 0.3074 0.4301 0. 027 8 0.3650 0.5193 0.04 12 0.6380 0. 922 8 0.1178 1.8714 2. 7681 MSE1− NN MSE2− NN MSE3− NN MSE4− NN MSE5− NN MSE10− NN MSE20− NN 0.0156 0.0194 0. 022 8 0. 024 2 0. 029 5 0.0500 0. 124 7 0.0315 0.0384 0.0485 0.0560 0.0684 0.1306 0. 328 2 h = 10 0.8575 1.67 62 0.8501 1.33 32 1.1830 2. 2703 6.4980... 20 h=1 2. 6038e-6 1.6569e-6 1.5344e-6 2. 0762e-6 2. 6 426 e-6 4.4688e-6 6. 427 2e-6 Chua double scroll h =2 h=3 h=5 1. 124 7e-5 3 .29 35e-5 1.3694e-4 5.5148e-6 1.5541e-5 6.1758e-5 5. 625 7e-6 1.5912e-5 6.3038e-5 6. 922 8e-6 1.9519e-5 7.4392e-5 8.7472e-6 2. 3965e-5 8.7017e-5 1.7896e-5 5 .21 98e-5 1.9949e-4 2. 7342e-5 9.3183e-5 4.4513e-4 h = 10 0.00 12 5.5566e-4 6.1618e-4 6.7 625 e-4 6. 624 4e-4 0.0014 0.00 42 MSE1− NN MSE2− NN... 0.03 62 [-0.3,-0.1] 1 116 [-0.1,0.1] 0.9918 24 1 0 2 0. 021 8 0.0074 0. 021 8 0.00 72 0. 028 5 0. 024 4 0. 024 1 [0.1,0.3] 0.9917 120 1 0.0097 0.0097 0.0 124 0. 022 8 [0.3,0.5] 0.9884 171 2 0.0199 0.0199 0.0183 0. 022 1 [0.5,0.7] 0.9740 150 4 0.0553 0.0508 0 .22 61 0. 023 9 [0.7,0.9] 0.8750 7 1 0.1333 0.0 721 0.5619 0.0981 [0.9,1.1] 1 2 0 0.0884 0.0884 - 0.0 025 [1.1,1.3] 1 2 0 0 .24 40 0 .24 40 - 0.0091 ˆ Table 4 Each row relates... our 36 Chaotic Systems k=1 k =2 k=3 k=4 k=5 k = 10 k = 20 h=1 0.0039 0.0 024 0.0 020 0.0014 0.0014 0.0016 0.0 021 Lorenz attractor h =2 h=7 h=8 0.0176 0.6 821 1.0306 0.01 02 0.4151 0. 624 9 0.0081 0.3387 0.5103 0.0057 0 .28 73 0.4347 0.0061 0.3179 0.48 52 0.0071 0.3374 05 124 0.0101 0.4 322 0.6474 h = 10 1.9406 1 .24 29 0.9955 0.8803 0.9 724 1.0 329 1 .23 33 MSE1− NN MSE2− NN MSE3− NN MSE4− NN MSE5− NN MSE10− NN MSE20− NN... 0.0 129 0. 020 7 0. 024 6 0. 022 6 0. 022 0 0. 023 1 0. 024 3 0.0349 0.05 62 0.8730 0.7318 0.7181 0.7133 0.7 123 0.8136 1.0 423 0.3485 0 .29 94 0 .29 51 0 .29 74 0 .29 91 0.3775 0.5397 0.4877 0.41 52 0.4087 0.4096 0.4104 0.5001 0.6893 Table 6 The top and bottom half of the table display MSE c and MSE NN as a function of the number of neighbors k and the prediction horizon, h, respectively k=1 k =2 k=3 k=4 k=5 k = 10 k = 20 h=1... 0. 024 2 0. 029 5 0.0500 0. 124 7 0.0315 0.0384 0.0485 0.0560 0.0684 0.1306 0. 328 2 h = 10 0.8575 1.67 62 0.8501 1.33 32 1.1830 2. 2703 6.4980 0 .23 92 0 .22 29 0 .27 84 0.3513 0. 422 4 0.9188 2. 2649 35 0.7 928 0.6318 0.7410 0.9 528 1.1133 2. 3539 5.4714 0.34 02 0.3017 0.3710 0.4711 0.5 623 1 .22 28 2. 9775 Table 5 The top and bottom half of the table display MSE c and MSE NN as a function of the number of neighbors k and the prediction... MSE4− NN MSE5− NN MSE10− NN MSE20− NN 5.1 729 e-6 4.3 528 e-6 5.9985e-6 8.6114e-6 1.1190e-5 1.7453e-5 5.5861e-5 8.7554e-6 7.9 723 e-6 1.1757e-5 1.7168e-5 2. 320 1e-5 4.5731e-5 1.6005e-4 4.8311e-4 4.3 521 e-4 4. 728 3e-4 5.6469e-4 6.7550e-4 0.0014 0.00 42 1.6178e-5 1.5174e-5 2. 2003e-5 3.1539e-5 4 .26 47e-5 9.4048e-5 3.3975e-4 5 .27 20e-5 4. 927 6e-5 6.4616e-5 8.6965e-5 1.1362e-4 2. 6532e-4 9. 420 8e-4 Table 7 The top and bottom... 98 2 0.0056 0.0054 0.014 0.0098 [-0.1,0.1] 0.93 108 8 0.0038 0.0039 0.0 026 0.00 52 [0.1,0.3] 0.94 109 7 0.0041 0.0036 0.011 0.0077 [0.3,0.5] 0.96 195 [0.5,0.7] 0.91 22 3 8 0.0 021 22 0.0044 0.0 020 0.0038 0.0049 0.01 02 0.0088 0.0079 [0.7,0.9] 0.90 18 2 0.0011 0.0008 0.0033 0.00 12 [0.9,1.1] 0.81 13 3 0.0006 0.0003 0.0016 0.0016 [1.1,1.3] 0. 82 9 2 0.0034 0.0031 0.0047 0.0 027 [1.3,1.5] 0.80 4 1 0.0 42 0.0 52. .. testing chaotic dynamics via Lyapunov exponents, Physica D, 89: 26 1 -26 6 [11] Guégan, D (20 03) Les Chaos en Finance: Approche Statistique, Economica Eds., Paris [ 12] Guégan, D & Leroux, J (20 09b) Forecasting chaotic systems: The role of local Lyapunov exponents, Chaos, Solitons and Fractals, 41: 24 01 -24 04 [13] Guégan, D & Leroux, J (20 09a) Local Lyapunov Exponents: A New Way to Predict Chaotic Systems, ... Prediction results n =1,000 predictions Rössler Lorenz Chua MSE c 0.0053 0.0039 2. 6038e-6 MSE NN MSE b 0.0156 0.00 52 0.0091 0.0037 5.1 729 e-6 2. 4947e-6 ρ 97.3% 94.30% 98.7% ˆ λt mean 0.13 02 0.1940 0.0593 (min;max) (-1 .24 53,0.9198) (-1.4353;1.4580) (-1.0593;1.1468) ˆ λt| f ail mean 0 .25 82 0.4354 0. 325 3 (min;max) (-0.4 824 ,09198) (-0.51 42; 1.3639) (-0.5648;0.5554) Table 1 MSE c , MSE NN and MSE b are as defined . 0.0 021 0.0 020 0.0049 0.0088 [0.5,0.7] 0.91 22 3 22 0.0044 0.0038 0.01 02 0.0079 [0.7,0.9] 0.90 18 2 0.0011 0.0008 0.0033 0.00 12 [0.9,1.1] 0.81 13 3 0.0006 0.0003 0.0016 0.0016 [1.1,1.3] 0. 82 9 2. 0.0054 [0.3,0.5] 0.97 22 2 8 0.0059 0.0056 0.01 32 0.0101 [0.5,0.7] 0.97 1 92 6 0.0051 0.00 52 0.0009 0.0 127 [0.7,0.9] - - - - - - - [0.9,1.1] 0 0 1 9.34e-10 - 9.34e-10 2. 79e-11 Table 2. Each row relates. 0 20 0 400 600 800 1000 K 0.7 0.8 0.9 1 ρ Fig. 22 . The limiting behavior of the correlation coefficient ρ with K increasing for the 5-min traffic volume. Chaotic Systems 22 0 20 0 400