Environmental and Hydrological Systems Modelling Tai Lieu Chat Luong A W Jayawardena Environmental and Hydrological Systems Modelling Environmental and Hydrological Systems Modelling A W Jayawardena Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Group, an informa business MATLAB® is a trademark of The MathWorks, Inc and is used with permission The MathWorks does not warrant the accuracy of the text or exercises in this book This book’s use or discussion of MATLAB® software or related products does not constitute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2014 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S Government works Version Date: 20131216 International Standard Book Number-13: 978-0-203-92744-1 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint Except as permitted under U.S Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers For permission to photocopy or use material electronically from this work, please access www.copyright.com (http:// www.copyright.com/) or contact the Copyright Clearance Center, Inc (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400 CCC is a not-for-profit organization that provides licenses and registration for a variety of users For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com Contents Preface Author xvii xix 1 Introduction 1 1.1 Some definitions 1.1.1 System 1 1.1.2 State of a system 1.2 General systems theory (GST) 1.3 Ecological systems (Ecosystems) 1.4 Equi-finality 4 1.5 Scope and layout References 7 Historical development of hydrological modelling 9 2.1 2.2 2.3 Basic concepts and governing equation of linear systems 2.1.1 Time domain analysis 2.1.1.1 Types of input functions 10 2.1.1.2 System response function – convolution integral 12 2.1.2 Frequency domain analysis 12 2.1.2.1 Fourier transform – frequency response function (FRF) 12 2.1.2.2 Laplace transform 14 2.1.2.3 z-Transform 15 Linear systems in hydrological modelling 16 2.2.1 Hydrological systems 16 2.2.2 Unit hydrograph 17 2.2.2.1 Unit hydrograph for a complex storm 18 2.2.2.2 Instantaneous unit hydrograph (IUH) 20 2.2.2.3 Empirical unit hydrograph 20 2.2.2.4 Unit pulse response function 21 2.2.3 Linear reservoir 21 2.2.4 Linear cascade 23 2.2.5 Linear channel 25 2.2.6 Time–area diagram 26 Random processes and linear systems 27 © 2010 Taylor & Francis Group, LLC v vi Contents 2.4 Non-linear systems 29 2.4.1 Determination of the kernel functions 29 2.5 Multilinear or parallel systems 31 2.6 Flood routing 31 2.6.1 Inventory method 31 2.6.2 Muskingum method 32 2.6.2.1 Estimation of the routing parameters K and c 33 2.6.2.2 Limitations of the Muskingum method 35 2.6.3 Modified Puls method 35 2.6.4 Muskingum–Cunge method 35 2.6.5 Hydraulic approach 37 2.6.5.1 Solution of the St Venant equations 37 2.6.5.2 Diffusion wave approximation 38 2.6.5.3 Kinematic wave approximation 38 2.7 Reservoir routing 41 2.8 Rainfall–runoff modelling 43 2.8.1 Conceptual-type hydrologic models 44 2.8.1.1 Stanford watershed model (SWM) 44 2.8.1.2 Tank model 44 2.8.1.3 HEC series 45 2.8.1.4 Xinanjiang model 47 2.8.1.5 Variable infiltration capacity (VIC) model 49 2.8.2 Physics-based hydrologic models 51 2.8.2.1 Système Hydrologique Europèen (SHE) model 51 2.8.3 Data-driven models 52 2.8.3.1 Why data-driven models? 53 2.8.3.2 Types of data-driven models 53 2.9 Guiding principles and criteria for choosing a model 53 2.10 Challenges in hydrological modelling 54 2.11 Concluding remarks 56 References 56 Population dynamics 61 3.1 3.2 3.3 3.4 3.5 3.6 3.7 Introduction 61 Malthusian growth model 61 Verhulst growth model 63 Predator–prey (Lotka–Volterra) model 64 Gompertz curve 65 Logistic map 66 3.6.1 Specific points in the logistic map 67 Cell growth 68 3.7.1 Cell division 69 3.7.2 Exponential growth 70 3.7.3 Cell growth models in a batch (closed system) bioreactor 70 © 2010 Taylor & Francis Group, LLC Contents vii 3.8 Bacterial growth 72 3.8.1 Binary fission 73 3.8.2 Monod kinetics 73 3.9 Radioactive decay and carbon dating 74 3.10 Concluding remarks 75 References 76 Reaction kinetics 77 4.1 Introduction 77 4.2 Michaelis–Menten equation 78 4.3 Monod equation 81 4.4 Concluding remarks 84 References 84 Water quality systems 85 5.1 Dissolved oxygen systems 85 5.1.1 Biochemical oxygen demand (BOD) 85 5.1.2 Nitrification 88 5.1.3 Denitrification 88 5.1.4 Oxygen depletion equation in a river due to a single point source of BOD 89 5.1.5 Reoxygenation coefficient 92 5.1.6 Deoxygenation coefficient 94 5.2 Water quality in a completely mixed water body 94 5.2.1 Governing equations for a completely mixed system 95 5.2.2 Step function input 96 5.2.3 Periodic input function 97 5.2.4 Fourier series input 98 5.2.5 General harmonic response 99 5.2.6 Impulse input 101 5.2.7 Arbitrary input 101 5.3 Water quality in rivers and streams 106 5.3.1 Point sources 106 5.3.2 Distributed sources 108 5.3.3 Effect of spatial flow variation 109 5.3.3.1 Exponential spatial flow variation 110 5.3.4 Unsteady state 111 5.3.4.1 Non-dispersive systems 111 5.3.4.2 Dispersive systems 111 5.3.5 Tidal reaches 113 5.3.5.1 Special case of no decay 113 5.3.5.2 Special case of no dispersion 114 5.4 Concluding remarks 114 References 114 © 2010 Taylor & Francis Group, LLC viii Contents Longitudinal dispersion 117 6.1 Introduction 117 6.2 Governing equations 117 6.2.1 Some characteristics of turbulent diffusion 118 6.2.2 Shear flow dispersion 119 6.2.3 Taylor’s approximation 120 6.2.4 Turbulent mixing coefficients 120 6.3 Dispersion coefficient 121 6.3.1 Routing method 123 6.3.2 Time scale – dimensionless time 124 6.4 Numerical solution 126 6.4.1 Finite difference method 127 6.4.2 Finite element methods 128 6.4.3 Moving finite elements 130 6.5 Dispersion through porous media 131 6.6 General-purpose water quality models 134 6.6.1 Enhanced Stream Water Quality Model (QUAL2E) 134 6.6.2 Water Quality Analysis Simulation Programme (WASP) 135 6.6.3 One Dimensional Riverine Hydrodynamic and Water Quality Model (EPD-RIV1) 135 6.7 Concluding remarks 136 References 136 Time series analysis and forecasting 139 7.1 Introduction 139 7.2 Basic properties of a time series 139 7.2.1 Stationarity 139 7.2.2 Ergodicity 140 7.2.3 Homogeneity 140 7.3 Statistical parameters of a time series 140 7.3.1 Sample moments 140 7.3.2 Moving averages – low-pass filtering 141 7.3.3 Differencing – high-pass filtering 142 7.3.4 Recursive means and variances 142 7.4 Tests for stationarity 143 7.5 Tests for homogeneity 144 7.5.1 von Neumann ratio 145 7.5.2 Cumulative deviations 145 7.5.3 Bayesian statistics 148 7.5.4 Ratio test 148 7.5.5 Pettit test 150 7.6 Components of a time series 151 7.7 Trend analysis 151 7.7.1 Tests for randomness and trend 151 7.7.1.1 Turning point test for randomness 152 © 2010 Taylor & Francis Group, LLC Contents ix 7.7.1.2 Kendall’s rank correlation test (τ test) 153 7.7.1.3 Regression test for linear trend 154 7.7.1.4 Mann–Kendall test 155 7.7.2 Trend removal 156 7.7.2.1 Splines 157 7.8 Periodicity 159 7.8.1 Harmonic analysis – cumulative periodogram 159 7.8.2 Autocorrelation analysis 164 7.8.3 Spectral analysis 167 7.8.3.1 Hanning method (after J von Hann) 171 7.8.3.2 Hamming method (after R.W Hamming, 1983) 171 7.8.3.3 Lag window method (after Tukey, 1965) 172 7.8.4 Cross correlation 173 7.8.5 Cross-spectral density function 173 7.9 Stochastic component 174 7.9.1 Autoregressive (AR) models 175 7.9.1.1 Properties of autoregressive models 175 7.9.1.2 Estimation of parameters 176 7.9.1.3 First-order model (lag-one Markov model) 177 7.9.1.4 Second-order model (lag-two model) 179 7.9.1.5 Partial autocorrelation function (PAF) 180 7.9.2 Moving average (MA) models 181 7.9.2.1 Properties of MA models 182 7.9.2.2 Parameters of MA models 182 7.9.2.3 MA(1) model 183 7.9.2.4 MA(2) model 184 7.9.3 Autoregressive moving average (ARMA) models 185 7.9.3.1 Properties of ARMA(p,q) models 185 7.9.3.2 ARMA(1,1) model 185 7.9.4 Backshift operator 186 7.9.5 Difference operator 187 7.9.6 Autoregressive integrated moving average (ARIMA) models 187 7.10 Residual series 188 7.10.1 Test of independence 188 7.10.2 Test of normality 188 7.10.3 Other distributions 189 7.10.4 Test for parsimony 190 7.10.4.1 Akaike information criterion (AIC) and Bayesian information criterion (BIC) 190 7.10.4.2 Schwartz Bayesian criterion (SBC) 190 7.11 Forecasting 191 7.11.1 Minimum mean square error type difference equation 191 7.11.2 Confidence limits 193 7.11.3 Forecast errors 193 7.11.4 Numerical examples of forecasting 193 © 2010 Taylor & Francis Group, LLC x Contents 7.12 Synthetic data generation 196 7.13 ARMAX modelling 197 7.14 Kalman filtering 198 7.15 Parameter estimation 202 7.16 Applications 204 7.17 Concluding remarks 204 Appendix 7.1: Fourier series representation of a periodic function 205 References 207 Artificial neural networks 211 8.1 Introduction 211 8.2 Origin of artificial neural networks 212 8.2.1 Biological neuron 212 8.2.2 Artificial neuron 212 8.2.2.1 Bias/threshold 213 8.3 Unconstrained optimization techniques 215 8.3.1 Method of steepest descent 215 8.3.2 Newton’s method (quadratic approximation) 216 8.3.3 Gauss–Newton method 216 8.3.4 LMS algorithm 217 8.4 Perceptron 218 8.4.1 Linear separability 219 8.4.2 ‘AND’, ‘OR’, and ‘XOR’ operations 220 8.4.3 Multilayer perceptron (MLP) 221 8.4.4 Optimal structure of an MLP 222 8.5 Types of activation functions 223 8.5.1 Linear activation function (unbounded) 223 8.5.2 Saturating activation function (bounded) 223 8.5.3 Symmetric saturating activation function (bounded) 228 8.5.4 Positive linear activation function 228 8.5.5 Hardlimiter (Heaviside function; McCulloch– Pitts model) activation function 229 8.5.6 Symmetric hardlimiter activation function 229 8.5.7 Signum function 229 8.5.8 Triangular activation function 229 8.5.9 Sigmoid logistic activation function 229 8.5.10 Sigmoid hyperbolic tangent function 230 8.5.11 Radial basis functions 230 8.5.11.1 Multiquadratic 230 8.5.11.2 Inverse multiquadratic 231 8.5.11.3 Gaussian 231 8.5.11.4 Polyharmonic spline function 231 8.5.11.5 Thin plate spline function 231 8.5.12 Softmax activation function 231 8.6 Types of artificial neural networks 232 © 2010 Taylor & Francis Group, LLC Fuzzy logic systems 483 After the estimation of the , bi , ci values by the least squares method, the rule set can be written as IF upstream discharge q1 is low AND upstream discharge q2 is low THEN downstream discharge Q = 1.38q1 + 0.47q2 + 7.77 IF upstream discharge q1 is low AND upstream discharge q2 is medium THEN downstream discharge Q = 2.34q1 + 0.70q2 − 137.23 IF upstream discharge q1 is low AND upstream discharge q2 is high THEN downstream discharge Q = 7.20q1 − 0.70q2 − 715.27 IF upstream discharge q1 is medium AND upstream discharge q2 is low THEN downstream discharge Q = −0.40q1 + 2.35q2 + 318.07 IF upstream discharge q1 is medium AND upstream discharge q2 is medium THEN downstream discharge Q5 = 1.02q1 + 1.26q2 + 117.67 IF upstream discharge q1 is medium AND upstream discharge q2 is high THEN downstream discharge Q = 2.20q1 + 1.02q2 + 1021.6 IF upstream discharge q1 is high AND upstream discharge q2 is low THEN downstream discharge Q = 0.92q1 + 0.76q2 + 356.45 IF upstream discharge q1 is high AND upstream discharge q2 is medium THEN downstream discharge Q = 1.05q1 + 0.83q2 − 1061.31 IF upstream discharge q1 is high AND upstream discharge q2 is high THEN downstream discharge Q = 1.18q1 + 0.88q2 + 196.34 For the product operator, ‘OR’, a different set of parameters is obtained They are as follows: IF upstream discharge q1 is low AND upstream discharge q2 is low THEN downstream discharge Q = 1.42q1 + 0.46q2 + 6.82 IF upstream discharge q1 is low AND upstream discharge q2 is medium THEN downstream discharge Q = 2.06q1 + 0.86q2 − 41.09 IF upstream discharge q1 is low AND upstream discharge q2 is high THEN downstream discharge Q = 17.68q1 − 3.29q2 + 3473.61 IF upstream discharge q1 is medium AND upstream discharge q2 is low THEN downstream discharge Q = −0.43q1 + 2.47q2 + 286.25 IF upstream discharge q1 is medium AND upstream discharge q2 is medium THEN downstream discharge Q5 = 1.14q1 + 1.16q2 + 66.4 IF upstream discharge q1 is medium AND upstream discharge q2 is high THEN downstream discharge Q = 3.02q1 + 0.72q2 + 1109.63 IF upstream discharge q1 is high AND upstream discharge q2 is low THEN downstream discharge Q = 2.81q1 − 4.07q2 + 1497.64 IF upstream discharge q1 is high AND upstream discharge q2 is medium THEN downstream discharge Q = 1.77q1 + 0.01q2 + 940.28 IF upstream discharge q1 is high AND upstream discharge q2 is high THEN downstream discharge Q = 1.22q1 + 0.84q2 + 149.33 Table 13.6 Performance indicators with three clustering centres Minimum Performance indicator MAE RMSE RRMSE EF CD Product Calibration Verification Calibration 84.176 172.711 0.436 0.931 0.964 84.495 135.945 0.4222 0.9468 0.9602 84.882 176.292 0.445 0.928 0.957 © 2010 Taylor & Francis Group, LLC Verification Optimum value 84.793 136.288 0.423 0.947 0.968 0 1.0 1.0 484 Environmental and hydrological systems modelling Table 13.7 Performance indicators with five clustering centres Min and max Performance indicator Calibration Verification Calibration 75.113 148.517 0.375 0.949 1.054 74.579 133.085 0.413 0.949 1.032 74.961 148.028 0.374 0.949 1.053 MAE RMSE RRMSE EF CD 1979 observed clustering product clustering product and probor 3000 Discharge (m3/s) Product and probability Verification Optimum value 73.664 128.143 0.398 0.953 1.031 0 1.0 1.0 clustering clustering and max 2500 2000 1500 1000 500 1/1/79 2/20/79 4/11/79 5/31/79 7/20/79 Time (days) 9/8/79 10/28/79 12/17/79 Figure 13.34 Time series plot of observed and predicted discharges for the year 1979 y = 0.9552x – 56.179 R2 = 0.9155 2500 discharge (m3/s) 1979 verification observed 3000 2000 1500 1000 500 0 500 1000 1500 2000 1979 calculated discharge from clustering 2500 product (m3/s) Figure 13.35 Scatter plot of observed and predicted discharges for the year 1979 using three cluster centres © 2010 Taylor & Francis Group, LLC Fuzzy logic systems 485 In this example, the partitioning was done by clustering the data The peak points in the triangular membership functions correspond to the centres of the clusters It can be seen that the accuracy of prediction increases as the number of clusters increase (Tables 13.6 and 13.7; Figures 13.34 and 13.35) Neural networks have been used in the design of membership functions of fuzzy systems The normal practice of designing membership functions in fuzzy systems is by using expert knowledge and/or a priori information about the system to design and tune the membership functions This approach is time consuming By using neural network learning capabilities, the process can be automated, thereby reducing time and cost and improving performance The idea of using neural networks to design the membership functions and the use of the gradient descent method for tuning the parameters that define the shape of the membership functions is equivalent to learning in a feed-forward neural network (Takagi and Hayashi, 1991) Many Japanese and Korean companies use fuzzy neural systems in the operation of equipment such as photocopying machines (e.g., Sanyo, Ricoh) (Morita et al., 1992), electric fans with remote controllers (Sanyo, 1991; Nikkei Electronics, 1991), washing machines (e.g., Hitachi, Toshiba, Sanyo) (Narita et al., 1991), and many other household electrical appliances 13.9 CONCLUDING REMARKS In this chapter, an attempt has been made to introduce the basic concepts of fuzzy logic systems in the context of systems approach of modelling Although by no means comprehensive, it is hoped that the material presented would be helpful for using the fuzzy logic approach for hydrological and environmental systems modelling REFERENCES Abonyi, J (2002): Modified Gath–Geva fuzzy clustering for identification of Takagi–Sugeno fuzzy models, systems, man and cybernetics, Part B: Cybernetics IEEE Transactions, 32(5), 612–621 Alata, M., Molhim, M and Ramini, A (2008): Optimizing of fuzzy c-means clustering algorithm using GA World Academy of Science, Engineering and Technology, 39, 224–229 Alvisi, S., Mascellani, G., Franchini, M and Bárdossy, A (2005): Water Level Forecasting Through Fuzzy Logic and Artificial Neural Network Approaches Published in Hydrology and Earth System Science Discussions, EGU Amarasekara, J.D (2012): Development of a flood forecasting model for the Gin River and Kelani River basins in Sri Lanka using fuzzy logic approach Thesis submitted in partial fulfilment for the Master’s degree in disaster management, National Graduate Institute for Policy Studies (GRIPS) and International Centre for Water Hazard and Risk Management (ICHARM) under the auspices of UNESCO (unpublished) Bárdossy, A and Disse, M (1993): Fuzzy rule-base models for infiltration Water Resources Research, 29(2), 373–382 Bárdossy, A and Duckstein, L (1995): Fuzzy Rule-Based Modelling with Applications to Geophysical, Biological and Engineering Systems CRC Press, Boca Raton, FL Berenji, H.R (1992): A reinforcement learning-based architecture for fuzzy logic control International Journal of Approximate Reasoning, 6, 267–292 Berenji, H.R and Khedkar, P (1992): Learning and tuning fuzzy logic controllers through reinforcements IEEE Transactions on Neural Networks, 3, 724–740 Bezdek, J.C (1981): Pattern Recognition with Fuzzy Objective Function Algorithms Plenum Press, New York © 2010 Taylor & Francis Group, LLC 486 Environmental and hydrological systems modelling Buckley, J.J and Hayashi, Y (1994a): Fuzzy neural networks: A survey Fuzzy Sets and Systems, 66(1), 1–13 Buckley, J.J and Hayashi, Y (1994b): Fuzzy neural networks In: Zadeh, L.A and Yager, R.R (Eds.), Fuzzy Sets, Neural Networks and Soft Computing Van Nostrand Reinhold, New York, pp. 233–249 Dunn, J.C (1973): A fuzzy relative of the ISODATA process and its use in detecting compact wellseparated clusters Journal of Cybernetics, 3, 32–57 Eklund, P., Fogström, M and Forsström, J (1992): A generic neuro-fuzzy tool for developing medical decision support In: Eklund, P (Ed.), Proceedings EPP92, International Seminar on Fuzzy Control through Neural Interpretations of Fuzzy Sets Åbo Akademis tryckeri, Åbo, pp. 1–27 Eklund, P., Virtanen, H and Riisssanen, T (1991): On the fuzzy logic nature of neural nets Proceedings of Neuro-Nimes, pp 293–300 Fullér, R (1995): Neural Fuzzy Systems, Åbo Akademis tryckeri, Åbo, 249 pp http://users.abo.fi/rfuller/ nfs.html Gath, I and Geva, A.B (1989): Unsupervised optimal fuzzy clustering IEEE Transactions on Pattern Analysis and Machine Intelligence, 7, 773–781 Gustafson, E.E and Kessel, W.C (1979): Fuzzy clustering with a fuzzy covariance matrix Proc IEEE CDC, San Diego, CA, 1979, pp 761–766 Höppner, F., Klawonn, R., Kruse, T and Runkler, T (1999): Fuzzy Cluster Analysis John Wiley, Chichester, UK Hundecha, Y., Bárdossy, A and Theisen, H (2001): Development of a fuzzy logic-based rainfall–runoff model Hydrological Sciences Journal, 46(3), 363–376 Ishikawa, M (1996): Structural learning with forgetting Neural Networks, 9, 501–521 Jang, J.S.R (1993): ANFIS: Adaptive-network-based fuzzy inference systems IEEE Transactions on Systems, Man and Cybernetics, 23(1993), 665–685 Jang, J.S.R., Sun, C.T and Mizutani, E (1997): Neuro-fuzzy and soft computing In: MATLAB Curriculum Series Prentice Hall, Upper Saddle River, NJ ISBN No 0-13-261066-3 Kasabov, N (1998): ECOS: A framework for evolving connectionist systems and the ECO learning paradigm Proc of ICONIP’98, Kitakyushu, Japan, IOS Press, vol 3, pp 1232–1235 Kasabov, N., Kim, J.S., Watts, M and Gray, A (1997): FuNN/2 – A fuzzy neural network architecture for adaptive learning and knowledge acquisition, information sciences applications Kwan, H.K and Cai, Y (1994): A fuzzy neural network and its application to pattern recognition IEEE Transactions on Fuzzy Systems, 2(3), 185–193 Larsen, H.L and Yager, R.R (2000): A framework for fuzzy recognition technology, systems, man and cybernetics, Part C: Applications and reviews IEEE Transactions, 30(1), 65–76 Mamdani, E.H and Assilian, S (1975): An experiment in linguistic synthesis with a fuzzy logic controller International Journal of Man Machine Studies, 7(1), 1–13 MathWorks Inc.: MATLAB User manual for MATLAB Fuzzy Logic Tool Box Morita, T., Kanaya, M and Inagaki, T (1992): Photo-copier image density control using neural network and fuzzy theory Proceedings of the Second International Workshop on Industrial Fuzzy Control and Intelligent Systems, pp 10–16 Narita, R., Tatsumi, H and Kanou, H (1991): Application of neural networks to household applications Toshiba Review, 46, 935–938 (in Japanese) Nauck, D (1994): A fuzzy perceptron as a generic model for neuro-fuzzy approaches In: Proceedings 2nd German GI-Workshop Fuzzy-Systems ’94, München Nauck, D and Kruse, R (1996): Neuro-fuzzy classification with NEFCLASS In: Kleinschmidt, P., Bachem, A., Derigs, U., Fischer, D., Leopold-Wildburger, U and Möhring, R (Eds.), Operations Research Proceedings, Berlin, 1995, pp 294–299 Nauck, D and Kruse, R (1997): Function approximation by NEFPROX Proc Second European Workshop on Fuzzy Decision Analysis and Neural Networks for Management, Planning, and Optimization (EFDAN’97), Dortmund, pp 160–169 Nayak, P.C., Sudheer, K.P., Rangan, D.M and Ramasastri, K.S (2004): A neuro-fuzzy computing technique for modelling hydrological time series Journal of Hydrology, 291, 52–66 © 2010 Taylor & Francis Group, LLC Fuzzy logic systems 487 Nayak, P.C., Sudheer, K.P., Rangan, D.M and Ramasastri, K.S (2005): Short-term flood forecasting with a neurofuzzy model Water Resources Research, 41, W04004 Negnevitsky, M (2005): Artificial Intelligence: A Guide to Intelligent Systems, Revision/New Edition Addison-Wesley, Harlow, England, 407 pp ISBN 0-321-20466-2 Nikkei Electronics (1991): New trend in consumer electronics: Combining neural networks and fuzzy logic Nikkei Electronics, 528, 165–169 (in Japanese) Pedrycz, W (2005): Knowledge-Based Clustering: From Data to Information Granules John Wiley & Sons, Chichester, UK, 336 pp Sanyo (1991): Electric fan series in 1991, Sanyo News Release (March 14, 1991) Sanyo (in Japanese) See, L and Openshaw, S (1999): Applying soft computing approaches to river level forecasting Hydrological Sciences Journal, 44(5), 763–779 See, L and Openshaw, S (2000): A hybrid multi-model approach to river level forecasting Hydrological Sciences Journal, 45(4), 523–535 Sen, Z (1998): Fuzzy algorithm for estimation of solar irrigation from sunshine duration Solar Energy, 63, 39–49 Sen, Z (2009): Fuzzy Logic and Hydrological Modelling CRC Press, Boca Raton, FL Takagi, H and Hayashi, I (1991): NN-driven fuzzy reasoning International Journal of Approximate Reasoning, 3, 191–212 Takagi, T and Sugeno, M (1974): Application of fuzzy algorithms for control of simple dynamic plant IEE Proceedings, 12, 1585–1588 Takagi, T and Sugeno, M (1985): Fuzzy identification of systems and its applications to modelling and control IEEE Transactions on Systems, Man and Cybernetics, 15, 116–132 Tareghian, R and Kashefipour, S.M (2007): Application of fuzzy systems and artificial neural networks for flood forecasting Journal of Applied Sciences, 7, 3451–3459 Tsukamoto, Y (1979): An Approach to Fuzzy Reasoning Method, Advances in Fuzzy Set Theory and Applications North Holland, Amsterdam, pp 137–149 Tsukamoto, Y (1994): Some issues of reasoning in fuzzy control: Principle, practice and perspective International Conference on Tools with Artificial Intelligence, pp 192–196 Umano, M and Ezawa, Y (1991): Execution of approximate reasoning by neural network Proceedings of FAN Symposium, pp 267–273 (in Japanese) Xiong, L., Shamseldin, A.Y and O’Connor, K.M (2001): A non-linear combination of the forecasts of rainfall-runoff models by the first-order Takagi–Sugeno fuzzy system Journal of Hydrology, 245, 196–217 Zadeh, L.A (1965): Fuzzy sets Information and Control, 8, 338–353 Zhu, B (2012): Hydrological forecasting based on TSK fuzzy logic system in Fu River basin Thesis submitted in partial fulfilment for the Master’s degree in disaster management, National Graduate Institute for Policy Studies (GRIPS) and International Centre for Water Hazard and Risk Management (ICHARM) under the auspices of UNESCO (unpublished) © 2010 Taylor & Francis Group, LLC Chapter 14 Genetic algorithms (GAs) and genetic programming (GP) Genetic algorithms are search algorithms based on the mechanics of natural selection and natural genetics They combine survival of the fittest among string structures with a structured yet randomized information exchange to form a search algorithm David E Goldberg Genetic Algorithms in Search, Optimization and Machine Learning, 1989 14.1 INTRODUCTION Since the original works by Holland (1975, 1993), genetic algorithms (GAs), being a form of evolutionary computing, have found applications in many areas of science and engineering They are inspired by Darwin’s theory of evolution and imitate nature’s selection of fitter or stronger genes according to a mechanism dictating the survival of the fittest This adaptive nature lends GAs to be applied to problems that require progressive modifications such as in parameter optimization The main components of GA consists of • Genetic representation for potential solutions • A way to create an initial population • An evaluation function that plays the role of the environment rating solutions in terms of their fitness • A selection method to choose two reproductive solutions • Genetic operators that alter the composition of the offsprings • Estimation of GA parameters (population size, probability of applying genetic operators, etc.) GAs operate on a population (a set of possible solutions) on the basis of biological genetics and natural selection, and are designed to produce successive populations having an increasing number of individuals (solutions) with desirable characteristics They are designed in such a way that best individuals proliferate over generations The objective function corresponding to each individual determines its ‘fitness’ The selection procedure possesses a guided randomness rather than being entirely random, and thus leads the populations of the subsequent generations increasingly towards the optimum The ‘best so far’ individual or the best set of parameters that correspond to the minimum/maximum objective function is recorded over the generations GAs operate on a coding of the parameters, rather than on the parameters themselves Each parameter is encoded into a string of finite length made up of binary numbers These strings are then concatenated to form one string that is regarded as one individual (structure) Several such individuals constitute a population © 2010 Taylor & Francis Group, LLC 489 490 Environmental and hydrological systems modelling All living organisms consist of cells that have the same set of chromosomes They are strings of DNA that identify the organism A chromosome consists of genes, which are blocks of DNA A complete set of genetic material (all chromosomes) is called a genome The fitness of an organism is measured by the success of the organism in its life In GA terminology, a set of solutions (represented by chromosomes) is called a population Successive populations are generated in such a way that the new population is better than the old one The new solutions, selected according to their fitness, are referred to as the ‘offsprings’ This procedure is repeated until some stopping criterion is satisfied In solving a particular problem, the first attempt is to look for some particular solution from a set of possible solutions The space containing all feasible solutions is known as the search space Each point (multidimensional) in the search space is a feasible solution and the aim is to look for the best solution from the search space that may not necessarily be the optimal Methods of finding the optimal solution include ‘hill climbing’, ‘simulated annealing’, and ‘genetic algorithms’ GAs consist of the following steps: Generation of a random population of ‘n’ chromosomes (feasible solutions for the problem) Evaluation of the fitness function f(x) of each chromosome x in the population Generation of a new population by repeating the following: • Select two parent chromosomes from a population according to their fitness • With a crossover probability, cross over the parents to form a new offspring If no crossover is performed, the offspring will be an exact copy of the parents • With a mutation probability, mutate new offspring at each locus (position in chromosome) • Place a new offspring in a new population Use new generation for repeating the algorithm If the stopping criterion is satisfied, stop Go to step The questions that need clarification include how to create chromosomes, what type of encoding to use, and how to select parents for crossover The concept of elitism is used to ensure that at least one best solution is copied without changes to a new population so that the best solution can survive to the end of the run 14.2 CODING The equivalent of chromosomes in an optimization problem are the parameters of the model or function Any possible set of parameters is represented by a set of genetic structures that collectively correspond to a point (multidimensional) in the search space Each structure can be represented by a string of binary numbers A string of length L has L bit positions each of which is occupied by a or For an ‘n’ parameter optimization problem, there will be ‘n’ binary structures, one for each parameter For example, let SiL = {BL , BL−1 , , B3 , B2 , B1 }i give the bit value corressponding to bit positions {L, L − 1, … , 3, 2, 1} where SiL is the i-th structure of the string of length L and Bk is a binary number at the k-th bit position The string attains its maximum value of 2L − when all the bits are occupied © 2010 Taylor & Francis Group, LLC Genetic algorithms (GAs) and genetic programming (GP) 491 by 1’s and its minimum value is zero The length of the string depends on the required precision If the i-th parameter xi that corresponds to the i-th structure is known to be in the range {ai ≤ xi ≤ bi}, then the precision of the representation, or the smallest possible division (ai − bi ) of the parameter range is L As can be seen, the precision increases with increasing (2 − 1) string length The mapping from the binary string SiL to the real value xi is accomplished first by converting the binary string SiL from base to base 10 as L k−1 SiL = {BL , BL−1 , , B3, B2 , B1 }i = B = xi (14.1) k 2 k=1 10 ∑ and then finding the corresponding real number as xi = bi + xi (ai − bi ) (14.2) (2L − 1) The initial population is randomly generated by L binary numbers to fill up the positions in one structure and repeating it for all n structures At this stage, some prior information about the feasible range of values of the parameters could help accelerate the search process The selection process for reproduction is based on an objective function that may be a measure of error The structures can be arranged according to the objective function and those with better (higher or lower depending on the objective function) values have a higher chance of propagating the desired characteristics to the next generation The basis for this is the schema theory (Holland, 1993), which asserts that structures having a fitness above that of the average for the whole population tend to occur more frequently in the next generation (Holland, 1993) More details of the schema theory can be found in Goldberg (1989) The roulette wheel method in which each set of structures in the ranking is assigned a pie-shaped slice in proportion to its fitness, and selecting the structures at which the pointer stops after an arbitrary rotation, is a widely adopted method 14.3 GENETIC OPERATORS The basic genetic operators are copying, crossover, masking, and mutation These operations are illustrated in Figure 14.1 for an example in which the string length is 10 For clarity, the binary numbers in the two structures are shown in upper and lower cases Copying is the complete replacement of all the bit positions of the resulting structure by those of the original Single-point crossover refers to the exchange of corresponding bit values beyond a certain point on the string The crossover point is randomly decided In Figure 14.1, the crossover points are indicated by vertical lines Multiple-point crossovers refer to the cases when there are more than one crossover points, which are again decided randomly For masking, a masking string that has the same length as the original structures has to be randomly generated The masking string determines which of the original structures dominate a certain bit position of the resulting structure If the bit position of the masking string is 1, then the corresponding bit positions of the two resulting structures will be those of the first and second original structures, respectively If the bit position of masking string is 0, then they will be those of the second and first, respectively Mutation corresponds to the rare perturbation of a population’s biological characteristics to prevent evolutionary dead ends Mutation means that the elements of DNA are a bit changed Such © 2010 Taylor & Francis Group, LLC 492 Environmental and hydrological systems modelling A B C D E F G H I J A B C D E F G H I J a b c d e f g h i j a b c d e f g h i j Original structures Copying A B Cd e f g h i j A B|c d e f |G HI J a b c |D E F G H I J a b|C DE F|g h i j Single point crossover Multiple point crossover 1 1 1 Masking string A b C D e F G h I j A B C D E F' G H I J a B c d E f g Hi J a b c d e f g h i j Masking Mutation Figure 14.1 Genetic operators changes are mainly caused by errors in copying genes from parents In GAs, it is usually adopted to prevent stagnation of a suboptimal population This operator is used less frequently compared with the others In Figure 14.1 above, one of the bits (F) becomes F′ after mutation It is to be noted that becomes and becomes when mutation takes place 14.4 PARAMETERS OF GA The parameters of GAs (population size, probability of applying genetic operators, probability distributions to be applied during ranking, etc.) are problem dependent as with many other optimization techniques They are usually not known a priori and very often a trialand-error procedure is needed to determine them 14.5 GENETIC PROGRAMMING (GP) In the recent past, genetic programming (GP), an evolutionary algorithm (EA)-based model, has been used to emulate the rainfall–runoff process (Liong et al., 2002; Whigham and Crapper 2001; Savic et al., 1999; Jayawardena et al., 2005, 2006) as a viable alternative to traditional rainfall–runoff models GP has the advantage of providing inherent functional input–output relationships as compared with traditional black box models, and therefore can offer some possible interpretations of the underlying processes © 2010 Taylor & Francis Group, LLC Genetic algorithms (GAs) and genetic programming (GP) 493 GP is an automatic programming technique for evolving computer programs to solve, or approximately solve, problems (Koza, 1992) In engineering applications, GP is frequently applied to model structure identification problems In such applications, GP is used to infer the underlying structure of either a natural or experimental process in order to model the process numerically GP is a member of the EA family EAs are based on Darwin’s natural selection theory of evolution where a population is progressively improved by selectively discarding the not-sofit populations and breeding new offsprings from better populations EAs work by defining a goal in the form of a quality criterion and then use this goal to measure and compare solution candidates in a stepwise refinement of a set of data structures and return an optimal or near-optimal solution after a number of generations Evolutionary strategies (ES), GAs, and evolutionary programs (EP) are three early variations of evolutionary algorithms, whereas GP is perhaps the most recent variant of EAs These techniques have become extremely popular owing to their success at searching complex non-linear spaces and their robustness in practical applications The basic search strategy in GP is a genetic algorithm (Goldberg, 1989) GP differs from the traditional GA in that it typically operates on parse trees instead of bit strings A parse tree is built up from a terminal set (the variables in the problem) and a function set (the basic operators used to form the function) Figure 14.2 shows such an example for the expression −b + b2 − ac represented as a parse tree The functions are mathematical or logical 2a operators, and terminals are constants and variables As in any standard evolutionary algorithm, in order to optimize its fitness value during the evolving process, the trees in GP are dynamically modified by genetic operators The ‘tree size’ of this expression is 7, where tree size is the maximum ‘node depth’ of a tree and node depth is the minimum number of nodes that must be traversed to get from the ‘root node’ of the tree (Figure 14.2) to the selected node The problem of finding a function in symbolic form that fits a given set of data is called symbolic regression Genetic symbolic regression is a special application of GP in the field of symbolic regression The aim of symbolic regression is to determine a functional relationship between input and output data sets Symbolic regression is error-driven evolution and it may be linear, quadratic, or higher-order polynomial Details of GP can be found in Liong et al (2002) The function set may contain operators such as +, −, ×, /, *, power(x,y), exp(x), etc The ends of a parse tree are called ‘Terminals’, which are actually the input variables The nodes are functions taken from the ‘Function Set’ As a GA, GP proceeds by initially generating a population of random parse trees, calculates their fitness – a measure of how well they solve the given problem, and subsequently ÷ Root node − √ * b * b * b * a Figure 14.2 GP parse tree for the quadratic equation © 2010 Taylor & Francis Group, LLC a − c −b + b − 4ac 2a 494 Environmental and hydrological systems modelling selects the better parse trees for reproduction and variation to form a new population Crossover in GP is similar to that in GA Genetic material is exchanged between two parents to create two offsprings In GP, subtrees are exchanged between the two parents This process of selection, reproduction, and variation is iterated until some stopping criterion is satisfied GP has the unique feature that it does not assume any functional form of the solution, and that it can optimize both the structure of the model and its parameters Since GP evolves an equation relating the output and input variables, it has the advantage of providing inherent functional relationship explicitly over techniques such as ANNs This gives the GP approach the automatic ability to select input variables that contribute beneficially to the model and to disregard those that not For rainfall–runoff modelling, the mathematical relationship may be expressed as Qt+δΔt = f(Rt, Rt−Δt,…, Rt−ωΔt, Qt, Qt−Δt,…, Qt−ωΔt) (14.3) where Q is the runoff, R is the rainfall intensity, δ (with δ = 1, 2, …) refers to how far into the future the runoff prediction is desired, ω (with ω = 1, 2, …) implies how far back the recorded data in the time series are affecting the runoff prediction, while Δt stands for time step 14.6 APPLICATION AREAS GAs have found applications in several areas of science and engineering such as pattern recognition, signal processing, machine learning, time series prediction, and optimization In the hydrological field, the applications have been mainly for parameter optimization Examples include their use for the calibration of Xinanjiang model (Wang et al., 1991), storm water management model (SWMM) (Liong et al., 1995), ARNO model (Franchini, 1996), moisture and contaminant transport model (Jayawardena et al., 1997), and tank model (Fernando, 1997) 14.7 CONCLUDING REMARKS This short and last chapter gives a basic introduction to GAs and GP GAs aim to successively generate superior solutions starting from an initial solution The approach can be used in many areas of science and engineering One of the main application areas is for parameter optimization in complex models GP, which has many common features as in GAs, has the added advantage of providing inherent functional input–output relationships as compared with traditional black box models, and therefore can offer some possible interpretations of the underlying processes REFERENCES Fernando, D.A.K (1997): On the application of artificial neural networks and genetic algorithms in hydro-meteorological modelling PhD thesis, The University of Hong Kong Franchini, M (1996): Use of genetic algorithms combined with a local search method for automatic calibration of conceptual rainfall–runoff models Hydrological Sciences Journal, 41(1), 21–39 © 2010 Taylor & Francis Group, LLC Genetic algorithms (GAs) and genetic programming (GP) 495 Goldberg, D (1989): Genetic Algorithms in Search, Optimization and Machine Learning Addison Wesley, Reading, MA Holland, J.H (1975): Adaptation in Natural and Artificial Systems The University of Michigan Press, Ann Arbor, MI Holland, J.H (1993): Adaptation in Natural and Artificial Systems MIT Press, Cambridge, MA Jayawardena, A.W., Fernando, D.A.K and Dissanayake, P.B.G (1997): Genetic algorithm approach of parameter optimisation for the moisture and contaminant transport problem in partially saturated porous media Proceedings of the 27th Congress of the International Association for Hydraulic Research, vol 1, ASCE, San Francisco, August 10–15, pp 761–766 Jayawardena, A.W., Muttil, N and Fernando, T.M.K.G (2005): Rainfall–runoff modelling using genetic programming In Zerger, A and Argent, R.M (Eds.), MODSIM 2005 International Congress on Modelling and Simulation: Advances and Applications for Management and Decision Making, Modelling and Simulation Society of Australia and New Zealand, Melbourne, Australia, December 12–15, pp 1841–1847 ISBN 0-9758400-2-9 Jayawardena, A.W., Muttil, N and Lee, J.H.W (2006): Comparative analysis of a data-driven and GISbased conceptual rainfall–runoff model Journal of Hydrologic Engineering, ASCE, 11(1), 1–11 Koza, J (1992): Genetic Programming: On the Programming of Computers by Natural Selection MIT Press, Cambridge, MA Liong, S.Y., Chan, W.T and ShreeRam, J (1995): Peak flow forecasting with genetic algorithms and SWMM Journal of Hydraulic Engineering, ASCE, 121(8), 613–617 Liong, S.Y., Gautam, T.R., Khu, S.T., Babovic, V and Muttil, N (2002): Genetic programming: A new paradigm in rainfall–runoff modelling Journal of American Water Resources Association, 38(3), 705–718 Savic, D.A., Walters, G.A and Davidson, G.W (1999): A genetic programming approach to rainfall– runoff modeling Water Resources Management, 13, 219–231 Wang, Q.J (1991): The genetic algorithm and its application to calibrating conceptual rainfall–runoff models Water Resources Research, 27(9), 2467–2471 Whigham, P.A and Crapper, P.F (2001): Modelling rainfall–runoff relationships using genetic programming Mathematical and Computer Modelling, 33, 707–721 © 2010 Taylor & Francis Group, LLC EnvironmEntal EnginEEring Mathematical modelling has become an indispensable tool for engineers, scientists, planners, decision makers and many other professionals to make predictions of future scenarios as well as real impending events As the modelling approach and the model to be used are problem specific, no single model or approach can be used to solve all problems, and there are constraints in each situation Modellers therefore need to have a choice when confronted with constraints such as lack of sufficient data, resources, expertise and time Environmental and Hydrological Systems Modelling provides the tools needed by presenting different approaches to modelling the water environment over a range of spatial and temporal scales Their applications are shown with a series of case studies, taken mainly from the Asia-Pacific Region Coverage includes: • Linear Systems • Conceptual Models • Data Driven Models • Process-Based Models • Risk-Management Models • Model Parameter Estimation • Model Calibration, Validation and Testing This book will be of great value to advanced students, professionals, academics and researchers working in the water environment A W Jayawardena is an Adjunct Professor at The University of Hong Kong and Technical Advisor to Nippon Koei Company Ltd (Consulting Engineers), Japan RU54409 ISBN-13: 978-0-415-46532-8 90000 780415 465328