Modeling Hydrologic Change: Statistical Methods - Chapter 1 pdf

LEWIS PUBLISHERS A CRC Press Company Boca Raton London New York Washington, D.C. Statistical Methods Richard H. McCuen Department of Civil and Environmental Engineering University of Maryland Modeling Hydrologic Change Modeling Hydrologic Change © 2003 by CRC Press LLC This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, microfilming, and recording, or by any information storage or retrieval system, without prior permission in writing from the publisher. The consent of CRC Press LLC does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific permission must be obtained in writing from CRC Press LLC for such copying. Direct all inquiries to CRC Press LLC, 2000 N.W. Corporate Blvd., Boca Raton, Florida 33431. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation, without intent to infringe. Visit the CRC Press Web site at www.crcpress.com © 2003 by CRC CRC Press LLC Lewis Publishers is an imprint of CRC Press LLC No claim to original U.S. Government works International Standard Book Number 1-56670-600-9 Library of Congress Card Number 2002073063 Printed in the United States of America 1 2 3 4 5 6 7 8 9 0 Printed on acid-free paper Library of Congress Cataloging-in-Publication Data McCuen, Richard H., 1941 Modeling hydrologic change: statistical methods / Richard H. McCuen. p. cm. Includes bibliographical references and index. ISBN 1-56670-600-9 1. Hydrologic models. 2. Hydrologic—Statistical methods. I. Title. GB656.2.H9 M33 2002 551.48′01′1—dc21 2002073063 CIP Catalog record is available from the Library of Congress Preface Modeling Hydrologic Change: Statistical Methods is about modeling systems where change has affected data that will be used to calibrate and test models of the systems and where models will be used to forecast system responses after change occurs. The focus is not on the hydrology. Instead, hydrology serves as the discipline from which the applications are drawn to illustrate the principles of modeling and the detection of change. All four elements of the modeling process are discussed: conceptualization, formulation, calibration, and verification. Analysis and synthesis are discussed in order to enable both model builders and users to appreciate the importance of both aspects of modeling. The book also focuses on the art and science of modeling. While modeling techniques may be of great interest to hydrology-oriented pro- fessionals, they have value to all disciplines involved in modeling changes. While the book is oriented toward the statistical aspects of modeling, a strong background in statistics is not required. Although the emphasis is on the analysis of temporal and spatial sequences of data, the fundamentals that comprise most of the book are far more applicable. Statistical and modeling methods can be applied to a broad array of problems. This book is not appropriate as a general text for an undergraduate introductory course in probability and statistics. It is intended for advanced under - graduates, graduate students, and practicing professionals. It includes topics that serve as background material for its central focus and topics related to the graphical and statistical detection of change and the fundamen - tals of modeling. While Chapters 2, 3, and 5 can be considered foundational, other chapters also introduce basic concepts. Chapters 4 and 6 through 9 are devoted to important graphical and statistical procedures used in modeling. Chapters 10 through 13 provide modeling tools useful in dealing with nonstationary systems. In Chapter 2, some fundamental time-series concepts are introduced, with a special emphasis on concepts relevant to changing systems. Changes to real systems affect data observations. The different forms that these changes introduce into data are defined and illustrated. In Chapter 3, basic concepts related to the fundamentals of hypothesis testing are introduced. While most of this material will serve as a review for readers with background in statistical analysis, the chapter includes the basic concepts important to understanding the statistical tests introduced in the middle chapters. Extreme events contained in measured data are the topics of Chapter 4. They can distort calibrated model parameters and predictions based on models. Thus, their proper assessment and handling are essential in the early stages of modeling. Frequency analysis is a rank-order statistical method widely used to connect the magnitudes and probabilities of occurrence of a random variable. The basic elements of frequency analysis as applied in hydrology are introduced in Chapter 5. © 2003 by CRC Press LLC While statistical methods are important tools for the detection of nonstationarity, they are less effective when not accompanied by graphical analyses. In Chapter 6, the uses of graphical methods in the modeling process are introduced. While graph - ical analysis alone is inadequate for characterizing hydrologic change, it is a nec- essary component of the modeling process. In Chapter 7, the fundamentals of detecting nonhomogeneity in time series are introduced. Special emphasis is placed on selecting the statistical method most sensitive to the types of changes to be evaluated. Hydrologic change may be evident in the moments of the measured data or more generally in the distribution of the data. The statistical detection of change to moments is discussed in Chapter 8, while the detection of changes in probability distribution is the topic of Chapter 9. Statistical methods sensitive to different types of change are introduced in these chapters. Chapter 10 covers many fundamentals of model calibration. Basic regression techniques along with advanced topics such as composite modeling and jackknifing are included. Computer simulation is a valuable tool for modeling expected watershed changes. The manipulation of a model to simulate alternative scenarios of change can be valuable to decision makers. The fundamentals of simulation are presented in Chapter 11. Sensitivity analysis is an important tool in modeling. It is useful for making error analyses and for assessing the relative importance of causative factors. The mathematical basis of sensitivity analysis and its uses are discussed in Chapter 12. Chapter 13 presents the role that geographic information systems (GIS) can play in the assessment of hydrologic change. The inclusion of large databases in modeling is discussed. The effects of urbanization on flood frequency analysis are shown. © 2003 by CRC Press LLC The Author Richard H. McCuen, professor of civil engineering at the University of Maryland at College Park, received degrees from Carnegie Mellon University and the Georgia Institute of Technology. He received the Icko Iben Award from the American Water Resource Association and was co-recipient of the 1988 Outstanding Research Award from the American Society of Civil Engineers Water Resources Planning and Man - agement Division. Topics in statistical hydrology and stormwater management are his primary research interest. He is the author of 17 books and more than 200 professional papers, including Modeling Hydrologic Change (CRC Press, 2002); Hydrologic Analysis and Design, Second Edition (Prentice-Hall, 1998); The Elements of Academic Research (ASCE Press, 1996); Estimating Debris Volumes for Flood Control (Lighthouse Publica - tions, 1996; with T.V. Hromadka); and Dynamic Communication for Engineers (ASCE Press, 1993; with P. Johnson and C. Davis). © 2003 by CRC Press LLC Acknowledgments Three people contributed greatly to this book. I very much appreciate Glenn Moglen’s willingness to contribute the chapter on GIS and its role in modeling change. This book initially was developed as a joint effort with Wilbert O. Thomas, Jr., Baker Engineering, Alexandria, Virginia, but his workload did not permit par - ticipation beyond planning and review of earlier material. His insights are appreci- ated. Finally, the assistance of Dominic Yeh, University of Maryland, for typing the many, many drafts was essential to the completion of the manuscript. His efforts are also very much appreciated. Richard H. McCuen College Park, Maryland © 2003 by CRC Press LLC Contents Chapter 1 Data, Statistics, and Modeling 1.1 Introduction 1.2 Watershed Changes 1.3 Effect on Flood Record 1.4 Watershed Change and Frequency Analysis 1.5 Detection of Nonhomogeneity 1.6 Modeling of Nonhomogeneity 1.7 Problems Chapter 2 Introduction to Time Series Modeling 2.1 Introduction 2.2 Components of a Time Series 2.2.1 Secular Trends 2.2.2 Periodic and Cyclical Variations 2.2.3 Episodic Variation 2.2.4 Random Variation 2.3 Moving-Average Filtering 2.4 Autocorrelation Analysis 2.5 Cross-Correlation Analysis 2.6 Identification of Random Components 2.7 Autoregression and Cross-Regression Models 2.7.1 Deterministic Component 2.7.2 Stochastic Element 2.7.3 Cross-Regression Models 2.8 Problems Chapter 3 Statistical Hypothesis Testing 3.1 Introduction 3.2 Procedure for Testing Hypotheses 3.2.1 Step 1: Formulation of Hypotheses 3.2.2 Step 2: Test Statistic and Its Sampling Distribution 3.2.3 Step 3: Level of Significance 3.2.4 Step 4: Data Analysis 3.2.5 Step 5: Region of Rejection 3.2.6 Step 6: Select Appropriate Hypothesis 3.3 Relationships among Hypothesis Test Parameters © 2003 by CRC Press LLC 3.4 Parametric and Nonparametric Tests 3.4.1 Disadvantages of Nonparametric Tests 3.4.2 Advantages of Nonparametric Tests 3.5 Problems Chapter 4 Outlier Detection 4.1 Introduction 4.2 Chauvenet’s Method 4.3 Dixon–Thompson Test 4.4 Rosner’s Outlier Test 4.5 Log-Pearson Type III Outlier Detection: Bulletin 17b 4.6 Pearson Type III Outlier Detection 4.7 Problems Chapter 5 Statistical Frequency Analysis 5.1 Introduction 5.2 Frequency Analysis and Synthesis 5.2.1 Population versus Sample 5.2.2 Analysis versus Synthesis 5.2.3 Probability Paper 5.2.4 Mathematical Model 5.2.5 Procedure 5.2.6 Sample Moments 5.2.7 Plotting Position Formulas 5.2.8 Return Period 5.3 Population Models 5.3.1 Normal Distribution 5.3.2 Lognormal Distribution 5.3.3 Log-Pearson Type III Distribution 5.4 Adjusting Flood Record for Urbanization 5.4.1 Effects of Urbanization 5.4.2 Method for Adjusting Flood Record 5.4.3 Testing Significance of Urbanization 5.5 Problems Chapter 6 Graphical Detection of Nonhomogeneity 6.1 Introduction 6.2 Graphical Analyses 6.2.1 Univariate Histograms 6.2.2 Bivariate Graphical Analysis 6.3 Compilation of Causal Information 6.4 Supporting Computational Analyses 6.5 Problems © 2003 by CRC Press LLC Chapter 7 Statistical Detection of Nonhomogeneity 7.1 Introduction 7.2 Runs Test 7.2.1 Rational Analysis of Runs Test 7.3 Kendall Test for Trend 7.3.1 Rationale of Kendall Statistic 7.4 Pearson Test for Serial Independence 7.5 Spearman Test for Trend 7.5.1 Rationale for Spearman Test 7.6 Spearman–Conley Test 7.7 Cox–Stuart Test for Trend 7.8 Noether’s Binomial Test for Cyclical Trend 7.8.1 Background 7.8.2 Test Procedure 7.8.3 Normal Approximation 7.9 Durbin–Watson Test for Autocorrelation 7.9.1 Test for Positive Autocorrelation 7.9.2 Test for Negative Autocorrelation 7.9.3 Two-Sided Test for Autocorrelation 7.10 Equality of Two Correlation Coefficients 7.11 Problems Chapter 8 Detection of Change in Moments 8.1 Introduction 8.2 Graphical Analysis 8.3 The Sign Test 8.4 Two-Sample t-Test 8.5 Mann–Whitney Test 8.5.1 Rational Analysis of the Mann–Whitney Test 8.6 The t-Test for Two Related Samples 8.7 The Walsh Test 8.8 Wilcoxon Matched-Pairs, Signed-Ranks Test 8.8.1 Ties 8.9 One-Sample Chi-Square Test 8.10 Two-Sample F-Test 8.11 Siegel–Tukey Test for Scale 8.12 Problems Chapter 9 Detection of Change in Distribution 9.1 Introduction 9.2 Chi-Square Goodness-of-Fit Test 9.2.1 Procedure 9.2.2 Chi-Square Test for a Normal Distribution 9.2.3 Chi-Square Test for an Exponential Distribution © 2003 by CRC Press LLC 9.2.4 Chi-Square Test for Log-Pearson III Distribution 9.3 Kolmogorov–Smirnov One-Sample Test 9.3.1 Procedure 9.4 The Wald–Wolfowitz Runs Test 9.4.1 Large Sample Testing 9.4.2 Ties 9.5 Kolmogorov–Smirnov Two-Sample Test 9.5.1 Procedure: Case A 9.5.2 Procedure: Case B 9.6 Problems Chapter 10 Modeling Change 10.1 Introduction 10.2 Conceptualization 10.3 Model Formulation 10.3.1 Types of Parameters 10.3.2 Alternative Model Forms 10.3.3 Composite Models 10.4 Model Calibration 10.4.1 Least-Squares Analysis of a Linear Model 10.4.2 Standardized Model 10.4.3 Matrix Solution of the Standardized Model 10.4.4 Intercorrelation 10.4.5 Stepwise Regression Analysis 10.4.6 Numerical Optimization 10.4.7 Subjective Optimization 10.5 Model Verification 10.5.1 Split-Sample Testing 10.5.2 Jackknife Testing 10.6 Assessing Model Reliability 10.6.1 Model Rationality 10.6.2 Bias in Estimation 10.6.3 Standard Error of Estimate 10.6.4 Correlation Coefficient 10.7 Problems Chapter 11 Hydrologic Simulation 11.1 Introduction 11.1.1 Definitions 11.1.2 Benefits of Simulation 11.1.3 Monte Carlo Simulation 11.1.4 Illustration of Simulation 11.1.5 Random Numbers 11.2 Computer Generation of Random Numbers 11.2.1 Midsquare Method © 2003 by CRC Press LLC [...].. .11 .3 11 .4 11 .5 11 .6 11 .2.2 Arithmetic Generators 11 .2.3 Testing of Generators 11 .2.4 Distribution Transformation Simulation of Discrete Random Variables 11 .3 .1 Types of Experiments 11 .3.2 Binomial Distribution 11 .3.3 Multinomial Experimentation 11 .3.4 Generation of Multinomial Variates 11 .3.5 Poisson Distribution 11 .3.6 Markov Process Simulation Generation... Distributed Random Variates 11 .4 .1 Uniform Distribution, U(α , β ) 11 .4.2 Triangular Distribution 11 .4.3 Normal Distribution 11 .4.4 Lognormal Distribution 11 .4.5 Log-Pearson Type III Distribution 11 .4.6 Chi-Square Distribution 11 .4.7 Exponential Distribution 11 .4.8 Extreme Value Distribution Applications of Simulation Problems Chapter 12 Sensitivity Analysis 12 .1 Introduction 12 .2 Mathematical Foundations... in Modeling Change 12 .7.2 Qualitative Sensitivity Analysis 12 .7.3 Sensitivity Analysis in Design 12 .8 Problems Chapter 13 Frequency Analysis under Nonstationary Land Use Conditions 13 .1 Introduction © 2003 by CRC Press LLC 13 .2 13 .3 13 .4 13 .5 13 .6 13 .7 13 .1. 1 Overview of Method 13 .1. 2 Illustrative Case Study: Watts Branch Data Requirements 13 .2 .1 Rainfall Data Records 13 .2.2 Streamflow Records 13 .2.3... Analysis 12 .2 .1 Definition 12 .2.2 The Sensitivity Equation 12 .2.3 Computational Methods 12 .2.4 Parametric and Component Sensitivity 12 .2.5 Forms of Sensitivity 12 .2.6 A Correspondence between Sensitivity and Correlation 12 .3 Time Variation of Sensitivity 12 .4 Sensitivity in Model Formulation 12 .5 Sensitivity and Data Error Analysis 12 .6 Sensitivity of Model Coefficients 12 .7 Watershed Change 12 .7 .1 Sensitivity... Land-Use Time Series Modeling Issues 13 .4 .1 Selecting a Model 13 .4.2 Calibration Strategies 13 .4.3 Simulating a Stationary Annual Maximum-Discharge Series Comparison of Flood-Frequency Analyses 13 .5 .1 Implications for Hydrologic Design 13 .5.2 Assumptions and Limitations Summary Problems Appendix A Statistical Tables Appendix B Data Matrices References © 2003 by CRC Press LLC 1 Data, Statistics, and Modeling. .. change? 1- 6 For a 15 0-acre watershed, how might the installation of numerous small stormwater-detention basins influence the flood record, including both peak discharges and baseflow? 1- 7 How might the effects of urbanization on a flood record be detected if specific knowledge of the timing and location of gradual watershed changes were not available? 1- 8 Discuss how an outlier due to a hurricane-generated... development of a 1- acre parcel of land affects the dominance of the physical watershed processes Assume that the predevelopment condition is woods, while after development the lot is residential 1- 2 Using Manning’s equation, discuss how clearing a stream reach of native vegetation changes the discharge characteristics 1- 3 Would you expect urban development of 10 acres in the upper reaches of a 500-acre watershed... occur under the current watershed conditions Two statistical advances are required to properly analyze nonstationary data First, more appropriate statistical tools are necessary to detect the effect of hydrologic change and the characteristics of the change For example, statistical methods are needed to identify trends, with one class of statistical methods needed for gradual trends and a second class... watershed urbanization, then an appropriate adjustment method must be used in place of the more data-intensive, watershed -modeling approach An annual-maximum flood series is characterized by the randomness introduced by the variations of both rainfall and the hydrologic conditions of the watershed when the flood-producing rainfall occurred The floods of an annual maximum series are produced by storms that... record? Explain Would the effect be noticeable if the 10 acres were near the watershed outlet? Explain 1- 4 Discuss how the effects of an episodic event, such as a hurricane, might influence the flood record of a 200-acre watershed Assume that the year in which the hurricane occurred was near the middle of the flood record © 2003 by CRC Press LLC 1- 5 What model components would be necessary for a model . Reliability 10 .6 .1 Model Rationality 10 .6.2 Bias in Estimation 10 .6.3 Standard Error of Estimate 10 .6.4 Correlation Coefficient 10 .7 Problems Chapter 11 Hydrologic Simulation 11 .1 Introduction 11 .1. 1 Definitions 11 .1. 2. Introduction 11 .1. 1 Definitions 11 .1. 2 Benefits of Simulation 11 .1. 3 Monte Carlo Simulation 11 .1. 4 Illustration of Simulation 11 .1. 5 Random Numbers 11 .2 Computer Generation of Random Numbers 11 .2 .1 Midsquare Method ©. H., 19 41 Modeling hydrologic change: statistical methods / Richard H. McCuen. p. cm. Includes bibliographical references and index. ISBN 1- 5 667 0-6 0 0-9 1. Hydrologic models. 2. Hydrologic Statistical