Aided Introduction to Econometrics

Computer-Aided Introduction to Econometrics Juan M. Rodriguez Poo In cooperation with Ignacio Moral, M. Teresa Aparicio, Inmaculada Villanua, Pavel ˇ C´ıˇzek, Yingcun Xia, Pilar Gonzalez, M. Paz Moral, Rong Chen, Rainer Schulz, Sabine Stephan, Pilar Olave, J. Tomas Alcala and Lenka Cizkova January 17, 2003 Preface This book is designed for undergraduate students, applied researchers and prac- titioners to develop professional skills in econometrics. The contents of the book are designed to satisfy the requirements of an undergraduate economet- rics course of about 90 hours. Although the book presents a clear and serious theoretical treatment, its main strength is that it incorporates an interactive computing internet based method that allows the reader to practice all the techniques he is learning theoretically along the different chapters of the book. It provides a comprehensive treatment of the theoretical issues related to lin- ear regression analysis, univariate time series modelling and some interesting extensions such as ARCH models and dimensionality reduction techniques. Furthermore, all theoretical issues are illustrated through an internet based interactive computing method, that allows the reader to learn from theory to practice the different techniques that are developed in the book. Although the course assumes only a modest background it moves quickly between different fields of applications and in the end, the reader can expert to have theoretical and computational tools that are deep enough and rich enough to be relied on throughout future professional careers. The computer inexperienced user of this book is softly introduced into the in- teractive book concept and will certainly enjoy the various practical examples. The e-book is designed as an interactive document: a stream of text and in- formation with various hints and links to additional tools and features. Our e-book design offers also a complete PDF and HTML file with links to world wide computing servers. The reader of this book may therefore without down- load or purchase of software use all the presented examples and methods via a local XploRe Quantlet Server (XQS). Such QS Servers may also be installed in a department or addressed freely on the web, click to www.xplore-stat.de and www.quantlet.com. ”Computer-Aided introduction to Econometrics” consists on three main parts: Linear Regression Analysis, Univariate Time Series Modelling and Computa- vi tional Methods. In the first part, Moral and Rodriguez-Poo provide the basic background for univariate linear regression models: Specification, estimation, testing and forecasting. Moreover, they provide some basic concepts on prob- ability and inference that are required to study fruitfully further concepts in regression analysis. Aparicio and Villanua provide a deep treatment of the multivariate linear regression model: Basic assumptions, estimation methods and properties. Linear hypothesis testing and general test procedures (Like- lihood ratio test, Wald test and Lagrange multiplier test) are also developed. Finally, they consider some standard extensions in regression analysis such as dummy variables and restricted regression. ˇ C´ıˇzek and Xia close this part with a chapter devoted to dimension reduction techniques and applications. Since the techniques developed in this section are rather new, this part of of higher level of difficulty than the preceding sections. The second part starts with an introduction to Univariate Time Series Anal- ysis by Moral and Gonzalez. Starting form the analysis of linear stationary processes, they jump to some particular cases of non-stationarity such as non- stationarity in mean and variance. They provide also some statistical tools for testing for unit roots. Furthermore, within the class of linear stationary processes they focus their attention in the sub-class of ARIMA models. Fi- nally, as a natural extension to the previous concepts to regression analysis, cointegration and error correction models are considered. Departing from the class of ARIMA models, Chen, Schulz and Stephan propose a way to deal with seasonal time series. Olave and Alcala end this part with an introduction to Autoregressive Conditional Heteroskedastic Models, which appear to be a nat- ural extension of ARIMA modelling to econometric models with a conditional variance that is time varying. In their work, they provide an interesting battery of tests for ARCH disturbances that appears as a nice example of the testing tools already introduced by Aparicio and Villanua in a previous chapter. In the last part of the book, ˇ Ciˇzkova develops several nonlinear optimization techniques that are of common use in Econometrics. The special structure of the e-book relying in a interactive computing internet based method makes it an ideal tool to comprehend optimization problems. I gratefully acknowledge the support of Deutsche Forschungsgemeinschaft, SFB 373 Quantifikation und Simulation ¨ Okonomischer Prozesse and Direcci´on Gen- eral de Investigaci´on del Ministerio de Ciencia y Tecnolog´ıa under research grant BEC2001-1121. For technical production of the e-book I would like to thank Zdenˇek Hl´avka and Rodrigo Witzel. Santander, October 2002, J. M. Rodriguez-Poo. Contributors Ignacio Moral Departamento de Econom´ıa, Universidad de Cantabria Juan M. Rodriguez-Poo Departamento de Econom´ıa, Universidad de Cantabria Teresa Aparicio Departamento de An´alisis Econ´omico, Universidad de Zaragoza Inmaculada Villanua Departamento de An´alisis Econ´omico, Universidad de Zaragoza Pavel ˇ C´ıˇzek Humboldt-Universit¨at zu Berlin, CASE, Center of Applied Statis- tics and Economics Yingcun Xia Department of Statistics and Actuarial Science, The University of Hong Kong Paz Moral Departamento de Econometr´ıa y Estad´ıstica, Universidad del Pa´ıs Vasco Pilar Gonzalez Departamento de Econometr´ıa y Estad´ıstica, Universidad del Pa´ıs Vasco Rong Chen Department of Information and Decision Sciences, University of Illinois at Chicago Rainer Schulz Humboldt-Universit¨at zu Berlin, CASE, Center of Applied Statistics and Economics Sabine Stephan German Institute for Economic Research Pilar Olave Departamento de m´etodos estad´ısticos, Universidad de Zaragoza Juan T. Alcal´a Departamento de m´etodos estad´ısticos, Universidad de Zaragoza Lenka ˇ C´ıˇzkov´a Humboldt-Universit¨at zu Berlin, CASE, Center of Applied Statistics and Economics Contents 1 Univariate Linear Regression Model 1 Ignacio Moral and Juan M. Rodriguez-Poo 1.1 Probability and Data Generating Process . . . . . . . . . . . . 1 1.1.1 Random Variable and Probability Distribution . . . . . 2 1.1.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1.3 Data Generating Process . . . . . . . . . . . . . . . . . 8 1.1.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.2 Estimators and Properties . . . . . . . . . . . . . . . . . . . . . 12 1.2.1 Regression Parameters and their Estimation . . . . . . . 14 1.2.2 Least Squares Method . . . . . . . . . . . . . . . . . . . 16 1.2.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.2.4 Goodness of Fit Measures . . . . . . . . . . . . . . . . . 20 1.2.5 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.2.6 Properties of the OLS Estimates of α, β and σ 2 . . . . . 23 1.2.7 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 28 1.3 Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 1.3.1 Hypothesis Testing about β . . . . . . . . . . . . . . . . 31 1.3.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 1.3.3 Testing Hypothesis Based on the Regression Fit . . . . 35 x Contents 1.3.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 1.3.5 Hypothesis Testing about α . . . . . . . . . . . . . . . . 37 1.3.6 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 1.3.7 Hypotheses Testing about σ 2 . . . . . . . . . . . . . . . 38 1.4 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 1.4.1 Confidence Interval for the Point Forecast . . . . . . . . 40 1.4.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 1.4.3 Confidence Interval for the Mean Predictor . . . . . . . 41 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 2 Multivariate Linear Regression Model 45 Teresa Aparicio and Inmaculada Villanua 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 2.2 Classical Assumptions of the MLRM . . . . . . . . . . . . . . . 46 2.2.1 The Systematic Component Assumptions . . . . . . . . 47 2.2.2 The Random Component Assumptions . . . . . . . . . . 48 2.3 Estimation Procedures . . . . . . . . . . . . . . . . . . . . . . . 49 2.3.1 The Least Squares Estimation . . . . . . . . . . . . . . 50 2.3.2 The Maximum Likelihood Estimation . . . . . . . . . . 55 2.3.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 2.4 Properties of the Estimators . . . . . . . . . . . . . . . . . . . . 59 2.4.1 Finite Sample Properties of the OLS and ML Estimates of β . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 2.4.2 Finite Sample Properties of the OLS and ML Estimates of σ 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 2.4.3 Asymptotic Properties of the OLS and ML Estimators of β 66 2.4.4 Asymptotic Properties of the OLS and ML Estimators of σ 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 [...]... 1.2 Estimators and Properties 13 fortunately, the sampling design and the linearity assumption in the PRF, are not sufficient conditions to ensure that there exists a precise statistical relationship between the estimators and its true corresponding values (see section 1.2.6 for more details) In order to do so, we need to know some additional features from the PRF Since we do not them, we decide to establish... problem ˆ ˆ ˆ ˆ (ˆ , β) = argmin ˆ ˆS(α, β) α ˆ ˆ α,β ˆ (1.31) In order to solve it, that is, to find the minimum values, the first conditions make the first partial derivatives have to be equal to zero ˆ ˆ ∂S(α, β) ˆ ˆ ˆ ∂α ˆ n = −2 ˆ ˆ (yi − α − βxi ) = 0 ˆ ˆ i=1 (1.32) ˆ ˆ ∂S(α, β) ˆ ˆ ˆ ˆ ∂β n = −2 ˆ ˆ (yi − α − βxi )xi = 0 ˆ ˆ i=1 To verify whether the solution is really a minimum, the matrix of second... and suppose we simply want to study the probability distribution of X, say f (x) How can we use the joint probability density function for (X, Y ) to obtain f (x)? The marginal distribution, f (x), of a discrete random variable X provides the probability that the variable X is equal to x, in the joint probability f (X, Y ), without considering the variable Y , thus, if we want to obtain the marginal distribution... aim of the statistical analysis is to obtain information from the population (its joint probability distribution) through the analysis of the sample Unfortunately, in many situations the aim of obtaining information about the 1.1 Probability and Data Generating Process 9 whole joint probability distribution is too complicated, and we have to orient our objective towards more modest proposals Instead... regression function, estimation of the parameter values is tantamount to the estimation of the entire regression function Therefore, once a sample is available, our task is considerably simplified since, in order to analyze the whole population, we only need to give correct estimates of the regression parameters One important issue related to the Population Regression Function is the so called Error term... · · · , n (1.22) ˆ then, given any estimator of α and β, namely β and α, we can substitute these ˆ estimators into the regression function yi = α + βxi , ˆ ˆ ˆ i = 1, · · · , n (1.23) obtaining the sample regression function (SRF) The relationship between the PRF and SRF is: yi = y i + u i , ˆ ˆ i = 1, · · · , n (1.24) where ui is denoted the residual ˆ Just to illustrate the difference between Sample... 285 7 Numerical Optimization Methods in Econometrics 287 ˇ ıˇ a Lenka C´zkov´ 7.1 Introduction 287 7.2 Solving a Nonlinear Equation 287 7.2.1 Termination of Iterative Methods 288 7.2.2 Newton-Raphson Method 288 Solving a System of Nonlinear Equations 290 7.3.1 Newton-Raphson Method for Systems 290... residual sum ˆ ˆ ˆ of squares–RSS n i=1 ˆ 2 (yi − yi ) → min ˆ (1.29) 1.2 Estimators and Properties 17 This criterion function has two variables with respect to which we are willing ˆ ˆ ˆ to minimize: α and β ˆ ˆ ˆ S(α, β) = ˆ ˆ n 2 ˆ ˆ (yi − α − βxi ) ˆ ˆ (1.30) i=1 Then, we define as Ordinary Least Squares (OLS) estimators, denoted by α ˆ ˆ and β, the values of α and β that solve the following optimization... concepts that are necessary to understand further developments in the chapter, the purpose is to highlight some of the more important theoretical results in probability, in particular, the concept of the random variable, the probability distribution, and some related 2 1 Univariate Linear Regression Model results Note however, that we try to maintain the exposition at an introductory level For a more formal... of the propositions against the behavior of observable data It is not our aim to include here a detailed discussion on econometric model building, this type of discussion can be found in Intriligator (1978), however, along the sequent subsections we will introduce, using monte carlo simulations, the main results related to estimation and inference in univariate linear regression models The next chapters . like to thank Zdenˇek Hl´avka and Rodrigo Witzel. Santander, October 2002, J. M. Rodriguez-Poo. Contributors Ignacio Moral Departamento de Econom´ıa, Universidad de Cantabria Juan M. Rodriguez-Poo. Computer -Aided Introduction to Econometrics Juan M. Rodriguez Poo In cooperation with Ignacio Moral, M. Teresa Aparicio, Inmaculada Villanua, Pavel ˇ C´ıˇzek, Yingcun Xia, Pilar Gonzalez, M. . addressed freely on the web, click to www.xplore-stat.de and www.quantlet.com. ”Computer -Aided introduction to Econometrics consists on three main parts: Linear Regression Analysis, Univariate