Class Notes in Statistics and Econometrics Part 1 pdf

Class Notes in Statistics and Econometrics Hans G. Ehrbar Economics Department, University of Utah, 1645 Campus Center Drive, Salt Lake City UT 84112-9300, U.S.A. URL: www.econ.utah.edu/ehrbar/ecmet.pdf E-mail address: ehrbar@econ.utah.edu Abstract. This is an attempt to make a carefully argued set of class notes freely available. The source code for these notes can be downloaded from www.econ.utah.edu/ehrbar/ecmet-sources.zip Copyright Hans G. Ehrbar un- der the GNU Public License Contents Chapter 1. Preface xxiii Chapter 2. Probability Fields 1 2.1. The Concept of Probability 1 2.2. Events as Sets 12 2.3. The Axioms of Probability 20 2.4. Objective and Subjective Interpretation of Probability 26 2.5. Counting Rules 28 2.6. Relationships Involving Binomial Coefficients 32 2.7. Conditional Probability 34 2.8. Ratio of Probabilities as Strength of Evidence 45 iii iv CONTENTS 2.9. Bayes Theorem 48 2.10. Independence of Events 50 2.11. How to Plot Frequency Vectors and Probability Vectors 57 Chapter 3. Random Variables 63 3.1. Notation 63 3.2. Digression about Infinitesimals 64 3.3. Definition of a Random Variable 68 3.4. Characterization of Random Variables 70 3.5. Discrete and Absolutely Continuous Probability Measures 77 3.6. Transformation of a Scalar Density Function 79 3.7. Example: Binomial Variable 82 3.8. Pitfalls of Data Reduction: The Ecological Fallacy 85 3.9. Independence of Random Variables 87 3.10. Location Parameters and Dispersion Parameters of a Random Variable 89 3.11. Entropy 100 Chapter 4. Random Number Generation and Encryption 121 4.1. Alternatives to the Linear Congruential Random Generator 125 4.2. How to test random generators 126 4.3. The Wichmann Hill generator 128 CONTENTS v 4.4. Public Key Cryptology 133 Chapter 5. Specific Random Variables 139 5.1. Binomial 139 5.2. The Hypergeometric Probability Distribution 146 5.3. The Poisson Distribution 148 5.4. The Exponential Distribution 154 5.5. The Gamma Distribution 158 5.6. The Uniform Distribution 164 5.7. The Beta Distribution 165 5.8. The Normal Distribution 166 5.9. The Chi-Square Distribution 172 5.10. The Lognormal Dis tribution 174 5.11. The Cauchy Distribution 174 Chapter 6. Sufficient Statistics and their Distributions 179 6.1. Factorization Theorem for Sufficient Statistics 179 6.2. The Exponential Family of Probability Distributions 182 Chapter 7. Chebyshev Inequality, Weak Law of Large Numbers, and Central Limit Theorem 189 vi CONTENTS 7.1. Chebyshev Inequality 189 7.2. The Probability Limit and the Law of Large Numbers 192 7.3. Central Limit Theorem 195 Chapter 8. Vector Random Variables 199 8.1. Expected Value, Variances, Covariances 203 8.2. Marginal Probability Laws 210 8.3. Conditional Probability Distribution and Conditional Mean 212 8.4. The Multinomial Distribution 216 8.5. Independent Random Vectors 218 8.6. Conditional Expectation and Variance 221 8.7. Expected Values as Predictors 226 8.8. Transformation of Vector Random Variables 235 Chapter 9. Random Matrices 245 9.1. Linearity of Expected Values 245 9.2. Means and Variances of Quadratic Forms in Random Matrices 249 Chapter 10. The Multivariate Normal Probability Distribution 261 10.1. More About the Univariate Case 261 10.2. Definition of Multivariate Normal 264 CONTENTS vii 10.3. Special Case: Bivariate Normal 265 10.4. Multivariate Standard Normal in Higher Dimensions 284 10.5. Higher Moments of the Multivariate Standard Normal 290 10.6. The General Multivariate Normal 299 Chapter 11. The Regression Fallacy 309 Chapter 12. A Simple Example of Estimation 327 12.1. Sample Mean as Estimator of the Location Parameter 327 12.2. Intuition of the Maximum Likelihood Estimator 330 12.3. Variance Estimation and Degrees of Freedom 335 Chapter 13. Estimation Principles and Classification of Estimators 355 13.1. Asymptotic or Large-Sample Properties of Estimators 355 13.2. Small Sample Properties 359 13.3. Comparison Unbiasedness Consistency 362 13.4. The Cramer-Rao Lower Bound 369 13.5. Best Linear Unbiased Without Distribution Assumptions 386 13.6. Maximum Likelihood Estimation 390 13.7. Method of Moments Estimators 396 13.8. M-Estimators 396 viii CONTENTS 13.9. Sufficient Statistics and Estimation 397 13.10. The Likelihood Principle 405 13.11. Bayesian Inference 406 Chapter 14. Interval Estimation 411 Chapter 15. Hypothesis Testing 425 15.1. Duality between Significance Tests and Confidence Regions 433 15.2. The Neyman Pearson Lemma and Likelihood Ratio Tests 434 15.3. The Runs Test 440 15.4. Pearson’s Goodness of Fit Test. 447 15.5. Permutation Tests 453 15.6. The Wald, Likelihood Ratio, and Lagrange Multiplier Tests 465 Chapter 16. General Principles of Econometric Modelling 469 Chapter 17. Causality and Inference 473 Chapter 18. Mean-Variance Analysis in the Linear Model 481 18.1. Three Versions of the Linear Model 481 18.2. Ordinary Least Squares 484 18.3. The Coefficient of Determination 499 CONTENTS ix 18.4. The Adjusted R- Square 509 Chapter 19. Digression about Correlation Coefficients 513 19.1. A Unified Definition of Correlation Coefficients 513 19.2. Correlation Coefficients and the Associated Least Squares Problem 519 19.3. Canonical Correlations 521 19.4. Some Remarks about the Sample Partial Correlation Coefficients 524 Chapter 20. Numerical Methods for c omputing OLS Estimates 527 20.1. QR Decomposition 527 20.2. The LINPACK Impleme ntation of the QR Decomposition 530 Chapter 21. About Computers 535 21.1. General Strategy 535 21.2. The Emacs Editor 542 21.3. How to Enter and Exit SAS 544 21.4. How to Transfer SAS Data Sets Between Computers 545 21.5. Instructions for Statistics 5969, Hans Ehrbar’s Section 547 21.6. The Data Step in SAS 557 Chapter 22. Specific Datasets 563 x CONTENTS 22.1. Cobb Douglas Aggregate Production Function 563 22.2. Houthakker’s Data 580 22.3. Long Term Data about US Economy 592 22.4. Dougherty Data 594 22.5. Wage Data 595 Chapter 23. The Mean Squared Error as an Initial Criterion of Precision 629 23.1. Comparison of Two Vector Estimators 630 Chapter 24. Sampling Properties of the Least Squares Estimator 637 24.1. The Gauss Markov Theorem 639 24.2. Digression about Minimax Estimators 643 24.3. Miscellaneous Properties of the BLUE 645 24.4. Estimation of the Variance 666 24.5. Mallow’s Cp-Statistic as Estimator of the Mean Squared Error 668 24.6. Optimality of Variance Estimators 670 Chapter 25. Variance Estimation: Should One Require Unbiasedness? 675 25.1. Setting the Framework Straight 678 25.2. Derivation of the Best Bounded MSE Quadratic Estimator of the Variance 682 [...]... Squares in the EV model Kalman’s Critique of Malinvaud Estimation if the EV Model is Identified P-Estimation Estimation When the Error Covariance Matrix is Exactly Known 10 99 10 99 11 08 11 11 111 6 11 26 11 32 11 46 11 52 11 65 Chapter 54 .1 54.2 54.3 54 Dynamic Linear Models Specification and Recursive Solution Locally Constant Model The Reference Model 11 69 11 69 11 75 11 81 xviii CONTENTS 54.4 Exchange Rate Forecasts... Measures of Income Inequality Properties of Inequality Measures 10 36 10 37 10 38 10 38 10 42 10 43 10 43 10 44 10 45 10 50 Chapter 49 Distributed Lags 49 .1 Geometric lag 49.2 Autoregressive Distributed Lag Models 10 51 1062 10 63 Chapter 50 .1 50.2 50.3 10 73 10 73 10 76 10 81 50 Investment Models Accelerator Models Jorgenson’s Model Investment Function Project CONTENTS xvii Chapter 51 Distinguishing Random Variables... Equality of Arrays and Extended Substitution B.5 Vectorization and Kronecker Product 15 43 15 44 15 49 15 59 15 61 1562 Appendix C Matrix Differentiation C .1 First Derivatives 15 83 15 83 Appendix 15 97 Bibliography CHAPTER 1 Preface These are class notes from several different graduate econometrics and statistics classes In the Spring 2000 they were used for Statistics 6869, syllabus on p ??, and in the Fall 2000... Growth Curve Models 13 13 13 13 13 15 13 29 Chapter 63 .1 63.2 63.3 63.4 63.5 63.6 63 Independent Observations from the Same Multivariate Population 13 33 Notation and Basic Statistics 13 33 Two Geometries 13 37 Assumption of Normality 13 39 EM-Algorithm for Missing Observations 13 41 Wishart Distribution 13 47 Sample Correlation Coefficients 13 49 Chapter 64 .1 64.2 64.3 64 Pooling of Cross Section and Time Series Data... in Milk Production Chapter 55 11 86 11 94 12 00 Numerical Minimization 12 07 Chapter 56 Nonlinear Least Squares 56 .1 The J Test 56.2 Nonlinear instrumental variables estimation 12 15 12 27 12 30 Chapter 57 .1 57.2 57.3 57 Applications of GLS with Nonspherical Covariance Matrix Cases when OLS and GLS are identical Heteroskedastic Disturbances Equicorrelated Covariance Matrix 12 33 12 34 12 35 12 38 Chapter 58 .1. .. Decomposition A.2 The Spectral Norm of a Matrix A.3 Inverses and g-Inverses of Matrices A.4 Deficiency Matrices A.5 Nonnegative Definite Symmetric Matrices A.6 Projection Matrices 15 01 15 01 1503 15 04 15 06 15 14 15 23 xxii CONTENTS A.7 Determinants A.8 More About Inverses A.9 Eigenvalues and Singular Value Decomposition 15 28 15 30 15 37 Appendix B Arrays of Higher Rank B .1 Informal Survey of the Notation B.2 Axiomatic... Estimators Confidence Bands 10 31 10 31 1032 10 34 10 34 10 36 10 36 xvi CONTENTS 47.7 47.8 47.9 47 .10 47 .11 Chapter 48 .1 48.2 48.3 48.4 Other Approaches to Density Estimation Two -and Three-Dimensional Densities Other Characterizations of Distributions Quantile-Quantile Plots Testing for Normality 48 Measuring Economic Inequality Web Resources about Income Inequality Graphical Representations of Inequality Quantitative... Matrix 13 75 13 76 13 80 13 86 13 89 13 95 Chapter 66 .1 66.2 66.3 66.4 66.5 66.6 66 Simultaneous Equations Systems Examples General Mathematical Form Indirect Least Squares Instrumental Variables (2SLS) Identification Other Estimation Methods 13 97 13 97 14 05 14 14 14 16 14 19 14 24 Chapter 67 Timeseries Analysis 67 .1 Covariance Stationary Timeseries 67.2 Vector Autoregressive Processes 14 35 14 35 14 50 CONTENTS 67.3... 67.4 Cointegration xxi 14 60 14 64 Chapter 68 Seasonal Adjustment 68 .1 Methods of Seasonal Adjustment 68.2 Seasonal Dummies in a Regression 14 67 14 72 14 74 Chapter 69 .1 69.2 69.3 14 87 14 87 14 89 14 95 69 Binary Choice Models Fisher’s Scoring and Iteratively Reweighted Least Squares Binary Dependent Variable The Generalized Linear Model Chapter 70 Multiple Choice Models 14 99 Appendix A Matrix Formulas A .1 A... for Numerical Variables More than One Explanatory Variable: Backfitting 997 998 10 02 10 14 Chapter 46 .1 46.2 46.3 46 Transformation of the Response Variable 10 19 Alternating Least Squares and Alternating Conditional Expectations 10 20 Additivity and Variance Stabilizing Transformations (avas) 10 27 Comparing ace and avas 10 29 Chapter 47 .1 47.2 47.3 47.4 47.5 47.6 47 Density Estimation How to Measure the . CONTENTS 13 .9. Sufficient Statistics and Estimation 397 13 .10 . The Likelihood Principle 405 13 .11 . Bayesian Inference 406 Chapter 14 . Interval Estimation 411 Chapter 15 . Hypothesis Testing 425 15 .1. Duality. Errors -in- Variables Model 10 99 53.2. General Definition of the EV Model 11 08 53.3. Particular Forms of EV Models 11 11 53.4. The Identification Problem 11 16 53.5. Properties of Ordinary Least Squares in. Dynamic Linear Models 11 69 54 .1. Specification and Recursive Solution 11 69 54.2. Locally Constant Model 11 75 54.3. The Reference Model 11 81 xviii CONTENTS 54.4. Exchange Rate Forecasts 11 86 54.5.