Linear models least squares and alternatives

Linear Models: Least Squares and Alternatives, Second Edition C Radhakrishna Rao Helge Toutenburg Springer Preface to the First Edition The book is based on several years of experience of both authors in teaching linear models at various levels It gives an up-to-date account of the theory and applications of linear models The book can be used as a text for courses in statistics at the graduate level and as an accompanying text for courses in other areas Some of the highlights in this book are as follows A relatively extensive chapter on matrix theory (Appendix A) provides the necessary tools for proving theorems discussed in the text and offers a selection of classical and modern algebraic results that are useful in research work in econometrics, engineering, and optimization theory The matrix theory of the last ten years has produced a series of fundamental results about the definiteness of matrices, especially for the differences of matrices, which enable superiority comparisons of two biased estimates to be made for the first time We have attempted to provide a unified theory of inference from linear models with minimal assumptions Besides the usual least-squares theory, alternative methods of estimation and testing based on convex loss functions and general estimating equations are discussed Special emphasis is given to sensitivity analysis and model selection A special chapter is devoted to the analysis of categorical data based on logit, loglinear, and logistic regression models The material covered, theoretical discussion, and a variety of practical applications will be useful not only to students but also to researchers and consultants in statistics We would like to thank our colleagues Dr G Trenkler and Dr V K Srivastava for their valuable advice during the preparation of the book We vi Preface to the First Edition wish to acknowledge our appreciation of the generous help received from Andrea Schă opp, Andreas Fieger, and Christian Kastner for preparing a fair copy Finally, we would like to thank Dr Martin Gilchrist of Springer-Verlag for his cooperation in drafting and finalizing the book We request that readers bring to our attention any errors they may find in the book and also give suggestions for adding new material and/or improving the presentation of the existing material University Park, PA Mă unchen, Germany July 1995 C Radhakrishna Rao Helge Toutenburg Preface to the Second Edition The first edition of this book has found wide interest in the readership A first reprint appeared in 1997 and a special reprint for the Peoples Republic of China appeared in 1998 Based on this, the authors followed the invitation of John Kimmel of Springer-Verlag to prepare a second edition, which includes additional material such as simultaneous confidence intervals for linear functions, neural networks, restricted regression and selection problems (Chapter 3); mixed effect models, regression-like equations in econometrics, simultaneous prediction of actual and average values, simultaneous estimation of parameters in different linear models by empirical Bayes solutions (Chapter 4); the method of the Kalman Filter (Chapter 6); and regression diagnostics for removing an observation with animating graphics (Chapter 7) Chapter 8, “Analysis of Incomplete Data Sets”, is completely rewritten, including recent terminology and updated results such as regression diagnostics to identify Non-MCAR processes Chapter 10, “Models for Categorical Response Variables”, also is completely rewritten to present the theory in a more unified way including GEE-methods for correlated response At the end of the chapters we have given complements and exercises We have added a separate chapter (Appendix C) that is devoted to the software available for the models covered in this book We would like to thank our colleagues Dr V K Srivastava (Lucknow, India) and Dr Ch Heumann (Mă unchen, Germany) for their valuable advice during the preparation of the second edition We thank Nina Lieske for her help in preparing a fair copy We would like to thank John Kimmel of viii Preface to the Second Edition Springer-Verlag for his effective cooperation Finally, we wish to appreciate the immense work done by Andreas Fieger (Mă unchen, Germany) with respect to the numerical solutions of the examples included, to the technical management of the copy, and especially to the reorganization and updating of Chapter (including some of his own research results) Appendix C on software was written by him, also We request that readers bring to our attention any suggestions that would help to improve the presentation University Park, PA Mă unchen, Germany May 1999 C Radhakrishna Rao Helge Toutenburg Contents Preface to the First Edition v Preface to the Second Edition vii Introduction Linear Models 2.1 Regression Models in Econometrics 2.2 Econometric Models 2.3 The Reduced Form 2.4 The Multivariate Regression Model 2.5 The Classical Multivariate Linear Regression 2.6 The Generalized Linear Regression Model 2.7 Exercises The 3.1 3.2 3.3 3.4 3.5 Model Linear Regression Model The Linear Model The Principle of Ordinary Least Squares (OLS) Geometric Properties of OLS Best Linear Unbiased Estimation 3.4.1 Basic Theorems 3.4.2 Linear Estimators 3.4.3 Mean Dispersion Error Estimation (Prediction) of the Error Term
and σ2 5 12 14 17 18 20 23 23 24 25 27 27 32 33 34 ... Different Linear Models 4.8 Supplements 4.9 Gauss-Markov, Aitken and Rao Least Squares Estimators 4.9.1 Gauss-Markov Least Squares 4.9.2 Aitken Least. .. of inference from linear models with minimal assumptions Besides the usual least- squares theory, alternative methods of estimation and testing based on convex loss functions and general estimating... linear models at various levels It gives an up-to-date account of the theory and applications of linear models The book can be used as a text for courses in statistics at the graduate level and