Mathematical statistics for economics and business (second edition) part 1

Ron C Mittelhammer Mathematical Statistics for Economics and Business Second Edition Mathematical Statistics for Economics and Business Second Edition Ron C Mittelhammer Mathematical Statistics for Economics and Business Second Edition With 93 Illustrations Ron C Mittelhammer School of Economic Sciences Washington State University Pullman, Washington USA ISBN 978-1-4614-5021-4 ISBN 978-1-4614-5022-1 (eBook) DOI 10.1007/978-1-4614-5022-1 Springer New York Heidelberg Dordrecht London Library of Congress Control Number: 2012950028 # Springer Science+Business Media New York 2013 This work is subject to copyright All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer Permissions for use may be obtained through RightsLink at the Copyright Clearance Center Violations are liable to prosecution under the respective Copyright Law The use of general descriptive names, registered names, trademarks, service marks, etc in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made The publisher makes no warranty, express or implied, with respect to the material contained herein Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) To my wife Linda, and to the memory of Carl, Edith, Dolly, and Ralph n Preface to the Second Edition of Mathematical Statistics for Economics and Business n n The general objectives of the second edition of Mathematical Statistics for Economics and Business remain the same as the first, namely, to provide a rigorous and accessible foundation in the principles of probability and in statistical estimation and inference concepts for beginning graduate students and advanced undergraduate students studying primarily in the fields of economics and business Since its publication, the first edition of the book has found use by those from other disciplines as well, including the social sciences (e.g., psychology and sociology), applied mathematics, and statistics, even though many of the applied examples in later chapters have a decidedly “economics and business” feel (although the examples are chosen in such a way that they are fairly well “self-contained” and understandable for those who have not studied either discipline in substantial detail) The general philosophy regarding how and why the book was originally written was presented in the preface to the first edition and in large measure could be inserted at this point for motivating the fundamental rationale for the second edition This philosophy includes the necessity of having a conceptual base of probability and statistical theory to be able to fully understand the application and interpretation of applied econometric and business statistics methods, coupled with the need to have a treatment of the subject that, while rigorous, also assumes an accessible level of prerequisites that can be expected to have been met by a large majority of graduate students entering the fields The choice of topic coverage is also deliberate and decidedly chosen to form the fundamental foundation on which econometric and business statistics methodology is built With the ongoing expansion, in both scope and depth, viii Preface to the Second Edition of Mathematical Statistics for Economics and Business of econometric and statistical methodology for quantitative analyses in both economics and business, it has never been more important, and many are now thinking absolutely essential, that a base of formal probability and statistics understanding become part of student training to enable effective reading of the literature and success in the fields Regarding the nature of the updates and revisions that have been made in producing the second edition, many of the basic probability and statistical concepts remain in common with the first edition The fundamental base of probability and statistics principles needed for later study of econometrics, business statistics, and a myriad of stochastic applications of economic and business theory largely intersects the topics covered in the first edition While a few topics were deleted in the second edition as being less central to that foundation, many more have been added These include the following: greater detail on the issue of parametric, semiparametric, and nonparametric models; an introduction to nonlinear least squares methods; Stieltjes integration has been added strategically in some contexts where continuous and discrete random variable properties could be clearly and efficiently motivated in parallel; additional testing methodology for the ubiquitous normality assumption; clearer differentiation of parametric and semiparametric testing of hypotheses; as well as many other refinements in topic coverage appropriate for applications in economics and business Perhaps the most important revision of the text has been in terms of the organization, exposition, and overall usability of the material Reacting to the feedback of a host of professors, instructors, and individual readers of the first edition, the presentation of both the previous and new material has been notably reorganized and rewritten to make the text easier to study and teach from At the highest level, the compartmentalization of topics is now better and easier to navigate through All theorems and examples are now titled to provide a better foreshadowing of the content of the results and/or the nature of what is being illustrated Some topics have been reordered to improve the flow of reading and understanding (i.e., the relatively more esoteric concept of events that cannot be assigned probability consistently has been moved to the end of a chapter and the review of elements of real analysis has been moved from the beginning of the asymptotic theory chapter to the appendix of the book), and in some cases, selected proofs of theorems that were essentially pure mathematics and that did little to bolster the understanding of statistical concepts were moved to chapter appendices to improve readability of the chapter text A large number of new and expanded exercises/problems have been added to the chapters While a number of texts focused on statistical foundations of estimation and inference are available, Mathematical Statistics for Economics and Business is a text whose level of presentation, assumed prerequisites, examples and problems, and topic coverage will continue to provide a solid foundation for future study of econometrics, business statistics, and general stochastic economic and business theory and application With its redesigned topic organization, additional topic coverage, revision of exposition, expanded set of problems, and continued focus on accessibility and motivation, the book will provide a conceptual foundation Preface to the Second Edition of Mathematical Statistics for Economics and Business ix on which students can base their future study and understanding of rigorous econometric and statistical applications, and it can also serve as an accessible refresher for practicing professionals who wish to reestablish their understanding of the foundations on which all of econometrics, business statistics, and stochastic economic and business theory are based Acknowledgments In addition to all of the acknowledgments presented in the first edition, which certainly remain deserving of inclusion here, I would like to thank Ms Danielle Engelhardt, whose enormous skills in typing, formatting, and proof-checking of the text material and whose always cheerful and positive “can-do” personality made the revision experience a much more enjoyable and efficient process I am also indebted to Dr Miguel Henry-Osorio for proofreading every character of every page of material and pointing out corrections, in addition to making some expositional suggestions that were very helpful to the revision process Mr Sherzod Akhundjanov also provided expert proof-checking, for which I am very grateful I also thank Haylee and Hanna Gecas for their constant monitoring of my progress on the book revision and for making sure that I did not stray too far from the targeted timeline for the effort I also wish to thank my colleague Dr Tom Marsh, who utilized the first edition of this book for many years in the teaching of his econometrics classes and who provided me with helpful feedback on student learning from and topic coverage in the book Finally, a deep thank you for the many comments and helpful suggestions I continued to receive over the years from my many doctoral students, the students who attended my statistics and econometrics classes here at the university; the many additional questions and comments I received from students elsewhere; and the input received from a host of individuals all over the world – the revision of the book has benefitted substantially from your input Thank you all n
Pn 2  N b; s n : □ i¼1 zi 272 Chapter Basic Asymptotics 5.8.4 Multivariate Central Limit Theorems The central limit theorems presented so far are applicable to sequences of random scalars Central limit theorems can be defined for sequences of random vectors, in which case conditions are established that ensure that an appropriate (vector) function of the random sequence converges in distribution to a multivariate normal distribution Due to a result discovered by H Cramer and H Wold,16 termed the Cramer-Wold device, questions of convergence in distribution for a multivariate random sequence can all be reduced to the question of convergence in distribution of sequences of random scalars, at least in principle Thus, all of the central limit theorems discussed to this point remain highly relevant to the multivariate case Theorem 5.36 Cramer-Wold Device The sequence of (k  1) random vectors {Xn} converges in distribution to the d random (k1) vector X iff ℓ0 Xn !ℓ0 X 8ℓ∈ℝk d Note that in applying Theorem 5.36, ℓ0 Xn !ℓ0 X is always trivially true when d ℓ ¼ 0, and so the condition ℓ0 Xn !ℓ0 X need only be checked for ℓ 6¼ We will be most concerned with convergence in distribution to members of the normal family of distributions In this context, Theorem 5.36 implies that to establish convergence in distribution of the sequence of random (k  1) vectors {Xn} to the d random (k  1) vector X ~ N(m,S), it suffices to demonstrate that ℓ Xn ! N(ℓ m, ℓ Sℓ) 8ℓ∈ℝk We formalize this observation as a corollary to Theorem 5.36 Corollary 5.4 CramerWold Device for Normal Limiting Distributions d d Xn ! N(m, S) iff ℓ0 Xn !N(ℓ0 m, ℓ0 Sℓ) 8ℓ∈ℝk The Cramer-Wold device can be used to define multivariate central limit theorems The following is a multivariate extension of the Lindberg-Levy CLT Theorem 5.37 Multivariate LindbergLevy CLT Let {Xn} be a sequence of iid (k  1) random vectors withE(Xi) ¼ m andCov(Xi) ¼ S i, where S is a (k  k) positive definite matrix Then n1=2 n1 n P i¼1 d Xi  m !N(0, S) a n  It follows from the multivariate Lindberg-Levy CLT that X N m; n1 S Example 5.45 CLT Applied to sum of Bernoulli vectors 16 Shipments of CPU chips from two different suppliers are to be inspected before being accepted Chips are randomly drawn, with replacement, from each shipment and are nondestructively tested in pairs, with (X1i, X2i) representing the outcome of the tests for pair i An outcome of x‘i ¼ indicates a faulty chip, x‘i ¼ indicates a nondefective chip, and the joint density of (X1i, X2i) is given by H Cramer and H Wold, (1936) Some Theorems on Distribution Functions, J London Math Soc., 11(1936), pp 290–295 5.9 Asymptotic Distributions of Differentiable Functions of Asymptotically Xi ¼ X 1i X 2i 273  px11i ð1  p1 Þ1x1i I f0;1g ðx 1i Þ px22i ð1  p2 Þ1x2i I f0;1g ðx2i Þ; where pi ∈ (0,1) for i ¼ 1, Note that p1 EXi ị ẳ ẳ p, and p2 p1 ð1  p1 Þ : CovðXi Þ ¼ S ¼ p2 ð1  p2 Þ n  n ¼ n1 P X 1i ; it follows from the multivariate Lindberg-Levy Letting X X 2i ð21Þ  i¼1  d a 1=2  n  N p; n1 S Xn  p !Z  N0; Sị and also that X CLT that Zn ẳ n If one were interested in establishing an asymptotic distribution for the difference in the number of defectives observed in n random pairs of  nÞ CPUs from shipments and 2, the random variable of interest would be c0 ðnX Pn Pn 0 1=2  ¼ i¼1 X 1i¼1  i¼1 X 2i ; , where c ¼ [1 1] Then c ðnXn Þ ¼ gðZn ; nị ẳ c ẵn Zn a  1=2  n  ỵnp, so that by Definition 5.2, c0 nX c n Z ỵ np  Nnc0 p; nc0 ScÞ Thus, the asymptotic distribution for the difference in the number of defectives is given a  nÞ  N np1  p2 ị; nẵp1  p1 ị ỵ p2 ð1  p2 ÞÞ □ by c0 ðnX Another useful multivariate CLT concerns the case where the elements in the sequence {Xn} are independent but not necessarily identically distributed (k  1) random vectors that exhibit uniform (i.e., across all n) absolute upper bounds with probability Theorem 5.38 Multivariate CLT for Independent Bounded Random Vectors Let {Xn} be a sequence of independent (k1) random vectors such that p( jx1i j m, jx 2i jm, ., jx ki j m) ¼ i, where m ∈(0,1) Let E(Xi) ¼ mi, Cov(Xi) ¼ Ci, P and suppose that limn!1 n1 ni¼1 Ci ¼ C , a finite positive definite (k  k) n P d ðXi  mi Þ!Nð0; CÞ: matrix Then n1=2 i¼1 Various other multivariate CLTs can be constructed using the Cramer-Wold device and CLTs for random scalars In practice, one often relies on the CramerWold device directly for establishing limiting distributions relating to statistical procedures of interest, and so we will not attempt to compile a list of additional multivariate CLTs here 5.9 Asymptotic Distributions of Differentiable Functions of Asymptotically Normally Distributed Random Variables: The Delta Method In this section we examine results concerning the asymptotic distributions of differentiable functions of asymptotically normally distributed random variables General conditions will be identified for which differentiable functions of asymptotically normally distributed random variables are 274 Chapter Basic Asymptotics themselves asymptotically normally distributed The utility of these results in practice is that once the asymptotic distribution of Xn is known, the asymptotic distributions of interesting functions of Xn need not be derived anew Instead, these asymptotic distributions can generally be defined by specifying the mean and covariance matrix of a normal distribution according to well-defined and straightforwardly implemented formulas All of the results that we will examine in this section are based on first-order Taylor series expansions of the function g(x) around a point m Being that the methods are based on derivatives, the methodology has come to be known as the delta method for deriving asymptotic distributions and associated asymptotic covariance matrices of functions of random variables We review the Taylor series expansion concept here, paying particular attention to the nature of the remainder term Recall that d(x,m) ¼ [(xm)0 (xm)]1/2 represents the distance between the vectors x and m Definition 5.11 First-Order Taylor Series Expansion and Remainder (Young’s Form) Let g: D ! ℝ be a function having partial derivatives in a neighborhood of the point m∈D that are continuous at m Let G ¼ ½@gðmÞ=@x1 ; ; @gðmÞ=@x k  represent the  k gradient vector of g(x) evaluated at the point x ¼ m Then for x∈D, g(x) ¼ g(m) + G(xm) + d(x, m)R(x) The remainder term R(x) is continuous at x ¼ m, and limx!m R(x) ¼ R(m) ¼ Young’s form of Taylor’s theorem is not prevalent in calculus texts The reader can find more details regarding this type of expansion in G.H Hardy (1952), A Course of Pure Mathematics, 10th ed., Cambridge, New York, The University Press, p 278, for the scalar case, and T.M Apostol (1957), Mathematical Analysis Cambridge, MA: Addison-Wesley, pp 110 and 118 for the multivariate case Our first result on asymptotic distributions of g(x) concerns the case where g(x) is a scalar-valued function As will be common to all of the results we will examine, the principal requirement on the function g(x) is that partial derivatives exist in a neighborhood of the point m and that they are continuous at m so that Lemma 5.6 can be utilized In addition, we will also make assumptions relating to the nature of the asymptotic distribution of X Theorem 5.39 Asymptotic Distribution of g(Xn) (Scalar Function Case) d Let {Xn} be a sequence of k  random vectors such that n1/2(Xn  m) ! Z ~ N(0,S) Let g(x) have first-order partial derivatives in a neighborhood of the point x ¼ m that are continuous at m, and suppose the gradient vector of g(x) evaluated at x ẳ m, G1kị ẳ ẵ@gmị=@x @gðmÞ=@xk  , is not the zero vector d a Then n1/2(g(Xn)  g(m)) !N(0,GSG0 ) and g(Xn)  N(g(m), n1GSG0 ) Example 5.46 Asymptotic Distribution of X n ð1  X n Þ for IID Bernoulli RVs Note from Example 5.39 that if {Xn} is a sequence of iid Bernoulli-type random d  n p) ! variables, then for p 6¼ or 1, n1/2 ( X N(0, p(1p)) Consider using an  ¼X  (1  X)  as an estimate of the variance p(1p) of the Bernoulli outcome of g(X)  n ) PDF, and consider defining an asymptotic distribution for g(X 5.9 Asymptotic Distributions of Differentiable Functions of Asymptotically 275  Theorem 5.39 applies with m ¼ p and S ¼ s2 ¼ p(1  p) Note that dg(p)/dX ¼ 12p, which is continuous in p and is nonzero so long as p 6¼ Also, s 6¼ if p 6¼ or Then for p 6¼ 0, 5, or 1, Theorem 5.39 implies that  a
n  n  X N pð1  pÞ; n1 ð1  2pÞ2 pð1  pÞ : X  n (1  X  n ) under the assumption p ¼ 1/2 can be An asymptotic density for X established using other methods (see Bickel and Doksum (1977), Mathematical Statistics, San Francisco: Holden-Day, p 53); however, convergence is not to a  d n  X  n  ð1=4Þ ! normal distribution Specifically, it can be shown that n X Z, where Z has the density of a w21 random variable that has been multiplied by (1/4) If p ¼ or p ¼ 1, the Xi’s are all degenerate random variables equal to  n is then degenerate at or or 1, respectively, and the limiting density of X as well □ By reinterpreting g(x) as a vector function and G as a Jacobian matrix, the conclusion of Theorem 5.39 regarding the asymptotic distribution of the vector function remains valid The extension allows one to define the joint asymptotic distribution of the random vector g(Xn) Theorem 5.40 Asymptotic Distribution of g(Xn) (Vector Function Case) d Let {Xn} be a sequence of k1 random vectors such that n1/2(Xn  m) ! Z ~ N(0, S) Let g(x) ¼ (g1(x), .,gm(x))0 be an (m1) vector function (m  k) having first order partial derivatives in a neighborhood of the point x ¼ m that are continuous at m Let the Jacobian matrix of g(x) evaluated at x ¼ m, 3 @g1 ðmÞ ðmÞ @g@x @g1 ðmÞ=@x0 @x1 k 7 7¼6 G ¼6 ; mk @gm ðmÞ @gm ðmÞ @gm ðmÞ=@x @x1 @xk have full row rank Then d a n1/2(g(Xn)  g(m)) !N(0,GSG0 ) and g(Xn)  N(g(m), n1 GSG0 ) Example 5.47 Asymptotic Distribution of Products and Ratios
d ^ n} be a sequence of (21) random vectors such that n1=2 b ^n  b ! Let {b N(0,S), where b ¼ [2 1]0 and S ¼ We seek an asymptotic distribution for the 1

0 ^½1b ^½2 b ^½2=b ^½1 : All of the conditions of Theorem ^ ¼ 3b vector function g b 5.40 are met, including the fact that b1 b2 dg1 bị=db0 G ẳ ẳ ẳ  b2 = b21 1= b1 1=4 dg2 ðbÞ=db0 22 1=2 has full row rank (note that the partial derivatives exist in an open rectangle containing b, and they are continuous at the point b) Then, since g(b) ẳ [6 ẵ] 90 1:5 and GSG0 ¼ , it follows from Theorem 5.40 that 1:5 :125 276 Chapter Basic Asymptotics " ^ nị ẳ gb ^ ẵ1 b ^ ẵ2 3b n n ^ ^ b ½2= b ½1 n # a  N n  90 ; n1 1:5 1=2 1:5 :125  : A specific distribution  is obtained once n is specified For example, if n ¼ 20,
 4:5 :075 a ^ 20  N ; : □ then g b 1=2 :075 :00625 The final result that we will examine concerning the asymptotic distribution of g(Xn) generalizes the previous two theorems to cases for which d V1=2 (Xnm) ! N(0,I), where {Vn} is a sequence of (mm) positive definite n matrices of real numbers such that Vn ! 0.17 Note this case subsumes the ðXn  mÞ previous cases upon defining Vn ¼ n1 S, in which case V1=2 n d ¼ S1=2 n1=2 ðXn  mÞ ! N(0,I) by Slutsky’s theorem The generalization allows additional flexibility in how the asymptotic distribution of Xn is initially established and is especially useful in the context of the least squares estimator to be discussed in Chapter Theorem 5.41 Asymptotic Distribution of g(Xn) (Generalized) d ðXn  mÞ!N(0,I), Let {Xn} be a sequence of (k1) random vectors such that V1=2 n where {Vn} is a sequence of (mm) positive definite matrices for which Vn ! Let g(x) be a (m1) vector function satisfying the conditions of Theorem 5.40 If there exists a sequence of positive real numbers {an} such that {[anGVnG0 ]1/2} is O(1) and d a an (Xn  m) is Op(1), then (GVnG)1/2 [g(Xn)  g(m)] !N(0,I) and g(Xn)  N(g(m), GVnG0 ) 1=2 5.10 17 Appendix: Proofs and Proof References for Theorems Theorem 5.1 The discrete case is left to the reader For the continuous case, see H Scheffe´, (1947), “A useful convergence theorem for probability distributions,” Ann Math Stat., 18, pp 434–438 Theorem 5.2 See E Lukacs (1970), Characteristic Functions, London: Griffin, pp 49–50, for a proof of this theorem for the more general case characterized by convergence of characteristic functions (which subsumes Theorem 5.2 as a special case) In the multivariate case, t will be a vector, and convergence of the MFG must hold 8ti∈(h,h), and 8i 1=2 Recall that V1=2 is the symmetric square root matrix of Vn, and V1=2 is the inverse of V1=2 is that n n n The defining property of Vn 1=2 1=2 1 ¼ Vn , while Vn Vn ¼ Vn Vn1=2 Vn1=2 5.10 Appendix: Proofs and Proof References for Theorems 277 Theorem 5.3 See the proof of Theorem 5.17 and R Serfling (1980), Approximation Theorems of Mathematical Statistics, New York: Wiley, pp 24–25 Theorem 5.4 See the proof of Theorem 5.16 Theorem 5.5 See the proof of Theorem 5.17 and R Serfling, op cit., pp 24–25 Theorem 5.6 All of the results follow from Theorem 5.5, and the fact that the functions being analyzed are continuous functions Note in particular that the matrix inverse function is continuous at all points for which the matrix is nonsingular Theorem 5.7 Y.S Chow and H Teicher (1978), Probability Theory, New York: SpringerVerlag, New York, p 249 Corollary 5.1 This follows immediately from Theorem 5.7 upon defining Xn¼X¼Y 8n d Theorem 5.8: Proof Let {Yn} be a sequence of scalar random variables and suppose Yn ! c, so that Fn(y) ! F(y) ¼ IA(y), where A ¼ {y: y  c} Then as n ! 1, P(|yn  c| < e)  Fn(c + t)  p Fn(c  t) ! 1, for t (0,e) and 8e > 0, which implies that Yn ! c The multivariate case can be proven similarly using marginal CDFs and the elementwise nature of the plim operator n Theorem 5.9 This follows from the proof in V Fabian and J Hannon (1985), Introduction to Probability and Mathematical Statistics, New York: John Wiley, p 159, and from Theorem 5.4 Theorem 5.10: Proof Each function on the left-hand side of (a), (b), and (c) is of the form g(Xn, Yn, an) and satisfies the conditions of Theorem 5.9, with an being a ghost in the definition of the function g n Theorem 5.11: Proof This follows directly from definitions of orders of magnitude in probability upon interpreting the sequence of real numbers or matrices as a sequence of degenerate random variables or random matrices n Theorem 5.12: Proof We provide a proof for the scalar case, which suffices to prove the matrix case given the elementwise definition of mean-square convergence (Definition 5.6.b) Necessity a E(Yn) ! E(Y) follows from the fact that

1=2 ! 0: jEðY n ị  EYịj ẳ jEY n  Yịj  EðjY n  YjÞ  E jY n  Y j2 To see this, note that the first inequality follows because ðyn yÞ  j y n yj Regarding the second inequality, note that g(z) ¼ z2 is a convex function on ℝ, and letting Z ¼ |Yn  Y|, Jensen’s inequality implies (E(|YnY|))2  E(|Yn  Y|2) (recall g(E(Z))  E(g(Z)) Convergence to zero occurs because E(|YnY|2) ¼ E((YnY)2) ! by convergence in mean square 278 Chapter Basic Asymptotics b E( Yn2 ) ¼ E((Yn  Y)2) ỵ E(Y2) ỵ 2E(Y(Yn  Y)), and since by the Cauchyh ... 4.2 .1 Family Name: Uniform 4.2.2 Family Name: Gamma, and Exponential and Chi-Square Subfamilies xvii 12 5 12 9 13 1 13 2 13 8 13 9 14 1 14 2 14 4 14 6 14 7 14 8 14 9 15 0 15 3 15 7 16 2 16 2 16 4 16 4 16 5 16 6 17 5 17 6... Uniform density, N ¼ Bernoulli density, p ¼ Binomial density, n ¼ 5, p ¼ 0.3 Partial poisson density, l ¼ 4.5 20 30 46 48 55 63 64 65 66 73 11 2 11 2 11 4 11 5 11 6 13 5 13 6 13 7 13 8 15 5 15 6 16 4 17 7 17 8... 87 89 90 90 91 94 95 96 98 10 0 10 1 11 1 11 1 11 7 11 8 12 2 12 2 12 3 Contents 3.5 Conditional Expectation 3.5 .1 Regression Function in the Bivariate Case 3.5.2 Conditional Expectation and Regression