THE LINEAR REGRESSION MODEL I

CHAPTER 19 The linear regression model I — specification, estimation and testing 19.1 Introduction The linear regression model forms the backbone of most other statistical models of particular interest in econometrics A sound understanding of the specification, estimation, testing and prediction in the linear regression model holds the key to a better understanding of the other statistical models discussed in the present book In relation to the Gauss linear model discussed in Chapter 18, apart from some apparent similarity in the notation and the mathematical manipulations involved in the statistical analysis, the linear regression model purports to model a very different situation from the one envisaged by the former In particular the Gauss linear model could be considered to be the appropriate statistical model for analysing estimable models of the form M,=%+a,t+ M,=ws+ 3 ¡=1 =1 Y cOint Y diQiat k } dit’, ¡=1 (19.1) (19.2) where M, refers to money and Q,,, i= 1,2, to quarterly dummy variables, in view of the non-stochastic nature of the x,,s involved On the other hand, estimable models such as M = AY*®P2y®, (19.3) referring to a demand for money function (M ~ money, Ÿ - income, P ~ price level, J — interest rate), could not be analysed in the context of the Gauss 369 370 Specification, estimation and testing linear model This is because it is rather arbitrary to discriminate on probabilistic grounds between the variable giving rise to the observed data chosen for M and those for Y, P and J For estimable models such as (3) the linear regression model as sketched in Chapter 17 seems more appropriate, especially if the observed data chosen not exhibit time dependence This will become clearer in the present chapter after the specification of the linear regression model in Section 19.2 The money demand function (3) is used to illustrate the various concepts and results introduced throughout this chapter 19.2 Specification Let {Z,, t¢ T} bea vector stochastic process on the probability space (S, F P(-) where Z, =(y,,X;)’ represents the vector of random variables giving rise to the observed data chosen, with y, being the variable whose behaviour we are aiming to explain The stochastic process {Z,,teT} is assumed to be normal, independent and identically distributed (NUD) with E(Z,)=m and Cov(Z,=Z Le Vt (x) N my\(O11 xi 12 (19.4) rel 2?) in an obvious notation (see Chapter 15) It is interesting to note at this stage that these assumptions seem rather restrictive for most economic data in general and time-series in particular On the basis of the assumption that {Z,, te T} isa NIID vector stochastic process we can proceed to reduce the joint distribution D(Z,, , Z;; w) in order to define the statistical GM of the linear regression model using the general form where and V=n+u,, ted, H,= E(y,/X,;=x,)_ (19.5) is the systematic component, u,=y,—E(y,/X,=x,) the non-systematic component (see Chapter 17) In view of the normality of {Z,,t¢1T! we deduce that H, = E(y,/X,=X,)= Bo + B’x, (Hnear in x,), (19.6) where — and — -1 ño=my,—ø;,>;ym,, _đâ-=l1 =E;72Ă Var(u,/X,=x,)= Vat(y,X,=x,)=ứ (homoskedastic), (19.7) 19.2 Specification where ứ?=ứĂiĂứiz;2ứ; 371 (see Chapter 15) The time inoariance of the parameters Øạ, B and oa? stems from the identically distributed (ID) assumption related to {Z,,t¢ 1} It is important, however, to note that the ID assumption provides only a sufficient condition for the time invariance of the statistical parameters In order to simplify the notation let us assume the m=0 without any loss of generality given that we can easily transform the original variables in mean derivation form (y,—m,) and (X,—m,) This implies that Bp, the coefficient of the constant, is zero and the systematic component becomes E(y,/X, = X,) = B’X, (19.8) In practice, however, unless the observed data are in mean deviation form the constant should never be dropped because the estimates derived ores are not estimates of the regression coefficients B= ¢,, but of B* = E(X,X/) | E(X/y,}; see Appendix 19.1 on the role of the constant The stattical GM of the linear regression model takes the particular form y,=Bx,+u,, teT, (19.9) with =(B, o”) being the statistical parameters of interest; the parameters in terms of which the statistical GM is defined By construction the systematic and non-systematic components of (9) satisfy the following properties: (i) E(u,/X, = X;) = E[U,— E(y,/X,= X,))/X = E(y,/X, = x,) — FX, (ii) ==x,] = X,) =0; E(u,u,/X, = X,) = E[(y, — Ely, /X,=x,))(ys — E/X= X,))/X, =X] — pm, t=s 0, (iti) t#s; E(u,u,/X, = X,) = 1, E(u,/X,=x,)=0, tseT The first two properties define {u,, t€ T} to be a white-noise process and (iii) establishes the orthogonality of the two components It is important to note that the above expectation operator E(-/X,=x,) is defined in terms of D(y,/X,; 6), which is the distribution underlying the probability model for (9) However, the above properties hold for E(-) defined in terms of D(Z,; w) as weil, given that: (i)’ Elu,) = Ey E(u,/X,=x,)} =0; (ii) E(u,u,) = EY E(uu,/X,=x,)} = a, t=s 0, t#s; 372 Specification, estimation and testing and („ E(uu,)= Et(Euu,X,=x,)}=0, ĐseT (see Section 7.2 on conditional expectation) The conditional distribution D(y,/X,; 0) 1s related to the joint distribution D(y,, X„; ý) via the decomposition DỤ,,X„; /⁄)=D(yX, W,) D(X,; Wf) (19.10) (see Chapter 5) Given that in defining the probability model of the linear regression model as based on D(},/X,; 8) we choose to ignore D(X,; ;) for the estimation of the statistical parameters of interest @ For this to be possible we need to ensure that X, is weakly exogenous with respect to for the sample period t=1, 2, , T (see Section 19.3, below) For the statistical parameters of interest =(B, a7) to be well defined we need to ensure that £,, is non-singular, in view of the formulae B= Z3;'02,, 67 =6,; — 6,273 6>,, at least for the sample period t= 1, 2, , T: This requires that the sample equivalent of £,,,(1/T)(X’X) where X =(x,,X), pe x)’ is indeed non-singular, i.e rank(X’X) = rank(X) =k, X, being a k x | vector As argued in Chapter (19.11) 17, the statistical parameters of interest not necessarily coincide with the theoretical parameters of interest € We need, however, to ensure that € is uniquely defined in terms of for € to be identifiable In constructing empirical econometric models we proceed from a well-defined estimated statistical GM (see Chapter 22) to reparametrise it in terms of the theoretical parameters of interest Any restrictions induced by the reparametrisation, however, should be tested for their validity For this reason no a priori restrictions are imposed on @ at the outset to make such restrictions testable at a later stage As argued above, the probability model underlying (9) is defined in terms of D(y,/X,; 8) and takes the form =| Dux: 9-5 am exp 5g py" 0cñ*xR., ret}, (19.12) Moreover, in view of the independence of {Z,, te 7} the sampling model takes the form of an independent sample, y=(y,, ,7)’, sequentially drawn from D(y,/X,; 8), t=1,2, , T, respectively Having defined all three components of the linear regression model let us 19.2 Specification collect all the assumptions properly 373 together specify and statistical the model The linear regression model: specification (1) Statistical GM, y,= Bx, +u,, te T [1] tt, = E(y,/X,=x,) — the systematic component; u, = y, ~ Ely,/X,= X,) — the non-systematic component [2] are the statistical 0=(B, 07) B=Xz76n;, 07 =0,,—G 174374, [3] [4] [51 parameters of interest (Note: Z),=Cov(X,), 62; =Cov(X,, y,), Øi¡= Vat(y,).) X, is weakly exogenous with respect to #,r=1,2, ,T No a priori information on Rank(X)=k, X=(x,, X2, , X7); Tx k data matrix, (T'>k) (II) Probability model 6= _ Tw a |PUu/XzØ)= 21.) 9P| 20) = ÿx,) | 0=(ÿ.ø”)cửxIR,, ret} [6] [7] () (11) (ili) @ is time D(y,/X„; 9) is normal; E(y,/X,=X,) = B’x, — linear in x,; =o? — homoskedastic (free of x,); Var(y,/X,=X,) (IIT) Sampling model invariant y=(\;, , yy) represents an independent sample sequentially drawn from D(y,/X,; 9), t= 1, 2, TAn important point to note about the above specification is that the model is specified directly in terms of D(y,/X,; 6) making no assumptions about D(Z,; w) For the specification of the linear regression model there is no need to make any assumptions related to {Z,,t¢ Tj The problem, however, is that the additional generality gained by going directly to D(y,/X,; 0) is more apparent than real Despite the fact that the assumption that {Z,,t¢ 1} is a NHD process is only sufficient (not necessary) for [6] to [8] above, it considerably enhances our understanding of econometric modelling in the context of the linear regression model This is, firstly, [8] because it is commonly easier in practice to judge the appropriateness of 374 Specification, estimation and testing probabilistic assumptions related to Z, rather than (),/X,=x,); and, secondly, in the context of misspecification analysis possible sources for the departures from the underlying assumptions are of paramount importance Such sources can commonly be traced to departures from the assumptions postulated for {Z,,te T} (see Chapters 21-22) Before we discuss the above assumptions underlying the linear regression it is of some interest to compare the above specification with the standard textbook approach where the probabilistic assumptions are made in terms of the error term Standard textbook specification of the linear regression model y=Xÿ+u (1) (u/X) ~ N(O, ø?1,); (2) (3) no a priori information on (8, ø?); rank (X)=k Assumption (1) implies the orthogonality E(X{u,/X,=x,)=0,t=1,2, , T, and assumptions [6] to [8] the probability and the sampling models respectively This is because (y/X) is a linear function of uand thus normally distributed (see Chapter 15), ie (y/X) ~~ N(XB, o°I,) (19.13) As we can see, the sampling model assumption of independence is ‘hidden’ behind the form of the conditional covariance o7/ Because of this the independence assumption and its implications are not clearly recognised in certain cases when the linear regression model is used in econometric modelling As argued in Chapter 17, the sampling model of an independent sample is usually inappropriate when the observed data come in the form of aggregate economic time series Assumptions (2) and (3) are identical to [4] and [5] above The assumptions related to the parameters of interest Ø=(, ø?) and the weak exogeneity of X, with respect to ([2] and [3] above) are not made in the context of the standard textbook specification These assumptions related to the parametrisation of the statistical GM play a very important role in the context of the methodology proposed in Chapter | (see also Chapter 26) Several concepts such as weak exogeneity (see Section 19.3, below) and collinearity (see Sections 20.5-6) are only definable with respect to a given parametrisation Moreover, the statistical GM is turned into an econometric model by reparametrisation, going from the statistical to the theoretical parameters of interest The most important difference between the specification [1]-[8] and (1)H{3), however, is the role attributed to the error term In the context of the 19.3 Discussion of the assumptions 375 latter the probabilistic and sampling model assumptions are made in terms of the error term not in terms of the observable random variables involved as in [1]-[8] This difference has important implications in the context of misspecification testing (testing the underlying assumptions) and action thereof The error term in the context of a statistical model as specified in the present book is by construction white-noise relative to a given information set ACF 19.3 Discussion of the assumptions [1] The systematic and non-systematic components As argued in Chapter 17 (see also Chapter 26) the specification of a statistical model is based on the joint distribution of Z,,t=1,2, , Tie D(Z,,Z3, , 275, )= Dữ»: ý) (19.14) which includes the relevant sample and measurement information The specification of the linear regression model can be viewed as directly related to (14) and derived by ‘reduction’ using the assumptions of normality and IID The independence assumption enables us to reduce D(Z; W) into the product of the marginal distributions D(Z,; w,), t= 1,2, , T, Le (19.15) D(Z; p) = H DứZ: Ú,) The identical distribution enables us to deduce that ,= for t= 1,2, , The next step in the reduction is the following decomposition of D(Z,; p): D(Z,; ÿ)= D(y,/X,; ÿ¡): DĨ Ú:) (19.16) The normality assumption with >0 and unrestricted enable us to deduce the weak exogeneity of X, relative to @ The choice of the relevant information set Y,={X,=x,} depends crucially on the NID assumptions; if these assumptions are invalid the choice of Y, will in general be inappropriate Given this choice of Y, the systematic and non-systematic components are defined by: E(y,/X,=%,), u,=y,— E(y/X,=X,) (19.17) Under the NITD assumptions y, and u, take the particular forms: HÈ=fX, u*=y,—X, (19.18) 376 Specification, estimation and testing Again, if the NIID u“u* assumptions and are invalid then E(u*u*/X,=x,)#0 (19.19) (see Chapters 21-22), [2] The parameters of interest As discussed in Chapter 17, the parameters in terms of which the statistical GM is defined constitute by definition the statistical parameters of interest and they represent a particular parametrisation of the unknown parameters of the underlying probability model In the case of the linear regression model the parameters of interest come in the form of 0=(B, a’) where =Š;ÿø;, ø?=0ii—Ø¡;Ÿ;jø¿, AS argued above the parametrisation depends not only on D(Z; W) but also on the assumptions of NIID Any changes in Z, or/and the NID assumptions will in general change the parametrisation [3] Exogeneity In the linear regression model we begin with D(y,, X,; w) and then we concentrate exclusively on D(y,/X,;,) where D(y,,X,/) = D(y,/X5 Wy) D(X; WH), (19.20) which implies that we choose to ignore the marginal distribution D(X,;w,) In order to be able to that, this distribution must contain no information relevant for the estimation of the parameters of interest, 0=(B,c7), i.e the stochastic structure of X, must be irrelevant for any inference on @ Formalising this intuitive idea we say that: X, 1s weakly exogenous over the sample period for @ if there exists a reparametrisation with ý=(Ú¡.¿) such that: (i) (ii) is a function of w, (@=h(y,)); yw, and y, are variation free ((w,, ¥,)eP, x W,) Variation free means that for any specific value ý; in ‘P,, w, cau take any other value in ‘¥, and vice versa For more details on exogeneity see Engle, Hendry and Richard (1983) When the above conditions are not satisfied the marginal distribution of X, cannot be ignored because it contains relevant information for any inference on Ø 19.3 [4] Discussion of the assumptions 377 No a priori information on =(B 0°) This assumption is made at the outset in order to avoid imposing invalid testable restrictions on At this stage the only relevant interpretation of is as statistical parameters, directly related to W, in D(y,/X,; #1) As such no a priori information seems likely to be available for Such information is commonly related to the theoretical parameters of interest § Before is used to define &, however, we need to ensure that the underlying statistical model is well defined (no misspecification) in terms of the observed data chosen [5] The observed data matrix X is of full rank For the data matrix X =(x,, X), x7), Tx k, we need to assume that rank(X)=k, k