Class Notes in Statistics and Econometrics Part 27 pdf

CHAPTER 53 Errors in Variables 53.1. The Simplest Errors-in-Variables Model We will explain here the main principles of errors in variables by the example of simple regression, in which y is regressed on one explanatory variable with a constant term. Assume the explanatory variable is a random variable, called x ∗ , and the disturbance term in the regression, which is a zero mean random variable independent of x ∗ , will be called v. In other words, we have the following relationship between random variables: (53.1.1) y = α + x ∗ β + v. 1099 1100 53. ERRORS IN VARIABLES If n observations of the variables y and x ∗ are available, one can obtain estimates of α and β and predicted values of the disturbances by running a regression of the vector of observations y on x ∗ : (53.1.2) y = ια + x ∗ β + v. But now let us assume that x ∗ can only be observed with a random error. I.e., we observe x = x ∗ +u. The error u is assumed to have zero mean, and to be independent of x ∗ and v. Therefore we have the model with the “latent” variable x ∗ : y = α + x ∗ β + v(53.1.3) x = x ∗ + u(53.1.4) This model is sometimes called “regression with both variables subject to error.” It is symmetric between the dependent and the explanatory variable, because one can also write it as y ∗ = α + x ∗ β(53.1.5) x = x ∗ + u(53.1.6) y = y ∗ + v(53.1.7) and, as long as β = 0, y ∗ = α + x ∗ β is equivalent to x ∗ = −α/β + y ∗ /β. 53.1. THE SIMPLEST ERRORS-IN-VARIABLES MODEL 1101 What happens if this is the true model and one regresses y on x? Plug x ∗ = x−u into (53.1.2): (53.1.8) y = ια + xβ + (v − uβ)    ε The problem is that the disturbance term ε is correlated with the explanatory vari- able: (53.1.9) cov[x, ε] = cov[x ∗ + u, v −uβ] = −β var[u]. Therefore OLS will give inconsistent estimates of α and β: ˆ β OLS =  (y i − ¯y)(x i − ¯x)  (x i − ¯x) 2 (53.1.10) plim ˆ β OLS = cov[y, x] var[x] = β  1 − var[u] var[x]  .(53.1.11) Since var[u] ≤ var[x], ˆ β OLS will have the right sign in the plim, but its absolute value will understimate the true β. Problem 468. 1 point [SM86, A3.2/3] Assume the variance of the measurement error σ 2 u is 10% of the variance of the unobserved exogenous variable σ 2 x ∗ . By how 1102 53. ERRORS IN VARIABLES many percent will then the OLS estimator ˆ β OLS asymptotically underestimate the absolute value of the true parameter β? Answer. 1 − var[u]/ var[x] = 1 − 0.1/1.1 = 0.90909, which is 9.09% below 1.  Although ˆ β OLS is not a consistent estimator of the underlying parameter β, it nevertheless converges towards a meaningful magnitude, namely, the best linear predictor of y on the basis of x, which characterizes the empirical relation between x and y in the above model. What is the difference between the underlying structural relationship between the two variables and their empirical relationship? Assume for a moment that x is observed and y is not observed, but one knows the means  E[x] E[y]  and the covari- ance matrix  var[x] cov[x, y] cov[x, y] var[y]  . Then the best linear predictor of y based on the observation of x is (53.1.12) y ∗ = E[y] + cov[y, x] var[x] (x −E[x]) =  E[y] − cov[y, x] var[x] E[x]  + cov[y, x] var[x] x This is a linear transformation of x, whose slope and intercept are not necessarity equal to the “underlying” α and β. One sees that the slope is exactly what ˆ β OLS converges to, and question 469 shows that the intercept is the plim of ˆα OLS . 53.1. THE SIMPLEST ERRORS-IN-VARIABLES MODEL 1103 Problem 469. Compute plim ˆα OLS . Is it possible to estimate the true underlying parameters α and β consistently? Not if they are jointly normal. For this look at the following two scenarios: y ∗ = 2x ∗ − 100 x ∗ ∼ N (100, 100) v ∼ N (0, 200) u ∼ N(0, 200) versus y ∗ = x ∗ x ∗ ∼ N (100, 200) v ∼ N (0, 400) u ∼ N(0, 100) (53.1.13) They lead to identical joint distributions of x and y, although the underlying param- eters are different. Therefore the model is unidentified. Problem 470. Compute means, variances, and the correlation coefficient of x and y in both versions of (53.1.13). Answer. First the joint distributions of y ∗ and x ∗ : (53.1.14)  y ∗ x ∗  ∼ N   100 100  ,  400 200 200 100   versus  y ∗ x ∗  ∼ N   100 100  ,  200 200 200 200   . Add to this the independent (53.1.15)  v u  ∼ N   0 0  ,  200 0 0 200   versus  v u  ∼ N   0 0  ,  400 0 0 100   . 1104 53. ERRORS IN VARIABLES  Problem 471. Compute a third specification of the underlying relationship be- tween x ∗ and y ∗ , the mean and variance of x ∗ , and the error variances, which leads again to the same joint distribution of x and y, and under which the OLS estimate is indeed a consistent estimate of the underlying relationship. 53.1.1. Three Restrictions on the True Parameteres. The lack of identi- fication means that the mean vector and dispersion matrix of the observed variables are compatible with many different values of the underlying parameters. But this lack of identification is not complete; the data give three important restrictions for the true parameters. Equation (53.1.5) implies for the means  µ y µ x  =  µ y ∗ µ x ∗  =  α + µ x ∗ β µ x ∗  ,(53.1.16) and variances and covariances satisfy, due to (53.1.6) and (53.1.7),  σ 2 y σ xy σ xy σ 2 x  =  σ 2 y ∗ σ x ∗ y ∗ σ x ∗ y ∗ σ 2 x ∗  +  σ 2 v 0 0 σ 2 u  =  β 2 σ 2 x ∗ βσ 2 x ∗ βσ 2 x ∗ σ 2 x ∗  +  σ 2 v 0 0 σ 2 u  .(53.1.17) 53.1. THE SIMPLEST ERRORS-IN-VARIABLES MODEL 1105 We know five moments of the observed variables: µ y , µ x , σ 2 y , σ xy , and σ 2 x ; but there are six independent parameters of the model: α, β, µ x ∗ , σ 2 x ∗ , σ 2 v , σ 2 u . It is therefore no wonder that the parameters cannot be determined uniquely from the knowledge of means and variances of the observed variables, as shown by counterexample (53.1.13). However α and β cannot be chosen arbitrarily either. The above equations imply three constraints on these parameters. The first restriction on the parameters comes from equation (53.1.16) for the means: From µ y ∗ = α + βµ x ∗ follows, since µ y = µ y ∗ and µ x = µ x ∗ , that (53.1.18) µ y = α + βµ x , i.e., all true underlying relationships compatible with the means and variances of the observed variables go through the same point  µ x µ y  . If σ xy = 0, this is the only restriction on the parameter vectors. To see this, remember σ xy = βσ 2 x ∗ . This product is zero if either σ 2 x ∗ = 0, or σ 2 x ∗ = 0 and β = 0. If σ 2 x ∗ = 0, then x ∗ and therefore also y ∗ are constants. Any two constants satisfy infinitely many affine relationships, and all α and β which satisfy the first constraint are possible parameter vectors which all describe the same affine relationship between x ∗ and y ∗ . In the other case, if σ 2 x ∗ = 0 and β = 0, then the linear relation underlying 1106 53. ERRORS IN VARIABLES the observations has coefficient zero, they are noisy observations of two linearly unrelated variables. In the regular case σ xy = 0, condition (53.1.17) for the dispersion matrices gives two more restrictions on the parameter vectors. From σ xy = βσ 2 x ∗ follows the second restriction on the parameters: (53.1.19) β must have the same sign as σ xy . And here is a derivation of the third restriction (53.1.23): from 0 ≤ σ 2 u = σ 2 x − σ 2 x ∗ and 0 ≤ σ 2 v = σ 2 y − β 2 σ 2 x ∗ (53.1.20) follows σ 2 x ∗ ≤ σ 2 x and β 2 σ 2 x ∗ ≤ σ 2 y .(53.1.21) Multiply the first inequality by |β| and substitute in both inequalities σ xy for βσ 2 x ∗ : |σ xy | ≤ |β|σ 2 x and |β||σ xy | ≤ σ 2 y (53.1.22) or |σ xy | σ 2 x ≤ |β| ≤ σ 2 y |σ xy | .(53.1.23) 53.1. THE SIMPLEST ERRORS-IN-VARIABLES MODEL 1107 The lower bound is the absolute value of the plim of the regression coefficient if one regresses the observations of y on those of x, and the reciprocal of the upper bound is the absolute value of the plim of the regression coefficient if one regresses the observed values of x on those of y. Problem 472. We have seen that the data generated by the two processes (53.1.13) do not determine the underlying relationship completely. What restrictions do these data impose on the parameters α and β of the underlying relation y ∗ = α + βx ∗ ? Problem 473. The model is y = α + x ∗ β + v, but x ∗ is not observed; one can only observe x = x ∗ + u. The errors u and v have zero expected value and are independent of each other and of x ∗ . You have lots of data available, and for the sake of the argument we assume that the joint distribution of x and y is known precisely: it is (53.1.24)  y x  ∼ N   1 −1  ,  6 −2 −2 3   . • a. 3 points What does the information about y and x given in equation (53.1.24) imply about α and β? Answer. (53.1.18) gives α − β = 1, (53.1.19) gives β ≤ 0, and (53.1.23) 2/3 ≤ |β| ≤ 3.  1108 53. ERRORS IN VARIABLES • b. 3 points Give the plims of the OLS estimates of α and β in the regression of y on x. Answer. plim ˆ β = cov[x, y]/ var[x] = − 2 3 , plim ˆα = E[y] − E[x]pl im ˆ β = 1 3 .  • c. 3 points Now assume it is known that α = 0. What can you say now about β, σ 2 u , and σ 2 v ? If β is identified, how would you estimate it? Answer. From y = (x − u)β + v follows, by taking expectations, E[y] = E[x]β (i.e., the true relationship still goes through the means), therefore β = −1, and a consistent estimate would be ¯y/¯x. Now if one knows β one gets var[x ∗ ] from cov[x, y] = cov[x ∗ +u, βx ∗ +v] = β var[x ∗ ], i.e., var[x ∗ ] = 2. Then one can get var[u] = var[x] − var[x ∗ ] = 3 − 2 = 1, and var[v] = var[y] − var[y ∗ ] = 6 − 2 = 4. Luckily, those variances came out to be positive; otherwise the restriction α = 0 would not be compatible with (53.1.24).  53.2. General Definition of the EV Model Given a n ×k matrix X whose c olumns represent the observed variables. These observations are generated by a linear EV model if the following holds: X = X ∗ + U,(53.2.1) X ∗ B = O(53.2.2) X ∗ is an n ×k matrix of the values of the unobserved “systematic” or “latent” vari- ables. We assume that Q ∗ = plim n→∞ 1 n X ∗  X ∗ exists. If the systematic variables [...]... form can always be achieved by rearranging the variables and/ or going over to linear combinations Any symmetric EV model is equivalent to one in which the parameter matrix has this form, after an appropriate linear transformation of the variables Partitioning the vectors of systematic variables and errors conformably, 1112 53 ERRORS IN VARIABLES one obtains the following form of the EV model: Y ∗ = X ∗B... constant term in the regressions, then Q and Q should not be considered to be the moments of the observed and systematic variables about the origin, but their covariance matrices We will use this rule extensively in the following examples 1126 53 ERRORS IN VARIABLES 53.5 Properties of Ordinary Least Squares in the EV model In the estimation of a univariate EV model it is customary to single out one... v + v v 2 σε = β Qβ + 2 σv and 53.5 ORDINARY LEAST SQUARES IN THE EV MODEL 1129 and 1 1 X ε = plim (X ∗ + U ) (−U β + v) = −Qβ n n Plugging (53.5.12) and (53.5.13) into (53.5.9) gives 1 2 (53.5.14) plim e e = σv + β (Q − QQ−1 Q)β n (53.5.13) plim Since Q − QQ−1 Q is nonnegative definite, this proves the first half of the inequality Problem 474 Assuming σ U v = o, show that in the plim, bOLS QbOLS ≤ β... Xγ + v; V ∼ (o, Ξ) with X and v independent (and again for simplicity all variables having zero mean), which gives the same joint distribution of X and y as the above specification Compute γ, V [xi ], and var[y i ] in terms of the structural data of the above specification Answer (53.5 .27) y ∗ = X ∗ β − Q−1 Qβ (53.5.28) X = X∗ (53.5.29) y = y∗ + v where x∗ i = xi ∼ N (o, Q) and var[v i ] = σ 2 +β Q∗... compare their theorem 4.1, and about the residual variance compare their theorem 4.3 1132 53 ERRORS IN VARIABLES 53.6 Kalman’s Critique of Malinvaud Problem 476 In Malinvaud’s econometrics textbook [Mal78] and [Mal70, pp 17–31 and 211–221], the following data about the French economy are used (all amounts in billion nouveaux francs, at 1959 prices): 53.6 KALMAN’S CRITIQUE OF MALINVAUD 1949 1950 1951 1952... coefficient in the linear relation to be estimated, and to normalize its coefficient to be −1 Writing this variable as the first variable, the symmetric form reads (53.5.1) y∗ X∗ y −1 =o β X = y∗ X∗ + v U ; but the usual way of writing this is, of course, y∗ = X ∗ β (53.5.2) y = y∗ + v X = X∗ + U In this situation it is tempting to write (53.5.3) y = Xβ + v − U β ε and to regress y on X This gives a biased and inconsistent... have nonzero variances and are uncorrelated, i.e., 1 if n U U is in the plim diagonal and nonsingular, then the identification problem of errors in the variables can be reduced to the following “Frisch Problem”: Given a positive definite symmetric matrix Q, how many ways are there to split it up as a sum (53.4.7) Q = Q∗ + Q where Q∗ is singular and nonnegative definite, and Q diagonal and positive definite?... following approach to estimation: first solve the Frisch problem in order to get an In other words, 1122 53 ERRORS IN VARIABLES estimate of the feasible parameter region compatible with the data, and then use additional information, not coming from the data, to narrow down this region to a single point The emphasis on the Frisch problem is due to Kalman, see [Kal82] Also look at [HM89] 53.4.2 “Sweeping... assumed to be nonsingular Defining Y ∗ = X ∗ BΓ−1 and V = EΓ−1 , 1114 53 ERRORS IN VARIABLES one can put this into the EV-form (53.3.5) (53.3.6) Y∗ X∗ Y −Γ =O B X = Y∗ X∗ + V O If one assumes that also X ∗ is observed with errors, one obtains a simultaneous equations model with errors in the variables The main difference between simultaneous equations systems and EV models is that in the former, identification... the kind (53.4.17) (in which the error variances of the first partition are constrained to be zero) and unconstrained Frisch decompositions of QZZ.X In this bijection, (53.4.17) corresponds to the decomposition 1124 53 ERRORS IN VARIABLES (53.4.20) QZZ.X = (Q∗ − QZX Q−1 QXZ ) + QZZ ZZ XX Proof: given that Q = QXX QZX QXZ QZZ is nonnegative definite, we have to QXX QXZ is nonnegative definite if and only . specification of the underlying relationship be- tween x ∗ and y ∗ , the mean and variance of x ∗ , and the error variances, which leads again to the same joint distribution of x and y, and under which the. is a random variable, called x ∗ , and the disturbance term in the regression, which is a zero mean random variable independent of x ∗ , will be called v. In other words, we have the following. CHAPTER 53 Errors in Variables 53.1. The Simplest Errors -in- Variables Model We will explain here the main principles of errors in variables by the example of simple regression, in which y is regressed