Chapter 04_Estimation By Instrumental Variables

Advanced Econometrics Chapter 4: Estimation By Instrumental Variables Chapter ESTIMATION BY INSTRUMENTAL VARIABLES (Instrumental Variable Estimators) I ENDOGENEITY: Now suppose ε, X are not independently generated: Cov(ε,X)≠ and E(ε|X) ≠ There are sources of this problem: Errors in measurement of independent variables: Suppose that the true regression equation is given by: yi = β0 + β1xi + εi where E(εi) = E(εixi) = Note: Cov(ε i , xi ) = E[ε i ( xi − x )] = E (ε i , xi ) − E (ε i , x ) = E (ε i , xi )    So if Cov(ε i , xi ) = ↔ E (ε i , xi ) = Suppose x *i = xi + ei Assume: → E(ei) = E(eixi) = estimate: yi = β0 + β1xi* + ui where: ui = εi - β1ei → correlated with x *i = xi + ei through terms ei Cov (ui , xi* ) ≠ Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables Variables on both sides of regression equation are jointly determined (endogenous) → RHS variables are endogenous hi = β + β1ei + ui  ei = α + α1hi + ε i → ei = α + α1 β α1 ui + εi + − α1 β1 − α1 β1 − α1 β1 → Cov(ui , ei ) ≠ Omitted variables: wi = β + β1 si + β + ε i Estimate: wi = β + β1 si + ui Where: ui = β + ε i , if and si are correlated → Cov(ui , si ) ≠ Lagged dependent variables (Yt-1) as a regressor and auto correlated errors Yt = α + βX t + λYt −1 + ε t   → Cov (ε t , Yt −1 ) ≠ because Yt-1 and εt both contain εt-1 ε t = ρε t −1 + ut  Model: (1) Y = X β + ε n ×k (2) X and ε are not generated independently (3) E(ε|X) ≠ (4) E(εε'|X) = σ2I (5) X consists of stationary random variables with: E  X i X i′  =  n ×k 1×k  Nam T Hoang University of New England - Australia p lim( XX ′ ) = Σ XX n University of Economics - HCMC - Vietnam Advanced Econometrics Now Chapter 4: Estimation By Instrumental Variables p lim( X ′ε ) = γ ≠ and n p lim βˆ = β + Σ −XX1 p lim( X ′ε ) = β + Σ −XX1 γ ≠ β n     ≠0 → βˆ is an inconsistent estimator βˆ is also no longer unbiased E ( βˆ X ) = β + ( X ′X ) −1 X ′E (ε X ) ≠ β  ≠0 II ESTIMATION BY INSTRUMENTAL VARIABLES: Suppose we can find a set of k variables W that have two properties: n ×k Exogeneity (validity): They are uncorrelated with the disturbance ε Relevance: They are correlated with the independent variable X Such that:  E (ε W ) = → E ( w' ε ) =   p lim W ' ε =  n   1   E  n W 'W  = E (WiWi′) = ΣWW      p lim n W ' X = ΣWW (W & X are stationary random variables) Then W is a set of instrumental variables and we define: βˆ IV = (W ' X ) −1W ' Y Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables βˆ IV : IV estimator Consistency: IV estimator βˆ IV is consistent: βˆ IV = (W ' X ) −1W ' Y = (W ' X ) −1W ' ( Xβ + ε ) −1 βˆ IV  W ' X   W 'ε  = β +     n   n  (Slutsky theorem) −1 W' X   W 'ε  p lim βˆ IV = β + p lim  p lim  n   n     −1 = β + ΣWX = β IV estimator is unbiased ( ) EW βˆ IV = β + E (W ' X )−1 E (W ' )E (ε W ) = β   −1 ΣWX III TWO-STAGE LEAST SQUARES ESTIMATION: W n ×k  E (ε X ) ≠   E (ε W ) = = p lim W ' ε  n    p lim W 'W = ΣWW n   p lim W ' X = ΣWX non singular  n Now we have a set of instruments Z , that are unrelated to ε n×q X consists two parts:  X =  X1 n×k n×( k −r )  X2 n×r  Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables X1: exogenous variables X2: endogenous variables Note: q must be ≥ r (if q < r → (W'W)-1 doesn't exist Z includes X1, We can define reduced form equations for X2: X2 = Z Π+ V n×r n×q q×r  X =  X 21 n×k  n×1  Π = Π q×r  q×1 So: n×r  X 22  X 2r    Π2  Πr    V = V1 V2 n×r  n×1   Vr    X 21 = ZΠ + V1   X = ZΠ + V2     X 2r = Z Π r + Vr  n ×1 n × q q×1 n ×1 Estimate this system by OLS, Π are estimators: q× r X  n ×12 X 22  X 2r   ˆ + Vˆ = Z Π n×q q×r ˆ = Z Π  q×1 n×r [ ˆ2  Π ˆ r  + Vˆ Vˆ Π    Vˆr ] ˆ → Xˆ is a good instrument Then we get: Xˆ = ZΠ ˆ , Cov( Z , ε ) = 0) ( Xˆ is correlated with X2 but not correlated with ε because Xˆ = ZΠ • [ ] After the first stage we get the set W = X Xˆ Apply OLS on [Y W ] we have: Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables βˆ2 SLS = (W 'W ) −1W ' Y → two-stage least squares estimator • [ ] We can also use W = X Xˆ as an instrument variable and get: βˆ IV = (W ' X ) −1W ' Y Π = ( Z ' Z ) −1 Z ' X We can show that: βˆ IV = βˆ2 SLS IV ASYMPTOTIC DISTRIBUTION OF βˆIV −1 1  1  n ( βˆ IV − β ) =  W ' X   W ' ε  n n  n   −1  = ΣWX  W 'ε  n n  1  w   i2  Wi =  wi       wik  n W ' ε = ∑Wi ε i i =1 E ( wi ε i ) = E ( wi ) E (ε i w) = Var ( wi ε i ) = E ( wi ε i ε i wi′ ) = σ ε2 E ( wi wi′ ) = σ ε2 ΣWW So by the central limit theorem: 1   W ' ε  n ~ N (0, σ ε Σ XX ) n   n ( βˆ IV − β )  → Σ −WX1 N (0, σ ε2 ΣWW ) Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables d −1 ′ ) σ ε2 ) N (0, ΣWX ΣWW (ΣWX  → −1   2  σ asy −1 ′ )−1 ε  ΣWW (ΣWX N  β , ΣWX → n     ˆ AsyVarCov ( β IV )   βˆIV Note: E ( βˆ IV ) = β → βˆIV is also an unbiased estimator βˆOLS is asymptotically efficient to βˆIV V HAUSMAN SPECIFICATION TEST AND AN APPLICATION TO IV ESTIMATION: Theorem: Let Z ~ N ( , Σ XX ) then: Z ' Σ −1 Z ~ χ [2r ] ( r ×1) r ×1 r ×r Proof: Recall: for λj: eigenvalue Cj : eigenvector λ1 0 λ Λ=    0 [ 0  0      λn   C = C1 C  C r ] r ×1 Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables Λ1 / we have: C ' Σ C = Λ = Λ1 / Λ1 / r ×r → C ' ΣC = ( Λ1 / )' ( Λ1 / ) → 1/ /2 (Λ ( Λ C 'ΣC ( Λ1 )'    ) = −1 / D'   =    λ1 0  λ2 0           λn  0  )' ( Λ1 / )' ( Λ1 / )( Λ−1 / ) = I D → D ' ΣD = I with D = CΛ−1 / → ( D ' ) −1 D ' ΣDD −1 = ( D ' ) −1 D −1 → Σ = ( D ' ) −1 D −1 → DD ' = Σ Note: C' = C-1, CC' = I Let W = D ' Z → W ~ N(0,DΣD') = N(0,I) r × r r ×1 r ×1 W ~ N(0,I) r ×1 → W 'W ~ χ [2r ] → ( D ' Z )' ( D ' Z ) ~ χ [2r ] → Z'  DD ' Z ~ χ [2r ] r ×1 Σ −1 Finally: Z ' Σ −1 Z ~ χ [2r ] Hausman Test: Y = Xβ + ε = X β + X β + ε n ×r Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables H0: E (ε X ) = r ×1 H0: E (ε X ) ≠ Two alternative estimators: βˆOLS : consistent under H0 but not under HA βˆ IV : consistent under both H0, HA (but inefficient compare to βˆOLS ) −1   βˆOLS = β + ( X ' X ) X ' ε  −1 ˆ   β IV = β + (W ' X ) W ' ε Under H0: βˆ IV = βˆOLS Construct the Hausman's test statistic: (βˆ IV )[ ( ′ − βˆOLS VarCov βˆ IV − βˆOLS )] (βˆ −1 IV ) − βˆOLS ~ χ [2r ] (Note: Z ~ N (0, Σ) → Z ' Σ −1 Z ~ χ [2r ] ) ( ) ( ) ( ) ( VarCov βˆ IV − βˆOLS = VarCov βˆ IV + VarCov βˆOLS − 2Cov βˆ IV , βˆOLS ( Cov βˆ IV , βˆOLS ) {[( )] )( ) } = E βˆ IV − β βˆOLS − β ' W , X [ ] = E {(W ' X ) −1W ' εε ' X ( X ' X ) −1 ) W , X } εε = (W ' X ) −1W '  E ( ' ) X ( X ' X ) −1 σ ε2 I = (W ' X ) −1W ' X ( X ' X ) −1 σ ε2 ( = ( X ' X ) −1 σ ε2 = VarCov βˆOLS ( ) ( ) ) ( So VarCov βˆ IV − βˆOLS = VarCov βˆ IV − VarCov βˆOLS Nam T Hoang University of New England - Australia ) University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables Then, the Hausman's test statistic is: (βˆ IV )[ ( ) ( ′ − βˆOLS VarCov βˆ IV − VarCov βˆOLS )] (βˆ −1 IV − βˆOLS ) ~χ [r] Under H0: H ~ χ [2r ] Wu's approach: Y = X β1 + X β + ε Do we have: E (ε X ) = In the first stage of IV estimation: ˆ +V X2 = Z Π × ×r n q q n×r n×r    r≤q → we get Xˆ Xˆ Y = X β1 + X β + Xˆ 2γ + ε * → Y = X β1 + X β + ( X − Vˆ )γ + ε * → Y = X β1 + X ( β + γ ) − Vˆγ + ε * Test: H0: γ = r ×1 r ×1 If reject H0 → E (ε X ) ≠ VI CHOOSING THE INSTRUMENTS: If we are working with time-series data, lagged values of regressors will generally provide appropriate instruments EX: y = β1 + β x + β x3 + ε Nam T Hoang University of New England - Australia 10 University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables 1 x12 X =    1 x n1 x03   x13     x n −1,3  1 x02  x12 W =    1 x n −1, x13     x n  Choice of Z affects asymptotic efficiency of βˆ IV Generally want to choose instruments to be highly corrected with the regressors (but uncorrelated with the errors) With the cross-section data, not always easy One option is to use the ranks of the data to form Z Example: y i = β1 + β xi + ε 1 14 1    1    X = 1  1    1  1 10 1 1  1  Z = 1 1  1 1 7 2  1  4 5  3 6 Appendix: Measurement Error in Linear Regression Y = X β +ε n ×1 n ×k (1) We don't observe X, but observe X* X* = X + V n ×k n ×k n ×k Where: E (ε X ) = n ×1 Nam T Hoang University of New England - Australia (2) E (εε ' X ) = σ I n ×n 11 University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables Put (2) into (1) yields: Y = ( X * −V ) β + ε n ×1 k ×1 Y = X * β + (ε − Vβ )  u The error term u = ε - Vβ is correlated with the regressor X* through the measurement error V Formally, we have: p lim *′ X u n = p lim *′ X (ε − Vβ ) n = p lim ( X + V )′(ε − Vβ ) n 1 1 = p lim X ' ε + p lim V ' ε − βp lim X 'V − βp lim V 'V ) n n  n n   0 = − βΣVV ≠ An OLS regression of Y on X will lead to an inconsistent estimate of β p lim *′ * X X = p lim ( X + V )′( X + V ) n n = p lim 1 1 X ' X + p lim X 'V + p lim V ' X + p lim V 'V n n n n = Σ XX + Σ vv ′ ′ ′ ′ βˆOLS =  X * X *  X * Y  =  X * X *   X * ( X * β + u )       ′ ′ = β +  X * X *  X * u   Nam T Hoang University of New England - Australia 12 University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables −1 p lim βˆ 1 ′  1 ′  = β + p lim X * X *   X * u  n  n  = β − (Σ XX + ΣVV ) ΣVV β −1  2 3  β =  5     4 −1 p lim βˆ = β − (Σ XX + ΣVV ) ΣVV β Clearly, OLS is inconsistent as long as there are measurement errors and ΣVV ≠ 1) βˆ is inconsistent as long as ΣVV ≠ k ×1 2) If there are some variables which are correctly measured → Their coefficient estimators are also inconsistent A badly measured variable contaminates all the least squares estimates → The effect of measurement errors is also called: "contamination bias" 3) For example if only one regressor is measured with errors ΣVV 0 0 =   0     2  σv      0  → the bias and inconsistent of all correctly measured variables depend on the form of ΣXX → unknown 4) In practice, it seems that the coefficients of the correctly measured variables are consistent but this depends on the special form of ΣXX Research questions: In practice → What kind of ΣXX we will count on the coefficients of correctly measured variables? Nam T Hoang University of New England - Australia 13 University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables → If we cannot find a good instrumental variable: omit wrongly measured variables or don't omit? Which form of ΣXX Computer programs could answer these questions (I guess) The form of ΣXX can be tested by simulations 5) There are other cases that endogeneity is a problem → what is the role of ΣXX in affecting the inconsistency of the coefficients in those cases 6) Endogeneity by measurement errors is a serious problem Nam T Hoang University of New England - Australia 14 University of Economics - HCMC - Vietnam ...Advanced Econometrics Chapter 4: Estimation By Instrumental Variables Variables on both sides of regression equation are jointly determined (endogenous) → RHS variables are endogenous hi... of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables X1: exogenous variables X2: endogenous variables Note: q must be ≥ r (if q < r → (W'W)-1 doesn't... correctly measured variables? Nam T Hoang University of New England - Australia 13 University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 4: Estimation By Instrumental Variables → If