Chapter 03_Stochastic Regression Model

Chapter 03_Stochastic Regression Model tài liệu, giáo án, bài giảng , luận văn, luận án, đồ án, bài tập lớn về tất cả cá...

Advanced Econometrics Chapter 3: Stochastic Regression Model Chapter STOCHASTIC REGRESSION MODEL I CONSISTENCY: Definition: • Let θˆn be a random variable If for any ∀ε > we have lim{θˆn − θ > 0} = then θ is probability limit of θˆn • If θˆn is an estimator for θ , then θˆn is said a consistent estimator of θ p lim θˆn = θ notation: n→∞ f (θˆ100 ) f (θˆn ) f (θˆ50 ) f (θˆ10 ) θ Note: A sufficient condition for this to hold is if Bias ( θˆn ) → θ and Var( θˆn ) → θ when n → ∞ Cramer Theorem:    (ii ) lim Var (θˆn ) = 0 n→∞  (i ) lim E (θˆn ) = θ If: n→∞ Nam T Hoang University of New England - Australia then p lim θˆn = θ n→∞ University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 3: Stochastic Regression Model xi ~ N ( µ , σ ) i = 1,2,3, n sample size Example:  σ2 n Var ( X ) = Get X = ∑ xi →  n n i =1 E ( X ) = µ   lim Var ( X ) = n → ∞ So  then X is a consistent estimator of µ: p lim ( X ) = µ n→∞ E( X ) = nlim Note: If an estimator is "inconsistent", then it is a useless estimator (unreliable) • There are many situations where OLS estimator is inconsistent Need to be clear with this Slutsky Theorem: Let F() be a continuous function, then: p lim F (θˆ1,n , θˆ2,n , , θˆk ,n ) = F [ p lim (θˆ1,n ), p lim (θˆ2,n ), , p lim (θˆk ,n )] n→∞ EX: if p lim(θˆn ) = C n→∞ → n→∞ n→∞ p lim[F (θˆn )] = F (C ) p lim[1 / θˆn ] = / C p lim[θˆn ] = C p lim[exp(θˆn ) = e C p lim(θˆ1,n θˆ1,n ) = p lim(θˆ1,n ) p lim(θˆ2,n ) A and B are stochastic matrices: p lim( AB ) = p lim( A) p lim( B ) also p lim( A−1 ) = ( p lim A) −1 if A is non-singular Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 3: Stochastic Regression Model II CLASSICAL STOCHASTIC REGRESSION MODEL: Now, consider the LS model, first under our standard assumption However, we will relax some of them • Don't need normality • X can be random, just assume that {xi, εi} is a random & independent sequence Model: (1) Y = X β + ε n ×k (2) X and ε are generated independently of each other and Rank ( X ) = k (3) E(ε|X) = (4) E(εε'|X) = σ2I (5) X consists of stationary random variables with: E  X i X i′  = Σ XX  n ×k 1×k  and 1 n p lim( XX ′ ) = p lim ∑ X i X i′ = E ( X i X i′) = Σ XX n n i =1 (Because X now is random) 1  x   i2  Stationary random variable: Xi =  xi       xik  First and second moments are constants: E  X i  = µ x X i  k ×1  k ×1 k ×1 : the ith row of X k ×1 = ith observation on all k variables Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 3: Stochastic Regression Model 1  x   i2  VarCov( X i ) = VarCov  xi  = E [( X i − µ X )( X i − µ X )'] = matrix of constants     k ×1 1× k     xik  Σ XX = population 2nd moment matrix X ' X = sample 2nd moment matrix n Recall: ∑X ∑X  n  ∑ X i2 X'X =      ∑ X ik → ∑X ∑X ∑X i2 2i i2  ∑X ∑X ik ∑X ∑X ∑X i3 i3 ik i2  i2 ∑X ∑X ik  i3 ∑X ik   ik     n X ' X = ∑ X i X i′ i =1 Unbiasedness of βˆOLS : ′X ) −1 X ′ε βˆ = β + ( X  ( βˆ = ( X ′X ) −1 X ′Y ) random → ˆ / X )] E ( βˆ ) = E X [  E ( β   (law of iterated expectation) ↓ expectation of βˆ conditional on X E X : Expectation over value of X Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 3: Stochastic Regression Model E ( βˆ X = X ) E ( βˆ X = X ) → E ( βˆ ) E ( βˆ X = X ) E ( βˆ ) = E X [ E{β + ( X ′X ) −1 X ′ε X }] = E X [ β + E{( X ′X ) −1 X ′ε X }] = E X [ β + E{( X ′X ) −1 X ′ X }.E (ε X )] (by assumption 2: X & ε are independent)  = E X [ β + ( X ′X ) −1 X ′.0] = β + E X (0)] = β VarCov of βˆOLS : ′ VarCov ( βˆ ) = E[( βˆ − E ( βˆ ))( βˆ − E ( βˆ )) ]   β β = E[( X ′X ) −1 X ′εε ′X ( X ′X ) −1 ] = E X [ E{( X ′X ) −1 X ′εε ′X ( X ′X ) −1 X }] = E X [ E{( X ′X ) −1 X ′ X }E (εε ′ X }E{ X ( X ′X ) −1 X }] = E X [( X ′X ) −1 X ′σ ε2 I X ' ( X ′X ) −1 ] = E X ( X ′X ) −1 σ ε2 I Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 3: Stochastic Regression Model Consistency of βˆOLS :   X ' X  −1 X ' ε  ˆ p lim β = β + p lim    n   n   X'X  Note: βˆ = β + ( X ′X ) −1 X ′ε = β +    n  −1 X 'ε n −1 1   X 'ε  p lim βˆ = β + p lim X ' X  p lim  (by Slutsky theorem: plimf(x) = f(plimx)) n   n   X 'ε  p lim βˆ = β + Σ −XX1 p lim   n  1   X'X  Note: E ( X i X i′) = Σ XX = p lim X i X i′  = p lim  n   n  Apply the Cramer theorem to (i) X 'ε n 1  lim E  X ' ε  = n  n→∞  1 1   Because: E  X ' ε  = E X  E  X ' ε X  n    n  1   = E X  E  X ' X  E (ε X )      n   1 = E X  X '.0  =  k ×1 n 1  (ii) lim CovVar  X ' ε  n→∞ n  1 1 = lim E  X ' εε ' X  n→∞ n n 1  = lim E X  E (X ' εε ' X X )  n→∞ n   1  = lim E X (X ' σ ε2 IX )  n→∞ n   Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 3: Stochastic Regression Model  σ2 = lim E X ( X ' X ) ε2  n→∞ n    X ' X σ ε2  = lim E X  ×  n→∞ n   n  1 n   n 1  ′ ′ E X ( X i X i )  = n.Σ XX = Σ XX Note: E  XX '  = E X  ∑ X i X i  = ∑     n  n   n i =1  n  i =1 Σ XX   X ' X σ ε2  Then: lim E X  ×  = lim σ Σ XX = n → ∞ n→∞ n n   n We have:  1  E X 'ε  = nlim  →∞  n    lim VarCov  X ' ε  = n → ∞ n  1  → p lim X ' ε  = (Cramer's Theorem) n   X 'ε  → p lim βˆ = β + Σ −XX1 p lim   n  = β + Σ −XX1 = β → βˆ is a consistent estimator of III LIMITING DISTRIBUTIONS AND ASYMPTOTIC DISTRIBUTIONS: Definition: Let zn be a random variable with probability distribution F(zn) and let z be another random variable with probability distribution F(z) If Fn(zn) converges to F(z) then F(z) is the limiting distribution of Fn(zn) Converges means: lim Fn ( z n ) − F ( z ) = n→∞ Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 3: Stochastic Regression Model d → F ( z) Notation Fn ( z )  d d zn  → z or equivalently write: z n  → F (z ) d Example: t[ r ]  → N (0,1) Central limit theorem: If X1, X2, Xn is a random sample from some distribution with mean µ, variance σ2 Then: d n ( X − µ)  → N (0, σ ) Proposition: Let wn be a random variable with plimwn = w and zn has limiting distribution of F(z) Then the limiting distribution of wnzn is equal to w.F(z) = plimwn.F(z) The asymptotic distribution of X is defined in terms of the limiting distribution of a related random variable n ( X − µ ) , which has a non-degenerate limiting distribution d n ( X − µ)  → N (0, σ )  σ2   is the asymptotic distribution of ( X − µ ) → N  0, ( X − µ)   n  d a  σ2   → X ~ N  µ , n   IV ASYMPTOTIC DISTRIBUTION OF βˆ Recall: βˆ = β + ( X ′X ) −1 X ′ε E ( βˆ ) = β p lim(βˆ ) = β → consistency VarCov ( βˆ ) = σ ε2 E X ( X ' X ) −1 E( X 'ε ) = Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 3: Stochastic Regression Model VarCov ( βˆ ) = σ ε2 E X ( X ' X ) −1 1  E  X ' εε ' X  = σ ε2 Σ XX n  Recall: d n ( X − µ)  → N (0, σ ) a  σ2   → X ~ N  µ , n   d n (θˆn − θ )  → N( , Q ) a   → θˆn ~ N θ , Q  where Q : asyVarCov θˆn n  n  k ×1 k × k ( ) For βˆ : −1 1  1  n ( βˆ − β ) =  X ' X   X ' ε  n = n  n  p lim Σ −XX   Because: E  X ' ε  = (E(X'ε) =  n    1  2 E X ' εε ' X  = σ ε IE  X ' X  = σ ε Σ XX n n   n Then by the central limit theorem: 1  n  X ' ε  ~ N ( , σ ε2 Σ XX ) k ×1 k ×k n  n   →  X ' ε  is a random sample from some distribution with mean & variance σ ε2 Σ XX k ×k  n  i =1 n Consider: X ' ε = ∑ wi → i =1   X   2i  wi = Xiεi =  X 3i ε i       X ki  → E ( wi ) = E ( X i ε i ) = E X ( X i ) E (ε i X i ) =      µX Nam T Hoang University of New England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 3: Stochastic Regression Model VarCov( wi ) = E ( X i ε i ε i X i′) = σ ε2 E ( X ' X ) = σ ε2 Σ XX Then by the central limit theorem: 1 n  n  ∑ wi −  ~ N ( , σ ε2 Σ XX ) k × k ×1 k ×k  n i =1  1  n  X 'ε  = n  −1 1  1  n ( βˆ − β ) =  X ' X   X ' ε  n n  n  Then: d  → Σ −XX1 N ( , σ ε2 Σ XX ) k ×1 k ×k d N ( , Σ −XX1 Σ XX Σ −XX1 ' σ ε2 )  →   k ×1  ( Σ XX symmetric) I Note: Z ~ N (0, Σ XX ) W = cZ → w ~ N (0, cΣ XX c' ) Σ XX symmetric µγ → ( β + µ X γ ) d N (0, Σ −XX1 σ ε2 ) n ( βˆ − β )  → So: a βˆ ~ N ( β , Σ −XX1 σ ε2 n    ) asyVarCov ( βˆ ) Σ −1 XX −1 1  σε = p lim X ' X  n n  n σ ε2 = n  p lim( X ' X )   −1 σ ε2 = E ( X ' X ) σ ε2 n Note: Σ XX = E X ( X i X i′)  n   n  n E ( X ' X ) = E  ∑ X i X i′  =  ∑ E ( X i X i′)  = ∑ Σ XX = n Σ XX k ×k  i =1   i =1  i =1 VarCov( wi ) = E ( X ' X ) −1 σ ε2 = σ ε2 [nΣ XX ]−1 = Σ −XX1 Nam T Hoang University of New England - Australia 10 σ ε2 n University of Economics - HCMC - Vietnam Advanced Econometrics Remember: Σ XX = E ( X i X i′) = E ( X ' X ) Chapter 3: Stochastic Regression Model n → E ( X ' X ) = nΣ XX → The more observations we have, the smaller variance of βˆ are Nam T Hoang University of New England - Australia 11 University of Economics - HCMC - Vietnam ... Economics - HCMC - Vietnam Advanced Econometrics Chapter 3: Stochastic Regression Model II CLASSICAL STOCHASTIC REGRESSION MODEL: Now, consider the LS model, first under our standard assumption However,...Advanced Econometrics Chapter 3: Stochastic Regression Model xi ~ N ( µ , σ ) i = 1,2,3, n sample size Example:  σ2 n Var ( X ) = Get... England - Australia University of Economics - HCMC - Vietnam Advanced Econometrics Chapter 3: Stochastic Regression Model 1  x   i2  VarCov( X i ) = VarCov  xi  = E [( X i − µ X )( X i −