Lecture Applied econometrics course - Chapter 2: Multiple regression model has content: Why we need multiple regression model, estimation, R – Square, assumption, variance and standard error of parameters, the issues of multiple regression model, Illustration by Computer.
APPLIED ECONOMETRICS COURSE CHAPTER II MULTIPLE REGRESSION MODEL NGUYEN BA TRUNG - 2016 Today’s talk Why we need multiple regression model Estimation R – Square Assumption Variance and Standard Error of Parameters The issues of multiple regression model Illustration by Computer NGUYEN BA TRUNG - 2016 I WHY WE NEED MULTIPLE REGRESSION MODEL? Reality and flexibility Avoid the biasedness caused by omitted variables Generally, multiple regression model is written by Yi 1 X 1i k X k u (2.1) The terminology in (2.1) is similar as simple regression model NGUYEN BA TRUNG - 2016 II ESTIMATION: OLS The matrix form of multiple regression model: Y1 Y Yn 1 Y 1 n X 21 X 22 X n 1 X 2n X k1 u1 1 X k u2 X kn 1 un 1 k X kn u n Compactness: Y X u (2.2) NGUYEN BA TRUNG - 2016 II ESTIMATION: OLS ˆ ˆ RSS uu (Y X ˆ )(Y X ˆ ) Y Y X ˆ Y Y X ˆ ˆ XX ˆ YY ˆ X Y ˆ XX ˆ Take derivative respect to parameters, we have: RSS ( ˆ ) 2 X Y X X ˆ ˆ X X ˆ X Y NGUYEN BA TRUNG - 2016 II ESTIMATION: OLS The parameters are estimated in form: ˆ X X X Y 1 (2.3) where: n X 2i X X X ki X X X ki 2i 2i X X X X 2i X 3i 2i 3i ki X 3i NGUYEN BA TRUNG - 2016 X X X 2i ki X ki ki Computer 1: WAGE1.wf Explain the meaning of education’s parameter? NGUYEN BA TRUNG - 2016 III R- SQUARE Y Yˆ uˆ X ˆ uˆ Take square both sides, we obtain: YY ( X ˆ uˆ )( X ˆ uˆ ) X ˆ X ˆ uˆ uˆ Remember again that: 2 ( Y Y ) Y nY i i Then, we have 2 Y Y nY XX nY uˆ uˆ TSS ESS RSS (2.4) NGUYEN BA TRUNG - 2016 III R - SQUARE ESS RSS R 1 TSS TSS R2 R2 is the non-decreasing function of the explained variables Therefore, R2 is not good indicator to measure the fit of your model You should use adjusted R-square instead: n 1 R (1 R ) n k-1 2 NGUYEN BA TRUNG - 2016 (2.5) Computer 2: WAGE1.wf NGUYEN BA TRUNG - 2016 IV ASSUMPTION Assumption 1: Yi 1 X 1i k X ki ui Assump 2: X is the randomness Asssump 3: No perfect multi-collinearity Assump 4: E(u | x1 , , x k ) Theorem 2: The unbiasedness of OLS Under the assumption 1- 4, the estimators of OLS are unbiased: E ( ˆ j ) j NGUYEN BA TRUNG - 2016 V VARIANCE AND STANDARD ERROR Assumption 5: Homoscedasticity Var(ui ) Theorem 3: The unbiasedness of estimator’s variance Under the assumption1- 5, the variance of estimator is unbiased: E (ˆ ) The variance of estimator is estimated by: Var(ˆ j ) 2 X X R ij j (2.6) Note that, R 2j is the R-square obtained by regressing all independent variables together NGUYEN BA TRUNG - 2016 Computer 3: WAGE1.wf NGUYEN BA TRUNG - 2016 VI THE ISSUES OF MULTIPLE REGRESSION MODEL 6.1 The redundancy of explained variables Suppose that we have the true model as: Yi 1 X 1i ui (2.7) However, we form the model as below: Yi 1 X 1i X i ui (2.8) This means that, X2i is a redundant variable in (2.8) Follow by the theorem 2, the estimators of (2.8) are still unbiased NGUYEN BA TRUNG - 2016 VI THE ISSUES OF MULTIPLE REGRESSION MODEL But, the variance of estimator will decreases Lose the degree of freedoms Difficulty to get statistically significant of estimator Increase probability of multi-collinearity NGUYEN BA TRUNG - 2016 VI THE ISSUES OF MULTIPLE REGRESSION MODEL 6.2 The omitted variables The true model: Yi 1 X X u (2.9) The wrong model : Yi 1 X u (2.10) We can show that: E ( 1 ) 1 21 (2.11) Where 1 is the parameter obtained by regressing X1 on X2 Thus, the omitted variable X2 will lead to biasedness: Bias (1) = 21 NGUYEN BA TRUNG - 2016 (2.12) VI THE ISSUES OF MULTIPLE REGRESSION MODEL The estimator of the model below will be positive or negative bias if omitted the povrate variable? NGUYEN BA TRUNG - 2016 Solution: Proxy variable The true model: y 1 x1 x2 x3* u (2.13) Suppose that we are unable to have 𝒙∗𝟑 variable, but we have a proxy variable for 𝒙𝟑∗ , called as: x3 Whether x3 is a good proxy variable or not? x3* 1 x3 v3 (2.14) If x3 is a good proxy variable, the estimators in (2.13) will be biased NGUYEN BA TRUNG - 2016 Solution: Proxy variable Substitute 𝒙𝟑∗ in (2.13) by (2.14), and rearrange again: y ( 3 ) 1 x1 x2 3 x3 (u 3v3 ) If e = 𝑢 + 𝛽3 𝑣3 , we have: y ( 3 ) 1 x1 x2 3 x3 e (2.16) Assume that E(e/x1, x2, x3) = 0, the estimators in (2.15) are unbiased: Example: log(wage) 1educ1 exp er ability u If omitted the ability variable, the model will be positive or negative bias? NGUYEN BA TRUNG - 2016 Example: Wage2.wf Assume that we have the true model as: Log(wage) = β0+ β1educ+ β2exper+ β3ability+u (true model) Due to omitting the ability variable, we only estimate the following model: Log(wage) = β0+ β1educ+ β2exper +u (omitted model) whether β1 will be biased or not? and why? Positive bias or negative bias? And why? NGUYEN BA TRUNG - 2016 Example: Wage2.wf NGUYEN BA TRUNG - 2016 APPLIED ECONOMETRICS COURSE END OF THE CHAPTER II NGUYEN BA TRUNG - 2016 ... NGUYEN BA TRUNG - 2016 I WHY WE NEED MULTIPLE REGRESSION MODEL? Reality and flexibility Avoid the biasedness caused by omitted variables Generally, multiple regression model is written... probability of multi-collinearity NGUYEN BA TRUNG - 2016 VI THE ISSUES OF MULTIPLE REGRESSION MODEL 6.2 The omitted variables The true model: Yi 1 X X u (2.9) The wrong model : Yi ... The terminology in (2.1) is similar as simple regression model NGUYEN BA TRUNG - 2016 II ESTIMATION: OLS The matrix form of multiple regression model: Y1 Y