Tài liệu Kinh tế ứng dụng_ Lecture 3: Outliers, Leverage and Influence docx

Applied Econometrics Outliers, Leverage and Influence 1 Applied Econometrics Lecture 3: Outliers, Leverage and Influence ‘Life is the art of drawing sufficient conclusions from insufficient premises’ SAMUEL BUTLER 1) Introduction The estimates of the regression parameters are influenced by a few extreme observations. The residual plot may let us pick out, which the individual data points are high or low. We may use the residual plot to find the outlier, which are inadequately captured by the regression model itself. 2) Identification of outliers ¾ The percentiles that cut the data up into four quarters have special names: The 25 th percentiles and the 75 th percentiles are called the lower and upper quartiles (Q L and Q U ) ¾ The lower quartile will be the [integer((n+1)/2)+1]/2 value from the bottom of the ordered list. the upper quartile is the [integer((n+1)/2)+1]/2 value from the top ¾ A data point Y 0 is considered to be an outliers if Y 0 < Q L – 1.5 IQR or Y 0 > Q U + 1.5 IQR where IQR is the inter – quartile range (IQR = Q U – Q L ) (Source: Hoaglin, 1983) 3) Outliers An outlier is a point, which is far removed from its fitted value (i.e., has large residual). Large in this context does not refer to the absolute size of a residual but to its size relative to most of the other residuals in the regression. When a point is an outlier in univariate analysis, it is defined with reference to its own mean. When a point is an outlier in bivariate analysis, it has a large residual (i.e., Y value is far removed from its fitted value). Apart from the graphical methods, we can also rely on special statistics to detect outliers. In order to compare the large residual to the other residual, we may calculate the standardized residual, which is simply the residual divided by the standard error of the estimate (e i /s). But an outlier in the data set will inflate the standard error of the regression. Hence we use the studentized residual Written by Nguyen Hoang Bao May 20, 2004 Applied Econometrics Outliers, Leverage and Influence 2 h1 s(i) e t i i i − = where e i is the residual (e i = y i – ) iy ˆ s(i) is the standard error of the estimation having dropped the ith observation from the sample h i is the hat statistic for the observation ith, which is defined as ∑ − − += = n 1i 2 2 i ) XX i ( ) XX i ( n 1 h The additional term in the denominator, h1 i − , is necessary since the variance of the residuals is assumed not to be constant. With the adjustment factor, we get a t – statistic, which tests whether the ith residual is significant different from zero and, hence, signals an outliers, which does not really fit the overall pattern. Alternatively, the t – statistic of the coefficient of the dummy variable pick out a single observation from the sample. 4) Leverage A data point has a high leverage if it is far removed in the X – direction (i.e., it is a disproportionate distance away from the middle range of the X – direction) (Myers, 1990). The points of high leverage can exert undue influence on the outcome of a least squares regression line. That is, points with high leverage are capable of exerting a strong pull on the slope of the regression line. In univariate analysis, the definition of an outlier and a point of leverage are the same. A point, which is an outlier, also has high leverage with respect to the mean. In bivariate analysis, a point of high leverage (with respect to the slope coefficient) is one which is far removed in the X – direction (as opposed to an outliers. which are far removed from Y – direction). A test statistic for the leverage is the hat statistic: ∑ − − += = n 1i 2 2 i ) XX i ( ) XX i ( n 1 h Written by Nguyen Hoang Bao May 20, 2004 Applied Econometrics Outliers, Leverage and Influence 3 which serves as a measure of leverage of the ith data point. It measures leverage because the numerator is the squared distance of the ith data point from its mean in the X – direction, while its denominator is a measure of overall variability of the data points along the X – axis. Therefore, the higher value of h i the higher is the leverage of the ith data point, the greater the distance of X i from its mean. h i can vary from 1/n (i.e., close to zero) for a point with no leverage and tend to one for very high leverage. It is suggested that the following guidelines are based on the maximum observed h i = max(h i ) (Huber, 1981): max(h i ) < 0.2 little to worry about 0.2 < max(h i ) < 0.5 risky 0.5 < max(h i ) too much leverage 5) Influence A data point is influential if removing it from the sample would markedly change the position of the least squares regression line (Moore and McCabe, 1989). Hence, influential data points pull the regression line in their regression. The influential data points do not necessarily produce large residuals. That is, they are not always outliers as well, although they can be. Conversely, an outlier is not necessarily an influential point, particularly when it is a point with little leverage. In univariate analysis, an outlier has high leverage and will be influential. In bivariate analysis, high leverage is a necessarily condition for influence on the slope, but not a sufficient one. Similarly, an outlier may not be influential if it has low leverage, nor a point of high leverage be an outlier if its leverage is strong enough. A test for influence is the DFBETA statistic, which is defined as 1 : )(i) β SE( (i) ββ DFBETA 1 11 i − = where bracket (i) refers to the value of the statistic when observation ith is excluded from the regression. The DFBETAs measure the sensitivity of the slope coefficient to the deletion of the ith data point 1 We suppose that the regression model can be specified as Y = β 0 + β 1 X Written by Nguyen Hoang Bao May 20, 2004 Applied Econometrics Outliers, Leverage and Influence 4 if DFBETA < 2/ n , the point has no influence if DFBETA > 3/ n , the point is influential if 2/ n < DFBETA < 3/ n , the point is inconclusive The regression analysis should capture general pattern in the data: an influential point can prevent this from being so. Hence, they are often best dropped from the regression. DFBETAs should always be used in conjunction with diagnostic regression graphics. It is always possible that a cluster of points is exerting influence rather than a single data point. Table 5: Summary measures outliers, leverage, and influence Statistic Formula Use Critical value Studentized residual (t i ) h1 s(i) e t i i i − = Outliers Critical values available (higher than usual t–test), but recommend use t i as an exploratory tool Hat statistic (h i ) ∑ − − += = n 1i 2 2 i ) XX i ( ) XX i ( n 1 h Leverage Bounded by 1/n (no leverage) and 1 (extremely leverage); values above 0.5 indicate excessive leverage and values over 0.2 indicate the observation may give problems DFBETA )(i) β SE( (i) ββ DFBETA 1 11 i − = Influence Under 2/ n , the point has no influence; over 3/ n, the point is influential and strongly so if DFBETA exceeds 2 Note: n is the sample size; k is the number of regressors; the subscript (i) (i.e., with parentheses) indicates an estimation from the sample omitting observation i. In each case you should use the absolute value of the calculated statistic. Source: Mukherjee Chandan, Howard White and Marc Wuyts (1998), ‘Econometrics and Data Analysis for Developing Countries’ published by Routledge, London, UK. Written by Nguyen Hoang Bao May 20, 2004 Applied Econometrics Outliers, Leverage and Influence 5 References Bao, Nguyen Hoang (1995), ‘Applied Econometrics’, Lecture notes and Readings, Vietnam-Netherlands Project for MA Program in Economics of Development. Hoaglin, David C., Mosteller F., Tukey J. (1983), Understanding Robust and Exploratory Data Analysis, New York: John Wiley. Huber, Peter J. (1981), Robust Statistics, New York: John Wiley. Maddala, G.S. (1992), ‘Introduction to Econometrics’, Macmillan Publishing Company, New York. Moore, D.S. and McCabe, G.P. (1989), Introduction to the Practice of Statistics, New York: Freeman. Mukherjee Chandan, Howard White and Marc Wuyts (1998), ‘Econometrics and Data Analysis for Developing Countries’ published by Routledge, London, UK. Myers R. H. (1990), Classical and Modern Regression with Application, Second Edition, Boston, M.A: PWS – Kent. Written by Nguyen Hoang Bao May 20, 2004 Applied Econometrics Outliers, Leverage and Influence 6 Workshop 3: Outliers, Leverage and Influence 1) Look carefully at the four plots in the attached figure. For each plot write down whether any of the points is: an outliers, a point of high leverage, an influential points or some combination of these. Briefly comment on your findings Hint:  Outliers are not necessarily influential (plot 4)  But they can be so (depending on leverage) (plot 3)  Yet high leverage points are not always influential (plot 1)  And influential points are not necessarily outliers (plot 2) Plot summary Plot Outliers Leverage Influence Plot 1 Plot 2 Plot 3 Plot 4 ______ ______ ______ ______ ______ ______ ______ ______ ______ ______ ______ ______ 2) An examination of residuals provides a diagnostic check on the model. When the regression model is inadequately specified, the residuals are not just pure noise. Instead they contain a message that can help us to specify a better model. Consider the four different relations between Y and X plotted below (Anscombe, 1973) – a simplified version of some common phenomena. 2.1) Calculate the regression line (Y against X), and graph it in panel 1, 2, 3 and 4 with the data points. 2.2) State which graph above corresponds to this situation: (i) The relation is really curved, rather than linear (ii) The positive relation is entirely the result of just one data point (iii) The residual variance is entirely the result of just one data point – which may very well be recorded in error (iv) It makes good sense to use the regression line for prediction 2.3) Briefly, what lesson does this show? Written by Nguyen Hoang Bao May 20, 2004 Applied Econometrics Outliers, Leverage and Influence 7 Regression 1 Regression 2 Regression 3 Regression 4 Y X 1 Y X 2 YX 3 Y X 4 8.04 6.95 7.58 8.81 8.33 9.96 7.24 4.26 10.84 4.82 5.68 10 8 13 9 11 14 6 4 12 7 5 9.14 8.14 8.74 8.77 9.26 8.1 6.13 3.1 9.13 7.26 4.74 10 8 13 9 11 14 6 4 12 7 5 7.46 6.77 12.74 7.11 7.81 8.84 6.08 5.39 8.15 6.42 5.73 10 8 13 9 11 14 6 4 12 7 5 6.58 5.76 7.71 8 8.47 7.04 5.25 12.5 5.56 7.91 6.89 8 8 8 8 8 8 8 19 8 8 8 3) The identification of outliers in univariate analysis Using the data file LEACCESS.WK1, identify if there is any outliers in each of the following data sets: 3.1) LE 3.2) Y 3.3) ln(Y) 4) The identification of outliers in bivariate analysis 4.1) Using the data file AIDSAV, test whether observation 26 (Lesotho) is: a) an outlier b) an point of high leverage c) an influential point 4.2) Draw the scatter plot of S/Y against A/Y showing the regression line with and without point 26 in the same graph 4.3) What happen to the R 2 when observation 26 is dropped from the data set? Explain 4.4) Are there any other problematic points in the sample? Written by Nguyen Hoang Bao May 20, 2004 Applied Econometrics Outliers, Leverage and Influence 8 4.5) Show algebraically that a point with no leverage cannot have any influence on the slope coefficient 5) Outliers in bivariate analysis 5.1) Using the data file HOLMQ, which contains the data for EDUEXP and EAID, examine the figure and test whether any possible points is: a) an outlier b) a point of high leverage c) an influential point 5.2) Draw the scatter plot, showing the fitted line. Briefly comment on your findings. Written by Nguyen Hoang Bao May 20, 2004 . Applied Econometrics Outliers, Leverage and Influence 1 Applied Econometrics Lecture 3: Outliers, Leverage and Influence ‘Life is the art. Hoang Bao May 20, 2004 Applied Econometrics Outliers, Leverage and Influence 6 Workshop 3: Outliers, Leverage and Influence 1) Look carefully at the four