Engineering Statistics Handbook Episode 1 Part 14 potx

********************************************************* ** normal Kolmogorov-Smirnov goodness of fit test y3 ** ********************************************************* KOLMOGOROV-SMIRNOV GOODNESS-OF-FIT TEST NULL HYPOTHESIS H0: DISTRIBUTION FITS THE DATA ALTERNATE HYPOTHESIS HA: DISTRIBUTION DOES NOT FIT THE DATA DISTRIBUTION: NORMAL NUMBER OF OBSERVATIONS = 1000 TEST: KOLMOGOROV-SMIRNOV TEST STATISTIC = 0.6119353E-01 ALPHA LEVEL CUTOFF CONCLUSION 10% 0.03858 REJECT H0 5% 0.04301 REJECT H0 1% 0.05155 REJECT H0 ********************************************************* ** normal Kolmogorov-Smirnov goodness of fit test y4 ** ********************************************************* KOLMOGOROV-SMIRNOV GOODNESS-OF-FIT TEST NULL HYPOTHESIS H0: DISTRIBUTION FITS THE DATA ALTERNATE HYPOTHESIS HA: DISTRIBUTION DOES NOT FIT THE DATA DISTRIBUTION: NORMAL NUMBER OF OBSERVATIONS = 1000 TEST: KOLMOGOROV-SMIRNOV TEST STATISTIC = 0.5354889 ALPHA LEVEL CUTOFF CONCLUSION 10% 0.03858 REJECT H0 5% 0.04301 REJECT H0 1% 0.05155 REJECT H0 Questions The Kolmogorov-Smirnov test can be used to answer the following types of questions: Are the data from a normal distribution? ● Are the data from a log-normal distribution?● Are the data from a Weibull distribution?● Are the data from an exponential distribution?● Are the data from a logistic distribution?● 1.3.5.16. Kolmogorov-Smirnov Goodness-of-Fit Test http://www.itl.nist.gov/div898/handbook/eda/section3/eda35g.htm (5 of 6) [5/1/2006 9:57:47 AM] Importance Many statistical tests and procedures are based on specific distributional assumptions. The assumption of normality is particularly common in classical statistical tests. Much reliability modeling is based on the assumption that the data follow a Weibull distribution. There are many non-parametric and robust techniques that are not based on strong distributional assumptions. By non-parametric, we mean a technique, such as the sign test, that is not based on a specific distributional assumption. By robust, we mean a statistical technique that performs well under a wide range of distributional assumptions. However, techniques based on specific distributional assumptions are in general more powerful than these non-parametric and robust techniques. By power, we mean the ability to detect a difference when that difference actually exists. Therefore, if the distributional assumptions can be confirmed, the parametric techniques are generally preferred. If you are using a technique that makes a normality (or some other type of distributional) assumption, it is important to confirm that this assumption is in fact justified. If it is, the more powerful parametric techniques can be used. If the distributional assumption is not justified, using a non-parametric or robust technique may be required. Related Techniques Anderson-Darling goodness-of-fit Test Chi-Square goodness-of-fit Test Shapiro-Wilk Normality Test Probability Plots Probability Plot Correlation Coefficient Plot Case Study Airplane glass failure times data Software Some general purpose statistical software programs, including Dataplot, support the Kolmogorov-Smirnov goodness-of-fit test, at least for some of the more common distributions. 1.3.5.16. Kolmogorov-Smirnov Goodness-of-Fit Test http://www.itl.nist.gov/div898/handbook/eda/section3/eda35g.htm (6 of 6) [5/1/2006 9:57:47 AM] with Y min denoting the minimum value. test whether the maximum value is an outlier with Y max denoting the maximum value. 2. Significance Level: . Critical Region: For the two-sided test, the hypothesis of no outliers is rejected if with denoting the critical value of the t-distribution with (N-2)/2 degrees of freedom and a significance level of /(2N). For the one-sided tests, we use a significance level of /N. In the above formulas for the critical regions, the Handbook follows the convention that is the upper critical value from the t-distribution and is the lower critical value from the t-distribution. Note that this is the opposite of what is used in some texts and software programs. In particular, Dataplot uses the opposite convention. Sample Output Dataplot generated the following output for the ZARR13.DAT data set showing that Grubbs' test finds no outliers in the dataset: ********************* ** grubbs test y ** ********************* GRUBBS TEST FOR OUTLIERS (ASSUMPTION: NORMALITY) 1. STATISTICS: NUMBER OF OBSERVATIONS = 195 MINIMUM = 9.196848 MEAN = 9.261460 1.3.5.17. Grubbs' Test for Outliers http://www.itl.nist.gov/div898/handbook/eda/section3/eda35h.htm (2 of 4) [5/1/2006 9:57:48 AM] MAXIMUM = 9.327973 STANDARD DEVIATION = 0.2278881E-01 GRUBBS TEST STATISTIC = 2.918673 2. PERCENT POINTS OF THE REFERENCE DISTRIBUTION FOR GRUBBS TEST STATISTIC 0 % POINT = 0.000000 50 % POINT = 2.984294 75 % POINT = 3.181226 90 % POINT = 3.424672 95 % POINT = 3.597898 97.5 % POINT = 3.763061 99 % POINT = 3.970215 100 % POINT = 13.89263 3. CONCLUSION (AT THE 5% LEVEL): THERE ARE NO OUTLIERS. Interpretation of Sample Output The output is divided into three sections. The first section prints the sample statistics used in the computation of the Grubbs' test and the value of the Grubbs' test statistic. 1. The second section prints the upper critical value for the Grubbs' test statistic distribution corresponding to various significance levels. The value in the first column, the confidence level of the test, is equivalent to 100(1- ). We reject the null hypothesis at that significance level if the value of the Grubbs' test statistic printed in section one is greater than the critical value printed in the last column. 2. The third section prints the conclusion for a 95% test. For a different significance level, the appropriate conclusion can be drawn from the table printed in section two. For example, for = 0.10, we look at the row for 90% confidence and compare the critical value 3.24 to the Grubbs' test statistic 2.92. Since the test statistic is less than the critical value, we accept the null hypothesis at the = 0.10 level. 3. Output from other statistical software may look somewhat different from the above output. Questions Grubbs' test can be used to answer the following questions: Does the data set contain any outliers?1. How many outliers does it contain?2. 1.3.5.17. Grubbs' Test for Outliers http://www.itl.nist.gov/div898/handbook/eda/section3/eda35h.htm (3 of 4) [5/1/2006 9:57:48 AM] Importance Many statistical techniques are sensitive to the presence of outliers. For example, simple calculations of the mean and standard deviation may be distorted by a single grossly inaccurate data point. Checking for outliers should be a routine part of any data analysis. Potential outliers should be examined to see if they are possibly erroneous. If the data point is in error, it should be corrected if possible and deleted if it is not possible. If there is no reason to believe that the outlying point is in error, it should not be deleted without careful consideration. However, the use of more robust techniques may be warranted. Robust techniques will often downweight the effect of outlying points without deleting them. Related Techniques Several graphical techniques can, and should, be used to detect outliers. A simple run sequence plot, a box plot, or a histogram should show any obviously outlying points. Run Sequence Plot Histogram Box Plot Normal Probability Plot Lag Plot Case Study Heat flow meter data. Software Some general purpose statistical software programs, including Dataplot, support the Grubbs' test. 1.3.5.17. Grubbs' Test for Outliers http://www.itl.nist.gov/div898/handbook/eda/section3/eda35h.htm (4 of 4) [5/1/2006 9:57:48 AM] Yates Order Before performing a Yates analysis, the data should be arranged in "Yates order". That is, given k factors, the kth column consists of 2 k-1 minus signs (i.e., the low level of the factor) followed by 2 k-1 plus signs (i.e., the high level of the factor). For example, for a full factorial design with three factors, the design matrix is - - - + - - - + - + + - - - + + - + - + + + + + Determining the Yates order for fractional factorial designs requires knowledge of the confounding structure of the fractional factorial design. Yates Output A Yates analysis generates the following output. A factor identifier (from Yates order). The specific identifier will vary depending on the program used to generate the Yates analysis. Dataplot, for example, uses the following for a 3-factor model. 1 = factor 1 2 = factor 2 3 = factor 3 12 = interaction of factor 1 and factor 2 13 = interaction of factor 1 and factor 3 23 = interaction of factor 2 and factor 3 123 =interaction of factors 1, 2, and 3 1. Least squares estimated factor effects ordered from largest in magnitude (most significant) to smallest in magnitude (least significant). That is, we obtain a ranked list of important factors. 2. A t-value for the individual factor effect estimates. The t-value is computed as where e is the estimated factor effect and is the standard deviation of the estimated factor effect. 3. The residual standard deviation that results from the model with the single term only. That is, the residual standard deviation from the model response = constant + 0.5 (X i ) 4. 1.3.5.18. Yates Analysis http://www.itl.nist.gov/div898/handbook/eda/section3/eda35i.htm (2 of 5) [5/1/2006 9:57:48 AM] where X i is the estimate of the ith factor or interaction effect. The cumulative residual standard deviation that results from the model using the current term plus all terms preceding that term. That is, response = constant + 0.5 (all effect estimates down to and including the effect of interest) This consists of a monotonically decreasing set of residual standard deviations (indicating a better fit as the number of terms in the model increases). The first cumulative residual standard deviation is for the model response = constant where the constant is the overall mean of the response variable. The last cumulative residual standard deviation is for the model response = constant + 0.5*(all factor and interaction estimates) This last model will have a residual standard deviation of zero. 5. Sample Output Dataplot generated the following Yates analysis output for the Eddy current data set: (NOTE DATA MUST BE IN STANDARD ORDER) NUMBER OF OBSERVATIONS = 8 NUMBER OF FACTORS = 3 NO REPLICATION CASE PSEUDO-REPLICATION STAND. DEV. = 0.20152531564E+00 PSEUDO-DEGREES OF FREEDOM = 1 (THE PSEUDO-REP. STAND. DEV. ASSUMES ALL 3, 4, 5, TERM INTERACTIONS ARE NOT REAL, BUT MANIFESTATIONS OF RANDOM ERROR) STANDARD DEVIATION OF A COEF. = 0.14249992371E+00 (BASED ON PSEUDO-REP. ST. DEV.) GRAND MEAN = 0.26587500572E+01 GRAND STANDARD DEVIATION = 0.17410624027E+01 99% CONFIDENCE LIMITS (+-) = 0.90710897446E+01 95% CONFIDENCE LIMITS (+-) = 0.18106349707E+01 99.5% POINT OF T DISTRIBUTION = 0.63656803131E+02 97.5% POINT OF T DISTRIBUTION = 0.12706216812E+02 IDENTIFIER EFFECT T VALUE RESSD: RESSD: MEAN + MEAN + TERM CUM TERMS 1.3.5.18. Yates Analysis http://www.itl.nist.gov/div898/handbook/eda/section3/eda35i.htm (3 of 5) [5/1/2006 9:57:48 AM] MEAN 2.65875 1.74106 1.74106 1 3.10250 21.8* 0.57272 0.57272 2 -0.86750 -6.1 1.81264 0.30429 23 0.29750 2.1 1.87270 0.26737 13 0.24750 1.7 1.87513 0.23341 3 0.21250 1.5 1.87656 0.19121 123 0.14250 1.0 1.87876 0.18031 12 0.12750 0.9 1.87912 0.00000 Interpretation of Sample Output In summary, the Yates analysis provides us with the following ranked list of important factors along with the estimated effect estimate. X1:1. effect estimate = 3.1025 ohms X2:2. effect estimate = -0.8675 ohms X2*X3:3. effect estimate = 0.2975 ohms X1*X3:4. effect estimate = 0.2475 ohms X3:5. effect estimate = 0.2125 ohms X1*X2*X3:6. effect estimate = 0.1425 ohms X1*X2:7. effect estimate = 0.1275 ohms Model Selection and Validation From the above Yates output, we can define the potential models from the Yates analysis. An important component of a Yates analysis is selecting the best model from the available potential models. Once a tentative model has been selected, the error term should follow the assumptions for a univariate measurement process. That is, the model should be validated by analyzing the residuals. Graphical Presentation Some analysts may prefer a more graphical presentation of the Yates results. In particular, the following plots may be useful: Ordered data plot1. Ordered absolute effects plot2. Cumulative residual standard deviation plot3. Questions The Yates analysis can be used to answer the following questions: What is the ranked list of factors?1. What is the goodness-of-fit (as measured by the residual standard deviation) for the various models? 2. 1.3.5.18. Yates Analysis http://www.itl.nist.gov/div898/handbook/eda/section3/eda35i.htm (4 of 5) [5/1/2006 9:57:48 AM] Related Techniques Multi-factor analysis of variance Dex mean plot Block plot Dex contour plot Case Study The Yates analysis is demonstrated in the Eddy current case study. Software Many general purpose statistical software programs, including Dataplot, can perform a Yates analysis. 1.3.5.18. Yates Analysis http://www.itl.nist.gov/div898/handbook/eda/section3/eda35i.htm (5 of 5) [5/1/2006 9:57:48 AM] [...].. .1. 3.5 .18 .1 Defining Models and Prediction Equations 97.5% POINT OF T DISTRIBUTION IDENTIFIER EFFECT = 0 .12 706 216 812 E+02 T VALUE RESSD: RESSD: MEAN + MEAN + TERM CUM TERMS -MEAN 2.65875 1. 7 410 6 1. 7 410 6 1 3 .10 250 21. 8* 0.57272 0.57272 2 -0.86750 -6 .1 1. 812 64 0.30429 23 0.29750 2 .1 1.87270 0.26737 13 0.24750 1. 7 1. 87 513 0.233 41 3 0. 212 50 1. 5 1. 87656 0 .19 1 21 123 0 .14 250 1. 0... (Here, X1 is either a +1 or -1, and similarly for the other factors and interactions (products).) has a residual standard deviation of 0.30429 ohms has a residual standard deviation of 0.26737 ohms has a residual standard deviation of 0.233 41 ohms has a residual standard deviation of 0 .19 1 21 ohms http://www.itl.nist.gov/div898 /handbook/ eda/section3/eda35i1.htm (2 of 3) [5 /1/ 2006 9:57:49 AM] 1. 3.5 .18 .1 Defining... http://www.itl.nist.gov/div898 /handbook/ eda/section3/eda35i2.htm (6 of 7) [5 /1/ 2006 9:57:49 AM] 1. 3.5 .18 .2 Important Factors Conclusions In summary, the seven criteria for specifying "important" factors yielded the following for the Eddy current data: 1 Effects, Engineering Significance: X1, X2 2 Effects, Numerically Significant: X1, X2 3 Effects, Statistically Significant: X1, X2, X2*X3 4 Effects, Probability Plots: X1, X2... standard deviation = 0.233 41) (with a cumulative residual standard deviation = 0 .19 1 21) (with a cumulative residual standard deviation = 0 .18 0 31) (with a cumulative residual standard deviation = 0.00000) Note that we must include all terms in order to drive the residual standard deviation below 0 .12 5 Again, the 5% rule is a rough-and-ready rule that has no basis in engineering or statistics, but is simply... 0 .14 250 1. 0 1. 87876 0 .18 0 31 12 0 .12 750 0.9 1. 87 912 0.00000 The last column of the Yates table gives the residual standard deviation for 8 possible models, each with one more term than the previous model Potential Models For this example, we can summarize the possible prediction equations using the second and last columns of the Yates table: q q q q q q has a residual standard deviation of 1. 7 410 6 ohms... interactions) to include in the model http://www.itl.nist.gov/div898 /handbook/ eda/section3/eda35i1.htm (3 of 3) [5 /1/ 2006 9:57:49 AM] 1. 3.5 .18 .2 Important Factors Effects: Engineering Significance The minimum engineering significant difference is defined as where is the absolute value of the parameter estimate (i.e., the effect) and engineering significant difference is the minimum That is, declare a... data: 1 Important Factors: X1 and X2 2 Parsimonious Prediction Equation: (with a residual standard deviation of 30429 ohms) Note that this is the initial model selection We still need to perform model validation with a residual analysis http://www.itl.nist.gov/div898 /handbook/ eda/section3/eda35i2.htm (7 of 7) [5 /1/ 2006 9:57:49 AM] 1. 3.6 .1 What is a Probability Distribution 1 Exploratory Data Analysis 1. 3... factors This criterion is neither engineering nor statistical, but it does offer some additional numerical insight For the current example, the largest effect is from X1 (3 .10 250 ohms), and so 10 % of that is 0. 31 ohms, which suggests keeping all factors whose effects exceed 0. 31 ohms Based on the order-of-magnitude criterion, we thus conclude that we should keep two terms: X1 and X2 A third term, X2*X3... an engineering point of view For the Eddy current example: 1 the engineer did not know ; 2 the design (a 23 full factorial) did not have replication; http://www.itl.nist.gov/div898 /handbook/ eda/section3/eda35i2.htm (2 of 7) [5 /1/ 2006 9:57:49 AM] 1. 3.5 .18 .2 Important Factors 3 ignoring 3-term interactions and higher interactions leads to an estimate of based on omitting only a single term: the X1*X2*X3... Averages, Youden Plot: X1, X2 6 Residual SD, Engineering Significance: all 7 terms 7 Residual SD, Statistical Significance: not applicable Such conflicting results are common Arguably, the three most important criteria (listed in order of most important) are: 4 Effects, Probability Plots: 1 Effects, Engineering Significance: 3 Residual SD, Engineering Significance: X1, X2 X1, X2 all 7 terms Scanning . -0.86750 -6 .1 1. 812 64 0.30429 23 0.29750 2 .1 1.87270 0.26737 13 0.24750 1. 7 1. 87 513 0.233 41 3 0. 212 50 1. 5 1. 87656 0 .19 1 21 12 3 0 .14 250 1. 0 1. 87876 0 .18 0 31 12 0 .12 750 0.9 1. 87 912 0.00000 . 1. 7 410 6 1. 7 410 6 1 3 .10 250 21. 8* 0.57272 0.57272 2 -0.86750 -6 .1 1. 812 64 0.30429 23 0.29750 2 .1 1.87270 0.26737 13 0.24750 1. 7 1. 87 513 0.233 41 3 0. 212 50 1. 5 1. 87656 0 .19 1 21 12 3 0 .14 250 1. 0. TERM CUM TERMS 1. 3.5 .18 . Yates Analysis http://www.itl.nist.gov/div898 /handbook/ eda/section3/eda35i.htm (3 of 5) [5 /1/ 2006 9:57:48 AM] MEAN 2.65875 1. 7 410 6 1. 7 410 6 1 3 .10 250 21. 8* 0.57272 0.57272