Engineering Statistics Handbook Episode 7 Part 3 potx

Normal or half-normal plots of effects (primarily for two-level full and fractional factorial experiments) ❍ Youden plots❍ Other methods❍ 5.4.4. How to test and revise DOE models http://www.itl.nist.gov/div898/handbook/pri/section4/pri44.htm (2 of 2) [5/1/2006 10:30:47 AM] 5. Process Improvement 5.4. Analysis of DOE data 5.4.6.How to confirm DOE results (confirmatory runs) Definition of confirmation runs When the analysis of the experiment is complete, one must verify that the predictions are good. These are called confirmation runs. The interpretation and conclusions from an experiment may include a "best" setting to use to meet the goals of the experiment. Even if this "best" setting were included in the design, you should run it again as part of the confirmation runs to make sure nothing has changed and that the response values are close to their predicted values. would get. At least 3 confirmation runs should be planned In an industrial setting, it is very desirable to have a stable process. Therefore, one should run more than one test at the "best" settings. A minimum of 3 runs should be conducted (allowing an estimate of variability at that setting). If the time between actually running the experiment and conducting the confirmation runs is more than a few hours, the experimenter must be careful to ensure that nothing else has changed since the original data collection. Carefully duplicate the original environment The confirmation runs should be conducted in an environment as similar as possible to the original experiment. For example, if the experiment were conducted in the afternoon and the equipment has a warm-up effect, the confirmation runs should be conducted in the afternoon after the equipment has warmed up. Other extraneous factors that may change or affect the results of the confirmation runs are: person/operator on the equipment, temperature, humidity, machine parameters, raw materials, etc. 5.4.6. How to confirm DOE results (confirmatory runs) http://www.itl.nist.gov/div898/handbook/pri/section4/pri46.htm (1 of 2) [5/1/2006 10:30:47 AM] Checks for when confirmation runs give surprises What do you do if you don't obtain the results you expected? If the confirmation runs don't produce the results you expected: check to see that nothing has changed since the original data collection 1. verify that you have the correct settings for the confirmation runs 2. revisit the model to verify the "best" settings from the analysis3. verify that you had the correct predicted value for the confirmation runs. 4. If you don't find the answer after checking the above 4 items, the model may not predict very well in the region you decided was "best". You still learned from the experiment and you should use the information gained from this experiment to design another follow-up experiment. Even when the experimental goals are not met, something was learned that can be used in a follow-up experiment Every well-designed experiment is a success in that you learn something from it. However, every experiment will not necessarily meet the goals established before experimentation. That is why it makes sense to plan to experiment sequentially in order to meet the goals. 5.4.6. How to confirm DOE results (confirmatory runs) http://www.itl.nist.gov/div898/handbook/pri/section4/pri46.htm (2 of 2) [5/1/2006 10:30:47 AM] 5. Process Improvement 5.4. Analysis of DOE data 5.4.7. Examples of DOE's 5.4.7.1.Full factorial example Data Source This example uses data from a NIST high performance ceramics experiment This data set was taken from an experiment that was performed a few years ago at NIST (by Said Jahanmir of the Ceramics Division in the Material Science and Engineering Laboratory). The original analysis was performed primarily by Lisa Gill of the Statistical Engineering Division. The example shown here is an independent analysis of a modified portion of the original data set. The original data set was part of a high performance ceramics experiment with the goal of characterizing the effect of grinding parameters on sintered reaction-bonded silicon nitride, reaction bonded silicone nitride, and sintered silicon nitride. Only modified data from the first of the 3 ceramic types (sintered reaction-bonded silicon nitride) will be discussed in this illustrative example of a full factorial data analysis. The reader may want to download the data as a text file and try using other software packages to analyze the data. Description of Experiment: Response and Factors Response and factor variables used in the experiment Purpose: To determine the effect of machining factors on ceramic strength Response variable = mean (over 15 repetitions) of the ceramic strength Number of observations = 32 (a complete 2 5 factorial design) Response Variable Y = Mean (over 15 reps) of Ceramic Strength Factor 1 = Table Speed (2 levels: slow (.025 m/s) and fast (.125 m/s)) Factor 2 = Down Feed Rate (2 levels: slow (.05 mm) and fast (.125 mm)) Factor 3 = Wheel Grit (2 levels: 140/170 and 80/100) Factor 4 = Direction (2 levels: longitudinal and transverse) Factor 5 = Batch (2 levels: 1 and 2) Since two factors were qualitative (direction and batch) and it was reasonable to expect monotone effects from the quantitative factors, no centerpoint runs were included. 5.4.7.1. Full factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri471.htm (1 of 15) [5/1/2006 10:30:49 AM] JMP spreadsheet of the data The design matrix, with measured ceramic strength responses, appears below. The actual randomized run order is given in the last column. (The interested reader may download the data as a text file or as a JMP file.) Analysis of the Experiment Analysis follows 5 basic steps The experimental data will be analyzed following the previously described 5 basic steps using SAS JMP 3.2.6 software. Step 1: Look at the data 5.4.7.1. Full factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri471.htm (2 of 15) [5/1/2006 10:30:49 AM] Plot the response variable We start by plotting the response data several ways to see if any trends or anomalies appear that would not be accounted for by the standard linear response models. First we look at the distribution of all the responses irrespective of factor levels. The following plots were generared: The first plot is a normal probability plot of the response variable. The straight red line is the fitted nornal distribution and the curved red lines form a simultaneous 95% confidence region for the plotted points, based on the assumption of normality. 1. The second plot is a box plot of the response variable. The "diamond" is called (in JMP) a "means diamond" and is centered around the sample mean, with endpoints spanning a 95% normal confidence interval for the sample mean. 2. The third plot is a histogram of the response variable.3. Clearly there is "structure" that we hope to account for when we fit a response model. For example, note the separation of the response into two roughly equal-sized clumps in the histogram. The first clump is centered approximately around the value 450 while the second clump is centered approximately around the value 650. 5.4.7.1. Full factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri471.htm (3 of 15) [5/1/2006 10:30:49 AM] Plot of response versus run order Next we look at the responses plotted versus run order to check whether there might be a time sequence component affecting the response levels. Plot of Response Vs. Run Order As hoped for, this plot does not indicate that time order had much to do with the response levels. Box plots of response by factor variables Next, we look at plots of the responses sorted by factor columns. 5.4.7.1. Full factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri471.htm (4 of 15) [5/1/2006 10:30:49 AM] Several factors, most notably "Direction" followed by "Batch" and possibly "Wheel Grit", appear to change the average response level. Step 2: Create the theoretical model Theoretical model: assume all 4-factor and higher interaction terms are not significant With a 2 5 full factorial experiment we can fit a model containing a mean term, all 5 main effect terms, all 10 2-factor interaction terms, all 10 3-factor interaction terms, all 5 4-factor interaction terms and the 5-factor interaction term (32 parameters). However, we start by assuming all three factor and higher interaction terms are non-existent (it's very rare for such high-order interactions to be significant, and they are very difficult to interpret from an engineering viewpoint). That allows us to accumulate the sums of squares for these terms and use them to estimate an error term. So we start out with a theoretical model with 26 unknown constants, hoping the data will clarify which of these are the significant main effects and interactions we need for a final model. Step 3: Create the actual model from the data Output from fitting up to third-order interaction terms After fitting the 26 parameter model, the following analysis table is displayed: Output after Fitting Third Order Model to Response Data Response: Y: Strength Summary of Fit RSquare 0.995127 RSquare Adj 0.974821 Root Mean Square Error 17.81632 Mean of Response 546.8959 Observations 32 Effect Test Sum Source DF of Squares F Ratio Prob>F X1: Table Speed 1 894.33 2.8175 0.1442 X2: Feed Rate 1 3497.20 11.0175 0.0160 X1: Table Speed* 1 4872.57 15.3505 0.0078 X2: Feed Rate X3: Wheel Grit 1 12663.96 39.8964 0.0007 X1: Table Speed* 1 1838.76 5.7928 0.0528 X3: Wheel Grit 5.4.7.1. Full factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri471.htm (5 of 15) [5/1/2006 10:30:49 AM] X2: Feed Rate* 1 307.46 0.9686 0.3630 X3: Wheel Grit X1:Table Speed* 1 357.05 1.1248 0.3297 X2: Feed Rate* X3: Wheel Grit X4: Direction 1 315132.65 992.7901 <.0001 X1: Table Speed* 1 1637.21 5.1578 0.0636 X4: Direction X2: Feed Rate* 1 1972.71 6.2148 0.0470 X4: Direction X1: Table Speed 1 5895.62 18.5735 0.0050 X2: Feed Rate* X4: Direction X3: Wheel Grit* 1 3158.34 9.9500 0.0197 X4: Direction X1: Table Speed* 1 2.12 0.0067 0.9376 X3: Wheel Grit* X4: Direction X2: Feed Rate* 1 44.49 0.1401 0.7210 X3: Wheel Grit* X4: Direction X5: Batch 1 33653.91 106.0229 <.0001 X1: Table Speed* 1 465.05 1.4651 0.2716 X5: Batch X2: Feed Rate* 1 199.15 0.6274 0.4585 X5: Batch X1: Table Speed* 1 144.71 0.4559 0.5247 X2: Feed Rate* X5: Batch X3: Wheel Grit* 1 29.36 0.0925 0.7713 X5: Batch X1: Table Speed* 1 30.36 0.0957 0.7676 X3: Wheel Grit* X5: Batch X2: Feed Rate* 1 25.58 0.0806 0.7860 X3: Wheel Grit* X5: Batch X4: Direction * 1 1328.83 4.1863 0.0867 X5: Batch X1: Table Speed* 1 544.58 1.7156 0.2382 X4: Directio* X5: Batch X2: Feed Rate* 1 167.31 0.5271 0.4952 X4: Direction* X5: Batch X3: Wheel Grit* 1 32.46 0.1023 0.7600 X4: Direction* X5: Batch This fit has a high R 2 and adjusted R 2 , but the large number of high (>0.10) p-values (in the "Prob>F" column) make it clear that the model has many unnecessary terms. 5.4.7.1. Full factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri471.htm (6 of 15) [5/1/2006 10:30:49 AM] [...]... Squares 894 .33 34 97. 20 4 872 . 57 F Ratio 3. 9942 15.6191 21 .76 18 Prob>F 0.0602 0.0009 0.0002 1 1 126 63. 96 1 838 .76 56.5595 8.2122 . Rate 1 34 97. 20 11.0 175 0.0160 X1: Table Speed* 1 4 872 . 57 15 .35 05 0.0 078 X2: Feed Rate X3: Wheel Grit 1 126 63. 96 39 .8964 0.00 07 X1: Table Speed* 1 1 838 .76 5 .79 28 0.0528 X3: Wheel Grit 5.4 .7. 1 example http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 471 .htm (5 of 15) [5/1/2006 10 :30 :49 AM] X2: Feed Rate* 1 3 07. 46 0.9686 0 .36 30 X3: Wheel Grit X1:Table Speed* 1 3 57. 05 1.1248 0 .32 97 X2: Feed Rate* X3: Wheel. Rate* X5: Batch X3: Wheel Grit* 1 29 .36 0.0925 0 .77 13 X5: Batch X1: Table Speed* 1 30 .36 0.09 57 0 .76 76 X3: Wheel Grit* X5: Batch X2: Feed Rate* 1 25.58 0.0806 0 .78 60 X3: Wheel Grit* X5: