Engineering Statistics Handbook Episode 7 Part 4 pdf

5. Process Improvement 5.4. Analysis of DOE data 5.4.7. Examples of DOE's 5.4.7.2.Fractional factorial example A "Catapult" Fractional Factorial Experiment A step-by-step analysis of a fractional factorial "catapult" experiment This experiment was conducted by a team of students on a catapult – a table-top wooden device used to teach design of experiments and statistical process control. The catapult has several controllable factors and a response easily measured in a classroom setting. It has been used for over 10 years in hundreds of classes. Below is a small picture of a catapult that can be opened to view a larger version. Catapult Description of Experiment: Response and Factors The experiment has five factors that might affect the distance the golf ball travels Purpose: To determine the significant factors that affect the distance the ball is thrown by the catapult, and to determine the settings required to reach 3 different distances (30, 60 and 90 inches). Response Variable: The distance in inches from the front of the catapult to the spot where the ball lands. The ball is a plastic golf ball. Number of observations: 20 (a 2 5-1 resolution V design with 4 center points). Variables: Response Variable Y = distance1. Factor 1 = band height (height of the pivot point for the rubber bands – levels were 2.25 and 4.75 inches with a centerpoint level of 3.5) 2. Factor 2 = start angle (location of the arm when the operator releases– starts the forward motion of the arm – levels were 0 and 20 degrees with a centerpoint level of 10 degrees) 3. Factor 3 = rubber bands (number of rubber bands used on the catapult– levels were 1 and 2 bands) 4. Factor 4 = arm length (distance the arm is extended – levels were 0 and 4 inches with a centerpoint level of 2 inches) 5. Factor 5 = stop angle (location of the arm where the forward motion of the arm is stopped and the ball starts flying – levels were 45 and 80 degrees with a centerpoint level of 62 degrees) 6. 5.4.7.2. Fractional factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri472.htm (1 of 18) [5/1/2006 10:30:51 AM] Design matrix and responses (in run order) The design matrix appears below in (randomized) run order. You can download the data in a spreadsheet Readers who want to analyze this experiment may download an Excel spreadsheet catapult.xls or a JMP spreadsheet capapult.jmp. One discrete factor Note that 4 of the factors are continuous, and one – number of rubber bands – is discrete. Due to the presence of this discrete factor, we actually have two different centerpoints, each with two runs. Runs 7 and 19 are with one rubber band, and the center of the other factors, while runs 2 and 13 are with two rubber bands and the center of the other factors. 5 confirmatory runs After analyzing the 20 runs and determining factor settings needed to achieve predicted distances of 30, 60 and 90 inches, the team was asked to conduct 5 confirmatory runs at each of the derived settings. Analysis of the Experiment Analyze with JMP software The experimental data will be analyzed using SAS JMP 3.2.6 software. Step 1: Look at the data 5.4.7.2. Fractional factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri472.htm (2 of 18) [5/1/2006 10:30:51 AM] Histogram, box plot, and normal probability plot of the response We start by plotting the data several ways to see if any trends or anomalies appear that would not be accounted for by the models. The distribution of the response is given below: We can see the large spread of the data and a pattern to the data that should be explained by the analysis. Plot of response versus run order Next we look at the responses versus the run order to see if there might be a time sequence component. The four highlighted points are the center points in the design. Recall that runs 2 and 13 had 2 rubber bands and runs 7 and 19 had 1 rubber band. There may be a slight aging of the rubber bands in that the second center point resulted in a distance that was a little shorter than the first for each pair. 5.4.7.2. Fractional factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri472.htm (3 of 18) [5/1/2006 10:30:51 AM] Plots of responses versus factor columns Next look at the plots of responses sorted by factor columns. 5.4.7.2. Fractional factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri472.htm (4 of 18) [5/1/2006 10:30:51 AM] 5.4.7.2. Fractional factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri472.htm (5 of 18) [5/1/2006 10:30:51 AM] Several factors appear to change the average response level and most have a large spread at each of the levels. Step 2: Create the theoretical model The resolution V design can estimate main effects and all 2-factor interactions With a resolution V design we are able to estimate all the main effects and all two-factor interactions cleanly – without worrying about confounding. Therefore, the initial model will have 16 terms – the intercept term, the 5 main effects, and the 10 two-factor interactions. Step 3: Create the actual model from the data Variable coding Note we have used the orthogonally coded columns for the analysis, and have abbreviated the factor names as follows: Bheight = band height Start = start angle Bands = number of rubber bands Stop = stop angle Arm = arm length. JMP output after fitting the trial model (all main factors and 2-factor interactions) The following is the JMP output after fitting the trial model (all main factors and 2-factor interactions). 5.4.7.2. Fractional factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri472.htm (6 of 18) [5/1/2006 10:30:51 AM] Use p-values to help select significant effects, and also use a normal plot The model has a good R 2 value, but the fact that R 2 adjusted is considerably smaller indicates that we undoubtedly have some terms in our model that are not significant. Scanning the column of p-values (labeled Prob>|t| in the JMP output) for small values shows 5 significant effects at the 0.05 level and another one at the 0.10 level. The normal plot of effects is a useful graphical tool to determine significant effects. The graph below shows that there are 9 terms in the model that can be assumed to be noise. That would leave 6 terms to be included in the model. Whereas the output above shows a p-value of 0.0836 for the interaction of bands and arm, the normal plot suggests we treat this interaction as significant. 5.4.7.2. Fractional factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri472.htm (7 of 18) [5/1/2006 10:30:51 AM] A refit using just the effects that appear to matter Remove the non-significant terms from the model and refit to produce the following output: R 2 is OK and there is no significant model "lack of fit" The R 2 and R 2 adjusted values are acceptable. The ANOVA table shows us that the model is significant, and the Lack of Fit table shows that there is no significant lack of fit. The Parameter estimates table is below. Step 4: Test the model assumptions using residual graphs (adjust and simplify as needed) 5.4.7.2. Fractional factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri472.htm (8 of 18) [5/1/2006 10:30:51 AM] Histogram of the residuals to test the model assumptions We should test that the residuals are approximately normally distributed, are independent, and have equal variances. First we create a histogram of the residual values. The residuals do appear to have, at least approximately, a normal distributed. Plot of residuals versus predicted values Next we plot the residuals versus the predicted values. There does not appear to be a pattern to the residuals. One observation about the graph, from a single point, is that the model performs poorly in predicting a short distance. In fact, run number 10 had a measured distance of 8 inches, but the model predicts -11 inches, giving a residual of 19. The fact that the model predicts an impossible negative distance is an obvious shortcoming of the model. We may not be successful at predicting the catapult settings required to hit a distance less 5.4.7.2. Fractional factorial example http://www.itl.nist.gov/div898/handbook/pri/section4/pri472.htm (9 of 18) [5/1/2006 10:30:51 AM] [...]... band fatigue?) http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 47 2 .htm ( 14 of 18) [5/1/2006 10:30:51 AM] 5 .4 .7. 2 Fractional factorial example Plot of residuals versus the factor variables Next we look at the residual values versus each of the factors http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 47 2 .htm (15 of 18) [5/1/2006 10:30:51 AM] 5 .4 .7. 2 Fractional factorial example The residuals... = 0, bands = -1, arm = 5 and stop = 5 http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 47 2 .htm ( 17 of 18) [5/1/2006 10:30:51 AM] 5 .4 .7. 2 Fractional factorial example Prediction profile plots for Y = 90 Finally, we set LN Y = LN 90 = 4. 4998 and obtain (see the figure below) a predicted log distance of 90.20 when bheight = -0. 87, start = -0.52, bands = 1, arm = 1, and stop = 0 "Confirmation" runs... values Recall that run numbers 2 and 13 had two rubber bands while run numbers 7 and 19 had only one rubber band Plots of residuals versus the factor variables Next we look at the residual values versus each of the factors http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 47 2 .htm (10 of 18) [5/1/2006 10:30:51 AM] 5 .4 .7. 2 Fractional factorial example The residual graphs are not ideal, although the... for this new model fit Step 4a: Test the (new) model assumptions using residual graphs (adjust and simplify as needed) Normal probability plot, box plot, and histogram of the residuals The following normal plot, box plot, and histogram of the residuals shows no problems http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 47 2 .htm (13 of 18) [5/1/2006 10:30:51 AM] 5 .4 .7. 2 Fractional factorial example... the coded columns of the matrix for the factor levels and Y = the natural logarithm of distance as the response, we initially obtain: http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 47 2 .htm (12 of 18) [5/1/2006 10:30:51 AM] 5 .4 .7. 2 Fractional factorial example A simpler model with just main effects has a satisfactory fit Examining the p-values of the 16 model coefficients, only the intercept... conducting a new experiment designed to fit a quadratic model Step 5: Use the results to answer the questions in your experimental objectives http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 47 2 .htm (16 of 18) [5/1/2006 10:30:51 AM] 5 .4 .7. 2 Fractional factorial example Final step: quantify the influence of all the significant effects and predict what settings should be used to obtain desired distances... log Y of 3.399351 (or a distance of 29. 94) The corresponding (coded) factor settings are: bheight = 0. 17, start = -1, bands = -1, arm = -1 and stop = 0 Prediction profile plots for Y = 30 Prediction profile plots for Y = 60 Repeating the profiler search for a Y value of 60 (or LN Y = 4. 0 94) yielded the figure below for which a natural log distance value of 4. 0 941 21 is predicted (a distance of 59.99)... fit, or sign of curvature at the centerpoint values The Lack of Fit table, however, indicates that the lack of fit is not significant http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 47 2 .htm (11 of 18) [5/1/2006 10:30:51 AM] 5 .4 .7. 2 Fractional factorial example Consider a transformation of the response variable to see if we can obtain a better model At this point, since there are several unsatisfactory... all 3 targets, but did not hit them all 5 times NOTE: The model discovery and fitting process, as illustrated in this analysis, is often an iterative process http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 47 2 .htm (18 of 18) [5/1/2006 10:30:51 AM] ... in the figure below shows the direction and strength of each of the main effects in the model Using natural log 30 = 3 .40 1 as the target value, the Profiler allows us to set up a "Desirability" function that gives 3 .40 1 a maximum desirability value of 1 and values above or below 3 .40 1 have desirabilities that rapidly decrease to 0 This is shown by the desirability graph on the right (see the figure . sorted by factor columns. 5 .4 .7. 2. Fractional factorial example http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 47 2 .htm (4 of 18) [5/1/2006 10:30:51 AM] 5 .4 .7. 2. Fractional factorial example http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 47 2 .htm. flying – levels were 45 and 80 degrees with a centerpoint level of 62 degrees) 6. 5 .4 .7. 2. Fractional factorial example http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 47 2 .htm (1 of 18). suggests we treat this interaction as significant. 5 .4 .7. 2. Fractional factorial example http://www.itl.nist.gov/div898 /handbook/ pri/section4/pri 47 2 .htm (7 of 18) [5/1/2006 10:30:51 AM] A refit using just