Tài liệu hướng dẫn chi tiết về nguyên tắc, phương pháp làm thí nghiệm cũng như xử lý số liệu thí nghiệm. Tài liệu rất hữu ích cho sinh viên và cáchi nhà khoa học nông nghiệp, nhưng đối tượng hay làm thí nghiệm đồng ruộng.
LEUVEN STATISTICS RESEARCH CENTER Internal DOE course Principles of Experimentation Basics of Statistics, Design and Analysis Experiments, Design of Computer Experiments of B De Ketelaere, K Rutten, P Darius 2014 Version Contents Introduction 11 Course Objectives 11 Course Contents 12 Basics of Statistics 13 1.1 1.2 1.3 What is Statistics 13 Basic Statistical Concepts 14 Describing a Sample 14 1.3.1 Graphical description using histogram 15 1.3.2 Numerical representation 15 1.4 1.5 1.6 Describing a population 16 The Central Limit Theorem 17 Confidence Intervals 19 1.6.1 Derivation of a 95% confidence interval for µ (σ known) 20 1.6.2 Changing the confidence level (in case σ is known) 22 1.6.3 Confidence intervals in case σ is not known (real life) 23 1.7 Hypothesis Testing 24 1.7.1 Prob-value (p-value) 25 1.7.2 Type I and Type II Errors 26 1.8 Some Important Distributions 27 1.8.1 Standard Normal Distribution 27 1.8.2 t-distribution 29 1.8.3 F-distribution 31 Basic Descriptive Statistics in JMP (version & 10) 32 2.1 2.2 2.3 Histogram 32 Display options 34 Highlighting bars and selecting rows 35 2.3.1 Histogram Scaling 35 2.3.2 Adjusting histogram bars 35 2.4 Options specific for each variable 36 2.4.1 JMP 2.4.2 JMP 10 2.5 2.6 2.7 2.8 36 37 Histogram Options 38 Curves 38 Normal Quantile Plot 40 Cumulative Density Function plot (CDF) 42 PRINCIPLES OF EXPERIMENTATION V7 CONTENTS | 2.9 Hypothesis Testing in JMP 43 2.9.1 Test whether the distribution is normal 43 2.9.2 Test on Mean and Standard Deviation 44 2.10 Example Displaying Data and Distributions - Waterbed 46 2.10.1 Example description 46 2.10.2 Solution – Approach 48 2.10.3 Solution – Approach 50 Simple Linear Regression 51 3.1 3.2 Introduction 51 The Statistical Approach to regression 55 3.2.1 Introduction 55 3.2.2 Inference 57 3.2.3 How good is the regression line 60 3.3 3.4 Correlation 62 Example: Regression Analysis Using JMP 63 3.4.1 The Fit Line Command 65 3.4.2 Density Ellipse 67 3.5 Regression With Curved Relationships 69 Multiple Regression 70 4.1 4.2 General 70 Model selection in Multiple Regression 71 4.2.1 Guiding principles for model building 72 4.2.2 Approach 1: All possible regressions 74 4.2.3 Approach 2: Stepwise Regression 80 Regression Examples & Exercises 81 5.1 Operational Data of Airline Company 81 5.1.1 Exercise description 81 5.1.2 Task description 82 5.1.3 Solution 84 5.2 Cholesterol in Adult Males 93 5.2.1 5.2.2 5.2.3 5.2.4 5.2.5 5.3 Exercise description 93 Solution – Overlay plot 94 Solution – Simple regression 95 First Method 96 Second Method 96 Methane Gas 97 5.3.1 Exercise description 97 5.3.2 Task description 99 5.3.3 Solutions 100 PRINCIPLES OF EXPERIMENTATION V7 CONTENTS | Design of Experiments – Basic Aspects 106 6.1 6.2 Observational Versus Designed Experiments 106 Design definitions 108 6.2.1 Experimental run 108 6.2.2 Design 108 6.2.3 Experimental region 108 6.3 6.4 6.5 What is a good design? 109 Design with Factor (Example of Simple Linear Regression) 109 Design with factors - An Illustrative Example 112 6.5.1 Experiment 113 6.5.3 Experiment 114 6.6 Orthogonality 115 6.6.1 Implications Of Orthogonality 117 6.6.2 Recipes For Orthogonality 119 6.7 6.8 Precision And Power 120 Protection Against Known Enemies – Blocking 121 6.8.1 Illustrative Example 121 6.8.2 Blocking 122 6.8.3 Examples Of Blocking Variables 123 6.9 Protection Against Unknown Enemies - Randomization 125 Planning The Experiments 126 7.1 7.2 7.3 7.4 7.5 Purpose 127 Responses 128 Experimental Factors and Region 128 Plan Details 130 After Performing The Experiment 131 Factorial Designs 132 8.1 8.2 8.3 Coding Of 2k Designs 133 Analysis of 2k Designs 134 Geometric Representation Of 22 And 23 Designs 136 8.3.1 Factor Effects 136 8.3.2 Interaction Effects 137 8.5 8.6 8.7 Sample Size 139 Centerpoints 141 Practical Application 142 Factorial Design Examples & Exercises 143 9.1 Example – A 24 Design 143 PRINCIPLES OF EXPERIMENTATION V7 CONTENTS | 9.1.1 Exercise description 143 9.1.2 Task description 143 9.1.3 Solution 144 9.2 9.3 Example – A 22 Design With Centerpoints 149 Exercise – Carbonation Soft Drinks 152 10 Screening Designs 155 10.1 Plackett-Burman Designs 158 10.1.1 Properties 158 10.1.2 Construction 158 10.1.3 Use 159 10.1.4 Combination With Reflected Designs 159 10.2 Fractional Factorial Designs 159 10.2.1 Properties 159 10.2.2 Construction 161 10.2.3 Use 161 10.3 Recommendation For Screening Designs 161 11 Screening Designs – Examples & Exercises 162 11.1 Example – Construction of a 23 screening design 162 11.2 Example – Plackett-Burman Design For 11 two-level factors 164 11.3 Exercise – Bread Baking Screening Experiment 165 12 Second Order Designs 168 12.1 Box-Behnken Designs 170 12.1.1 Construction 170 12.1.3 Construction in JMP 171 12.2 Face-Centered Cube Designs 173 12.2.1 Construction 174 12.3 Central Composite Designs 175 12.3.1 Construction 176 12.4 How To Analyze a Response Surface Design in JMP 177 12.5 Useful Second-Order Composite Designs 181 13 Response Surface Methodology 182 13.1 Outline Of RSM Strategy 182 13.2 RSM Overview: Putting It All Together 184 13.2.1 Preliminary Stage 184 13.2.2 Stage - Screening 184 13.2.3 Stage - Limited Response Surface 185 13.2.4 Stage - Full Response Surface 187 PRINCIPLES OF EXPERIMENTATION V7 CONTENTS | 14 Illustration Of The RSM-Methodology 188 14.1 Problem Description 188 14.2 Task description 192 14.3 Solution 193 14.3.1 Limited Response Surface 198 14.3.2 Full Response Surface 201 15 Comprehensive Case Study – Garden Sprinkler 208 15.1 Exercise Description 208 15.2 The Simulator 211 15.2.1 Running a design 211 16 One factor Analysis of Variance 213 16.1 16.2 16.3 16.4 16.5 16.6 16.7 Introduction 213 Terminology 216 ANOVA: Different Model Types 217 One Factor ANOVA Model 218 Estimation Of The One Factor ANOVA Model 220 Testing Whether All Means Are Equal 223 Testing Which Treatments Are Different 224 16.7.1 Visualizing Differences 224 16.7.2 Confidence Intervals For 𝝁𝒊′𝒔 225 16.7.3 Confidence Intervals for A difference D 225 16.7.4 Confidence Intervals For A Contrast L 226 16.8 Multiple Comparisons 228 16.8.1 Tukey Multiple Comparisons Procedure 229 16.8.2 Scheffé multiple comparisons procedure 229 16.8.3 Bonferroni multiple comparisons procedure 229 16.8.4 Dunnett’s Multiple Comparisons Procedure 230 16.9 JMP® example 230 17 One Factor ANOVA Exercises 243 17.1 Rolling Resistance Of Tires 243 17.1.1 Exercise Description 243 17.1.2 Solution 243 18 Two factor Analysis of Variance 246 18.1 18.2 18.3 18.4 Introduction 246 Graphical representation 247 Additive model – a special case 248 General two way ANOVA model 249 PRINCIPLES OF EXPERIMENTATION V7 CONTENTS | 18.5 Analysis of a two factor ANOVA model 252 18.5.1 Equal sample size, 𝒏𝒊𝒋 = 𝒏 > 𝟏 253 18.5.2 Unequal sample size, 𝒏𝒊𝒋 > 𝟏 256 19 Two factor ANOVA Exercises 262 19.1 Rubber Modulus 262 19.1.1 Exercise Description 262 19.1.2 Solution 263 20 Experiments with Mixtures 270 20.1 Introduction 270 20.2 Designs for Unconstrained Mixtures 272 20.2.1 Simplex-lattice designs 272 20.2.2 Simplex-centroid designs 273 20.2.3 Augmented simplex-centroid designs 274 20.3 Models for mixtures designs 274 20.4 Designs for constrained mixtures 276 20.4.1 Designs with pseudocomponents 277 20.4.2 Example - Exploring Concrete Mixtures with Pseudocomponents 279 20.4.3 Factorial ratios designs 280 21 Mixture Experiments – Examples & Exercises 281 21.1 Unconstrained Mixture Design - Fruit Juice 281 21.1.1 Exercise Description 281 21.1.2 Task Description 281 21.1.3 Solution 283 21.2 Example Constrained Mixture Designs – Harvey Wallbangers 288 22 Introduction to Computer Experimentation 291 22.1 22.2 22.3 22.4 Introduction 291 Computer experiments versus physical experiments 293 Desirable Properties for Computer Designs 295 A Simple Example 297 23 Common Designs For Computer Experiments 299 23.1 Latin Hypercube Designs (LHDs) 300 23.1.1 Understanding Latin Hypercube Designs 300 23.1.2 Examples 305 23.1.3 Properties of LHDs 309 23.2 Uniform Designs 310 23.2.1 Understanding Uniform Designs 310 23.2.2 Properties of Uniform Designs 313 PRINCIPLES OF EXPERIMENTATION V7 CONTENTS | 23.3 Distance based designs 314 23.4 Other types of designs 317 23.5 Choosing an Appropriate Number of Runs 320 24 The Design of Computer Experiments - Exercises 321 25 The Analysis of Computer Experiments 323 25.1 Introduction 323 25.2 Response Surface Methodology 326 25.2.1 Use of Least Squares for Deterministic Responses 326 25.2.2 Consequences of the Use of Least Squares 328 25.2.3 Assessing Model Adequacy 329 25.2.4 Final note 330 25.3 Kriging Approach 330 25.3.1 Introduction 330 25.3.2 The Kriging Model - Theoretical Aspects 331 25.3.3 The Role of the Distance Function 𝑹(∙) 337 25.3.4 The Gaussian Correlation Function 338 25.3.5 The Cubic Correlation Function 338 25.3.6 Kriging Types and Relation to Other Methods 340 25.3.7 Visual Representation 341 25.4 RSM Versus Kriging Approach 344 26 Analysis of Computer Experiments - Exercises 345 Literature 353 PRINCIPLES OF EXPERIMENTATION V7 CONTENTS | 25.3.7 Visual Representation A first glimpse on the data approximation / data interpolation capability of kriging is given in figure 25.2 The black curve represents the true function, whereas the red line gives the kriging approximation for different sample sizes It is clear that the number of samples and the exact positioning of the samples is crucial to obtain good approximation properties Figure 25.2 - Kriging approximations for different sample sizes Black: true function; red: approximation PRINCIPLES OF EXPERIMENTATION V8 PAGE 341 Following three figures demonstrate the kriging approach for three very different values of 𝜃𝑘 If 𝜃𝑘 is close to zero, it means that parameter 𝑥𝑘 does not have a big impact on the response Indeed, the 𝑟𝑖𝑗 ~ in that case so that responses are equal no matter the difference between the factor levels 𝑥𝑖𝑘 and 𝑥𝑗𝑘 In contrary, large 𝜃𝑘 values result in a rapid decrease of the influence of one point on another with increasing distance (a) PRINCIPLES OF EXPERIMENTATION V8 PAGE 342 (b) (c) Figure 25.3 - Three examples of kriging, with a very high value of θk, a medium value and a low value PRINCIPLES OF EXPERIMENTATION V8 PAGE 343 25.4 RSM Versus Kriging Approach We have now discussed the two most often used approaches to analyzing computer simulation data based on a metamodel We summarize in the Table below the main (dis-)advantages of both approaches Table 25.1 – RSM versus kriging RSM Interpretable in terms of factors: slopes allow to quantify how much the response changes with a unit change in the considered factor Software allows for automatic screening of main effects and their possible interactions Based on assumptions that not hold Can handle simple parametric forms Available in every statistical software PRINCIPLES OF EXPERIMENTATION V8 Kriging Hard to interprete the effects of changing a factor level Requires manual work, no formal tests for checking parameter significance Takes into account the particularities of deterministic code outputs Can handle highly nonlinear forms Requires specialized software PAGE 344 26 Analysis of Computer Experiments - Exercises EXAMPLE BASED ON Jones, B and Johnson, RT, Design and Analysis for the Gaussian Process Model Qual Reliab Engng Int 2009; 25:515–524 We have discussed several ways of designing an experiment for computer simulations, as well as two ways of analyzing them The usefulness of any computer simulation experiment depends on both the design and the model that is fit We thus consider two designs and two modeling approaches In this exercise, we will now apply them to a nonlinear, known function As an example, we will take the noncentral F distribution Its cumulative density function (cdf) is given by where 𝐼((𝑥 |𝑎, 𝑏)) is the incomplete beta function with parameters 𝑎 and 𝑏 It has inputs In our exercise, we will use its quantile function It can be generated in JMP as follows: PRINCIPLES OF EXPERIMENTATION V8 PAGE 345 Figure 26.1 – The noncentral F Quantile function in JMP Take as limits for the analyses that we will perform the following values: Table 26.1 – Limits for the design Parameter Numerator df Denominator df Probability Noncentrality PRINCIPLES OF EXPERIMENTATION V8 Minimum 2 0.5 Maximum 10 10 0.99 PAGE 346 Perform the following in JMP: Let us start with RSM and take one of the most complex RSM models that is usually applied, being a full cubic function, i.e with all possible interactions a Generate a D-optimal custom design given (1) the limits from Table 15.1 and (2) the fact that we want to fit the full cubic model Save it to a data table Remember the number of runs in the design b Generate the corresponding noncentral F Quantile in the Y column using the formula node explained above c Fit the cubic model and retain only the significant terms that you find Save the predicted values in a separate column d Write down the prediction formula in a Word® document and interpret PRINCIPLES OF EXPERIMENTATION V8 PAGE 347 Let us now look at Gaussian Processes Go through the following steps: a Generate a LHD with as many runs as you obtained in the RSM approach Take the same design limits as above Save the design to a new data table b Generate the corresponding noncentral F Quantile in the Y column using the formula node explained above c Fit a Gaussian Process model to the data Use the default settings proposed by JMP d Interpret the output table you obtained Save the predicted values to a new column in the data table e Write down the prediction formula in a Word® document and interpret PRINCIPLES OF EXPERIMENTATION V8 PAGE 348 Let us now perform a validation step, i.e we will now generate 204 = 160 000 design points within the limits from above You can use the JMP script that we provided and that we labeled ‘ff script.JSL’ Copy now the prediction formulas from the RSM and Gaussian Process modeling into two new columns We can now compare both methods a Compute the procentual error for each observation and for each method Use 100 × |𝑌 − 𝑌̂|/𝑌 b Plot the distribution of the procentual error for both methods c What you conclude? If time permits, try also to fit the Gaussian Process modeling based on the custom design, and the RSM based on the LHD, and add those to your comparison scheme PRINCIPLES OF EXPERIMENTATION V8 PAGE 349 PRINCIPLES OF EXPERIMENTATION V8 PAGE 350 Let us now perform a validation step, i.e we will now generate 204 = 160 000 design points within the limits from above You can use the JMP script that we provided and that we labeled ‘ff script.JSL’ Copy now the prediction formulas from the RSM and Gaussian Process modeling into two new columns We can now compare both methods a Compute the procentual error for each observation and for each method Use 100 × |𝑌 − 𝑌̂|/𝑌 b Plot the distribution of the procentual error for both methods c What you conclude? If time permits, try also to fit the Gaussian Process modeling based on the custom design, and the RSM based on the LHD, and add those to your comparison scheme PRINCIPLES OF EXPERIMENTATION V8 PAGE 351 Literature Box, G E P., Hunter, J S., Hunter, W G (2005) Statistics for Experimenters Wiley-Interscience Haaland, P D (1989) Experimental Design in Biotechnology Marcel Dekker Kutner, M H., Nachtsheim, C J., Neter, J & Li, W (2004) Applied Linear Statistical Models McGraw-Hill Irwin Mee, R W (2009) A Comprehensive Guide to Factorial Two-Level Experimentation Springer Montgomery, D C (2009) Design and Analysis of Experiments Wiley Myers, R H., Montgomery, D C & Anderson-Cook, C.M (2009) Response surface methodology: Process and product optimization using designed experiments Wiley PRINCIPLES OF EXPERIMENTATION V7 ERROR! USE THE HOME TAB TO APPLY KOP TO THE TEXT THAT YOU WANT TO APPEAR HERE | 353 Oehlert, G W (2000) A First Course in Design and Analysis of Experiments W H Freemand and Company Wu, C F J & Hamada, M (2009) Experiments: Planning, Analysis and Parameter Design Optimization Wiley Series in Probability and Statistics PRINCIPLES OF EXPERIMENTATION V7 ERROR! USE THE HOME TAB TO APPLY KOP TO THE TEXT THAT YOU WANT TO APPEAR HERE | 354 LSTAT LEUVEN STATISTICS RESEARCH CENTRE Celestijnenlaan 200 B 3001 Heverlee, BELGIË tel + 32 16 32 22 42 bart.deketelaere@biw.kuleuven.be lstat.kuleuven.be [...]... uncertainty: a review of some basic statistical principles; Relating the behavior of one variable to the setting of other variables: basics of regression; Reasons for designing experiments Examples of good and bad designs; Basic principles of experimental design: replication, randomization, blocking Factorial designs: set-up, analysis, interpretation How to determine the necessary number of experimental... Population variance = 2 PRINCIPLES OF EXPERIMENTATION V8 PAGE 16 1.5 The Central Limit Theorem Whatever the population from which we sample looks like, the distribution of the sample mean X approaches, as n tends to infinity, a normal distribution with mean and standard deviation n Example 1.2 - Illustration of the CLT in case of a N(0,1) population distribution PRINCIPLES OF EXPERIMENTATION V8 PAGE... 9997 9997 9997 9997 9997 9997 9998 PRINCIPLES OF EXPERIMENTATION V8 PAGE 28 1.8.2 t-distribution X has a t-distribution with n – 1 degrees of freedom S n Estimating σ through S has induced an error, hence the heavier tails Figure 1.10 - t-distribution with n-1 degrees of freedom PRINCIPLES OF EXPERIMENTATION V8 PAGE 29 Table 1.3 - t-distribution with v degrees of freedom 𝝂 60 70 80 90 95 975 99... Distribution menu JMP 9 PRINCIPLES OF EXPERIMENTATION V8 PAGE 32 In this way, you can obtain a graphical display of your data and some descriptive statistics Figure 2.3 - Histograms and descriptive statistics The graphical display includes histograms of the three variables and an outlier box plot of these variables The ends of the box are the 25th and 75th quantiles The line across the middle of the box identifies... estimate; hence replication increases precision (precision is inversely related to the width of the distribution of the estimator The standard deviation of the estimator is called the standard error); PRINCIPLES OF EXPERIMENTATION V8 PAGE 18 Figure 1.3 - Increase in precision by replication a small amount of replication helps a lot, too much replication is wasteful 1.6 Confidence Intervals X is an... probability of 0.95 PRINCIPLES OF EXPERIMENTATION V8 PAGE 19 1.6.1 Derivation of a 95% confidence interval for µ (σ known) CLT: X of 0,95 X n N 0,1 n lies between -1,96 and 1,96 with a probability X P 1,96 n 1,96 0,95 or P 1,96 X 1,96 0,95 n n or 95% confidence interval for (in case is known) X 1,96 ; X 1,96 n n PRINCIPLES. .. given a significance level PRINCIPLES OF EXPERIMENTATION V8 PAGE 25 1.7.2 Type I and Type II Errors Table 1.1 - Type I and II Errors State of the world Decision H0 true H0 acceptable CORRECT H0 false Prob = 1 – α Prob = α (confidence level) TYPE II ERROR CORRECT Prob = β H0 rejected TYPE I ERROR Prob = 1 – β (power) Figure 1.8 - Type I and II Errors graphically PRINCIPLES OF EXPERIMENTATION V8 PAGE 26... Xn PRINCIPLES OF EXPERIMENTATION V8 PAGE 14 1.3.1 Graphical description using histogram 1.3.2 Numerical representation Location: 1 n X Xi n i 1 Sample mean Scatter: 1 n Sample variance S Xi X n 1 i 1 2 2 Sample standard deviation S S 2 PRINCIPLES OF EXPERIMENTATION V8 PAGE 15 1.4 Describing a population Figure 1.2 - From sample to population Histogram Density Function Fraction of. .. 1.282 1.645 1.960 2.326 2.576 PRINCIPLES OF EXPERIMENTATION V8 PAGE 30 1.8.3 F-distribution To test whether S12 and S 22 , two sample variances computed with respectively dfl and df2 in their numerator, could result from populations with the same variance: S12 df1 2 2 if 1 2 then Fdf 2 2 has an F-distribution with df1 S2 and df2 degrees of freedom PRINCIPLES OF EXPERIMENTATION V8 PAGE 31 2 Basic... Objectives The aim of the course is to provide a sound motivation for the use of statistical techniques in experimentation Rather than to insist on the technicalities of some analysis techniques, the course tries to give a broad view of the role statistics can play in helping the experimenter to understand or optimize a system subject to experimental error, using a minimal amount of experimentation The