Advanced Analysis of Variance WILEY SERIES IN PROBABILITY AND STATISTICS Established by Walter A Shewhart and Samuel S Wilks Editors: David J Balding, Noel A C Cressie, Garrett M Fitzmaurice, Geof H Givens, Harvey Goldstein, Geert Molenberghs, David W Scott, Adrian F M Smith, Ruey S Tsay Editors Emeriti: J Stuart Hunter, Iain M Johnstone, Joseph B Kadane, Jozef L Teugels The Wiley Series in Probability and Statistics is well established and authoritative It covers many topics of current research interest in both pure and applied statistics and probability theory Written by leading statisticians and institutions, the titles span both state-of-the-art developments in the field and classical methods Reflecting the wide range of current research in statistics, the series encompasses applied, methodological and theoretical statistics, ranging from applications and new techniques made possible by advances in computerized practice to rigorous treatment of theoretical approaches This series provides essential and invaluable reading for all statisticians, whether in academia, industry, government, or research A complete list of titles in this series can be found at http://www.wiley.com/go/wsps Advanced Analysis of Variance Chihiro Hirotsu This edition first published 2017 © 2017 John Wiley & Sons, Inc All rights reserved No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions The right of Chihiro Hirotsu to be identified as the author of this work has been asserted in accordance with law Registered Office John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA Editorial Office 111 River Street, Hoboken, NJ 07030, USA For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com Wiley also publishes its books in a variety of electronic formats and by print-on-demand Some content that appears in standard print versions of this book may not be available in other formats Limit of Liability/Disclaimer of Warranty The publisher and the authors make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties; including without limitation any implied warranties of fitness for a particular purpose This work is sold with the understanding that the publisher is not engaged in rendering professional services The advice and strategies contained herein may not be suitable for every situation In view of on-going research, equipment modifications, changes in governmental regulations, and the constant flow of information relating to the use of experimental reagents, equipment, and devices, the reader is urged to review and evaluate the information provided in the package insert or instructions for each chemical, piece of equipment, reagent, or device for, among other things, any changes in the instructions or indication of usage and for added warnings and precautions The fact that an organization or website is referred to in this work as a citation and/or potential source of further information does not mean that the author or the publisher endorses the information the organization or website may provide or recommendations it may make Further, readers should be aware that websites listed in this work may have changed or disappeared between when this works was written and when it is read No warranty may be created or extended by any promotional statements for this work Neither the publisher nor the author shall be liable for any damages arising here from Library of Congress Cataloging-in-Publication Data Names: Hirotsu, Chihiro, 1939– author Title: Advanced analysis of variance / by Chihiro Hirotsu Description: Hoboken, NJ : John Wiley & Sons, 2017 | Series: Wiley series in probability and statistics | Includes bibliographical references and index | Identifiers: LCCN 2017014501 (print) | LCCN 2017026421 (ebook) | ISBN 9781119303343 (pdf) | ISBN 9781119303350 (epub) | ISBN 9781119303336 (cloth) Subjects: LCSH: Analysis of variance Classification: LCC QA279 (ebook) | LCC QA279 H57 2017 (print) | DDC 519.5/38–dc23 LC record available at https://lccn.loc.gov/2017014501 Cover Design: Wiley Cover Image: © KTSDESIGN/SCIENCE PHOTO LIBRARY/Gettyimages; Illustration Courtesy of the Author Set in 10/12pt Times by SPi Global, Pondicherry, India Printed in the United States of America 10 Contents Preface Notation and Abbreviations xi xvii Introduction to Design and Analysis of Experiments 1.1 Why Simultaneous Experiments? 1.2 Interaction Effects 1.3 Choice of Factors and Their Levels 1.4 Classification of Factors 1.5 Fixed or Random Effects Model? 1.6 Fisher’s Three Principles of Experiments vs Noise Factor 1.7 Generalized Interaction 1.8 Immanent Problems in the Analysis of Interaction Effects 1.9 Classification of Factors in the Analysis of Interaction Effects 1.10 Pseudo Interaction Effects (Simpson’s Paradox) in Categorical Data 1.11 Upper Bias by Statistical Optimization 1.12 Stage of Experiments: Exploratory, Explanatory or Confirmatory? References 1 5 7 8 10 10 Basic Estimation Theory 2.1 Best Linear Unbiased Estimator 2.2 General Minimum Variance Unbiased Estimator 2.3 Efficiency of Unbiased Estimator 2.4 Linear Model 2.5 Least Squares Method 2.5.1 LS method and BLUE 2.5.2 Estimation space and error space 2.5.3 Linear constraints on parameters for solving the normal equation 2.5.4 Generalized inverse of a matrix 2.5.5 Distribution theory of the LS estimator 11 11 12 14 18 19 19 22 24 28 29 vi CONTENTS 2.6 2.7 Maximum Likelihood Estimator Sufficient Statistics References 31 34 39 Basic Test Theory 3.1 Normal Mean 3.1.1 Setting a null hypothesis and a rejection region 3.1.2 Power function 3.1.3 Sample size determination 3.1.4 Nuisance parameter 3.1.5 Non-parametric test for median 3.2 Normal Variance 3.2.1 Setting a null hypothesis and a rejection region 3.2.2 Power function 3.3 Confidence Interval 3.3.1 Normal mean 3.3.2 Normal variance 3.4 Test Theory in the Linear Model 3.4.1 Construction of F-test 3.4.2 Optimality of F-test 3.5 Likelihood Ratio Test and Efficient Score Test 3.5.1 Likelihood ratio test 3.5.2 Test based on the efficient score 3.5.3 Composite hypothesis References 41 41 41 45 47 48 49 53 53 55 56 56 57 58 58 61 62 62 63 64 68 Multiple Decision Processes and an Accompanying Confidence Region 4.1 Introduction 4.2 Determining the Sign of a Normal Mean – Unification of One- and Two-Sided Tests 4.3 An Improved Confidence Region Reference 71 71 Two-Sample Problem 5.1 Normal Theory 5.1.1 Comparison of normal means assuming equal variances 5.1.2 Remark on the unequal variances 5.1.3 Paired sample 5.1.4 Comparison of normal variances 5.2 Non-parametric Tests 5.2.1 Permutation test 5.2.2 Rank sum test 75 75 75 78 79 81 84 84 86 71 73 74 CONTENTS 5.3 5.2.3 Methods for ordered categorical data Unifying Approach to Non-inferiority, Equivalence and Superiority Tests 5.3.1 Introduction 5.3.2 Unifying approach via multiple decision processes 5.3.3 Extension to the binomial distribution model 5.3.4 Extension to the stratified data analysis 5.3.5 Meaning of non-inferiority test and a rationale of switching to superiority test 5.3.6 Bio-equivalence 5.3.7 Concluding remarks References vii 88 92 92 93 98 100 104 107 109 110 One-Way Layout, Normal Model 6.1 Analysis of Variance (Overall F-Test) 6.2 Testing the Equality of Variances 6.2.1 Likelihood ratio test (Bartlett’s test) 6.2.2 Hartley’s test 6.2.3 Cochran’s test 6.3 Linear Score Test (Non-parametric Test) 6.4 Multiple Comparisons 6.4.1 Introduction 6.4.2 Multiple comparison procedures for some given structures of sub-hypotheses 6.4.3 General approach without any particular structure of sub-hypotheses 6.4.4 Closed test procedure 6.5 Directional Tests 6.5.1 Introduction 6.5.2 General theory for unifying approach to shape and change-point hypotheses 6.5.3 Monotone and step change-point hypotheses 6.5.4 Convexity and slope change-point hypotheses 6.5.5 Sigmoid and inflection point hypotheses 6.5.6 Discussion References 113 113 115 115 116 116 118 121 121 One-Way Layout, Binomial Populations 7.1 Introduction 7.2 Multiple Comparisons 7.3 Directional Tests 7.3.1 Monotone and step change-point hypotheses 165 165 166 167 167 122 125 128 128 128 130 136 152 158 161 161 viii CONTENTS 7.3.2 Maximal contrast test for convexity and slope change-point hypotheses 7.3.3 Cumulative chi-squared test for convexity hypothesis 7.3.4 Power comparisons 7.3.5 Maximal contrast test for sigmoid and inflection point hypotheses References 171 181 185 187 190 Poisson Process 8.1 Max acc t1 for the Monotone and Step Change-Point Hypotheses 8.1.1 Max acc t1 statistic in the Poisson sequence 8.1.2 Distribution function of max acc t1 under the null model 8.1.3 Max acc t1 under step change-point model 8.2 Max acc t2 for the Convex and Slope Change-Point Hypotheses 8.2.1 Max acc t2 statistic in the Poisson sequence 8.2.2 Max acc t2 under slope change-point model References 193 193 193 194 195 197 197 198 199 Block Experiments 9.1 Complete Randomized Blocks 9.2 Balanced Incomplete Blocks 9.3 Non-parametric Method in Block Experiments 9.3.1 Complete randomized blocks 9.3.2 Incomplete randomized blocks with block size two References 201 201 205 211 211 226 234 10 Two-Way Layout, Normal Model 10.1 Introduction 10.2 Overall ANOVA of Two-Way Data 10.3 Row-wise Multiple Comparisons 10.3.1 Introduction 10.3.2 Interaction elements 10.3.3 Simultaneous test procedure for obtaining a block interaction model 10.3.4 Constructing a block interaction model 10.3.5 Applications 10.3.6 Discussion on testing the interaction effects under no replicated observation 10.4 Directional Inference 10.4.1 Ordered rows or columns 10.4.2 Ordered rows and columns 10.5 Easy Method for Unbalanced Data 10.5.1 Introduction 10.5.2 Sum of squares based on cell means 237 237 238 244 244 247 248 250 254 255 256 257 259 260 260 260 CONTENTS ix 10.5.3 Testing the null hypothesis of interaction 10.5.4 Testing the null hypothesis of main effects under Hαβ 10.5.5 Accuracy of approximation by easy method 10.5.6 Simulation 10.5.7 Comparison with the LS method on real data 10.5.8 Estimation of the mean μij References 261 263 264 264 264 269 270 11 Analysis of Two-Way Categorical Data 11.1 Introduction 11.2 Overall Goodness-of-Fit Chi-Square 11.3 Row-wise Multiple Comparisons 11.3.1 Chi-squared distances among rows 11.3.2 Reference distribution for simultaneous inference in clustering rows 11.3.3 Clustering algorithm and a stopping rule 11.4 Directional Inference in the Case of Natural Ordering Only in Columns 11.4.1 Overall analysis 11.4.2 Row-wise multiple comparisons 11.4.3 Multiple comparisons of ordered columns 11.4.4 Re-analysis of Table 11.1 taking natural ordering into consideration 11.5 Analysis of Ordered Rows and Columns 11.5.1 Overall analysis 11.5.2 Comparing rows References 273 273 275 276 276 12 Mixed and Random Effects Model 12.1 One-Way Random Effects Model 12.1.1 Model and parameters 12.1.2 Standard form for test and estimation 12.1.3 Problems of negative estimators of variance components 12.1.4 Testing homogeneity of treatment effects 12.1.5 Between and within variance ratio (SN ratio) 12.2 Two-Way Random Effects Model 12.2.1 Model and parameters 12.2.2 Standard form for test and estimation 12.2.3 Testing homogeneity of treatment effects 12.2.4 Easy method for unbalanced two-way random effects model 12.3 Two-Way Mixed Effects Model 12.3.1 Model and parameters 12.3.2 Standard form for test and estimation 299 299 299 300 302 303 303 306 306 307 308 278 278 281 281 283 284 288 291 291 292 296 309 314 314 316 396 ADVANCED ANALYSIS OF VARIANCE Table 15.11 Estimated cell frequencies under Hφ Factor levels Workability D F G Total 2 2 1 2 1.6020 0.3980 2.3980 0.6020 2.1465 2.8535 0.8535 1.1465 0.2515 0.7485 0.7485 2.2515 4.0000 4.0000 4.0000 4.0000 5.0000 7.0000 4.0000 16.0000 Total Now, the worked example is only slightly different from the welding experiment, but there are still two, one, and one degrees of freedom for the conditional analyses of factors F, D, and G, respectively To test the null hypothesis of the factor F, the conditional MLEs of the cell frequencies are calculated as in Table 15.11 From these fitted values we can calculate the conditional expectation of the sufficient statistics s = y 1 , y for φ = φ11 , φ12 as E s = 6020 + 6020 1465 + 1465 2040 = 2930 The variance–covariance matrix is similarly obtained via (14.12) again as V s = 55148 −0 39300 −0 39300 681956 The variance–covariance matrix of the accumulated statistics is easily obtained as V∗ = 1 V s 1 = 55148 15848 (15.7) 15848 44744 Then we obtain the cumulative chi-squared χ ∗2 conditional = 1− 2040 55148 + + − 2040 + 2930 4474 = 637∗ The correlation matrix of the components of χ ∗ is easily obtained from (15.7) as C∗ C∗ = 3190 3190 DESIGN AND ANALYSIS OF EXPERIMENTS BY ORTHOGONAL ARRAYS 397 Then the constants for the chi-squared approximation are obtained from df = tr C∗ C∗ = 2, 2d2 f = 2tr C∗ C∗ = 4070 as d = 1018 and f = 8152 The upper 0.05 point is calculated as 1018 × 628 = 20 Therefore, the crude analysis of the marginal table of Table 15.10 will be supported The conditional standardized residuals under the null hypothesis of factor D, conditional on the factors F and G, are all ± 249 excluding zero residuals Hence, a somewhat large χ ∗ value in the marginal table of Table 15.10 should better be considered as spurious Similarly, the conditional standardized residuals under the null hypothesis of factor G, conditional on the factors D and F, are ± 2334, suggesting that the effects of factor G are negligible (see Hirotsu, 1990 for details) 15.3 Optimality of an Orthogonal Array We generalize the optimality of the weighing experiment with balance in Section 1.1 Now the problem is to estimate the weight of p materials μ = μ1 … , μp by n measurements with a balance In an experiment we can put each material on the left or right plate, or there is a choice not to weigh the material in the experiment This is formulated mathematically by considering a variable xij which takes 1, −1, or according to whether the jth material is weighed on the left or right plate, or not weighed at the ith experiment Then, introducing an n × p design matrix X = xij , the statistical model is expressed in linear form as y = Xμ + e, (15.8) where the error e is assumed to be uncorrelated and of equal variance, namely V e = σ2 I The problem is to select an optimal design X of weighing There are several definitions of optimality First, the rank of X must be p for the unbiased estimator of μ for all −1 components to be available Then, μ = X X y is an unbiased estimator of μ with variance V μ = M −1 σ , where M = X X is Fisher’s information matrix If there are two designs X1 and X2, and supposing X X − X X to be semi-positive definite, then the design X1 is strongly recommended since for any parameter l μ, the variance of the estimator l μ by X1 is always less than or equal to that by X2 However, it is rarely the case that such an optimal design X1 exists Therefore, usually the following four criteria are used, where λj , j = 1, …, p are the eigenvalues of M 398 ADVANCED ANALYSIS OF VARIANCE D-optimum Maximize the determinant M = Πλj It is equivalent to minimizing the generalized variance V μ D is the initial of determinant A-optimum Minimize tr M − = λj− It is equivalent to minimizing the average of the variance of μj A is the initial of average E-optimum Maximize the minimum of λj , j = 1, …, p It is equivalent to minimizing the maximum variance of the standardized linear combination l μ, l l = E is the initial of eigenvalue Mini-max criterion Minimize the maximum diagonal element of M − It is equivalent to minimizing the maximum variance of μj , j = 1, …, p An orthogonal array of the two levels satisfies all criteria (1) ~ (4) simultaneously We start by stating Hotelling’s theory Theorem 15.1 Hotelling’s theorem (1944) (1) By n weighings of p materials with a balance, V μj ≥ σ n, j = 1, …, p That is, the variance cannot be made smaller than σ 2/n for each material (2) The necessary and sufficient condition for V μj = σ n for some j is xij = or − j for all i and i xij xij = for all j Proof Without any loss of generality, we can discuss μ1 assuming j = Let the first column of X (15.8) be x1 and the rest of the columns X2, so that X = x1 X Then, by simple algebra we have V μ1 = x1 x1 − x1 X X2 X −1 X x1 −1 σ2, (15.9) where an obvious modification is necessary in the case of rank X < p Now in (15.9) it is obvious that x1 x1 ≤ n, x1 X X X −1 X x1 ≥ This immediately proves the first part of Theorem 15.1 Then V μ1 is minimized when x1 x1 = n (15.10) and x1 X X X −1 X x1 = (15.11) DESIGN AND ANALYSIS OF EXPERIMENTS BY ORTHOGONAL ARRAYS 399 Equation (15.10) holds if and only if xi1 = or − and not for all i; that means for material j to be weighed every time, either on the left or right plate Equation (15.11) holds if and only if material is weighed the same number of times on the same or opposite plates to other materials μj , j = 2, …, p This is an orthogonality relationship between two columns of X, and implies the latter part of the necessary and sufficient condition of Theorem 15.1 In the case of n = 4n the design matrix X composed of any p columns of a Hadamard matrix gives X X = nIp and satisfies the necessary and sufficient condition of Theorem 15.1 for each column Therefore, it satisfies optimality criteria (2) and (4) Further, in the weighing problem we have obviously λj = tr X X ≤ pn Therefore we have Πλj ≤ λj p p ≤ np (15.12) However, the design matrix from a Hadamard matrix satisfies the equality of equation (15.12) showing D-optimality Then it is also E-optimal since it satisfies the equality of equation (15.13), minλj ≤ λj p ≤ n (15.13) Definition 15.1 Hadamard matrix A square matrix composed of ± 1, each row of which is orthogonal to other rows A Hadamard matrix of order n satisfies H H = nIn References Hirotsu, C (1990) Discussion on a critical look at accumulation analysis and related methods Technometrics 32, 133–136 Hotelling, H (1944) Some improvements in weighing and other experimental techniques Ann Math Statist 15, 297–305 Moriguti, S (1989) Statistical methods, new edn Japanese Standards Association, Tokyo (in Japanese) Taguchi, G (1962) Design of experiments (1), (2) Maruzen, Tokyo (in Japanese) Taguchi, G and Wu, Y (1980) Introduction to off-line quality control Central Quality Control Association, Nagoya, Japan Wu, C F J and Hamada, M (1990) A critical look at accumulation analysis and related methods Technometrics 32, 119–130 Appendix Table A Upper percentiles tα(a, f) of max acc t1 α = 01 a f 10 15 20 25 30 40 60 120 ∞ 3.900 3.115 2.908 2.813 2.758 2.723 2.680 2.638 2.598 2.558 4.203 3.309 3.075 2.968 2.907 2.867 2.819 2.772 2.726 2.682 4.39 3.44 3.19 3.07 3.01 2.97 2.91 2.86 2.81 2.764 4.56 3.54 3.27 3.15 3.08 3.04 2.98 2.93 2.88 2.824 4.70 3.62 3.34 3.21 3.14 3.09 3.04 2.98 2.92 2.871 4.78 3.68 3.39 3.26 3.19 3.14 3.08 3.02 2.96 2.909 (continued overleaf ) Advanced Analysis of Variance, First Edition Chihiro Hirotsu © 2017 John Wiley & Sons, Inc Published 2017 by John Wiley & Sons, Inc 402 Table A APPENDIX (Continued) α = 05 a f 10 15 20 25 30 40 60 120 ∞ 2.441 2.151 2.067 2.027 2.004 1.989 1.970 1.952 1.934 1.916 2.676 2.333 2.235 2.188 2.161 2.143 2.121 2.100 2.079 2.058 2.84 2.46 2.35 2.30 2.26 2.24 2.22 2.20 2.18 2.151 2.95 2.54 2.43 2.37 2.34 2.32 2.29 2.27 2.24 2.219 3.05 2.62 2.49 2.43 2.40 2.38 2.35 2.32 2.30 2.271 3.12 2.67 2.54 2.48 2.45 2.43 2.40 2.37 2.34 2.314 α = 10 a f 10 15 20 25 30 40 60 120 ∞ 1.873 1.713 1.666 1.643 1.629 1.620 1.609 1.598 1.588 1.577 2.089 1.894 1.836 1.808 1.792 1.781 1.768 1.755 1.742 1.730 2.23 2.02 1.95 1.92 1.90 1.89 1.87 1.86 1.84 1.829 2.34 2.10 2.03 2.00 1.98 1.96 1.95 1.93 1.92 1.901 2.43 2.17 2.10 2.06 2.04 2.02 2.01 1.99 1.97 1.956 2.49 2.23 2.15 2.11 2.09 2.07 2.05 2.04 2.02 2.001 Table B Upper percentiles of max acc χ PÃa α = 01 b m a 5 5 10 ∞ 17.74 11.47 10.04 9.405 8.468 7.808 15.96 11.46 10.33 9.822 9.044 8.480 14.97 11.46 10.53 10.10 9.435 8.944 21.12 16.33 15.03 14.42 13.48 12.77 20.10 16.48 15.44 14.95 14.17 13.58 19.51 16.58 15.72 15.31 14.64 14.14 24.31 20.12 18.90 18.32 17.39 16.69 23.63 20.38 19.40 18.93 18.19 17.59 23.22 20.56 19.75 19.35 18.75 18.21 27.51 23.60 22.41 21.84 20.94 20.21 27.01 23.94 22.99 22.53 21.82 21.19 26.71 24.18 23.38 22.99 22.14 21.86 α = 05 b m a 5 5 10 ∞ 8.195 6.262 5.753 5.520 5.159 4.894 8.256 6.695 6.261 6.058 5.739 5.500 8.287 6.971 6.595 6.416 6.134 5.919 12.83 10.86 10.27 9.995 9.550 9.210 13.00 11.39 10.91 10.67 10.29 9.994 13.10 11.74 11.32 11.12 10.79 10.53 16.34 14.41 13.81 13.52 13.05 12.69 16.62 15.04 14.54 14.30 13.90 13.58 16.79 15.45 15.02 14.81 14.48 14.20 19.55 17.63 17.02 16.73 16.24 15.86 19.91 18.34 17.83 17.58 17.18 16.85 20.14 18.80 18.37 18.16 17.75 17.52 (continued overleaf ) Table B (Continued) α = 10 b m a 5 5 10 ∞ 5.440 4.440 4.161 4.031 3.826 3.672 5.806 4.934 4.685 4.566 4.378 4.235 6.002 5.255 5.034 4.927 4.756 4.625 9.841 8.654 8.292 8.117 7.835 7.615 10.29 9.288 8.976 8.824 8.575 8.382 10.56 9.702 9.429 9.296 9.076 8.904 13.21 11.98 11.60 11.41 11.09 10.85 13.75 12.72 12.39 12.23 11.96 11.75 14.09 13.20 12.92 12.77 12.54 12.35 16.28 15.03 14.62 14.42 14.09 13.83 16.90 15.85 15.51 15.34 15.05 14.83 17.30 16.39 16.09 15.94 15.67 15.50 Table C Upper percentiles of the largest eigenvalue of Wishart matrix W Imin a− 1, b− , max a − 1, b − Upper column for α = 05 and lower column for α = 01 b−1 a−1 10 10 15 20 5.99 9.21 7.82 11.35 9.45 13.28 11.07 15.09 12.59 16.81 14.07 18.48 15.51 20.09 18.31 23.21 8.59 12.16 10.74 14.57 12.68 16.73 14.49 18.73 16.21 20.64 17.88 22.47 19.49 24.23 22.61 27.63 10.74 14.57 13.11 17.18 15.24 19.50 17.21 21.65 19.09 23.69 20.88 25.64 22.62 27.52 25.96 31.12 12.68 16.73 15.24 19.50 17.52 21.96 19.63 24.24 21.62 26.38 23.53 28.43 25.37 30.40 28.90 34.18 14.49 18.73 17.21 21.65 19.63 24.24 21.86 26.62 23.95 28.86 25.96 31.00 27.88 33.05 31.58 36.98 16.21 20.64 19.09 23.69 21.62 26.38 23.95 28.86 26.14 31.19 28.23 33.40 30.24 35.53 34.07 39.59 17.88 22.47 20.88 25.64 23.53 28.43 25.96 31.00 28.23 33.40 30.40 35.69 32.48 37.89 36.45 42.07 19.49 24.23 22.62 27.52 25.37 30.41 27.88 33.05 30.24 35.53 32.48 37.89 34.63 40.15 38.72 44.45 21.06 25.95 24.31 29.34 27.15 32.32 29.75 35.04 32.18 37.59 34.50 40.01 36.70 42.33 40.91 46.74 22.61 27.63 25.96 31.12 28.90 34.18 31.58 36.98 34.07 39.59 36.45 42.07 38.72 44.45 43.04 48.95 29.97 35.60 33.80 39.51 37.13 42.95 40.14 46.05 42.96 48.96 45.62 51.71 48.15 54.33 52.94 59.28 36.94 43.08 41.18 47.38 44.84 51.10 48.14 54.48 51.22 57.64 54.10 60.60 56.86 63.42 62.04 68.76 Index acceptance region, 56, 94, 131 accumulation analysis by Taguchi, 170, 393 Behrens–Fisher problem, 79 Bernoulli sequence, 34, 38 best linear unbiased predictor (BLUP), 326, 332, 333 Bonferroni’s inequality, 125, 126, 233 categorical data (response), 7, 8, 31 one-way, 347, 375 ordered, 118, 217, 218, 275, 292, 358, 393 three-way, 348 two-way, 273, 347, 361 cause-and- effect diagram, change-point contrast, 130, 147, 153 slope, 153 step, 258, 259, 282, 284, 291, 292 change-point model inflection, 130, 187 slope, 130, 151–154, 156, 172, 173, 196–199 step, 129, 130, 135, 142, 167, 185, 193, 195–197, 218 chi-squared approximation, 150, 170, 182, 188, 275, 282–284, 290, 355, 359, 361, 397 conditional analysis, 393, 396 covariance, 367 distribution, 34–36, 38, 48, 49, 50, 137, 140 expectation, 37, 38, 50, 276, 367 independence, 348 variance, 50, 276, 364, 366 confidence coefficient, 57, 73, 93, 94 interval, 56, 57, 76, 83, 102, 103, 108, 109, 118 region, 71, 73, 75, 76, 93, 94, 108 simultaneous interval, 124 unbiased interval, 58 cumulative chi-squared statistic χ ∗ 2, 90, 129, 150, 152, 170, 171, 181, 185, 218, 221, 224, 282, 288, 289, 360, 361, 369, 394, 396 χ ∗ ∗ 2, 291, 359, 360, 368 χ ∗ ∗ ∗ 2, 358, 364 χ† 2, 158, 182, 184, 185 χ # 2, 158, 161, 188 two-way, 259 degrees of freedom (df), 14, 30, 141, 228–230, 249, 263, 385, 387 design matrix, 18, 20, 323, 385, 397 distribution binomial, 98, 101, 103, 133, 165–167, 172, 173, 175, 183, 185, 189, 194, 195, 212, 213, 216, 287 bivariate chi-squared, 259, 288, 294 Advanced Analysis of Variance, First Edition Chihiro Hirotsu © 2017 John Wiley & Sons, Inc Published 2017 by John Wiley & Sons, Inc 408 INDEX distribution (cont’d) Cauchy, 14, 17 chi-squared (χ2 -), 30, 61, 63, 67, 119, 171, 222, 275, 302, 319, 352, 355 conditional, 34–36, 38, 49, 50, 137, 140, 168, 173, 174, 194, 195, 231, 353 exponential, 228 F-, 57, 82, 151, 256, 303, 319 hyper geometric, 168, 169 logistic, 31, 87 lognormal, 17, 79 long-tailed, 87 multinomial, 65, 67, 165, 275, 347, 376, 394 multivariate hyper geometric, 88, 119, 215, 216, 218, 275, 276, 279, 354 multivariate normal, 30, 58, 130, 133, 137, 301 non-central chi-squared, 30, 288 non-central F-, 61, 319 normal, 1, 2, 13, 39, 43, 49, 53, 58, 201, 256, 300 Poisson, 133, 172, 173, 193, 347 standard normal, 48 Student’s t-, 24, 48 uniform, 36 t-, 14, 48, 49, 72, 77, 78, 84, 87, 303 Wishart, 280, 281 downturn, 171, 198 effect(s) additive, 2, fixed, 6, 299, 332 interaction, 2, 3, 7, 239, 376, 383, 385, 387, 391 linear mixed, 322 main, 7, 239, 300, 306, 309, 314, 376, 383, 385, 387 mixed, 323, 331, 332 random, 6, 299 effective repetition number BIBD, 209 orthogonal array, 392 efficiency of BIBD, 208 efficient score, 62–66, 103, 104, 129, 133, 136–137, 142, 156, 173, 195, 196, 199, 324, 353, 360, 393 equivalence at least, 95, 104, 106, 107 bio-, 74, 107, 110 test, 74, 92 theorem, 138 error space, 21–23, 31 estimable function(s), 19, 24, 27, 29, 31, 58 estimation space, 22, 23, 31, 60 estimator best linear unbiased (BLUE), 2, 11, 12, 17, 19, 20, 22–27, 29, 30, 269 efficiency of, 16, 17, 32 least squares (LS), 20, 29, 32, 57–59, 76, 252 maximum likelihood (MLE) (see maximum likelihood) minimum variance (best) unbiased, 1, 12–14, 16, 23, 24, 37, 39, 78, 79, 301, 307, 309 experiment(s) confirmatory, 10, 392 explanatory, 10 exploratory, 10 follow-up, 10 one-factor-at-a-time, 2, simultaneous, 2, stages of, 10 exponential family, 39, 129, 133, 172 factor block, 5, 81, 201 controllable, 5, 245 covariate, indicative, 5, 245 noise, response, variation, 5, 299, 333 false positive, 121, 122 F-approximation, 151, 161 Fisher’s amount of information, 15, 16, 33 information matrix, 34, 65, 397 INDEX three principles of experiments, F-statistic, 59, 118, 123, 242, 261, 263, 268 Gauss Markov’s model, 20 theorem, 20, 24, 28 generalized inverse of a matrix, 21, 28 general mean, 3, 135, 202, 239, 299, 303, 305, 306, 314, 322, 326, 392 goodness-of-fit chi-square(s), 67, 68, 90, 166, 227, 230, 231, 233, 273, 275, 276, 282, 288, 290, 291, 352, 356, 357, 362 test, 65, 152, 161, 171, 181, 183, 184, 188, 213, 214 Hadamard matrix, 2, 399 heredity principle, 349 Hotelling’s T2-statistic, 319 hypothesis alternative, 41, 44, 60, 61, 92 composite, 45, 48, 64, 66, 101 convexity (concavity), 130, 132, 157, 160, 171, 185, 186, 195, 197 handicapped, 93, 99 homogeneity, 113, 119, 165, 341 independence, 166, 273, 275, 287, 348, 356, 357 inflection point, 158 left one-sided, 48, 50, 53 linear, 58, 61 monotone, 92, 128, 129, 134–136, 156, 160, 167, 185, 193, 258 null, 41, 48, 53, 213 null for interaction, 239, 240, 245, 257, 316, 317, 362 one-sided, 45 order restricted, 129 restricted alternative, 133, 134 right one-sided, composite, 47–50, 53 sigmoid(icity), 130, 159–161, 187 simple, 45, 62 two-sided, 45, 47, 49, 50, 53, 76 two-way ordered alternative T1, 256 409 two-way ordered alternative T2, 256, 259 two-way ordered alternative T3, 257 ICH E9, 95, 97, 106, 110 identification condition, 239, 240, 266, 269, 349, 375, 385 incidence matrix, 200 incomplete block design, interaction controllable vs controllable, indicative and variation, diagram, 387–389 effects, 2, 3, 7, 101, 238, 348 element generalized, 248 element pair-wise, 247 element two sub-groups, 247 generalized, linear by linear, 256 plot(s), 329, 331, 336, 337 pseudo, removal by transformation, 256 three-way, 350, 370, 375 treatment and block, 201, 212 treatment and response, two-way, 354 interval estimation, 56, 57 intra-block analysis, 211 isotonic regression, 128, 129 iterative scaling procedure, 350–352, 364, 378, 394 law of large numbers, 11, 33 least squares method, 11, 19, 31, 264 likelihood function, 31, 62, 133, 324, 349, 352, 362, 375 linear estimable function see estimable function linear trend, 92, 142, 147, 149, 161, 167 link function, 31 maximum likelihood equation, 32 estimator (MLE), 31, 32, 62–67, 101, 133, 230, 349, 352, 355–357, 372, 378, 381, 394, 396 410 INDEX maximum likelihood (cont’d) method, 31 residual (REML), 324 restricted, 129, 380 mean squared error (MSE), 12, 37 model additive, 7, 9, 17, 238, 253 block interaction, 248–254, 256, 290 generalized linear, 11, 31 hierarchical, 322, 326 linear, 11, 17–20, 22, 31, 58, 75, 385, 389, 391 linear mixed effects, 322 logit linear l, 31, 183, 377 log linear, 7, 31, 101, 348, 354, 364, 373, 375, 394 multiplicative, 9, 378 multivariate normal, 336 non-linear, 255 one-way ANOVA, 19, 64, 121, 123 one-way layout, 25, 61, 64, 75, 114, 129, 143, 145, 165 one-way random effects, 299, 300, 302, 322, 324 saturated, 349, 352, 357 three-way layout, 391 Tukey’s df non-additivity, 255 two-stage, 322, 331, 332 two-way ANOVA, 237, 238, 284 two-way layout, 255 two-way mixed effects, 314 two-way random effects, 306, 309, 312 modified log likelihood, 324 multiple comparisons closed test procedure, 122, 125, 128, 143, 144, 147 column-wise, 245, 262, 273, 321 Dunnett(’s) method, 124, 147, 217, 220 row-wise, 238, 244, 245, 254, 255, 258, 262, 273–276, 283, 289, 321, 333 Scheffé(’s) method, 123, 124, 217, 220, 274, 275, 335 Tukey(’s) method, 122–124, 217, 219 multiple decision processes, 71, 75, 92, 99 natural parameter, 134, 167 Neyman–Pearson’s fundamental lemma, 42 non-centrality parameter, 48, 49, 61, 62, 114, 157, 186, 242, 261, 319 non-inferiority, 95, 98, 104 margin, 93, 96, 97, 99, 104, 105, 110 strong, 95, 104 test, 74, 92, 93, 95, 96, 98, 100, 103–105, 109, 110 weak, 95, 96, 103, 110 normal approximation, 284 normal equation, 20, 22, 24, 25, 29, 32, 59, 206 nuisance parameter, 48, 53, 103, 197, 202, 249, 250 optimal design A-, 398 D-, 398, 399 E-, 398, 399 mini-max criterion, 398 ordered statistic(s), 36 orthogonal array, 383, 385–387, 392 paired sample, 79, 204 power function, 46, 49, 55, 130 profile log likelihood, 324 p-value, 45, 49, 77, 78, 80, 84, 90, 101, 124, 127, 140, 141, 147, 169, 174, 178, 180, 181, 185, 190, 195, 213, 214, 268, 287, 342, 367, 369, 379 recovery of inter-block information, 6, 211, 314 rejection region, 42–48, 54, 56, 62–64, 82, 303, 308, 310, 311, 320 risk consumer’s, 96–98, 100, 104, 107, 108, 110 producer’s, 96, 108, 110 of test, 43 INDEX sample space, 72, 175, 189 Schwarz’s inequality, 15, 123 signal-to-noise (SN) ratio, 303, 304, 326 significance level, 42, 43, 45, 48, 56, 63, 68, 71, 72 Simpson’s paradox, 89, 101, 349, 378, 379, 393 skewness correction, 101, 104 sparsity principle, 385 standard form BIBD, 207 complete randomized blocks, 202 for interaction, 238 linear model, 21 one-way layout, 27 one-way random effects model, 300 two-way layout, 241 two-way mixed effects model, 316 two-way random effects model, 307 stratified analysis, 102, 121 sufficient statistic(s), 11, 34–39, 132, 168, 173, 194, 275, 347, 350, 352, 354 complete, 39, 301, 307 minimal, 39, 312 sum of squares based on cell means, 260, 261 BIBD, 207, 209 complete randomized blocks, 202 for interaction, 252, 254, 257 one-way layout, 114 one-way random effects model, 301, 304 orthogonal array, 389–392 residual, 23, 60–62, 114, 267, 268, 389 total, 241, 269, 389 two-way layout, 241–244 Taguchi method, test Abelson-Tukey, 147, 156, 157 Bartlett(’s), 114, 117 Birch(’s), 355, 362, 378 Breslow-Day, 378 χ 2-, 7, 273 411 Cochran(’s), homogeneity, 114, 117, 215 conditional, 349, 350 convexity (concavity), 178, 181, 184, 185, 197 directional, 166 efficient score, 62–66, 129, 137, 156 equivalence, 74, 92 F-, 7, 58, 60–62, 65, 76, 113, 128, 208, 237, 245, 256, 276 Friedman’s, 219, 224 function, 42, 43, 114, 130 handicapped, 104 Hartley(’s), 114, 117 homogeneity, 62, 166, 215, 208, 218, 226, 308 independence, 356, 357 Kruskal–Wallis, 119 left one-sided, 47, 71, 79 likelihood ratio, 62–64, 67, 101, 114, 166, 213, 227, 230, 355–357, 362, 378 linear score, 92, 118, 119, 219, 222 Mantel-Haenszel, 378 max acc χ 2, 259, 287, 290, 294, 295 max acc t1, 90, 92, 129, 135–137, 141–150, 154, 160, 167, 169, 185, 186, 193–195, 199, 287, 294, 381, 382 max acc t2, 156–159, 173, 183, 185, 186, 197–199 max acc t3, 160, 161, 187, 188, 190 max max χ, χ 2, 259, 294, 296 max one-to-others χ 2, 369, 372, 373 McNemar’s, 213 modified likelihood ratio, 336 modified Wiiliams, 147, 148 most powerful similar, 42–45, 48 most stringent, 62 non-inferiority, 74, 92, 93, 95, 96, 98, 100, 103–105, 109, 110 non-parametric, 49, 84, 85, 92, 118, 184, 201 one-sided, 45, 72, 92, 95, 110 paired t-, 75, 81 permutation, 36, 49, 51, 53, 77, 84, 85, 87, 118 412 INDEX test (cont’d) power of, 42 rank sum, 86 right one-sided, 47, 55, 71, 82, 83 signed rank sum, 49, 51, 53 similar, 48, 173, 350 size of, 43, 264, 265 Smirnov–Grubbs, 77 Student’s t-, 48 superiority, 74, 92, 103–105, 109, 110 t-, 48–51, 58, 61, 76, 79, 80, 85, 92, 121 two-sided, 47, 50, 55, 71, 72, 81–83, 92, 93, 95, 110, 171 two-way max acc t1, 372 u-, 50 unbiased, 45, 48, 58 unconditional, 349, 352, 356 uniformly most powerful unbiased, 44, 45, 48, 53–55, 57, 58, 76 Wilcoxon–Mann–Whitney rank sum, 87 Wilcoxon (rank), 89, 92, 121, 292, 295 Williams, 147, 148 theorem central limit, 17, 33 Cramér–Rao’s, 14, 17, 24, 32, 39 factorization, 35 Gauss–Markov’s, 20, 24, 28 Holm’s, 127 Hotelling’s, 398 Marcus-Peritz-Gabriel, 128 Rao–Blackwell, 37, 39 Rao’s, 13 three-way contingency table, 354 three-way deadlock, 229, 231, 232 t-statistic, 48, 59, 76, 80, 124 two-way contingency table, 67, 166, 275, 282 unbalanced data, 260, 264 unbalanced two-way data, 309 unbiased variance, 23, 24, 48, 59, 65, 72, 76, 77, 83, 114, 125, 150, 203, 208, 209, 242, 249, 257, 266, 390, 391 unlike pair, 212, 213, 215 weighted least squares (WLS), 325 Wishart matrix, 249, 258, 275, 280, 283, 335, 336, 341, 385 ... ABBREVIATIONS H(y R1, C1, N): MH(yij yi , y j), MH(yij Ri, Cj, N): f(y, ? ?) and p(y, ? ?): Pr (A), Pr {A}: L(y, ? ?), L(? ?): E(y) and E(y): E(y B) and E(y B): V(y) and V(y): V(y B) and V(y B): Cov(x, y): Cor(x,... paradox Young j = Drug (i = 1) Drug (i = 2) Old j = Young + old (k = 1) (k = 2) (k = 1) (k = 2) (k = 1) (k = 2) 120 30 40 10 10 40 30 120 130 70 70 130 effects, see Darroch (1 97 4) In most cases an... y1 (2 .3 5) ya Equation (2 .3 5) cannot be solved since rank X = a, whereas there are a + unknown parameters Therefore, we apply methods (1 ), (2 ), and (3 ) 26 ADVANCED ANALYSIS OF VARIANCE (1 ) Direct