John wiley sons numerical issues in statistical computing for the social scientist (2004) tlf

TLFeBOOK Numerical Issues in Statistical Computing for the Social Scientist TLFeBOOK ii WILEY SERIES IN PROBABILITY AND STATISTICS Established by WALTER A SHEWHART and SAMUEL S WILKS Editors: David J Balding, Noel A C Cressie, Nicholas I Fisher, Iain M Johnstone, J B Kadane, Louise M Ryan, David W Scott, Adrian F M Smith, Jozef L Teugels; Editors Emeriti: Vic Barnett, J Stuart Hunter, David G Kendall A complete list of the titles in this series appears at the end of this volume TLFeBOOK Numerical Issues in Statistical Computing for the Social Scientist MICAH ALTMAN JEFF GILL MICHAEL P McDONALD A JOHN WILEY & SONS, INC., PUBLICATION TLFeBOOK Copyright c 2004 by John Wiley & Sons, Inc All rights reserved Published by John Wiley & Sons, Inc., Hoboken, New Jersey Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax 978-750-4470, or on the web at www.copyright.com Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, e-mail: permreq@wiley.com Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose No warranty may be created or extended by sales representatives or written sales materials The advice and strategies contained herein may not be suitable for your situation You should consult with a professional where appropriate Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages For general information on our other products and services please contact our Customer Care Department within the U.S at 877-762-2974, outside the U.S at 317-572-3993 or fax 317-572-4002 Wiley also publishes its books in a variety of electronic formats Some content that appears in print, however, may not be available in electronic format Library of Congress Cataloging-in-Publication Data: Altman, Micah Numerical issues in statistical computing for the social scientist / Micah Altman, Jeff Gill, Michael P McDonald p cm.—(Wiley series in probability and statistics) Includes bibliographical references and index ISBN 0-471-23633-0 (acid-free paper) Statistics–Data processing Social sciences–Statistical methods–Data processing I Gill, Jeff II McDonald, Michael P., 1967–III Title IV Series QA276.4.A398 2004 519.5–dc21 2003053470 Printed in the United States of America 10 TLFeBOOK Contents Preface xi Introduction: Consequences of Numerical Inaccuracy 1.1 1.2 1.3 1.4 Importance of Understanding Computational Statistics Brief History: Duhem to the Twenty-First Century Motivating Example: Rare Events Counts Models Preview of Findings Sources of Inaccuracy in Statistical Computation 2.1 2.2 2.3 2.4 2.5 2.6 Introduction 2.1.1 Revealing Example: Computing the Coefficient Standard Deviation 2.1.2 Some Preliminary Conclusions Fundamental Theoretical Concepts 2.2.1 Accuracy and Precision 2.2.2 Problems, Algorithms, and Implementations Accuracy and Correct Inference 2.3.1 Brief Digression: Why Statistical Inference Is Harder in Practice Than It Appears Sources of Implementation Errors 2.4.1 Bugs, Errors, and Annoyances 2.4.2 Computer Arithmetic Algorithmic Limitations 2.5.1 Randomized Algorithms 2.5.2 Approximation Algorithms for Statistical Functions 2.5.3 Heuristic Algorithms for Random Number Generation 2.5.4 Local Search Algorithms Summary 10 12 12 12 13 15 15 15 18 20 21 22 23 29 30 31 32 39 41 v TLFeBOOK vi CONTENTS Evaluating Statistical Software 3.1 3.2 3.3 3.4 3.5 3.6 Introduction 3.1.1 Strategies for Evaluating Accuracy 3.1.2 Conditioning Benchmarks for Statistical Packages 3.2.1 NIST Statistical Reference Datasets 3.2.2 Benchmarking Nonlinear Problems with StRD 3.2.3 Analyzing StRD Test Results 3.2.4 Empirical Tests of Pseudo-Random Number Generation 3.2.5 Tests of Distribution Functions 3.2.6 Testing the Accuracy of Data Input and Output General Features Supporting Accurate and Reproducible Results Comparison of Some Popular Statistical Packages Reproduction of Research Choosing a Statistical Package Robust Inference 4.1 4.2 4.3 4.4 4.5 44 44 47 48 49 51 53 54 58 60 63 64 65 69 71 Introduction Some Clarification of Terminology Sensitivity Tests 4.3.1 Sensitivity to Alternative Implementations and Algorithms 4.3.2 Perturbation Tests 4.3.3 Tests of Global Optimality Obtaining More Accurate Results 4.4.1 High-Precision Mathematical Libraries 4.4.2 Increasing the Precision of Intermediate Calculations 4.4.3 Selecting Optimization Methods Inference for Computationally Difficult Problems 4.5.1 Obtaining Confidence Intervals with Ill-Behaved Functions 4.5.2 Interpreting Results in the Presence of Multiple Modes 4.5.3 Inference in the Presence of Instability Numerical Issues in Markov Chain Monte Carlo Estimation 5.1 5.2 5.3 44 Introduction Background and History Essential Markov Chain Theory TLFeBOOK 71 71 73 73 75 84 91 92 93 95 103 104 106 114 118 118 119 120 vii CONTENTS 5.3.1 Measure and Probability Preliminaries 5.3.2 Markov Chain Properties 5.3.3 The Final Word (Sort of) Mechanics of Common MCMC Algorithms 5.4.1 Metropolis–Hastings Algorithm 5.4.2 Hit-and-Run Algorithm 5.4.3 Gibbs Sampler Role of Random Number Generation 5.5.1 Periodicity of Generators and MCMC Effects 5.5.2 Periodicity and Convergence 5.5.3 Example: The Slice Sampler 5.5.4 Evaluating WinBUGS Absorbing State Problem Regular Monte Carlo Simulation So What Can Be Done? 120 121 125 126 126 127 128 129 130 132 135 137 139 140 141 Numerical Issues Involved in Inverting Hessian Matrices 143 5.4 5.5 5.6 5.7 5.8 Jeff Gill and Gary King 6.1 6.2 6.3 6.4 6.5 6.6 6.7 Introduction Means versus Modes Developing a Solution Using Bayesian Simulation Tools What Is It That Bayesians Do? Problem in Detail: Noninvertible Hessians Generalized Inverse/Generalized Cholesky Solution Generalized Inverse 6.7.1 Numerical Examples of the Generalized Inverse 6.8 Generalized Cholesky Decomposition 6.8.1 Standard Algorithm 6.8.2 Gill–Murray Cholesky Factorization 6.8.3 Schnabel–Eskow Cholesky Factorization 6.8.4 Numerical Examples of the Generalized Cholesky Decomposition 6.9 Importance Sampling and Sampling Importance Resampling 6.9.1 Algorithm Details 6.9.2 SIR Output 6.9.3 Relevance to the Generalized Process 6.10 Public Policy Analysis Example 6.10.1 Texas 6.10.2 Florida TLFeBOOK 143 145 147 148 149 151 151 154 155 156 156 158 158 160 160 162 163 163 164 168 viii CONTENTS 6.11 Alternative Methods 6.11.1 Drawing from the Singular Normal 6.11.2 Aliasing 6.11.3 Ridge Regression 6.11.4 Derivative Approach 6.11.5 Bootstrapping 6.11.6 Respecification (Redux) 6.12 Concluding Remarks Numerical Behavior of King’s EI Method 7.1 7.2 7.3 7.4 7.5 Introduction Ecological Inference Problem and Proposed Solutions Numeric Accuracy in Ecological Inference 7.3.1 Case Study 1: Examples from King (1997) 7.3.2 Nonlinear Optimization 7.3.3 Pseudo-Random Number Generation 7.3.4 Platform and Version Sensitivity Case Study 2: Burden and Kimball (1998) 7.4.1 Data Perturbation 7.4.2 Option Dependence 7.4.3 Platform Dependence 7.4.4 Discussion: Summarizing Uncertainty Conclusions Some Details of Nonlinear Estimation 171 171 173 173 174 174 175 176 177 177 179 180 182 186 187 188 189 191 194 195 196 197 199 B D McCullough 8.1 8.2 8.3 8.4 8.5 8.6 8.7 Introduction Overview of Algorithms Some Numerical Details What Can Go Wrong? Four Steps 8.5.1 Step 1: Examine the Gradient 8.5.2 Step 2: Inspect the Trace 8.5.3 Step 3: Analyze the Hessian 8.5.4 Step 4: Profile the Objective Function Wald versus Likelihood Inference Conclusions TLFeBOOK 199 200 204 206 210 211 211 212 212 215 217 ix CONTENTS Spatial Regression Models 219 James P LeSage 9.1 9.2 9.3 9.4 9.5 Introduction Sample Data Associated with Map Locations 9.2.1 Spatial Dependence 9.2.2 Specifying Dependence Using Weight Matrices 9.2.3 Estimation Consequences of Spatial Dependence Maximum Likelihood Estimation of Spatial Models 9.3.1 Sparse Matrix Algorithms 9.3.2 Vectorization of the Optimization Problem 9.3.3 Trade-offs between Speed and Numerical Accuracy 9.3.4 Applied Illustrations Bayesian Spatial Regression Models 9.4.1 Bayesian Heteroscedastic Spatial Models 9.4.2 Estimation of Bayesian Spatial Models 9.4.3 Conditional Distributions for the SAR Model 9.4.4 MCMC Sampler 9.4.5 Illustration of the Bayesian Model Conclusions 10 Convergence Problems in Logistic Regression 219 219 219 220 222 223 224 225 226 228 229 230 231 232 234 234 236 238 Paul Allison 10.1 10.2 10.3 10.4 Introduction Overview of Logistic Maximum Likelihood Estimation What Can Go Wrong? Behavior of the Newton–Raphson Algorithm under Separation 10.4.1 Specific Implementations 10.4.2 Warning Messages 10.4.3 False Convergence 10.4.4 Reporting of Parameter Estimates and Standard Errors 10.4.5 Likelihood Ratio Statistics 10.5 Diagnosis of Separation Problems 10.6 Solutions for Quasi-Complete Separation 10.6.1 Deletion of Problem Variables 10.6.2 Combining Categories 10.6.3 Do Nothing and Report Likelihood Ratio Chi-Squares 10.6.4 Exact Inference TLFeBOOK 238 238 240 243 244 244 246 247 247 247 248 248 248 249 249 316 SUBJECT INDEX combinatorial optimization, 31 computational efficiency, 250 computation error examples, 1–2 computer, conditioning, 47–48 condition number, 47 confidence interval, 77, 86, 104 bootstrapped, 106 credible interval, see credible interval likelihood ratio, 211, 247, 249, 251 likelihood ratio inference, 104 likelihood ratio test (LRT), 105 Wald inference, 104–105, 212, 213 Wald vs likelihood inference, 215, 216 conjugate gradient, see optimization algorithm, conjugate gradient constrained maximum likelihood, see maximum likelihood estimation (MLE), constrained Cook’s curvature, see likelihood curvature correlation matrix, standardized, 154 countable collection, 120 coupling from the past, 264 coverage, 216 credible interval, 110 cross-level-inference, see ecological inference forward vs central differences, 205 numerical, 205 DFP, see optimization algorithm, Davidson–Fletcher–Powell (DFP) distribution functions benchmarks, 58–60, 65 cumulative bivariate normal, 180, 181, 194 truncated normal, 146 DSSTAB, 59, 92 Data Documentation Initiative (DDI), 256 data input and output, 256 benchmarks, 60–63 canonical data, 255 cyclic redundancy checks, 62–63 truncation, 256 Davidson–Fletcher–Powell method (DFP), see optimization algorithm, Davidson–Fletcher–Powell (DFP) DDI, see Data Documentation Initiative derivatives analytic vs numerical, 201 automatic, 201 forward difference method, 32 ecological inference, 11 choice of optimization algorithm, 187 choice of PRNGS, 187–188 Goodman’s regression, 178 method of bounds, 178, 179 sensitivity to environment, 195–197 sensitivity to implementation, 188–189, 194–195 sensitivity to perturbation, 185–186, 191–194 two-stage models, 190 ecological inference (EI), 176 background, 177–179 definition, 179–180 numerical issues, 180–181 EI-R, 196–197 ELV, 59 EM algorithm, 114, 142 empirical cdf, 113 ergodic theorem, 124 E-Views, 199 exact inference, logistic regression, 249 Excel, 12–14, 50, 52, 74, 207, 209 solver, 206–209, 209 EzI , 182, 188, 189, 191, 195, 196 execution bugs, 189 simulation variance, 195 versions, 196 floating point arithmetic, 23–24, 28 approximation, 94 bits, 23 cancellation, 25 compilers, 29 example, 25–27 TLFeBOOK 317 SUBJECT INDEX guard bits, 24 inaccuracies, 42 in EI, 180 intermediate calculation, 28 machine epsilon, 24 mantissa, 24 misconceptions, 28 normalized number, 24 overflow, 23 precision, 24–25 reproducibility, 28–29 rounding, 24, 93, 94 rounding error, 17 truncation, 24 underflow, 23, 25 FORTRAN, xv, 50, 51, 92, 265 function maximization, see optimization algorithm GIS, see geographic information systems (GIS) GLIM, 244, 246, 246 global optimality, see optimization algorithm, global optimality Gnu Multiple Precision (GMP), 93 Goodman’s regression, see ecological inference, Goodman’s regression Gauss, xv, 7, 9, 66–68, 74, 153, 155, 156, 176, 181, 182, 187, 188, 199, 213, 258 Gauss–Markov, 79–80 Gauss–Newton method, see optimization algorithm, Gauss–Newton method generalized Cholesky, 147, 151, 155–159 numerical examples, 158–159 generalized inverse, 147, 152, 151–155 minimum norm, 153 Moore–Penrose, 151 normalized, 153 numerical examples, 154–155 reflexive, 153 weak, 153 generalized linear model, 82 interactions, 82 link function, 82 genetic algorithm (GA), see optimization algorithm, genetic algorithm geographic information systems (GIS), 219, 266 Gerschgorin bounds, 158 Gerschgorin circle theorem, 158 Gibbs sampler, 6, 119, 128, 139, 232 full conditional distributions, 129 hat matrix, 251 Hessian, 103, 105, 143 bootstrapping approach, 174–175 derivative approach, 174 inversion, 10–11, 194–195 invertibility, 143 nonpositive definite, 149 noninvertible, 262–263 causes, 149 numerical example, 166, 169 positive definite, 262 processing, numerical example, 163–171 respecification, 146, 175 singular, 149 heuristic, 30, 32–40 defined, 16 highest posterior density, 109–113, 148, 263 average coverage criterion, 110 average length criterion, 110 definition, 109 highest average coverage criterion, 110 simulation methods, 112 worst outcome criterion, 111 HPD, see highest posterior density ICPSR, see Inter-University Consortium for Political and Social Research implementation defined, 16, 19 error, 21–22, 258 implementation uncertainty, 197 importance ratio, 160 importance sampling, 147, 160–163 algorithm, 160–161 TLFeBOOK 318 importance sampling (continued) applied to pseudo-variance matrix, 163 approximation distribution, 160 diagnostics, 163 illustration, 162 importance weight, 161 independence chain, 125 inference, 20–21 Bayesian, 18, 145, 148 logistic regression, 250 likelihood model, 18, 145 influence, 71, 72 inlier, 140 instrumental variables, 83 integer linear programming, 31 Inter-University Consortium for Political and Social Research (ICPSR), 60, 182, 189, 191, 255, 256, 259 inverse probability model, 18 jackknifing, 81 Java, 63 JMP, 247 KISS, see pseudo-random number generator, KISS Kolmogorov’s zero-one law, 123 LAPACK, 92 Leamer bounds, 116, 197, 261 likelihood curvature, 77 likelihood discplacement, 77 likelihood function, 76 core region, 90 curvature, 77, 85 nonconcavity, 85 quadratic, 261 sensitive, 77 likelihood inference, see inference, likelihood model likelihood mean-shift, 77 likelihood principle, 108 likelihood ratio, see confidence intervals, likelihood ratio Limdep, 199, 244, 246 SUBJECT INDEX linear model Gauss–Markov assumptions, 78 generalized, see generalized linear model logistic regression, 265 Bayesian inference, 250 convergence criteria, 246 diagnostics, 247 example, Florida, 168 example, Texas, 164 likelihood, 168 MLE, 238 separation, 240 detection, 245 treatment, 248 software problems, 244 logit, see logistic regression log relative error (LRE), 19, 53, 59 LogXact, 250 lunch, no free, 40, 99, 102 m − n poly-t form, 107 machine epsilon, 24, 157 machine precision, 83 Maple, 93–95, 261 Markov chain, 119 absorbing condition, 122 absorbing set, 122, 139 absorbing state, 139–140, 142 auxiliary variables, 135 burn-in period, 133 chain period, 122 closed set, 122 convergence bounds, 264 conditions, 126 diagnostics, 264 convergence of empirical averages, 134 convergence parameter, 123 convergence to stationarity, 134 cycle, 122 definition, 121 degeneracy, 140 detailed balance equation, 127, 135 empirical averages, 134 equilibrium distribution, 123, 124 ergodic, 133, 265 geometric convergence, 135 TLFeBOOK 319 SUBJECT INDEX geometrically ergodic, 129, 134 Gibbs sampler, 128, 130, 135 steps, 129 hit-and-run algorithm, 127 steps, 128 homogeneous, 129 invariant distribution, 124 irreducible set, 121 limiting distribution, 124 marginal distribution, 124 measure conditions, 120 memory-less property, 121 Metropolis–Hastings algorithm, 126, 130, 142 acceptance ratio, 140 actual transaction function, 127 candidate generating distribution, 126 instrumental distribution, 126 jumping distribution, 126 proposal distribution, 126 transition kernel, 127 minorization condition, 131 mixing, 264 movement, 120 obtainable set, 122 partial sums, 134 periodicity of generator, 131 properties aperiodic, 122 communicating, 122 ergodic, 124 Harris recurrent, 123, 134 Harris R-recurrent, 133 homogeneous, 122 irreducible, 121 periodic, 122 persistent, 122 positive recurrent, 123 recurrent, 122, 123 transient, 123 pseudo-random number generator problems, 137 ψ-communicating, 121 random number generation, 129 random number generators, 130 problems, 130 reversibility, 127 σ -finite kernel, 120 slice sampler, 135 stationary distribution, 124, 133 symmetry, 127 transient, 123 transitioning, 120 transition kernel, 131 trapping set, 139 unbounded and continuous state spaces, 123 uniform ergodicity, 125 unobtainable set, 122 Markov chain Monte Carlo (MCMC), 10, 32, 35, 38, 58, 119, 145, 250, 263 background, 118–120 mechanics, 126–129 properties, 121–126 spatial models, 231–234 theory, 120–121 Markovian property, 121, 129, 135, 139 Mathematica, 59, 93–95, 207, 261 Matlab, 59, 93, 226, 237 MATLAB, 48 matrix decomposition diagonalization, 152 LDU, 151, 152 matrix inverse conditions, 151 maximum likelihood estimation (MLE), constrained, 181 convergence failure, 243 nonexistence, 243 measurable functions, 120 measurable space, 120 measure space, 132 measurement error, 18 generalized linear model, 82 linear model, 78–82 induced covariance, 79 multivariate, 80 model effect, 78 zero mean, 78 measures positive, 120 signed, 120 Metropolis–Hastings algorithm, 119, 126, 135, 234, 264 acceptance ratio, 127 steps, 127 symmetry, 127 TLFeBOOK 320 SUBJECT INDEX Minitab, 244, 246 misspecification, 20 mixture distribution, 113 mixture model, 113 MLE, see maximum likelihood estimation (MLE) model specification, 257 MODULA-2, 138 Monte Carlo arithmetic, 77–78 Monte Carlo simulation, 140–141, 147, 259 error term, 141 expected value, 141 steps, 141 Moore–Penrose theorem, 152, 153 MRC Biostatistics Unit, 137 multicollinearity, 46, 144, 173 multiplicative congruential generator, 138 MuPad, 93 National Election Survey, 90, 255 National Institute of Standards and Technology (NIST), 23, 45, 49–50, 53, 88, 92, 199 National Science Foundation, 253 Newton’s method, see optimization algorithm, Newton’s method Newton–Raphson, 239, 240, 243, 244, 246, 247, 251 NIST, see National Institute of Standards and Technology nonlinear estimation diagnostics, 210 function profiling, 212–213 gradient analysis, 211 Hessian condition, 212 Tobit model, 203 trace analysis, 211 nonlinear least squares, see optimization algorithm, nonlinear least squares nonlinear maximum likelihood, 199 nonlinear regression, see optimization algorithm, nonlinear least squares norm operator, 132 North Carolina, 6–8 NP-completeness, 31 numerical accuracy, 4–6, 19 backward error, 48 benchmarks, 48–49 test results, 64–65 error detection, example, rare event counts model, 6, examples, stability, 20 numerical expected value, 141 omitted variable bias, 169 open source software (OSS), see statistical software, open source optimization algorithm, 30, 39–41, 75, 84, 94, 200–202, 258–259, 261–262 artificial neural networks (ANN), 101–102 BFGS, 67, 97–99, 103, 186, 202, 203, 206, 217 BHHH, 67, 202, 204 conjugate gradient, 67, 96 convergence, 102 convergence criteria, 98, 103, 205 convergence rates, 205 Davidson–Fletcher–Powell (DFP), 67, 97, 151, 202 downhill simplex method, 98 example, ecological inference, 186–187 Gauss–Newton, 204, 208, 217 genetic algorithms, 101 global optimality, 84–92 global optimality tests Starr test, 86, 88–90 Veall test, 87–90 global optimum, 39–40, 51, 84 gradient, 67 grid search, 87, 109 implementation, 99 Levenberg–Marquardt, 204, 217 line search, 96, 97, 201 local optima, 84, 103 multiple modes, 89, 106 example, 107–109 Nelder–Mead algorithm, 99 Newton’s method, 96, 98, 201 Newton–Raphson, 67, 204 TLFeBOOK 321 SUBJECT INDEX NL2SOL, 217 nonlinear, see nonlinear estimation nonlinear least squares, 199, 261–262 numerical accuracy, 206–209 optimization method, 95–96 options, 52 quadratic hill-climbing, 204 quasi-Newton, 202, 204–206 simplex algorithm, 99 simulated annealing, 99–101 solvers, 52–53, 67 starting values, 51, 53, 199, 240 steepest descent, 67, 96 stopping rules, 246 trust region, 204 trust-region algorithm, 96 outliers, 71, 72 parallel analyses, 144 penalized log likelihood, 251 penalized maximum likelihood estimation, 250 perfect sampling, 264 PERL, 51, 258 posterior distribution as objective function, 145 plateau, 150 ridge, 145 saddlepoint, 150 precedence, 51 precision, 24 defined, 15 prior, 18 PRNG, see pseudo-random number generator probability measure, 121 pseudo-random number generator, 32–39, 75 add with carry, 33–35 benchmarks, 54–58, 65 birthday-spacings test, 55 compound, 36 crypto-strength, 132 defined, 32–33 DIEHARD, 55, 56, 64 distribution of draws, 37 hardware generators, 39 independence, 37 inverse CDF method, 36 inverse congruential, 35 KISS, 132 lagged Fibonnaci, 33–35 linear congruential, 33, 131, 136 long-runs test, 55 Markov chains, see Markov chain, pseudo-random number generator Mersenne twister, 35, 132, 264 mother-of-all generator, 132 multiple recursive, 33, 132 parallel application, 58 parallel processor, 37 period, 33, 36, 57, 57 PRNGS, 181 KISS, 188 RANDMAX, 132 RANDU, 33–34, 130, 135, 137 RANROT, 132 rejection method, 36 reproducibility, 37 seed, 37, 57 shuffle, 35 SPRNG, 56–57, 93 TESTU01, 56, 93 pseudo-variance matrix, 147, 151, 155, 160, 171 pseudoprime, 30–31 ψ-irreducibility, 121 QR factorialization, 153 quasi-Newton, 202, 204 R, xv, 7, 9, 37, 44, 51, 70, 74, 136, 137, 176, 207, 215, 244, 246, 246 R Development Core Team, 51, 294 R-square, 81 racial bloc voting, 177–180 random number generation, 32 hardware sources, 38 Markov chains, see Markov chain, random number generation pseudo, see pseudo-random number generator true, 57 random walk chain, 125 TLFeBOOK 322 SUBJECT INDEX randomized algorithm, 29–31 example, prime number, 30 recreation, 255 register overflow, 138 rejection sampling, 147, 173 reliability, replication, 2, 5, 10, 253–254, 254 example, failure to replicate, 254 example, scobit, 66 reproduction, 254–255; see also replication independent, 254 secondary analysis, 254 tasks, 255 resistance, 71, 72 ridge regression, 173–174 robustness, 71, 72 Bayesian, 72–73 global, 71 linear model, 72 local, 71 rodeo, prison, 255 roll-off, 190 S-Plus, 7, 9, 37, 74, 153, 155, 199, 208, 213, 251 sample space, 119 sampling importance resampling (SIR), see importance sampling SAS, 7, 9, 42, 44, 74, 199, 213, 244–247, 247, 249, 251, 296 Schrage algorithm, 138 scobit, 65–69, 88–90 Scythe Library, 132 sensitivity analysis data perturbation, 182–185 bounded, 185 proportional data, 185 definition, 73 perturbations estimation effect, 76, 77 maximizing, 77 minimizing, 77 perturbation tests, 46, 75–84 software, 74 tests, 73–91 SHAZAM, 66–68, 74, 199 σ -algebra, 120, 121 signed measure, 132 simulated annealing (SA), see optimization algorithm, simulated annealing simulation, 4, error, 141 variance, 190–191 smalltalk, 51 software bugs, 22–23, 41, 46, 257 ecological inference, 189 examples, 64 solution, defined, 15 spatial dependence definition, 219–220 specification using weight matrices, 220–221 spatial regression, 266 Bayesian heteroskedastic models, 229–231 estimation, 231–236 estimation issues, 222–223 limited dependent variables, 222 maximum likelihood estimation, 223–224 numerical accuracy, 226–229 sparse matrix algorithms, 224–225 spatial autoregressive model (SAR), 221–222 spatial Durbin model (SDM), 222 spatial error model (SEM), 222 vectorization, 225–226 SPSS, xv, 44, 199, 244, 245, 246, 247 standard deviation, 12–14 starting values, see optimization algorithm, starting values Stata, xv, 7, 9, 50, 63, 66–68, 74, 244, 246, 247, 258 state space, 120 statistical physics, 119, 126 Statistical Reference Datasets (StRD), 49–51, 70, 199, 199, 259 BoxBOD test, 41, 88, 89, 95 Misra1a test, 52, 206–210, 213 π-digits test, 50 statistical software, 44, 60, 65, 74 arbitrary precision, 94 computer algebra, 94 TLFeBOOK 323 SUBJECT INDEX external library, 63 high precision, 91 multiple precision, 93 open source, 70, 74 selecting, 69–70 speed, 91 statistics textbooks, xiv steepest descent, see optimization algorithm, steepest descent stochastic process, 119, 121 H -valued, 121 history, 121 random elements, 121 stochastic simulation, 118 stopping rules, see optimization algorithm, stopping rules StRD, see Statistical Reference Datasets symmetry, 127 Systat, 244, 246 Thornburg v Gingles, 178 Tobit, 203 total variation norm theorem, 133 Transactions on Mathematical Software, 92 transition kernel, 120 truncated singular normal distribution, 171, 173 TSP, 207 U.S Census, 219, 228 variance reduction, 141 VDC, see Virtual Data Center verification, 254; see also replication recommendations, 259–260 verification data, 258 Virtual Data Center (VDC), 256 Wald inference, 249 Wald test, see confidence intervals, Wald inference; confidence intervals, likelihood ratio warning messages, 7, 14, 246 Web site, xv WinBUGS, xv, 6, 112, 131, 135, 137, 142, 250 evaluation, 137–139 period of generator, 131 random seed, 139 within-chain versus cross-chain correlation, 265 Yacas, 93, 95 TLFeBOOK WILEY SERIES IN PROBABILITY AND STATISTICS ESTABLISHED BY WALTER A SHEWHART AND SAMUEL S WILKS Editors David J Balding, Noel A C Cressie, Nicholas I Fisher, Iain M Johnstone, J B Kadane, Louise M Ryan, David W Scott, Adrian F M Smith, Jozef L Teugels Editors Emeriti: Vic Barnett, J Stuart Hunter, David G Kendall The Wiley Series in Probability and Statistics is well established and authoritative It covers many topics of current research interest in both pure and applied statistics and probability theory Written by leading statisticians and institutions, the titles span both state-of-the-art developments in the field and classical methods Reflecting the wide range of current research in statistics, the series encompasses applied, methodological and theoretical statistics, ranging from applications and new techniques made possible by advances in computerized practice to rigorous treatment of theoretical approaches This series provides essential and invaluable reading for all statisticians, whether in academia, industry, government, or research ABRAHAM and LEDOLTER · Statistical Methods for Forecasting AGRESTI · Analysis of Ordinal Categorical Data AGRESTI · An Introduction to Categorical Data Analysis AGRESTI · Categorical Data Analysis, Second Edition ALTMAN, GILL, and McDONALD · Numerical Issues in Statistical Computing for the Social Scientist ˇ · Mathematics of Chance ANDEL ANDERSON · An Introduction to Multivariate Statistical Analysis, Third Edition ∗ ANDERSON · The Statistical Analysis of Time Series ANDERSON, AUQUIER, HAUCK, OAKES, VANDAELE, and WEISBERG · Statistical Methods for Comparative Studies ANDERSON and LOYNES · The Teaching of Practical Statistics ARMITAGE and DAVID (editors) · Advances in Biometry ARNOLD, BALAKRISHNAN, and NAGARAJA · Records ∗ ARTHANARI and DODGE · Mathematical Programming in Statistics ∗ BAILEY · The Elements of Stochastic Processes with Applications to the Natural Sciences BALAKRISHNAN and KOUTRAS · Runs and Scans with Applications BARNETT · Comparative Statistical Inference, Third Edition BARNETT and LEWIS · Outliers in Statistical Data, Third Edition BARTOSZYNSKI and NIEWIADOMSKA-BUGAJ · Probability and Statistical Inference BASILEVSKY · Statistical Factor Analysis and Related Methods: Theory and Applications BASU and RIGDON · Statistical Methods for the Reliability of Repairable Systems BATES and WATTS · Nonlinear Regression Analysis and Its Applications BECHHOFER, SANTNER, and GOLDSMAN · Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons BELSLEY · Conditioning Diagnostics: Collinearity and Weak Data in Regression BELSLEY, KUH, and WELSCH · Regression Diagnostics: Identifying Influential Data and Sources of Collinearity BENDAT and PIERSOL · Random Data: Analysis and Measurement Procedures, Third Edition BERRY, CHALONER, and GEWEKE · Bayesian Analysis in Statistics and Econometrics: Essays in Honor of Arnold Zellner BERNARDO and SMITH · Bayesian Theory ∗ Now available in a lower priced paperback edition in the Wiley Classics Library TLFeBOOK BHAT and MILLER · Elements of Applied Stochastic Processes, Third Edition BHATTACHARYA and JOHNSON · Statistical Concepts and Methods BHATTACHARYA and WAYMIRE · Stochastic Processes with Applications BILLINGSLEY · Convergence of Probability Measures, Second Edition BILLINGSLEY · Probability and Measure, Third Edition BIRKES and DODGE · Alternative Methods of Regression BLISCHKE AND MURTHY (editors) · Case Studies in Reliability and Maintenance BLISCHKE AND MURTHY · Reliability: Modeling, Prediction, and Optimization BLOOMFIELD · Fourier Analysis of Time Series: An Introduction, Second Edition BOLLEN · Structural Equations with Latent Variables BOROVKOV · Ergodicity and Stability of Stochastic Processes BOULEAU · Numerical Methods for Stochastic Processes BOX · Bayesian Inference in Statistical Analysis BOX · R A Fisher, the Life of a Scientist BOX and DRAPER · Empirical Model-Building and Response Surfaces ∗ BOX and DRAPER · Evolutionary Operation: A Statistical Method for Process Improvement BOX, HUNTER, and HUNTER · Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building ˜ · Statistical Control by Monitoring and Feedback Adjustment BOX and LUCENO BRANDIMARTE · Numerical Methods in Finance: A MATLAB-Based Introduction BROWN and HOLLANDER · Statistics: A Biomedical Introduction BRUNNER, DOMHOF, and LANGER · Nonparametric Analysis of Longitudinal Data in Factorial Experiments BUCKLEW · Large Deviation Techniques in Decision, Simulation, and Estimation CAIROLI and DALANG · Sequential Stochastic Optimization CHAN · Time Series: Applications to Finance CHATTERJEE and HADI · Sensitivity Analysis in Linear Regression CHATTERJEE and PRICE · Regression Analysis by Example, Third Edition CHERNICK · Bootstrap Methods: A Practitioner’s Guide CHERNICK and FRIIS · Introductory Biostatistics for the Health Sciences ` and DELFINER · Geostatistics: Modeling Spatial Uncertainty CHILES CHOW and LIU · Design and Analysis of Clinical Trials: Concepts and Methodologies, Second Edition CLARKE and DISNEY · Probability and Random Processes: A First Course with Applications, Second Edition ∗ COCHRAN and COX · Experimental Designs, Second Edition CONGDON · Bayesian Statistical Modelling CONOVER · Practical Nonparametric Statistics, Second Edition COOK · Regression Graphics COOK and WEISBERG · Applied Regression Including Computing and Graphics COOK and WEISBERG · An Introduction to Regression Graphics CORNELL · Experiments with Mixtures, Designs, Models, and the Analysis of Mixture Data, Third Edition COVER and THOMAS · Elements of Information Theory COX · A Handbook of Introductory Statistical Methods ∗ COX · Planning of Experiments CRESSIE ã Statistics for Spatial Data, Revised Edition ă ă and HORVATH ´ CSORG O · Limit Theorems in Change Point Analysis DANIEL · Applications of Statistics to Industrial Experimentation DANIEL · Biostatistics: A Foundation for Analysis in the Health Sciences, Sixth Edition ∗ DANIEL · Fitting Equations to Data: Computer Analysis of Multifactor Data, Second Edition DASU and JOHNSON · Exploratory Data Mining and Data Cleaning ∗ Now available in a lower priced paperback edition in the Wiley Classics Library TLFeBOOK DAVID and NAGARAJA · Order Statistics, Third Edition FIENBERG, and KADANE · Statistics and the Law DEL CASTILLO · Statistical Process Adjustment for Quality Control DETTE and STUDDEN · The Theory of Canonical Moments with Applications in Statistics, Probability, and Analysis DEY and MUKERJEE · Fractional Factorial Plans DILLON and GOLDSTEIN · Multivariate Analysis: Methods and Applications DODGE · Alternative Methods of Regression ∗ DODGE and ROMIG · Sampling Inspection Tables, Second Edition ∗ DOOB · Stochastic Processes DOWDY and WEARDEN · Statistics for Research, Second Edition DRAPER and SMITH · Applied Regression Analysis, Third Edition DRYDEN and MARDIA · Statistical Shape Analysis DUDEWICZ and MISHRA · Modern Mathematical Statistics DUNN and CLARK · Applied Statistics: Analysis of Variance and Regression, Second Edition DUNN and CLARK · Basic Statistics: A Primer for the Biomedical Sciences, Third Edition DUPUIS and ELLIS · A Weak Convergence Approach to the Theory of Large Deviations ∗ ELANDT-JOHNSON and JOHNSON · Survival Models and Data Analysis ENDERS · Applied Econometric Time Series ETHIER and KURTZ · Markov Processes: Characterization and Convergence EVANS, HASTINGS, and PEACOCK · Statistical Distributions, Third Edition FELLER · An Introduction to Probability Theory and Its Applications, Volume I, Third Edition, Revised; Volume II, Second Edition FISHER and VAN BELLE · Biostatistics: A Methodology for the Health Sciences ∗ FLEISS · The Design and Analysis of Clinical Experiments FLEISS · Statistical Methods for Rates and Proportions, Second Edition FLEMING and HARRINGTON · Counting Processes and Survival Analysis FULLER · Introduction to Statistical Time Series, Second Edition FULLER · Measurement Error Models GALLANT · Nonlinear Statistical Models GHOSH, MUKHOPADHYAY, and SEN · Sequential Estimation GIFI · Nonlinear Multivariate Analysis GLASSERMAN and YAO · Monotone Structure in Discrete-Event Systems GNANADESIKAN · Methods for Statistical Data Analysis of Multivariate Observations, Second Edition GOLDSTEIN and LEWIS · Assessment: Problems, Development, and Statistical Issues GREENWOOD and NIKULIN · A Guide to Chi-Squared Testing GROSS and HARRIS · Fundamentals of Queueing Theory, Third Edition ∗ HAHN and SHAPIRO · Statistical Models in Engineering HAHN and MEEKER · Statistical Intervals: A Guide for Practitioners HALD · A History of Probability and Statistics and their Applications Before 1750 HALD · A History of Mathematical Statistics from 1750 to 1930 HAMPEL · Robust Statistics: The Approach Based on Influence Functions HANNAN and DEISTLER · The Statistical Theory of Linear Systems HEIBERGER · Computation for the Analysis of Designed Experiments HEDAYAT and SINHA · Design and Inference in Finite Population Sampling HELLER · MACSYMA for Statisticians HINKELMAN and KEMPTHORNE: · Design and Analysis of Experiments, Volume 1: Introduction to Experimental Design HOAGLIN, MOSTELLER, and TUKEY · Exploratory Approach to Analysis of Variance HOAGLIN, MOSTELLER, and TUKEY · Exploring Data Tables, Trends and Shapes ∗ HOAGLIN, MOSTELLER, and TUKEY · Understanding Robust and Exploratory Data Analysis ∗ DEGROOT, ∗ Now available in a lower priced paperback edition in the Wiley Classics Library TLFeBOOK HOCHBERG and TAMHANE · Multiple Comparison Procedures HOCKING · Methods and Applications of Linear Models: Regression and the Analysis of Variance, Second Edition HOEL · Introduction to Mathematical Statistics, Fifth Edition HOGG and KLUGMAN · Loss Distributions HOLLANDER and WOLFE · Nonparametric Statistical Methods, Second Edition HOSMER and LEMESHOW · Applied Logistic Regression, Second Edition HOSMER and LEMESHOW · Applied Survival Analysis: Regression Modeling of Time to Event Data HØYLAND and RAUSAND · System Reliability Theory: Models and Statistical Methods HUBER · Robust Statistics HUBERTY · Applied Discriminant Analysis HUNT and KENNEDY · Financial Derivatives in Theory and Practice HUSKOVA, BERAN, and DUPAC · Collected Works of Jaroslav Hajek— with Commentary IMAN and CONOVER · A Modern Approach to Statistics JACKSON · A User’s Guide to Principle Components JOHN · Statistical Methods in Engineering and Quality Assurance JOHNSON · Multivariate Statistical Simulation JOHNSON and BALAKRISHNAN · Advances in the Theory and Practice of Statistics: A Volume in Honor of Samuel Kotz ă JUDGE, GRIFFITHS, HILL, LUTKEPOHL, and LEE · The Theory and Practice of Econometrics, Second Edition JOHNSON and KOTZ · Distributions in Statistics JOHNSON and KOTZ (editors) · Leading Personalities in Statistical Sciences: From the Seventeenth Century to the Present JOHNSON, KOTZ, and BALAKRISHNAN · Continuous Univariate Distributions, Volume 1, Second Edition JOHNSON, KOTZ, and BALAKRISHNAN · Continuous Univariate Distributions, Volume 2, Second Edition JOHNSON, KOTZ, and BALAKRISHNAN · Discrete Multivariate Distributions JOHNSON, KOTZ, and KEMP · Univariate Discrete Distributions, Second Edition ˇ ´ and SEN · Robust Statistical Procedures: Asymptotics and Interrelations JURECKOV A JUREK and MASON · Operator-Limit Distributions in Probability Theory KADANE · Bayesian Methods and Ethics in a Clinical Trial Design KADANE AND SCHUM · A Probabilistic Analysis of the Sacco and Vanzetti Evidence KALBFLEISCH and PRENTICE · The Statistical Analysis of Failure Time Data, Second Edition KASS and VOS · Geometrical Foundations of Asymptotic Inference KAUFMAN and ROUSSEEUW · Finding Groups in Data: An Introduction to Cluster Analysis KEDEM and FOKIANOS · Regression Models for Time Series Analysis KENDALL, BARDEN, CARNE, and LE · Shape and Shape Theory KHURI · Advanced Calculus with Applications in Statistics, Second Edition KHURI, MATHEW, and SINHA · Statistical Tests for Mixed Linear Models KLEIBER and KOTZ · Statistical Size Distributions in Economics and Actuarial Sciences KLUGMAN, PANJER, and WILLMOT · Loss Models: From Data to Decisions KLUGMAN, PANJER, and WILLMOT · Solutions Manual to Accompany Loss Models: From Data to Decisions KOTZ, BALAKRISHNAN, and JOHNSON · Continuous Multivariate Distributions, Volume 1, Second Edition KOTZ and JOHNSON (editors) · Encyclopedia of Statistical Sciences: Volumes to with Index ∗ Now available in a lower priced paperback edition in the Wiley Classics Library TLFeBOOK KOTZ and JOHNSON (editors) · Encyclopedia of Statistical Sciences: Supplement Volume KOTZ, READ, and BANKS (editors) · Encyclopedia of Statistical Sciences: Update Volume KOTZ, READ, and BANKS (editors) · Encyclopedia of Statistical Sciences: Update Volume KOVALENKO, KUZNETZOV, and PEGG · Mathematical Theory of Reliability of Time-Dependent Systems with Practical Applications LACHIN · Biostatistical Methods: The Assessment of Relative Risks LAD · Operational Subjective Statistical Methods: A Mathematical, Philosophical, and Historical Introduction LAMPERTI · Probability: A Survey of the Mathematical Theory, Second Edition LANGE, RYAN, BILLARD, BRILLINGER, CONQUEST, and GREENHOUSE · Case Studies in Biometry LARSON · Introduction to Probability Theory and Statistical Inference, Third Edition LAWLESS · Statistical Models and Methods for Lifetime Data, Second Edition LAWSON · Statistical Methods in Spatial Epidemiology LE · Applied Categorical Data Analysis LE · Applied Survival Analysis LEE and WANG · Statistical Methods for Survival Data Analysis, Third Edition LEPAGE and BILLARD · Exploring the Limits of Bootstrap LEYLAND and GOLDSTEIN (editors) · Multilevel Modelling of Health Statistics LIAO · Statistical Group Comparison LINDVALL · Lectures on the Coupling Method LINHART and ZUCCHINI · Model Selection LITTLE and RUBIN · Statistical Analysis with Missing Data, Second Edition LLOYD · The Statistical Analysis of Categorical Data MAGNUS and NEUDECKER · Matrix Differential Calculus with Applications in Statistics and Econometrics, Revised Edition MALLER and ZHOU · Survival Analysis with Long Term Survivors MALLOWS · Design, Data, and Analysis by Some Friends of Cuthbert Daniel MANN, SCHAFER, and SINGPURWALLA · Methods for Statistical Analysis of Reliability and Life Data MANTON, WOODBURY, and TOLLEY · Statistical Applications Using Fuzzy Sets MARDIA and JUPP · Directional Statistics MASON, GUNST, and HESS · Statistical Design and Analysis of Experiments with Applications to Engineering and Science, Second Edition McCULLOCH and SEARLE · Generalized, Linear, and Mixed Models McFADDEN · Management of Data in Clinical Trials McLACHLAN · Discriminant Analysis and Statistical Pattern Recognition McLACHLAN and KRISHNAN · The EM Algorithm and Extensions McLACHLAN and PEEL · Finite Mixture Models McNEIL · Epidemiological Research Methods MEEKER and ESCOBAR · Statistical Methods for Reliability Data MEERSCHAERT and SCHEFFLER · Limit Distributions for Sums of Independent Random Vectors: Heavy Tails in Theory and Practice ∗ MILLER · Survival Analysis, Second Edition MONTGOMERY, PECK, and VINING · Introduction to Linear Regression Analysis, Third Edition MORGENTHALER and TUKEY · Configural Polysampling: A Route to Practical Robustness MUIRHEAD · Aspects of Multivariate Statistical Theory MURRAY · X-STAT 2.0 Statistical Experimentation, Design Data Analysis, and Nonlinear Optimization ∗ Now available in a lower priced paperback edition in the Wiley Classics Library TLFeBOOK MURTHY, XIE, and JIANG · Weibull Models MYERS and MONTGOMERY · Response Surface Methodology: Process and Product Optimization Using Designed Experiments, Second Edition MYERS, MONTGOMERY, and VINING · Generalized Linear Models With Applications in Engineering and the Sciences NELSON · Accelerated Testing, Statistical Models, Test Plans, and Data Analyses NELSON · Applied Life Data Analysis NEWMAN · Biostatistical Methods in Epidemiology OCHI · Applied Probability and Stochastic Processes in Engineering and Physical Sciences OKABE, BOOTS, SUGIHARA, and CHIU · Spatial Tesselations: Concepts and Applications of Voronoi Diagrams, Second Edition OLIVER and SMITH · Influence Diagrams, Belief Nets and Decision Analysis PALTA · Quantitative Methods in Population Health: Extensions of Ordinary Regressions PANKRATZ · Forecasting with Dynamic Regression Models PANKRATZ · Forecasting with Univariate Box-Jenkins Models: Concepts and Cases ∗ PARZEN · Modern Probability Theory and Its Applications ˜ PENA, TIAO, and TSAY · A Course in Time Series Analysis PIANTADOSI · Clinical Trials: A Methodologic Perspective PORT · Theoretical Probability for Applications POURAHMADI · Foundations of Time Series Analysis and Prediction Theory PRESS · Bayesian Statistics: Principles, Models, and Applications PRESS · Subjective and Objective Bayesian Statistics, Second Edition PRESS and TANUR · The Subjectivity of Scientists and the Bayesian Approach PUKELSHEIM · Optimal Experimental Design PURI, VILAPLANA, and WERTZ · New Perspectives in Theoretical and Applied Statistics PUTERMAN · Markov Decision Processes: Discrete Stochastic Dynamic Programming ∗ RAO · Linear Statistical Inference and Its Applications, Second Edition RENCHER · Linear Models in Statistics RENCHER · Methods of Multivariate Analysis, Second Edition RENCHER · Multivariate Statistical Inference with Applications RIPLEY · Spatial Statistics RIPLEY · Stochastic Simulation ROBINSON · Practical Strategies for Experimenting ROHATGI and SALEH · An Introduction to Probability and Statistics, Second Edition ROLSKI, SCHMIDLI, SCHMIDT, and TEUGELS · Stochastic Processes for Insurance and Finance ROSENBERGER and LACHIN · Randomization in Clinical Trials: Theory and Practice ROSS · Introduction to Probability and Statistics for Engineers and Scientists ROUSSEEUW and LEROY · Robust Regression and Outlier Detection RUBIN · Multiple Imputation for Nonresponse in Surveys RUBINSTEIN · Simulation and the Monte Carlo Method RUBINSTEIN and MELAMED · Modern Simulation and Modeling RYAN · Modern Regression Methods RYAN · Statistical Methods for Quality Improvement, Second Edition SALTELLI, CHAN, and SCOTT (editors) · Sensitivity Analysis ∗ SCHEFFE · The Analysis of Variance SCHIMEK · Smoothing and Regression: Approaches, Computation, and Application SCHOTT · Matrix Analysis for Statistics SCHUSS · Theory and Applications of Stochastic Differential Equations SCOTT · Multivariate Density Estimation: Theory, Practice, and Visualization ∗ Now available in a lower priced paperback edition in the Wiley Classics Library TLFeBOOK ∗ SEARLE · Linear Models SEARLE · Linear Models for Unbalanced Data SEARLE · Matrix Algebra Useful for Statistics SEARLE, CASELLA, and McCULLOCH · Variance Components SEARLE and WILLETT · Matrix Algebra for Applied Economics SEBER and LEE · Linear Regression Analysis, Second Edition SEBER · Multivariate Observations SEBER and WILD · Nonlinear Regression SENNOTT · Stochastic Dynamic Programming and the Control of Queueing Systems ∗ SERFLING · Approximation Theorems of Mathematical Statistics SHAFER and VOVK · Probability and Finance: Its Only a Game! SMALL and MCLEISH · Hilbert Space Methods in Probability and Statistical Inference SRIVASTAVA · Methods of Multivariate Statistics STAPLETON · Linear Statistical Models STAUDTE and SHEATHER · Robust Estimation and Testing STOYAN, KENDALL, and MECKE · Stochastic Geometry and Its Applications, Second Edition STOYAN and STOYAN · Fractals, Random Shapes and Point Fields: Methods of Geometrical Statistics STYAN · The Collected Papers of T W Anderson: 1943–1985 SUTTON, ABRAMS, JONES, SHELDON, and SONG · Methods for MetaAnalysis in Medical Research TANAKA · Time Series Analysis: Nonstationary and Noninvertible Distribution Theory THOMPSON · Empirical Model Building THOMPSON · Sampling, Second Edition THOMPSON · Simulation: A Modeler’s Approach THOMPSON and SEBER · Adaptive Sampling THOMPSON, WILLIAMS, and FINDLAY · Models for Investors in Real World Markets ˜ and STIGLER (editors) · Box on Quality and TIAO, BISGAARD, HILL, PENA, Discovery: with Design, Control, and Robustness TIERNEY · LISP-STAT: An Object-Oriented Environment for Statistical Computing and Dynamic Graphics TSAY · Analysis of Financial Time Series UPTON and FINGLETON · Spatial Data Analysis by Example, Volume II: Categorical and Directional Data VAN BELLE · Statistical Rules of Thumb VIDAKOVIC · Statistical Modeling by Wavelets WEISBERG · Applied Linear Regression, Second Edition WELSH · Aspects of Statistical Inference WESTFALL and YOUNG · Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment WHITTAKER · Graphical Models in Applied Multivariate Statistics WINKER · Optimization Heuristics in Economics: Applications of Threshold Accepting WONNACOTT and WONNACOTT · Econometrics, Second Edition WOODING · Planning Pharmaceutical Clinical Trials: Basic Statistical Principles WOOLSON and CLARKE · Statistical Methods for the Analysis of Biomedical Data, Second Edition WU and HAMADA · Experiments: Planning, Analysis, and Parameter Design Optimization YANG · The Construction Theory of Denumerable Markov Processes ∗ ZELLNER · An Introduction to Bayesian Inference in Econometrics ZHOU, OBUCHOWSKI, and McCLISH · Statistical Methods in Diagnostic Medicine ∗ Now available in a lower priced paperback edition in the Wiley Classics Library TLFeBOOK ... of the titles in this series appears at the end of this volume TLFeBOOK Numerical Issues in Statistical Computing for the Social Scientist MICAH ALTMAN JEFF GILL MICHAEL P McDONALD A JOHN WILEY. .. on numerical computing issues, and the comprehensive work in statistical computing has hindered research in a number of social science fields There are two pathologies that can result In one instance,... 4.5.1 Obtaining Confidence Intervals with Ill-Behaved Functions 4.5.2 Interpreting Results in the Presence of Multiple Modes 4.5.3 Inference in the Presence of Instability Numerical Issues in Markov