Statistics for physical sciences an introduction

STATISTICS FOR PHYSICAL SCIENCES AN INTRODUCTION This page intentionally left blank STATISTICS FOR PHYSICAL SCIENCES AN INTRODUCTION B R MARTIN Department of Physics and Astronomy University College London AMSTERDAM • BOSTON • HEIDELBERG • LONDON NEW YORK • OXFORD • PARIS • SAN DIEGO SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO Academic Press is an imprint of Elsevier Academic Press is an imprint of Elsevier 225 Wyman Street, Waltham, MA 02451, USA 525 B Street, Suite 1900, San Diego, CA 92101-4495, USA The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, UK Radarweg 29, PO Box 211, 1000 AE Amsterdam, The Netherlands First edition 2012 Copyright Ó 2012 Elsevier Inc All rights reserved No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means electronic, mechanical, photocopying, recording or otherwise without the prior written permission of the publisher Permissions may be sought directly from Elsevier’s Science & Technology Rights Department in Oxford, UK: phone (+44) (0) 1865 843830; fax (+44) (0) 1865 853333; email: permissions@elsevier.com Alternatively you can submit your request online by visiting the Elsevier web site at http://elsevier.com/locate/permissions, and selecting Obtaining permission to use Elsevier material Notice No responsibility is assumed by the publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein Because of rapid advances in the medical sciences, in particular, independent verification of diagnoses and drug dosages should be made Library of Congress Cataloging-in-Publication Data Application submitted British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library For information on all Academic Press publications visit our web site at elsevierdirect.com Printed and bound in USA 12 13 14 15 10 ISBN: 978-0-12-387760-4 Contents 3.4 Functions of a Random Variable Problems 55 Preface ix 51 Probability Distributions II: Examples 57 Statistics, Experiments, and Data 1.1 Experiments and Observations 1.2 Displaying Data 1.3 Summarizing Data Numerically 1.3.1 Measures of Location 1.3.2 Measures of Spread 1.3.3 More than One Variable 12 1.4 Large Samples 15 1.5 Experimental Errors 17 Problems 19 4.1 Uniform 57 4.2 Univariate Normal (Gaussian) 4.3 Multivariate Normal 63 4.3.1 Bivariate Normal 65 4.4 Exponential 66 4.5 Cauchy 68 4.6 Binomial 69 4.7 Multinomial 74 4.8 Poisson 75 Problems 80 Probability 21 59 Sampling and Estimation 83 2.1 Axioms of Probability 21 2.2 Calculus of Probabilities 23 2.3 The Meaning of Probability 27 2.3.1 Frequency Interpretation 27 2.3.2 Subjective Interpretation 29 Problems 32 5.1 Random Samples and Estimators 83 5.1.1 Sampling Distributions 84 5.1.2 Properties of Point Estimators 86 5.2 Estimators for the Mean, Variance, and Covariance 90 5.3 Laws of Large Numbers and the Central Limit Theorem 93 5.4 Experimental Errors 97 5.4.1 Propagation of Errors 99 Problems 103 Probability Distributions I: Basic Concepts 35 3.1 Random Variables 35 3.2 Single Variates 36 3.2.1 Probability Distributions 36 3.2.2 Expectation Values 40 3.2.3 Moment Generating, and Characteristic Functions 42 3.3 Several Variates 45 3.3.1 Joint Probability Distributions 45 3.3.2 Marginal and Conditional Distributions 45 3.3.3 Moments and Expectation Values 49 Sampling Distributions Associated with the Normal Distribution 105 6.1 6.2 6.3 6.4 v Chi-Squared Distribution 105 Student’s t Distribution 111 F Distribution 116 Relations Between c2, t, and F Distributions 119 Problems 121 vi CONTENTS Parameter Estimation I: Maximum Likelihood and Minimum Variance 123 7.1 Estimation of a Single Parameter 123 7.2 Variance of an Estimator 128 7.2.1 Approximate methods 130 7.3 Simultaneous Estimation of Several Parameters 133 7.4 Minimum Variance 136 7.4.1 Parameter Estimation 136 7.4.2 Minimum Variance Bound 137 Problems 140 Parameter Estimation II: Least-Squares and Other Methods 143 8.1 Unconstrained Linear Least Squares 143 8.1.1 General Solution for the Parameters 145 8.1.2 Errors on the Parameter Estimates 149 8.1.3 Quality of the Fit 151 8.1.4 Orthogonal Polynomials 152 8.1.5 Fitting a Straight Line 154 8.1.6 Combining Experiments 158 8.2 Linear Least Squares with Constraints 159 8.3 Nonlinear Least Squares 162 8.4 Other Methods 163 8.4.1 Minimum Chi-Square 163 8.4.2 Method of Moments 165 8.4.3 Bayes’ Estimators 167 Problems 171 Interval Estimation 173 9.1 Confidence Intervals: Basic Ideas 174 9.2 Confidence Intervals: General Method 177 9.3 Normal Distribution 179 9.3.1 Confidence Intervals for the Mean 180 9.3.2 Confidence Intervals for the Variance 182 9.3.3 Confidence Regions for the Mean and Variance 183 9.4 Poisson Distribution 184 9.5 Large Samples 186 9.6 Confidence Intervals Near Boundaries 187 9.7 Bayesian Confidence Intervals 189 Problems 190 10 Hypothesis Testing I: Parameters 193 10.1 Statistical Hypotheses 194 10.2 General Hypotheses: Likelihood Ratios 198 10.2.1 Simple Hypothesis: One Simple Alternative 198 10.2.2 Composite Hypotheses 201 10.3 Normal Distribution 204 10.3.1 Basic Ideas 204 10.3.2 Specific Tests 206 10.4 Other Distributions 214 10.5 Analysis of Variance 215 Problems 10 218 11 Hypothesis Testing II: Other Tests 221 11.1 Goodness-of-Fit Tests 221 11.1.1 Discrete Distributions 222 11.1.2 Continuous Distributions 225 11.1.3 Linear Hypotheses 228 11.2 Tests for Independence 231 11.3 Nonparametric Tests 233 11.3.1 Sign Test 233 11.3.2 Signed-Rank Test 234 11.3.3 Rank-Sum Test 236 11.3.4 Runs Test 237 11.3.5 Rank Correlation Coefficient 239 Problems 11 241 Appendix A Miscellaneous Mathematics 243 A.1 Matrix Algebra 243 A.2 Classical Theory of Minima 247 Appendix B Optimization of Nonlinear Functions 249 B.1 General Principles 249 B.2 Unconstrained Minimization of Functions of One variable 252 B.3 Unconstrained Minimization of Multivariable Functions 253 B.3.1 Direct Search Methods 253 B.3.2 Gradient Methods 254 B.4 Constrained Optimization 255 vii CONTENTS Appendix C Statistical Tables 257 C.1 C.2 C.3 C.4 C.5 C.6 C.7 C.8 Normal Distribution 257 Binomial Distribution 259 Poisson Distribution 266 Chi-squared Distribution 273 Student’s t Distribution 275 F Distribution 277 Signed-Rank Test 283 Rank-Sum Test 284 C.9 Runs Test 285 C.10 Rank Correlation Coefficient 286 Appendix D Answers to Odd-Numbered Problems 287 Bibliography 293 Index 295 This page intentionally left blank Preface Almost all physical scientists e physicists, astronomers, chemists, earth scientists, and others e at some time come into contact with statistics This is often initially during their undergraduate studies, but rarely is it via a full lecture course Usually, some statistics lectures are given as part of a general mathematical methods course, or as part of a laboratory course; neither route is entirely satisfactory The student learns a few techniques, typically unconstrained linear leastsquares fitting and analysis of errors, but without necessarily the theoretical background that justifies the methods and allows one to appreciate their limitations On the other hand, physical scientists, particularly undergraduates, rarely have the time, and possibly the inclination, to study mathematical statistics in detail What I have tried to in this book is therefore to steer a path between the extremes of a recipe of methods with a collection of useful formulas, and a detailed account of mathematical statistics, while at the same time developing the subject in a reasonably logical way I have included proofs of some of the more important results stated in those cases where they are fairly short, but this book is written by a physicist for other physical scientists and there is no pretense to mathematical rigor The proofs are useful for showing how the definitions of certain statistical quantities and their properties may be used Nevertheless, a reader uninterested in the proofs can easily skip over these, hopefully to come back to them later Above all, I have contained the size of the book so that it can be read in its entirety by anyone with a basic exposure to mathematics, principally calculus and matrices, at the level of a firstyear undergraduate student of physical science Statistics in physical science is principally concerned with the analysis of numerical data, so in Chapter there is a review of what is meant by an experiment, and how the data that it produces are displayed and characterized by a few simple numbers This leads naturally to a discussion in Chapter of the vexed question of probability e what we mean by this term and how is it calculated There then follow two chapters on probability distributions: the first reviews some basic concepts and in the second there is a discussion of the properties of a number of specific theoretical distributions commonly met in the physical sciences In practice, scientists rarely have access to the whole population of events, but instead have to rely on a sample from which to draw inferences about the population; so in Chapter the basic ideas involved in sampling are discussed This is followed in Chapter by a review of some sampling distributions associated with the important and ubiquitous normal distribution, the latter more familiar to physical scientists as the Gaussian function The next two chapters explain how estimates are inferred for individual parameters of a population from sample statistics, using several practical techniques This is called point estimation It is generalized in Chapter by considering how to obtain estimates for the interval ix vf / vx2 vxi : Di ¼ vx2 vx1 vx22 « « « v2 f v2 f v2 f .. .STATISTICS FOR PHYSICAL SCIENCES AN INTRODUCTION This page intentionally left blank STATISTICS FOR PHYSICAL SCIENCES AN INTRODUCTION B R MARTIN Department of Physics and Astronomy... binned data, but for the whole population, i.e., N ¼ 100 students The variances and covariance are easiest to calculate from formulas analogous to (1.6) and (1.12b), but for binned data For the population,... Distribution 179 9.3.1 Confidence Intervals for the Mean 180 9.3.2 Confidence Intervals for the Variance 182 9.3.3 Confidence Regions for the Mean and Variance 183 9.4 Poisson Distribution 184 9.5