Chapter 16 Statistical Tests Copyright © 2011 Pearson Education, Inc 16.1 Concepts of Statistical Tests A manager is evaluating software to filter SPAM e-mails (cost $15,000) To make it profitable, the software must reduce SPAM to less than 20% Should the manager buy the software? Use a statistical test to answer this question Consider the plausibility of a specific claim (claims are called hypotheses) of 40 Copyright © 2011 Pearson Education, Inc 16.1 Concepts of Statistical Tests Null and Alternative Hypotheses Statistical hypothesis: claim about a parameter of a population Null hypothesis (H0): specifies a default course of action, preserves the status quo Alternative hypothesis (Ha): contradicts the assertion of the null hypothesis of 40 Copyright © 2011 Pearson Education, Inc 16.1 Concepts of Statistical Tests SPAM Software Example Let p = email that slips past the filter H0: p ≥ 0.20 Ha: p < 0.20 These hypotheses lead to a one-sided test of 40 Copyright © 2011 Pearson Education, Inc 16.1 Concepts of Statistical Tests One- and Two-Sided Tests One-sided test: the null hypothesis allows any value of a parameter larger (or smaller) than a specified value Two-sided test: the null hypothesis asserts a specific value for the population parameter of 40 Copyright © 2011 Pearson Education, Inc 16.1 Concepts of Statistical Tests Type I and II Errors Reject H0 incorrectly (buying software that will not be cost effective) Retain H0 incorrectly (not buying software that would have been cost effective) of 40 Copyright © 2011 Pearson Education, Inc 16.1 Concepts of Statistical Tests Type I and II Errors indicates a correct decision of 40 Copyright © 2011 Pearson Education, Inc 16.1 Concepts of Statistical Tests Other Tests Visual inspection for association, normal quantile plots and control charts all use tests of hypotheses For example, the null hypothesis in a visual test for association is that there is no association between two variables shown in the scatterplot of 40 Copyright © 2011 Pearson Education, Inc 16.1 Concepts of Statistical Tests Sampling Distribution Statistical tests rely on the sampling distribution of the statistic that estimates the parameter specified in the null and alternative hypotheses Key question: What is the chance of getting a sample that differs from H0 by as much as this one if H0 is true? 10 of 40 Copyright © 2011 Pearson Education, Inc 16.3 Testing the Mean Null and Alternative Hypotheses Let µ = mean monthly rent for all rental properties in the Denver area Set up hypotheses as: H0: µ ≤ µ0 = $500 Ha: µ > µ0 = $500 26 of 40 Copyright © 2011 Pearson Education, Inc 16.3 Testing the Mean t - Statistic Used in the t-test for µ (since s estimates σ) The t-statistic, with n-1 df, is x − µ0 t= s/ n 27 of 40 Copyright © 2011 Pearson Education, Inc 16.3 Testing the Mean Example: Denver Rental Properties The firm obtained rents for a sample of size n=45; the average rent was $647 with s = $299 647 − 500 t= 299 / 45 t = 3.298 with 44 df; p-value = 0.00097 Reject H0 ; mean rent exceeds break-even value 28 of 40 Copyright © 2011 Pearson Education, Inc 16.3 Testing the Mean Finding the p-Value in the t-Table t = 3.298 is larger than any value in the row 29 of 40 Copyright © 2011 Pearson Education, Inc 16.3 Testing the Mean Summary 30 of 40 Copyright © 2011 Pearson Education, Inc 16.3 Testing the Mean Checklist SRS condition: the sample is a simple random sample from the relevant population Sample size condition Unless the population is normally distributed, a normal model can be used to approximate the sampling distribution of if n is larger than 10 times both the squared skewness and absolute value of X kurtosis 31 of 40 Copyright © 2011 Pearson Education, Inc 4M Example 16.2: COMPARING RETURNS Motivation ON INVESTMENTS Does stock in IBM return more, on average, than T-Bills? From 1980 through 2005, TBills returned 5% each month 32 of 40 Copyright © 2011 Pearson Education, Inc 4M Example 16.2: COMPARING Method RETURNS ON INVESTMENTS Let µ = mean of all future monthly returns for IBM stock Set up the hypotheses as H0: µ ≤ 0.005 Ha: µ > 0.005 Sample consists of monthly returns on IBM for 312 months (January 1980 – December 2005) 33 of 40 Copyright © 2011 Pearson Education, Inc 4M Example 16.2: COMPARING Mechanics RETURNS ON INVESTMENTS Sample yields = 0.0106 with s = 0.0805 x x − µ0 0.0106 − 0.0050 t= t= t = 1.22 with 311 df; p-value = 0.111 s/ n 0.0805 / 312 34 of 40 Copyright © 2011 Pearson Education, Inc 4M Example 16.2: COMPARING Message RETURNS ON INVESTMENTS Monthly IBM returns from 1980 through 2005 not bring statistically significantly higher earnings than comparable investments in US Treasury Bills during this period 35 of 40 Copyright © 2011 Pearson Education, Inc 16.4 Other Properties of Tests Significance versus Importance Statistical significance does not mean that you have made an important or meaningful discovery The size of the sample affects the p-value of a test With enough data, a trivial difference from H0 leads to a statistically significant outcome 36 of 40 Copyright © 2011 Pearson Education, Inc 16.4 Other Properties of Tests Confidence Interval or Test? A confidence interval provides a range of parameter values that are compatible with the observed data A test provides a precise analysis of a specific hypothesized value for a parameter 37 of 40 Copyright © 2011 Pearson Education, Inc Best Practices Pick the hypotheses before looking at the data Choose the null hypothesis on the basis of profitability Pick the α level first, taking into account both types of error Think about whether α = 0.05 is appropriate for each test 38 of 40 Copyright © 2011 Pearson Education, Inc Best Practices (Continued) Make sure to have an SRS from the right population Use a one-sided test Report a p–value to summarize the outcome of a test 39 of 40 Copyright © 2011 Pearson Education, Inc Pitfalls Do not confuse statistical significance with substantive importance Do not think that the p–value is the probability that the null hypothesis is true Avoid cluttering a test summary with jargon 40 of 40 Copyright © 2011 Pearson Education, Inc ... 2011 Pearson Education, Inc 16. 1 Concepts of Statistical Tests Type I and II Errors indicates a correct decision of 40 Copyright © 2011 Pearson Education, Inc 16. 1 Concepts of Statistical... Tests Visual inspection for association, normal quantile plots and control charts all use tests of hypotheses For example, the null hypothesis in a visual test for association is that there... simple random sample from the relevant population Sample size condition (for proportion): both np0 and n(1 p0 ) are larger than 10 19 of 40 Copyright © 2011 Pearson Education, Inc 4M Example 16. 1: