Once the parameters of an equation are estimated, the analyst must address the question of whether the parameter estimates ( a and ˆ ˆb ) are significantly different from 0. If the estimated coefficient is far enough away from 0—either sufficiently greater than 0 (a positive estimate) or sufficiently less than 0 (a negative estimate)—
the estimated coefficient is said to be statistically significant. The question of statisti- cal significance arises because the estimates are themselves random variables. The parameter estimates are random because they are calculated using values of Y and X that are collected in a random fashion (remember, the sample is a random sample).
Because the values of the parameters are estimates of the true parameter values, the estimates are rarely equal to the true parameter values. In other words, the estimates calculated by the computer are almost always going to be either too large or too small.
Because the estimated values of the parameters ( ˆa and ˆb ) are unlikely to be the true values (a and b), it is possible that a parameter could truly be equal to 0 even though the computer calculates a parameter estimate that is not equal to 0. Fortunately, statistical techniques exist that provide a tool for making probabilistic statements about the true values of the parameters. This tool is called hypothesis testing.
To understand fully the concept of hypothesis testing, you would need to take at least one course, and probably two, in statistics. In this text we intend only to motivate through intuition the necessity and process of performing a test of statistical significance. Our primary emphasis will be to show you how to test the hypothesis that Y is truly related to X. If Y is indeed related to X, the true value of the slope parameter b will be either a positive or a negative number. (Remember, if b 5 DY/
DX 5 0, no change in Y occurs when X changes.) Thus, the explanatory variable X has a statistically significant effect on the dependent variable Y when b ị 0.1
Now try Technical Problem 2.
statistically significant There is sufficient evidence from the sample to indicate that the true value of the coefficient is not 0.
hypothesis testing A statistical technique for making a probabilistic statement about the true value of a parameter.
1Testing for statistical significance of the intercept parameter a is typically of secondary impor- tance to testing for significance of the slope parameters. As you will see, it is the slope parameters rather than the intercept parameter that provide the most essential information for managerial deci- sion making. Nevertheless, it is customary to test the intercept parameter for statistical significance in exactly the same manner as the slope parameter is tested.
We will now discuss the procedure for testing for statistical significance by describing how to measure the accuracy, or precision, of an estimate. Then we will introduce and explain a statistical test (called a t-test) that can be used to make a probabilistic statement about whether Y is truly related to the explanatory variable X—that is, whether the true value of the parameter b is zero.
The Relative Frequency Distribution for b^
As noted, the necessity of testing for statistical significance arises because the ana- lyst does not know the true values of a and b—they are estimated from a random sample of observations on Y and X. Consider again the relation between sales of travel packages and advertising expenditures estimated in the previous sec- tion. The least-squares estimate of the slope parameter b from the sample of seven travel agencies shown in Table 4.2 is 4.9719. Suppose you collected a new sample by randomly selecting seven other travel agencies and use their sales and ad- vertising expenditures to estimate b. The estimate for b will probably not equal 4.9719 for the second sample. Remember, b ˆ is computed using the values of S and A in the sample. Because of randomness in sampling, different samples generally result in different values of S and A, and thus different estimates of b. Therefore, b ˆ is a random variable—its value varies in repeated samples.
The relative frequency with which ˆb takes on different values provides infor- mation about the accuracy of the parameter estimates. Even though researchers seldom have the luxury of taking repeated samples, statisticians have been able to determine theoretically the relative frequency distribution of values that b ˆ would take in repeated samples. This distribution is also called the probability density function, or pdf, by statisticians. Figure 4.3 shows the relative frequency distribu- tion for ˆb , when the true value of b is equal to 5.
Now try Technical Problem 3.
relative frequency distribution The distribution (and relative frequency) of values b ˆcan take because observations on Y and X come from a random sample.
1
0 1 2 3 4 5 6 7 8 9 10 bˆ
Relative frequency of bˆ
Least-squares estimate of b (b)ˆ F I G U R E 4.3
Relative Frequency Distribution for b ˆ When b 5 5
Notice that the distribution of values that ˆb might take in various samples is centered around the true value of 5. Even though the probability of drawing a sample for which ˆb exactly equals 5 is extremely small, the average (mean or expected) value of all possible values of ˆb is 5. The estimator ˆb is said to be an unbiased estimator if the average (mean or expected) value of the estimator is equal to the true value of the parameter. Statisticians have demonstrated that the least-squares estimators of a and b (â and ˆb ) are unbiased estimators in a wide variety of statistical circumstances. Unbiasedness does not mean that any one estimate equals the true parameter value. Unbiasedness means only that, in repeated samples, the estimates tend to be centered around the true value.
The smaller the dispersion of b ˆ around the true value, the more likely it is that an estimate of b ˆ is close to the true value. In other words, the smaller the variance of the distribution of ˆb , the more accurate estimates are likely to be. Not surpris- ingly, the variance of the estimate of b plays an important role in the determina- tion of statistical significance. The square root of the variance of ˆb is called the standard error of the estimate, which we will denote S ˆb .2 All computer regression routines compute standard errors for the parameter estimates.
The Concept of a t -Ratio
When we regressed sales on advertising expenditures for the seven travel agencies in Table 4.2, we obtained an estimate of b equal to 4.9719. Because 4.9719 is not equal to 0, this seems to suggest that the level of advertising does indeed affect sales. (Remember that if b 5 0, there is no relation between sales and advertising.) As explained earlier, the estimate of b calculated using a random sample may take on a range of values. Even though 4.9719 is greater than 0, it is possible that the true value of b is 0. In other words, the analyst runs some risk that the true value of b is 0 even when ˆb is not calculated to be 0.
The probability of drawing a sample for which the estimate of b is much larger than 0 is very small when the true value of b is actually 0. How large does ˆb have to be for an analyst to be quite sure that b is not really 0 (i.e., advertising does play a significant role in determining sales)? The answer to this question is obtained by performing a hypothesis test. The hypothesis that one normally tests is that b 5 0.
Statisticians use a t-test to make a probabilistic statement about the likelihood that the true parameter value b is not equal to 0. Using the t-test, it is possible to determine statistically how large b ˆ must be in order to conclude that b is not equal to 0.
To perform a t-test for statistical significance, we form what statisticians call a t-ratio:
t 5 ___S b ˆˆb
where ˆb is the least-squares estimate of b and S ˆb is the standard error of the estimate, both of which are calculated by the computer. The numerical value of the t-ratio is called a t-statistic.
unbiased estimator An estimator that produces estimates of a parameter that are on average equal to the true value of the parameter.
t-test
A statistical test used to test the hypothesis that the true value of a parameter is equal to 0 (b 5 0).
t-ratio
The ratio of an estimated regression parameter divided by the standard error of the estimate.
t-statistic
The numerical value of
the t-ratio. 2More correctly, the standard error of the estimate is the square root of the estimated variance of b .ˆ
By combining information about the size of ˆb (in the numerator) and the accuracy or precision of the estimate (in the denominator), the t-ratio indicates how much confidence one can have that the true value of b is actually larger than ( significantly different from) 0. The larger the absolute value of the t-ratio, the more confident one can be that the true value of b is not 0. To show why this is true, we must examine both the numerator and the denominator of the t-ratio.
Consider the numerator when the estimate b is positive. When b is actually 0, ˆ drawing a random sample that will produce an estimate of b that is much larger than 0 is unlikely. Thus the larger the numerator of the t-ratio, the less likely it is that b really does equal 0. Turning now to the denominator of the t-ratio, recall that S ˆb , the standard error of the estimate, measures the accuracy of the estimate of b. The smaller the standard error of ˆb (and thus the more accurate ˆb is), the smaller the error in estimation is likely to be. Consequently, the farther from 0 ˆb is (i.e., the larger the numerator) and the smaller the standard error of the estimate (i.e., the smaller the denominator), the larger the t-ratio, and the more sure we are that the true value of b is greater than 0.
Now consider the situation when the estimate ˆb is negative (e.g., if we had esti- mated the relation between profits and shoplifting). In this case we would be more certain that b was really negative if the t-ratio had a more negative magnitude.
Regardless of whether b ˆ is positive or negative, the following important statistical relation is established:
Relation The larger the absolute value of ˆ b yS ˆb (the t-ratio), the more probable it is that the true value of b is not equal to 0.
Performing a t-Test for Statistical Significance
The t-statistic is used to test the hypothesis that the true value of b equals 0. If the calculated t-statistic or t-ratio is greater than the critical value of t (to be explained later), then the hypothesis that b 5 0 is rejected in favor of the alternative hypothesis that b ị 0. When the calculated t-statistic exceeds the critical value of t, b is signifi- cantly different from 0, or, equivalently, b is statistically significant. If the hypothesis that b 5 0 cannot be rejected, then the sample data are indicating that X, the explana- tory variable for which b is the coefficient, is not related to the dependent variable Y (DYyDX 5 0). Only when a parameter estimate is statistically significant should the associated explanatory variable be included in the regression equation.
Although performing a t-test is the correct way to assess the statistical significance of a parameter estimate, there is always some risk that the t-test will indicate b ị 0 when in fact b 5 0. Statisticians refer to this kind of mistake as a Type I error—finding a parameter estimate to be significant when it is not.3 The probability of making a Type I error when performing a t-test is referred to
Now try Technical Problem 4.
critical value of t The value that the t-statistic must exceed in order to reject the hypothesis that b 5 0.
Type I error Error in which a parameter estimate is found to be statistically significant when it is not.
3Statisticians also recognize the possibility of committing a Type II error, which occurs when an analyst fails to find a parameter estimate to be statistically significant when it truly is significant.
In your statistics class you will study both types of errors, Type I and Type II. Because it is usually impossible to determine the probability of committing a Type II error, tests for statistical significance typically consider only the possibility of committing a Type I error.
as the level of significance of the t-test. The level of significance associated with a t-test is the probability that the test will indicate b ị 0 when in fact b 5 0. Stated differently, the significance level is the probability of finding the parameter to be statistically significant when in fact it is not. As we are about to show you, an ana- lyst can control or select the level of significance for a t-test. Traditionally, either a 0.01, 0.02, 0.05, or 0.10 level of significance is selected, which reflects the ana- lyst’s willingness to tolerate at most a 1, 2, 5, or 10 percent probability of finding a parameter to be significant when it is not. In practice, however, the significance level tends to be chosen arbitrarily. We will return to the problem of selecting the appropriate level of significance later in this discussion of t-tests.
A concept closely related to the level of significance is the level of confidence.
The level of confidence equals one minus the level of significance, and thus gives the probability that you will not make a Type I error. The confidence level is the probability a t-test will correctly find no relation between Y and X (i.e., b 5 0).
The lower the level of significance, the greater the level of confidence. If the level of significance chosen for conducting a t-test is 0.05 (5 percent), then the level of confidence for the test is 0.95 (95 percent), and you can be 95 percent confident that the t-test will correctly indicate lack of significance. The levels of significance and confidence provide the same information, only in slightly different ways: The significance level gives the probability of making a Type I error, while the confi- dence level gives the probability of not making a Type I error. A 5 percent level of significance and a 95 percent level of confidence mean the same thing.
Relation In testing for statistical significance, the level of significance chosen for the test determines the probability of committing a Type I error, which is the mistake of finding a parameter to be significant when it is not truly significant. The level of confidence for a test is the probability of not committing a Type I error. The lower (higher) the significance level of a test, the higher (lower) the level of confidence for the test.
The t-test is simple to perform. First, calculate the t-statistic (t-ratio) from the parameter estimate and its standard error, both of which are calculated by the computer. (In most statistical software, the t-ratio is also calculated by the computer.) Next, find the appropriate critical value of t for the chosen level of significance. (Critical values of t are provided in a t-table at the end of this book, along with explanatory text.) The critical value of t is defined by the level of significance and the appropriate degrees of freedom. The degrees of freedom for a t-test are equal to n 2 k, where n is the number of observations in the sample and k is the number of parameters estimated.4 (In the advertising example, there are 7 2 2 5 5 degrees of freedom, since we have seven observations and estimated two parameters, a and b.)
Once the critical value of t is found for, say, the 5 percent level of significance or 95 percent level of confidence, the absolute value of the calculated t-statistic
level of significance The probability of finding the parameter to be statistically significant when in fact it is not.
level of confidence The probability of correctly failing to reject the true hypothesis that b 5 0; equals one minus the level of significance.
degrees of freedom The number of observations in the sample minus the number of parameters being estimated by the regression analysis (n 2 k).
4Occasionally you may find other statistics books (or t-tables in other books) that define k as the
“number of explanatory variables” rather than the “number of parameters estimated,” as we have done in this text. When k is not defined to include the estimated intercept parameter, then the number of degrees of freedom must be calculated as n 2 (k 1 1). No matter how k is defined, the degrees of freedom for the t-test are always equal to the number of observations minus the number of parameters estimated.
is compared with the critical value of t. If the absolute value of the t-statistic is greater than the critical value of t, we say that, at the 95 percent confidence level, the estimated parameter is (statistically) significantly different from zero. If the absolute value of the calculated t-statistic is less than the critical value of t, the estimated value of b cannot be treated as being significantly different from 0 and X plays no statistically significant role in determining the value of Y.
Returning to the advertising example, we now test to see if 4.9719, the estimated value of b, is significantly different from 0. The standard error of ˆb , which is calculated by the computer, is equal to 1.23. Thus the t-statistic is equal to 4.04 (5 4.9719/1.23). Next we compare 4.04 to the critical value of t, using a 5 percent significance level (a 95 percent confidence level). As noted, there are 5 degrees of freedom. If you turn to the table of critical t-values at the end of the text, you will find that the critical value of t for 5 degrees of freedom and a 0.05 level of significance is 2.571. Because 4.04 is larger than 2.571, we reject the hypothesis that b is 0 and can now say that 4.9719 ( ˆb ) is significantly different from 0. This means that advertising expenditure is a statistically significant variable in determining the level of sales. If 4.04 had been less than the critical value, we would not have been able to reject the hypothesis that b is 0 and we would not have been able to conclude that advertising plays a significant role in determining the level of sales.
The procedure for testing for statistical significance of a parameter estimate is summarized in the following statistical principle:
Principle To test for statistical significance of a parameter estimate b , compute the t-ratioˆ t 5 ___b ˆ
S ˆb
where S ˆb is the standard error of the estimate b . Next, for the chosen level of significance, find the critical ˆ t-value in the t-table at the end of the text. Choose the critical t-value with n 2 k degrees of freedom for the chosen level of significance. If the absolute value of the t-ratio is greater (less) than the critical t-value, then b is (is not) statistically significant.ˆ
Using p-Values to Determine Statistical Significance
Using a t-test to determine whether a parameter estimate is statistically signifi- cant requires that you select a level of significance at which to perform the test.
In most of the situations facing a manager, choosing the significance level for the test involves making an arbitrary decision. We will now show you an alternative method of assessing the statistical significance of parameter estimates that does not require that you “preselect” a level of significance (or, equivalently, the level of confidence) or use a t-table to find a critical t-value. With this alternative method, the exact degree of statistical significance is determined by answering the question, “Given the t-ratio calculated for ˆb , what would be the lowest level of significance—or the highest level of confidence—that would allow the hypothesis b 5 0 to be rejected in favor of the alternative hypothesis b ị 0?”
Consider the t-test for the parameter estimate 4.9719. In the previous section, the effect of advertising (A) on sales (S) was found to be statistically significant
because the calculated t-ratio 4.04 exceeded 2.571, the critical t-value for a 5 percent level of significance (a 95 percent level of confidence). A t-ratio only as large as 2.571 would be sufficient to achieve a 5 percent level of significance that b ị 0. The calculated t-ratio 4.04 is much larger than the critical t for the 5 percent significance level. This means a significance level lower than 5 percent (or a confi- dence level higher than 95 percent) would still allow one to reject the hypothesis of no significance (b 5 0). What is the lowest level of significance or, equivalently, the greatest level of confidence that permits rejecting the hypothesis that b 5 0 when the computer calculates a t-ratio of 4.04? The answer is given by the p-value for 4.04, which most statistical software, and even spreadsheets, can calculate.
The p-value associated with a calculated t-ratio gives the exact level of significance for a t-ratio associated with a parameter estimate.5 In other words, the p-value gives the exact probability of committing a Type I error—finding significance when none exists—if you conclude that b ị 0 on the basis of the t-ratio calculated by the computer. One minus the p-value is the exact degree of confidence that can be assigned to a particular parameter estimate.
The p-value for the calculated t-ratio 4.04 (5 4.9719/1.23) is 0.010. A p-value of 0.010 means that the exact level of significance for a t-ratio of 4.04 is 1 percent and the exact level of confidence is 99 percent. Rather than saying b is statisti- cally significant at the 5 percent level of significance (or the 95 percent level of confidence), using the p-value we can make a more precise, and stronger, state- ment: ˆb is statistically significant at exactly the 1 percent level of significance. In other words, at the 99 percent confidence level advertising affects sales (b ị 0);
that is, there is only a 1 percent chance that advertising does not affect sales.
While t-tests are the traditional means of assessing statistical significance, most computer software packages now routinely print the p-values associated with t-ratios. Rather than preselecting a level of significance (or level of confidence) for t-tests, it is now customary to report the p-values associated with the estimated parameters—usually along with standard errors and t-ratios—and let the users of the statistical estimations decide whether the level of significance is acceptably low or the level of confidence is acceptably high.
Relation The exact level of significance associated with a t -statistic, its p -value, gives the exact (or minimum) probability of committing a Type I error—finding significance when none exists—if you conclude that b ị 0 on the basis of the t -ratio calculated by the computer. One minus the p -value is the exact degree of confidence that can be assigned to a particular parameter estimate.