Business Statistics: A Decision-Making Approach 6th Edition Chapter 14 Multiple Regression Analysis and Model Building Business Statistics: A Decision-Making Approach, 6e © 2005 PrenticeHall, Inc Chap 14-1 Chapter Goals After completing this chapter, you should be able to: understand model building using multiple regression analysis apply multiple regression analysis to business decision-making situations analyze and interpret the computer output for a multiple regression model test the significance Business Statistics: A Decision- of the independent variables a multiple model Makingin Approach, 6e ©regression 2005 Prentice-Hall, Inc Chap 14-2 Chapter Goals (continued) After completing this chapter, you should be able to: use variable transformations to model nonlinear relationships recognize potential problems in multiple regression analysis and take the steps to correct the problems incorporate qualitative variables into the regression by using dummy variables Business Statistics: Amodel Decision Making Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-3 The Multiple Regression Model Idea: Examine the linear relationship between dependent (y) & or more independent variables (xi) Population model: Y-intercept Population slopes Random Error y =â +â1x1 +â x +K +â k x k +å Estimated multiple regression model: Estimated (or predicted) value of y Estimated intercept Business Statistics: A DecisionMaking Approach, 6e © 2005 1 Prentice-Hall, Inc Estimated slope coefficients yˆ =b +b x +b x +K + b x k k Chap 14-4 Multiple Regression Model Two variable model y pe o Sl fo i ar v r le ab yˆ =b0 +b1x1 +b x x1 x2 iable x r a v r o f Slope Business Statistics: A DecisionMaking Approach, 6e © 2005 x1 Prentice-Hall, Inc Chap 14-5 Multiple Regression Model Two variable model y yi Sample observation yˆ =b0 +b1x1 +b x < < yi e = (y – y) x2i < x1i Business Statistics: A DecisionMaking Approach, 6e © 2005 x1 Prentice-Hall, Inc x2 The best fit equation, y , is found by minimizing the sum of squared errors, e2 Chap 14-6 Multiple Regression Assumptions Errors (residuals) from the regression model: < e = (y – y) The errors are normally distributed The mean of the errors is zero Errors have a constant variance The model errors are independent Business Statistics: A DecisionMaking Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-7 Model Specification Decide what you want to and select the dependent variable Determine the potential independent variables for your model Gather sample data (observations) for all variables Business Statistics: A DecisionMaking Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-8 The Correlation Matrix Correlation between the dependent variable and selected independent variables can be found using Excel: Tools / Data Analysis… / Correlation Can check for statistical significance of correlation with a t test Business Statistics: A DecisionMaking Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-9 Example A distributor of frozen desert pies wants to evaluate factors thought to influence demand Dependent variable: Pie sales (units per week) Independent variables: Price (in $) Advertising ($100’s) Data is collected for 15 weeks Business Statistics: A DecisionMaking Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-10 Testing for Significance: Quadratic Model Test for Overall Relationship F test statistic = MSR MSE Testing the Quadratic Effect Compare quadratic model y =â +â1x j +â x 2j +å with the linear model y =â +â1x j +å Hypotheses (No 2nd order polynomial term) H : β = 0 Statistics: A Decision Business Making Approach, © 2005 (2nd order polynomial term is needed) H6e A: β2 Prentice-Hall, Inc Chap 14-54 Higher Order Models y x If p = the model is a cubic form: j j y =â +â1x j +â x +â x +å Business Statistics: A DecisionMaking Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-55 Interaction Effects Hypothesizes interaction between pairs of x variables Response to one x variable varies at different levels of another x variable Contains two-way cross product terms y =â +â1x1 +â x12 +â x +â x1x +â x12 x Basic Terms Business Statistics: A DecisionMaking Approach, 6e © 2005 Prentice-Hall, Inc Interactive Terms Chap 14-56 Effect of Interaction Given: y =â +â1x1 +â x +â x1x +å Without interaction term, effect of x1 on y is measured by β1 With interaction term, effect of x1 on y is measured by β1 + β3 x2 Effect changes as x2 increases Business Statistics: A DecisionMaking Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-57 Interaction Example y = + 2x1 + 3x2 + 4x1x2 y 12 where x2 = or (dummy variable) x2 = y = + 2x1 + 3(1) + 4x1(1) = + 6x1 x2 = y = + 2x1 + 3(0) + 4x1(0) = + 2x1 x1 Statistics: 0.5 A Decision1 1.5 Business MakingEffect Approach, 6e ©of2005 (slope) x1 on y does depend on x2 value Prentice-Hall, Inc Chap 14-58 Interaction Regression Model Worksheet Case, i yi x1i x2i x1i x2i : : : : 40 30 : multiply x1 by x2 to get x1x2, then Business Statistics: A Decisionrun regression with y, x1, x2 , x1x2 Making Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-59 Evaluating Presence of Interaction Hypothesize interaction between pairs of independent variables y =â +â1x1 +â x +â x1x +å Hypotheses: H : β = (no interaction between x and x ) HA: β3 ≠ (x1 interacts with x2) Business Statistics: A DecisionMaking Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-60 Model Building Goal is to develop a model with the best set of independent variables Stepwise regression procedure Easier to interpret if unimportant variables are removed Lower probability of collinearity Provide evaluation of alternative models as variables are added Best-subset approach Try all combinations Business Statistics: A Decision- and select the best using the and lowest s highest adjusted R ε Making Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-61 Stepwise Regression Idea: develop the least squares regression equation in steps, either through forward selection, backward elimination, or through standard stepwise regression The coefficient of partial determination is the measure of the marginal contribution of each independent variable, given that other independent variables are in the model Business Statistics: A DecisionMaking Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-62 Best Subsets Regression Idea: estimate all possible regression equations using all possible combinations of independent variables Choose the best fit by looking for the highest adjusted R2 and lowest standard error sε Stepwise regression and best subsets regression can be performed using PHStat, Business Statistics: A DecisionMinitab, or other statistical software packages Making Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-63 Aptness of the Model Diagnostic checks on the model include verifying the assumptions of multiple regression: Each x is linearly related to y i Errors have constant variance Errors are independent Error are normally distributed Business Statistics: A DecisionErrors (or Residuals) are given by Making Approach, 6e © 2005 Prentice-Hall, Inc ei =( y - yˆ ) Chap 14-64 residuals residuals Residual Analysis x Constant variance x residuals residuals Non-constant variance Business Statistics: A DecisionNot Independent Making Approach, 6e © 2005 Prentice-Hall, Inc x x Independent Chap 14-65 The Normality Assumption Errors are assumed to be normally distributed Standardized residuals can be calculated by computer Examine a histogram or a normal probability plot of the standardized residuals to check for normality Business Statistics: A DecisionMaking Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-66 Chapter Summary Developed the multiple regression model Tested the significance of the multiple regression model Developed adjusted R2 Tested individual regression coefficients Used dummy variables Examined interaction in a multiple regression model Business Statistics: A DecisionMaking Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-67 Chapter Summary (continued) Described nonlinear regression models Described multicollinearity Discussed model building Stepwise regression Best subsets regression Examined residual plots to check model assumptions Business Statistics: A DecisionMaking Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-68 ... multiple regression analysis to business decision-making situations analyze and interpret the computer output for a multiple regression model test the significance Business Statistics: A Decision-... correct the problems incorporate qualitative variables into the regression by using dummy variables Business Statistics: Amodel Decision Making Approach, 6e © 2005 Prentice-Hall, Inc Chap 14-3 The... +å Estimated multiple regression model: Estimated (or predicted) value of y Estimated intercept Business Statistics: A DecisionMaking Approach, 6e © 2005 1 Prentice-Hall, Inc Estimated slope coefficients