Business Statistics: A Decision-Making Approach 7th Edition Chapter 15 Multiple Regression Analysis and Model Building Business Statistics: A Decision-Making Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-1 Chapter Goals After completing this chapter, you should be able to: explain model building using multiple regression analysis apply multiple regression analysis to business decision-making situations analyze and interpret the computer output for a multiple regression model test the significance Business Statistics: A Decision- of the independent variables a multiple model Makingin Approach, 7e ©regression 2008 Prentice-Hall, Inc Chap 15-2 Chapter Goals (continued) After completing this chapter, you should be able to: recognize potential problems in multiple regression analysis and take steps to correct the problems incorporate qualitative variables into the regression model by using dummy variables use variable transformations to model nonlinear Business Statistics: A Decisionrelationships Making Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-3 The Multiple Regression Model Idea: Examine the linear relationship between dependent (y) & or more independent variables (xi) Population model: Y-intercept Population slopes Random Error y β0 β1x1 β x βk x k ε Estimated multiple regression model: Estimated (or predicted) value of y Estimated intercept Business Statistics: A DecisionMaking Approach, 7e © 2008 1 Prentice-Hall, Inc Estimated slope coefficients yˆ b b x b x Chap bk15-4 xk Multiple Regression Model Two variable model y pe o Sl fo i ar v r le ab yˆ b0 b1x1 b x x1 x2 iable x r a v r o f Slope Business Statistics: A DecisionMaking Approach, 7e © 2008 x1 Prentice-Hall, Inc Chap 15-5 Multiple Regression Model Two variable model y yi Sample observation yˆ b0 b1x1 b x < < yi e = (y – y) x2i < x1i Business Statistics: A DecisionMaking Approach, 7e © 2008 x1 Prentice-Hall, Inc x2 The best fit equation, y , is found by minimizing the sum of squared errors, e2 Chap 15-6 Multiple Regression Assumptions Errors (residuals) from the regression model: < e = (y – y) The model errors are independent and random The errors are normally distributed The mean of the errors is zero Errors have a constant variance Business Statistics: A DecisionMaking Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-7 Model Specification Decide what you want to and select the dependent variable Determine the potential independent variables for your model Gather sample data (observations) for all variables Business Statistics: A DecisionMaking Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-8 The Correlation Matrix Correlation between the dependent variable and selected independent variables can be found using Excel: Formula Tab: Data Analysis / Correlation Can check for statistical significance of correlation with a t test Business Statistics: A DecisionMaking Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-9 Example A distributor of frozen desert pies wants to evaluate factors thought to influence demand Dependent variable: Pie sales (units per week) Independent variables: Price (in $) Advertising ($100’s) Data are collected for 15 weeks Business Statistics: A DecisionMaking Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-10 Testing for Significance: Quadratic Model Test for Overall Relationship F test statistic = MSR MSE Testing the Quadratic Effect Compare quadratic model y β0 β1x j β x 2j ε with the linear model y β0 β1x j ε Hypotheses (No 2nd order polynomial term) H : β = 0 Statistics: A Decision Business Making Approach, © 2008 (2nd order polynomial term is needed) H7e A: β2 Prentice-Hall, Inc Chap 15-56 Higher Order Models y x If p = the model is a cubic form: j j y β0 β1x j β x β3 x ε Business Statistics: A DecisionMaking Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-57 Interaction Effects Hypothesizes interaction between pairs of x variables Response to one x variable varies at different levels of another x variable Contains two-way cross product terms y β0 β1x1 β x12 β3 x β x1x β5 x12 x Basic Terms Business Statistics: A DecisionMaking Approach, 7e © 2008 Prentice-Hall, Inc Interactive Terms Chap 15-58 Effect of Interaction Given: y β0 β1x1 β2 x β3 x1x ε Without interaction term, effect of x1 on y is measured by β1 With interaction term, effect of x1 on y is measured by β1 + β3 x2 Effect changes as x2 increases Business Statistics: A DecisionMaking Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-59 Interaction Example y = + 2x1 + 3x2 + 4x1x2 y 12 where x2 = or (dummy variable) x2 = y = + 2x1 + 3(1) + 4x1(1) = + 6x1 x2 = y = + 2x1 + 3(0) + 4x1(0) = + 2x1 x1 Statistics: 0.5 A Decision1 1.5 Business MakingEffect Approach, 7e ©of2008 (slope) x1 on y does depend on x2 value Prentice-Hall, Inc Chap 15-60 Interaction Regression Model Worksheet multiply x1 by x2 to get x1x2, then Business Statistics: A Decisionrun regression with y, x1, x2 , x1x2 Making Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-61 Evaluating Presence of Interaction Hypothesize interaction between pairs of independent variables y β0 β1x1 β x β3 x1x ε Hypotheses: H : β = (no interaction between x and x ) HA: β3 ≠ (x1 interacts with x2) Business Statistics: A DecisionMaking Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-62 Model Building Goal is to develop a model with the best set of independent variables Stepwise regression procedure Easier to interpret if unimportant variables are removed Lower probability of collinearity Provide evaluation of alternative models as variables are added Best-subset approach Try all combinations Business Statistics: A Decision- and select the best using the and lowest s highest adjusted R ε Making Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-63 Stepwise Regression Idea: develop the least squares regression equation in steps, either through forward selection, backward elimination, or through standard stepwise regression The coefficient of partial determination is the measure of the marginal contribution of each independent variable, given that other independent variables are in the model Business Statistics: A DecisionMaking Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-64 Best Subsets Regression Idea: estimate all possible regression equations using all possible combinations of independent variables Choose the best fit by looking for the highest adjusted R2 and lowest standard error sε Stepwise regression and best subsets regression can be performed using PHStat, Business Statistics: A DecisionMinitab, or other statistical software packages Making Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-65 Aptness of the Model Diagnostic checks on the model include verifying the assumptions of multiple regression: Errors are independent and random Error are normally distributed Errors have constant variance Each x is linearly related to y i Business Statistics: A DecisionErrors (or Residuals) are given by Making Approach, 7e © 2008 Prentice-Hall, Inc ei ( y yˆ ) Chap 15-66 residuals residuals Residual Analysis x Constant variance x residuals residuals Non-constant variance Business Statistics: A DecisionNot Independent Making Approach, 7e © 2008 Prentice-Hall, Inc x x Independent Chap 15-67 The Normality Assumption Errors are assumed to be normally distributed Standardized residuals can be calculated by computer Examine a histogram or a normal probability plot of the standardized residuals to check for normality Business Statistics: A DecisionMaking Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-68 Chapter Summary Developed the multiple regression model Tested the significance of the multiple regression model Developed adjusted R2 Tested individual regression coefficients Used dummy variables Examined interaction in a multiple regression model Business Statistics: A DecisionMaking Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-69 Chapter Summary (continued) Described nonlinear regression models Described multicollinearity Discussed model building Stepwise regression Best subsets regression Examined residual plots to check model assumptions Business Statistics: A DecisionMaking Approach, 7e © 2008 Prentice-Hall, Inc Chap 15-70 ... multiple regression analysis to business decision-making situations analyze and interpret the computer output for a multiple regression model test the significance Business Statistics: A Decision-... incorporate qualitative variables into the regression model by using dummy variables use variable transformations to model nonlinear Business Statistics: A Decisionrelationships Making Approach,... b1x1 b x < < yi e = (y – y) x2i < x1i Business Statistics: A DecisionMaking Approach, 7e © 2008 x1 Prentice-Hall, Inc x2 The best fit equation, y , is found by minimizing the sum of squared errors,