an investigation of accuracy, learning and biases in judgmental adjustments of statistical forecasts

AN INVESTIGATION OF ACCURACY, LEARNING AND BIASES IN JUDGMENTAL ADJUSTMENTS OF STATISTICAL FORECASTS DISSERTATION Presented in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy In the Graduate School of The Ohio State University By Cuneyt Eroglu * * * * * The Ohio State University 2006 Dissertation Committee: Approved by Professor Douglas M. Lambert, Adviser Professor Keely L. Croxton _____________________________ Professor A. Michael Knemeyer Adviser Graduate Program in Business Administration Copyright by Cuneyt Eroglu 2006 ii ABSTRACT Judgmentally adjusting a statistical forecast before using it is a widespread practice in business. The goal of this study is to provide a deeper understanding of judgmental adjustments of statistical forecasts to improve forecasting performance. The forecasting performance is measured with three dependent variables. The first dependent variable is accuracy improvement which represents the positive change in forecast accuracy after a judgmental adjustment. The second dependent variable is learning which refers to the continuous improvement of a forecaster’s performance over time. The third variable is actually a set of variables that includes various biases. A bias is a systematic deviation in forecasts that is introduced by a particular forecaster. The independent variables included in this study are personality variables, motivational variables and situational variables. Personality is measured by the Big-Five model that analyzes an individual’s personality in five dimensions including extraversion, conscientiousness, neuroticism, agreeableness and openness to experience. Motivational variables are measured in terms of motivational orientation and various motivational stimuli. Situational variables include feedback, supervision, timing of adjustment, and demographics. iii Data were collected from a company where store managers used judgmental adjustments of statistical forecasts to improve the forecast accuracy for their stores. The data collection covered a period of 12 months and 390 stores over several Midwestern and southeastern states. The data for dependent variables were obtained from forecasting records and the data for independent variables were collected using a survey instrument. The results indicate that, on average, judgmental adjustments of statistical forecasts result in accuracy improvement. The extent of the accuracy improvement is affected by personality, motivational and situational variables. Furthermore, there was evidence for biases that were introduced through judgmental adjustments. Biases were also moderated by personality, motivational and situational variables. This study detected no evidence of learning. iv ACKNOWLEDGMENTS A doctoral dissertation is rarely an exclusive work of a single individual. This dissertation is no exception. I am deeply indebted to several people whose support, help and encouragement made this dissertation a reality. Their contributions to this study were invaluable. First and foremost, I would like to thank Professor Douglas M. Lambert, Director of The Global Supply Chain Forum at The Ohio State University and Chairman of my dissertation committee. He has been very supportive throughout the entire course of my doctoral program in countless ways. His expert judgment and constructive critiquing of my work enabled me to navigate safely and productively through every phase of my dissertation from selecting a promising research topic to applying rigorous academic standards to my data collection and analysis to communicating the research results in a well-written dissertation. He not only guided me from an academic perspective, but also helped me obtain business data for my dissertation and make my research relevant to practitioners. His insights, feedback, encouragement and guidance have made this dissertation remarkably better. Above all, he has provided me with an excellent role model as a researcher, a teacher and an academician, which I will always look up to and which will surely shape my academic career for the years to come. v Together with Professor Lambert, Professor Keely L. Croxton and Professor A. Michael Knemeyer formed my dissertation committee that guided me through the dissertation phase of my studies. Their feedback and insights contributed immensely to the quality of the end product. They were always ready and willing to assist me with any challenges and dilemmas that I was facing. Furthermore, they dedicated many hours of their time to review my work and provide me with further guidance. I will forever owe Professor Croxton and Professor Knemeyer a debt of gratitude. I also would like to thank Professor Martha C. Cooper, Professor Walter Zinn and Professor Thomas J. Goldsby who have made significant contributions to my personal and professional growth during my Ph.D. program. During my interactions with them, both inside and outside the class room, our discussions have always been intellectually stimulating and broadened my horizons. Their support and guidance will forever be remembered with much gratefulness. As a person who has always gone beyond the call of duty, Shirley J. Gaddis deserves much credit. Without her crucial assistance and extraordinary organizational skills, it would have been much more difficult for me to communicate with my dissertation committee members, schedule meetings and presentations, and circulate drafts of my work. She has been an exceptional facilitator between my dissertation committee members and myself, which, in turn, substantially shortened the time required for writing this dissertation. I would also like to express my gratitude for my fellow doctoral students who have helped create a friendly working atmosphere where we shared good times, debated research ideas, and vented during more challenging times. Furthermore, I have received vi much help from the staff at the Marketing and Logistics Department, the Graduate Programs Office and the Office of International Education. Their assistance is greatly appreciated. This dissertation is based on the data obtained from a company that is a member of The Global Supply Chain Forum at The Ohio State University. I would like to express my deepest thanks to the executives at this company and other member companies of The Global Supply Chain Forum who have provided me with data, stimulated my thinking with their questions, and enriched my work with their suggestions and feedback. Their contributions added immensely to the value of this dissertation. I shall forever be grateful to them. Last but not least, I would like to thank Steven Robeano, a dear friend who has always been there for me throughout the Ph.D. program. I could always count on him to listen to me, to share his insights and ideas, to offer advice and provide encouragement. His friendship is greatly appreciated. vii VITA EDUCATION 1992 Bachelor of Science, Industrial Engineering, Middle East Technical University, Ankara, Turkey 1994 Master of Science, Management Science, University of Miami, Coral Gables, Florida 2004 Master of Arts, Business Administration, Logistics, Fisher College of Business, The Ohio State University 2002-present Graduate Teaching and Research Associate, The Ohio State University PROFESSIONAL EXPERIENCE 1994-1996 Logistics Analyst, Ryder Dedicated Logistics, Miami, Florida 1997-1998 Analyst, Istanbul Gold Exchange, Istanbul, Turkey 1999-2001 Business Development Manager, Federal Express, Istanbul, Turkey 2001-2001 Project Manager, Ericsson Telecommunications, Istanbul, Turkey viii FIELDS OF STUDY Major Field: Business Administration Areas of Specialization: Logistics and Marketing ix TABLE OF CONTENTS ABSTRACT ii ACKNOWLEDGMENTS iv VITA vii LIST OF TABLES xiii LIST OF FIGURES xviii CHAPTER 1 INTRODUCTION 1 1.1 Background 1 1.1.1 Theoretical Background 4 1.1.2 Business Background 7 1.2 Research Design 9 1.2.1 Research Purpose 10 1.2.2 Model Building and Hypotheses 11 1.2.3. Data Collection and Analysis 15 1.3 Limitations 16 1.4 Potential Contributions 19 1.4.1 Managerial Contributions 19 1.4.2 Academic Contributions 20 1.5 Organization 21 LIST OF REFERENCES 23 CHAPTER 2 LITERATURE REVIEW 25 2.1 Prevalence of Judgmental Adjustments 26 2.2 Efficacy of Judgmental Adjustments 27 2.2.1 The Case Against Judgmental Adjustments 28 2.2.2 The Case for Judgmental Adjustments 33 2.2.3 Domain Knowledge and Judgmental Adjustments 36 2.3 Factors Affecting the Accuracy of Judgmental Adjustments 38 2.4 A Critique of Current Literature 51 [...]... researchers maintain that further research is needed to explore numerous aspects of judgmental adjustments of statistical forecasts A fundamental finding in previous studies is that human intervention in the form of judgmental adjustments can help improve accuracy of statistical forecasts in two ways First, it can detect pattern changes Sanders (1992) shows instances when judgmental adjustments can improve... improve accuracy and reduce biases by recognizing the patterns in data 4 and incorporating that information into the statistical forecast Second, it can incorporate expert knowledge about the data series when a statistical forecasting method ignores such information For example, an industry expert can add customer, product and market knowledge to a statistical sales forecast Sanders and Ritzman (1991) studied... trends Judgmental methods, on the other hand, make use of both quantitative and qualitative data even though they are inefficient in dealing with large data sets, and can be subjective and inconsistent Therefore, forecasters often face a choice between statistical and judgmental methods when implementing a forecasting system 2 Knowledge Source Judgmental Expert Opinions Delphi Method Conjoint Analysis... generalizable The purpose of this research is to address the previously mentioned shortcomings in the literature In other words, it is to deepen the understanding of judgmental adjustments by (1) proposing a working model for the process which generates judgmental adjustments to statistical forecasts, (2) exploring certain variables that affect the accuracy, learning and biases in judgmental adjustments Moreover,... common one in business As such, the success of many managerial decisions depends on the accuracy of forecasts that are judgmentally adjusted Furthermore, the significance of this practice has led to proliferation of academic studies on this subject Hence, a better understanding of judgmental adjustments is of interest to researchers and practitioners This research is an investigation of judgmental adjustments. .. “amplification of demand” (Forrester 1961) or the “bullwhip” effect (Lee, Padmanabhan and Wang 1997) The need for increased forecast accuracy has fostered interest in better forecasting methods both in academia and in industry There is a significant body of literature that proposes many alternative forecasting methods However, all of these methods can be grouped in one of two categories: Statistical or judgmental. .. the understanding of the internal functioning of the judgmental adjustment process and provide important input in devising operating policies geared towards improving the accuracy of these adjustments 1.2.2 Model Building and Hypotheses The first step involved proposing a working model for the process which generates judgmental adjustments This model provided the groundwork for identifying the major... financial well-being and competitive position of a firm As such, improving the accuracy of forecasts is an important goal for a company Second, the effects of poor forecasting transcend the boundaries of a single 1 firm and reach other members of the supply chain Forecast errors of a firm cause larger fluctuations in demand for the suppliers in the upstream tiers of the supply chain This phenomenon... working model is provided in Figure 4 The inputs to the working model are statistical forecast and contextual information When the store manager generates a judgmental adjustment, he or she also takes into account relevant previous experiences After the judgmental adjustment, actual sales are observed and forecasting performance can be determined in terms of accuracy improvement, learning and biases. .. 63: Motivational stimuli and optimism, conservatism and overreaction biases 242 xv Table 64: Effects of motivational variables on accuracy improvement and biases 243 Table 65: Directional bias and its effects on accuracy and other biases 244 Table 66: Information sharing and accuracy 246 Table 67: Information sharing and biases 246 Table 68: Feedback and accuracy improvement . AN INVESTIGATION OF ACCURACY, LEARNING AND BIASES IN JUDGMENTAL ADJUSTMENTS OF STATISTICAL FORECASTS DISSERTATION Presented in Partial Fulfillment of the Requirements. of information and presentation of adjustments 251 Table 76: Presentation of information, accuracy improvement and biases 251 Table 77: Presentation of adjustments, accuracy improvement and. Judgmentally adjusting a statistical forecast before using it is a widespread practice in business. The goal of this study is to provide a deeper understanding of judgmental adjustments of