ACCC Working Paper no 9 - Estimating the Market Risk Premium in Regulatory Decisions

Estimating the Market Risk Premium in Regulatory Decisions: Conditional versus Unconditional Estimates Peter Gibbard Working Paper no 9, September 2013 ACCC/AER WORKING PAPER SERIES © Commonwealth of Australia 2013 ISBN 978-1-921973-87-1 This work is copyright Apart from any use permitted by the Copyright Act 1968, no part may be reproduced without permission of the Australian Competition and Consumer Commission Requests and enquiries concerning reproduction and rights should be addressed to the Director of Publishing, Australian Competition and Consumer Commission, GPO Box 3131, Canberra ACT 2601 Citation details: ACCC/AER Working Paper, September 2013 Series Note The Australian Competition and Consumer Commission (ACCC) is an independent statutory authority formed in 1995 to administer the Competition and Consumer Act 2010 and a range of additional legislation, promoting competition, fair trading and regulates national infrastructure for the benefit of all Australians The Australian Energy Regulator (AER) is a constituent part of the ACCC with an independent board Working papers are intended to disseminate the results of current research by ACCC staff and consultants The aim of the series is to facilitate discussion and comment The working papers come from across the ACCC covering competition law, economic regulation, and consumer protection They are available at no cost in electronic format only from the ACCC website, www.accc.gov.au To subscribe to this series, email working.papers@accc.gov.au The papers in this series reflect the views of the individual authors The views expressed in the paper not necessarily reflect the views of the ACCC or the AER Enquiries may be addressed to: The Editor ACCC/AER working paper series Australian Competition and Consumer Commission GPO Box 520 Melbourne Vic 3001 Email: workingpapers@accc.gov.au About the author Peter Gibbard currently holds the position of Principal Economic Adviser with the ACCC/AER He joined the ACCC/AER in 2010, and works for the Regulatory Development Branch, providing economic consultancy advice across the ACCC and AER Peter has a Bachelor of Economics (First Class Honours) and a Bachelor of Laws from the University of Adelaide, an M.Phil in Economics from Oxford University and a PhD in Philosophy, focusing on mathematical logic, from the University of Michigan Prior to joining the ACCC, he worked in the Financial Stability Division of the Bank of England in London Contents Estimating the Market Risk Premium in Regulatory Decisions: Conditional versus Unconditional Estimates Series Note Contents Introduction Predictability versus unpredictability: three phases of research 2.1 The first phase: the unpredictability of returns and random walks .9 2.2 The second phase: from unpredictability to predictability 10 2.3 The third phase: a renewed ‘healthy skepticism’ about predictability 12 The effect on the MRP of the dividend yield, risk-free rate and volatility 16 3.1 Dividend yields and the MRP .16 3.1.1 Theory 16 3.1.2 Evidence 17 3.2 The risk-free rate and the MRP 19 3.2.1 Theory 19 3.2.2 Evidence 20 3.3 Volatility and the MRP 21 3.3.1 Theory 21 3.3.2 Evidence 22 Reasons for regulators to have practical concerns about conditional estimates .25 4.1 The diversity and complexity of recent models of return predictability 25 4.2 Instability in models of return predictability .27 4.3 Data mining 28 Conclusion 31 REFERENCES 32 Introduction Regulators have discussed at least four different kinds of methodologies for estimating the market risk premium (MRP) for the purpose of determining regulatory prices First, MRP estimates can be informed by survey evidence, drawing on surveys of corporate executives, academics, auditors and accountants Second, the MRP can be calculated using dividend growth models Third, estimates of the MRP can be obtained from historical averages of annual excess returns (equity returns less the risk-free rate) Fourth, estimates of the MRP may be specified to be conditional on currently available information – that is, they may be specified to be a function of information such as market volatility, dividend yields and the risk-free rate This paper compares the third and fourth methods, identifying the key issues in the debate between conditional estimates of the MRP and historical, unconditional estimates This debate is closely tied to the debate about the predictability of excess returns, on account of the relationship between expected returns and the required rate of return Accordingly, in the annual report on investment returns by Dimson, Marsh and Staunton, Credit Suisse Global Investment Returns Sourcebook 2012, when the authors evaluate the question of whether estimates of the MRP should be conditional, they discuss the debate about the predictability of returns In particular, they observe that the debate about predictability is ‘far from settled’: Yet despite extensive research, this debate [about predictability] is far from settled In a special issue of the Review of Financial Studies, leading scholars expressed opposing views, with Cochrane (2008) and Campbell and Thompson (2008) arguing for predictability, whereas Goyal and Welch (2008) find that ‘these models would not have helped an investor with access only to available information to profitably time the market’ Cochrane’s (2011) recent Presidential Address demonstrates the persistence of this controversy (Dimson et al., 2012, p 36) In their contribution to the debate, Welch and Goyal (2008) argue that, in forecasting excess returns, investors cannot better than use a historical average The implication that can be drawn from their study is that estimates of the MRP should not be conditioned upon current information but, instead, a historical average should be used Dimson et al (2012, p 37) themselves conclude that, for ‘practical purposes’, it is ‘hard’ for predictors of equity premia to outperform a long-term historical average: In summary, there are good reasons to expect the equity premium to vary over time Market volatility clearly fluctuates, and investors’ risk aversion also varies over time However these effects are likely to be brief Sharply lower (or higher) If markets are in equilibrium and efficient, expected returns are equal to the required rate of return Thus if, in addition, returns are predictable on the basis of current information, then (given that expectations are rational) not only expected returns but also the required rate of return is dependent on current information stock prices may have an impact on immediate returns, but the effect on longterm performance will be diluted Moreover volatility does not usually stay at abnormally high levels for long, and investor sentiment is also mean reverting For practical purposes, we conclude that for forecasting the long run equity premium, it is hard to improve on extrapolation from the longest history that is available at the time the forecast is being made When they refer to ‘the long run equity premium’, they have in mind forecast horizons of about five years.2 While Welch and Goyal (2008) is representative of an important recent strand of the research literature, the debate on predictability, as Dimson et al (2012) observe, is ‘far from settled’, and John Cochrane’s influential defense of predictability, in particular, ‘demonstrates the persistence of this controversy’ There is an extensive and complex literature on the predictability of equity returns; and this working paper attempts to summarise the literature by identifying key phases in research on predictability since the 1960s Three distinct phases of the literature are identified, and are discussed in Section of the paper The transition from the first to the second phase was highlighted by Cochrane in 2001 in the first edition of his book Asset Pricing: Cochrane proposed that whereas the first generation of research on asset pricing had emphasised the unpredictability of returns, a ‘new generation’ of research instead supported the view that returns are predictable The transition from the second to a third phase of research is noted by Ang and Bekaert (2007, p 653), who suggest that ‘the literature is converging to a new consensus, substantially different from the old view’ This third phase of research called for a renewed scepticism about the predictability of returns, especially in the medium- and long-run In the debate between the second and third phases, a critical event is the 2008 issue of Review of Financial Studies that is described in the quotation above from Dimson et al (2012) The contribution to the debate by Welch and Goyal (2008) was especially influential in challenging the claims for predictability Whereas the second phase of research provides a basis for conditional estimates of the MRP, the third phase of research questions whether the MRP can be estimated conditionally – that is, it questions whether there are better estimates of the MRP than the unconditional historical average This third phase of research addresses a concern of the regulated businesses that an MRP estimate based on a historical average is ‘backward-looking’ The regulated businesses have questioned whether the use of a historical average of excess returns is consistent with the Capital Asset Pricing Model (CAPM): the concern is that a historical average is backwardlooking, whereas the CAPM is forward-looking But if the study of Welch and Goyal (2008) is accepted, then the historical average can be construed as a forward-looking measure: it is forward-looking because it can be construed as a good predictor of future excess returns, and because it is not clear that there are better predictors While Section of the paper identifies the key phases in the general debate about predictability, Section examines specific debates about which explanatory variables can be Dimson et al (2012, pp 36-37) present evidence in support of their conclusion which relates to forecast horizons of five or fewer years used to predict excess returns It focuses on the debates about three predictor variables – dividend yields, interest rates and volatility Whereas Sections and are concerned primarily with the academic literature on predictability, Section has a more practical focus: it identifies problems that may arise in practice if a regulator attempts to estimate a conditional MRP Even if it were conceded that excess returns are predictable from some given set of variables, a regulator faces at least three practical problems with using that set of variables to estimate a conditional MRP (1) In response to scepticism about predictability in the third phase of research, the recent literature has investigated a range of models of returns that is increasingly (i) diverse, and (ii) complex If a regulator were considering conditional models of the MRP, it would be difficult for the regulator to select and implement such a model not only because of the diversity of touted models but also because of their increasing complexity (2) The third-phase of research has particularly emphasised concerns about the stability of models of excess returns A number of studies have found that the values of the parameters in the models of returns tend to change over time If, in fact, the relationship between excess returns and a variable changes over time, it is unclear how the regulator can set the MRP as a function of that variable (3) Apparently significant relationships between variables and excess returns may reflect data-mining The conclusion, therefore, is that the debate among researchers on predictability is, as Dimson et al (2012, p 36) put it, ‘far from settled’: whereas the second phase of research might be used to support the case for a conditional estimate of the MRP, the third phase of research might be used in support of an unconditional estimate Nevertheless, there are at least three reasons why in practice regulators may have grounds for using an unconditional rather than a conditional estimate of the MRP Predictability versus unpredictability: three phases of research 2.1 The first phase: the unpredictability of returns and random walks In his evaluation of debates about the predictability of returns over time, Cochrane (2005) distinguishes between two different phases of research on asset pricing The ‘first revolution in finance’ (which, he says, peaked ‘in the early 1970s’) emphasised the ‘near unpredictability of stock returns’, whereas ‘a new generation of empirical research’ has tended to find that stock returns are predictable at least ‘over the business cycle and longer horizons’ (Cochrane, 2005, p 389-90) While Cochrane aligns his own position with the second phase of research, he provides a helpful summary of some of the key propositions that characterise the first phase: Stock returns are close to unpredictable Prices are close to random walks; expected returns not vary greatly through time… Any apparent predictability is either a statistical artifact which will quickly vanish out of sample, or cannot be exploited after transaction costs (Cochrane, 2005, p 389) In general, during this early phase of research, the unpredictability of stock returns was seen as a consequence of efficient – or at least near-efficient – markets Fama (1970, p 383) provides a helpful definition of an efficient market: ‘A market in which prices always “fully reflect” available information is called “efficient”’ (Fama (1970), p 383) Why does market efficiency make it difficult to predict returns? In his article ‘Proof that Properly Anticipated Prices Fluctuate Randomly’, Paul Samuelson presents a theoretical account of conditions under which returns will be unpredictable The intuition for this result is encapsulated in the following remark: ‘If one could be sure that a price will rise, it would have already risen’ (Samuelson 1965, p 41) Cornell (1999, p.2) provides an example that illustrates this intuition: Suppose, for example, that someone were to write a convincing book entitled The Crash of 2000, explaining why the new millennium will be accompanied by a dramatic drop in share prices If the arguments were truly convincing, then investors who read the book would clearly want to sell their stock before the dawn of the millennium Assuming that enough investors read the book and acted in accordance with its predictions, stock markets would not fall at the start of the millennium but at the time the book was widely distributed – but this would mean the predictions of the book are false As noted above, the first edition of Cochrane’s book appeared in 2001 The argument, in brief, is as follows: if markets are efficient – so that the price falls when the information in the book becomes public – then the supposition that prices are predictable implies a contradiction In the first phase of research, such theoretical arguments for unpredictability found support in empirical studies of stock markets Fama (1991, p 1578) draws attention to empirical studies of short-run correlations – correlations between daily, weekly and monthly returns While such correlations tended to be positive, researchers concluded that there were not good statistical grounds for rejecting the assumption of constant expected returns Fama (1991, p 1578) summarises the empirical findings of the first phase of research: The evidence for predictability in the early work often lacks statistical power, however, and the portion of the variance of returns explained by the variation in expected returns is so small…that the hypothesis of market efficiency and constant expected returns is typically accepted as a good working model 2.2 The second phase: from unpredictability to predictability In the late 1980s, however, a second body of research began to accumulate, reacting against the first phase In his survey article on market efficiency, Fama (1991, p 1609) provides a summary of this ‘new evidence’ on predictability: The recent evidence on the predictability of returns from other variables seems to give a more reliable picture of the variation through time of expected returns….In contrast to the autocorrelation tests on long-horizon returns, the forecast power of D/P, E/P, and the term-structure variables is reliable for periods after the Great Depression D/P, E/P, and the default spread track autocorrelated variation in expected returns that becomes a larger fraction of the variance in returns for longer return horizons These variables typically account for less than 5% of the variance of monthly returns but around 25-30% of the variances of 2- to 5-year returns In short, the recent work suggests that expected returns take large, slowly decaying swings away from their unconditional means Like Fama, Cochrane (2005, p 390) emphasises the contrast between short- and long-horizon predictability He outlines the findings of this ‘new generation’ of empirical evidence as follows: Variables including the dividend/price ratio and term premium can in fact predict substantial amounts of stock return variation This phenomenon occurs over the business cycle and longer horizons Daily, weekly, and monthly stock returns are still close to unpredictable Whereas the second phase of research acknowledges the findings of the earlier phase – that returns are ‘close to unpredictable’ over shorter horizons – the new claim is that returns are predictable over ‘longer horizons’ 10 mention that this correlation is only significant over horizons significantly shorter than the regulatory period, so this finding is of limited relevance for the purpose of estimating the regulatory cost of capital In his recent paper, Zhu (2013, pp 209-11) finds that the risk-free rate is even a poor predictor of excess returns over short horizons As discussed in Section 2.3, when Zhu applies his jackknife technique to correct for short-term bias, the evidence of the predictive power of the risk-free rate ‘vanishes completely’ 3.3 Volatility and the MRP 3.3.1 Theory The theoretical relationship between the MRP and volatility can be derived from the seminal paper of Merton (1973), in which he presents his ‘intertemporal CAPM’ (ICAPM) model In Merton’s model, the MRP changes over time Armitage (2005, p 82) provides a good introduction to the ICAPM, and he offers the following helpful characterisation of the model: the ICAPM ‘is a multifactor model in which an asset’s risk premium is determined by its sensitivity to state variables’ While Merton (1973) is frequently cited in discussions of the relationship between the MRP and volatility, Merton does not specifically explore this relationship in his 1973 paper Rather, this relationship is a focus of Merton (1980), which draws upon, and examines special cases of, his earlier 1973 paper Under certain assumptions, Merton asserts, the MRP at a given time is a positive linear function of the conditional volatility at that time The coefficient θ depends, in turn, upon investors’ coefficients of relative risk aversion.12 On the one hand, a positive relationship between the MRP and volatility might seem to be intuitively plausible After all, the MRP rewards investors for bearing risk, so if the risk increases it might be expected that the premium increases On the other hand, the existence of such a relationship is not uniformly supported by the data (see section 3.3.2 below) In response, several authors have suggested that the positive relationship between the MRP and volatility is not as intuitively and theoretically plausible as it initially seems to be For instance, Cornell (1999, p 51) asserts that: It turns out that the economic intuition that periods of high price variability should also be characterized by high stock-market returns is false 12 See Merton (1980, pp 329-330) for this result For examples of studies which rely on Merton (1973, 1980) to derive a relationship between expected returns and conditional volatility, see Poterba and Summers (1986), p 1143; Harvey (1989), p 291; Dean and Faff (2001), pp 172-73; Glosten, Jagannathan and Runkle (1993), pp.1781-82; Cuthbertson and Nitzsche (2004), p 658 21 Cornell suggests, furthermore, that the ambiguity of the relationship between returns and volatility ‘is not so surprising’ in the light of the following observation of Glosten et al (1993, pp 1779-80) At first blush, it may appear that the rational risk-averse investors would require a relatively larger risk premium during times when the payoff of the security is more risky A larger risk premium may not be required, however, because time periods which are relatively more risky could coincide with times when investors are better able to bear particular types of risk Further, a larger risk premium may not be required because investors may want to save relatively more during periods when the future is more risky If all the productive assets available for transferring income to the future carry risk and no risk-free investment opportunities are available, then the price of the risky asset may be bid up considerably, thereby reducing the risk premium Hence a positive as well as a negative sign for the covariance between the conditional mean and the conditional variance of the excess return on stocks would be consistent with theory Despite the intuitive plausibility of the positive relationship between volatility and the MRP, and despite the influence of Merton’s analysis, the relationship should be regarded as theoretically ambiguous As Glosten et al (1993) argue, theory alone is not sufficient to establish a positive relationship between the MRP and volatility The question about the relationship between the MRP and volatility must be resolved by empirical evidence and not by theory 3.3.2 Evidence As well as pointing out the ambiguity in the theoretical relation, Cornell (1999, p 51) also makes it clear that the empirical evidence on this relationship is decidedly mixed Some econometric studies find the relationship to be insignificant; and those which identify a significant relationship disagree over its sign: The literature on the relation between stock returns and the variability of returns includes contributions by Black (1976); Merton (1980); French, Schwert, and Stambaugh (1987); Poterba and Summers (1988); Breen, Glosten and Jagannathan (1989); Turner, Startz, and Nelson (1989); Nelson (1991); Campbell and Hetschel (1992); and Glosten, Jagannathan and Runckle (1993) All of these articles document the fact that the variability of returns changes over time Unfortunately, their authors disagree as to how the changing variability is related to the risk premium Some present findings that indicate a positive relation, others present findings that indicate a negative relation, and still others find no significant relation at all If there is a bottom line, it is that the relation between stock returns and the variability of returns is remarkably weak…Whatever the explanation for the weak relation between the ex-post risk premium and the variability of returns, it means that return variability is not a good variable for modeling possible changes in the risk premium (underlining added) 22 A number of other researchers echo Cornell’s conclusion that the empirical literature on the relationship between volatility and the equity premium is inconclusive Thus Scruggs (1998, p 575) observes that It is surprising, however, that the empirical nature of this important relationship [between the market risk premium and conditional market variance] has not been resolved Theory generally predicts a positive relation between the market risk premium and conditional market variance if investors are risk averse Yet, empirical studies to date fail to agree on the sign of this important relation Scruggs provides a helpful summary of the findings in the empirical literature The following table is abstracted from Scruggs (1998, p 577), and it shows the divergent empirical findings on the relationship between the risk premium and variance Table 1: Survey of Empirical Research on the Relation between the Risk Premium and Volatility13 Paper French, Schwert and Stambaugh (1987) Campbell (1987) Harvey (1989) Turner, Startz and Nelson (1989) Baillie and DeGennaro (1990) Glosten, Jagannathan and Runkle (1993) Empirical relation between risk premium and market variance Insignificant positive Insignificant positive Insignificant positive Significant negative Significant positive Significant negative Significant positive Significant positive Significant positive Insignificant positive Insignificant positive Insignificant positive Insignificant positive Significant negative In another summary of the empirical literature, Whitelaw (1994, p 515-516) also emphasises the variety of divergent findings about the relationship between the MRP and volatility: On a market-wide level, strong intuition suggests that risk and returns should be positively related Consequently, researchers have searched for both a positive relation between expected returns and the conditional volatility of returns…Yet, prior empirical investigations into the contemporaneous correlation between the first two moments of stock market returns yield decidedly mixed results 13 Some papers provide multiple estimates because they may use multiple proxies, data sets, models of variance, estimation methodologies or specifications of the risk-return relation 23 Dean and Faff (2001, p 169-170) provide a similar characterisation of the empirical literature on the relationship between expected returns and volatility: Although financial theory predicts that there should be a positive relationship between expected return and its variance, researchers cannot find consensus even on estimates of the sign of the relationship, let alone predict its magnitude More recent authors continue to emphasise the inconclusive character of the empirical literature on the relationship between volatility and expected returns For instance, Cornell’s conclusion that ‘the relation between stock returns and the variability of returns is remarkably weak’ is cited with approval in Armitage (2005, p 88) Bollerslev et al (2009, p 4465) provide a similar description of the empirical literature: The classical intertemporal CAPM model of Merton (1973) is often used to motivate the existence of a traditional risk-return tradeoff in aggregate market returns Despite an extensive empirical literature devoted to the estimation of such a premium, the search for a significant time-invariant expected returnvolatility tradeoff type relationship has largely proved elusive When Bollerslev et al (2009) talk of ‘a significant time-invariant expected return-volatility tradeoff’ they mean a significant time-invariant positive relationship between expected returns and volatility On the basis of these observations of Cornell (1999), Scruggs (1998), Whitelaw (1994), Dean and Faff (2001), Armitage (2005) and Bollerslev et al (2009), it can be concluded that there is no consensus in the empirical literature that there is a robust positive relationship between the MRP and volatility It should be noted that in response to this conclusion, at least some researchers have attempted to devise more complex models of expected returns, in which various measures of volatility are used to model returns: see, for example, Guo and Savickas (2006) and Bollerslev et al (2009) These models represent a considerable departure, however, from the simple relationship between volatility and the MRP that is typically presented in submissions by the regulated businesses As will be discussed below in Section 4.1, given the diversity and complexity of the conditional models of the MRP in the current academic literature, it is unclear how regulators can (i) make an evidence-based choice of a particular conditional model of the MRP and (ii) implement such models 24 Reasons for regulators to have practical concerns about conditional estimates Sections and above summarise the debate about the predictability of excess returns But even if it were conceded that excess returns are to some degree predictable, it would not follow that regulators should use a conditional estimate of the MRP There are two kinds of reasons why, despite some evidence of predictability, regulators might have reason to avoid conditional models of the MRP First, if markets are not perfectly efficient, return predictability does not imply a conditional MRP 14 Second, there are reasons why, in practice, regulators may face difficulties in applying the findings of economic research to their regulatory decision making It is the second kind of reason that is the focus of this section Sections 4.1 to 4.3 below specify three reasons why conditional estimates may potentially be especially problematical for regulators 4.1 The diversity and complexity of recent models of return predictability The current research literature on return predictability is characterised by a diversity of distinct and complex models of excess returns The third phase of the research literature has re-evaluated the claims of the second phase of research on predictability For some researchers, such as Welch and Goyal (2008), this reevaluation has taken the form of a broad scepticism about claims of return predictability For 14 This observation is a commonplace of the academic literature, and is articulated by Peseran and Timmermann (1995, pp 1201-2) in the passage below They note that there is an interpretation of return predictability – their ‘second interpretation’ – according to which markets are inefficient, and the MRP is constant despite the predictability of excess returns: Many recent studies conclude that stock returns can be predicted by means of publicly available information…However, the economic interpretation of these results is controversial and far from evident First, it is possible that the predictable components in stock returns reflect time-varying expected returns, in which case predictability is, in principle, consistent with an efficient stock market A second interpretation takes expected returns as roughly constant and regards predictability of stock returns as evidence of stock market inefficiency…Inevitably, all theoretical attempts at interpretation of excess return predictability will be model-dependent, and hence inconclusive (see Fama (1991)) Peseran and Timmermann’s quotation refers to Fama (1991, p 1577), which makes the same point: In brief, the new work says that returns are predictable from past returns, dividend yields, and the various term-structure variables…This means, however, that the new results run head-on into the joint-hypothesis problem: Does return predictability reflect rational variations through time in expected returns, irrational deviations of price from fundamental value, or some combination of the two? 25 other researchers, however, this re-evaluation has not taken the form of a denial of predictability, but rather the investigation of a range of novel – and generally more complex – models of predictability The following sample of six recent studies of predictability illustrates (i) the influence of phrase-three research on recent academic literature and (ii) the complexity and the diversity of the models formulated as a response to phase-three research Rapach et al (2010) open by recognising the current scepticism about return predictability, citing phrase-three research, including Bossaerts and Hillion (1999), Goyal and Welch (2003) and Welch and Goyal (2008) Their response to the problems identified by phase-three research is to move away from ‘individual forecasts’ – forecasts based on a single observed variable Instead, they use fifteen different variables to estimate fifteen individual forecasts They recommend using a forecast based on a combination of these fifteen individual forecasts Timmermann (2008) similarly motivates his paper by pointing to the scepticism about predictability articulated in Bossaerts and Hillion (1999), Goyal and Welch (2003) and Pesaran and Timmermann (1995) He suggests that nevertheless there may be ‘local predictability: ‘most of the time stock returns are not predictable, but there appear to be pockets in time where there is modest evidence of local predictability’ (Timmermann, 2008, p 17) If an econometrician runs a range of models, he or she may be able to make local predictions if it is possible to measure which models are working well at different points of time: the econometrician must obtain ‘some indication of when different models produce valuable forecasts and when they fail to so e.g in the form of a real-time monitoring system tracking how reliable the forecasts have been over the recent time’ (Timmermann, 2008, pp 16-17) Cooper and Priestley (2009) also begin by acknowledging the skepticism about return predictability that arises from phase-three research, citing the papers by Bossaerts and Hillion (1999) and Goyal and Welch (2003) They respond by suggesting a novel variable for predicting excess returns − the output gap (the deviation of industrial production from trend) Like the previous three studies, Pettenuzzo et al (2012) opens by pointing to the phasethree research of Bossaerts, Hillion, Goyal and Welch They suggest that the performance of forecasts can be improved by introducing constraints on the forecasting models – in particular, the conditional mean of the equity premium is constrained to be non-negative, and the conditional Sharpe ratio is constrained to lie within certain bounds Bollerslev et al (2009) examine the relationship between volatility and the risk premium They open their discussion by pointing to the failure of the empirical literature to establish such a relation, concluding that ‘the search for a significant time-invariant expected returnvolatility tradeoff type relationship has largely proved elusive’ (Bollerslev et al 2009, p 4465) This encourages the authors to investigate a more complex relationship between expected returns and volatility Rather than explaining returns by using a straightforward measure of volatility, they focus on the difference between two measures of volatility: their 26 central conclusion is that ‘the difference between the “model free” implied and realized variances is able to explain a nontrivial fraction of the variation in quarterly stock returns over the 1990-2007 sample period’ (Bollerslev et al 2009, p 4465) Like Bollerslev et al (2009), Guo and Savickas (2006, p 43) begin by pointing to empirical research which ‘failed to uncover a positive risk-return relation in the stock market across time’ Their response is to model expected returns not as a function of a single measure of volatility, but rather as a function of two distinct measures of volatility, idiosyncratic volatility and stock market volatility They find that whereas these two measures of volatility ‘individually have negligible forecasting power in the in-sample regression, they jointly provide a significant predictor of excess stock market returns’ (Guo and Savickas, 2006, p 43) These six papers all were published in or after 2006, and provide a representative sample of recent attempts to model excess returns Responding to scepticism about predictability in the phase-three literature, these papers offer novel – and frequently more complex – model specifications As a result of the phase-three literature, there is a considerable range of novel and complex models of excess returns in the academic literature In this literature, there is no consensus – or anything approaching a consensus – on the appropriate set of methodologies for modeling future excess returns Thus if a regulator were considering providing a timevarying, conditional model of the MRP, it is unclear how the regulator would make an evidence-based selection on the basis of this literature Given the diversity and complexity of forecasting models, it would be difficult for the regulator not only to select but also to implement a model of the changes in the MRP over time 4.2 Instability in models of return predictability A number of studies have found instability in models of return predictability – that is, the models tend to change over time A parameter in a model is said to be unstable if it changes over time The third phase of research on return predictability has emphasized the instability of models of excess returns This instability provides a reason for a regulator to avoid conditional models of the MRP – to avoid setting the MRP as a function of some variable If that function is unstable over time, it is difficult, if not impossible, for the regulator to measure accurately how the MRP should be adjusted in response to changes in the variable This problem was alluded to in Section 2.3 (in the discussion of Welch and Goyal (2008)) and in section 4.1 (in the discussion of Timmerman (2008)), but it is a sufficiently important problem to warrant a more detailed discussion In Goyal and Welch (2003, p 653), the diagnosis for the poor out-of-sample forecasting performance of dividend yields is that the underlying relationships are unstable: The primary source of poor predictive ability is parameter instability The estimated dividend-price ratio autoregression coefficient has increased from about 0.4 in 1945 to about 0.9 in 2002 27 Bossaerts and Hillion (1999, p 407) similarly suggest that model instability accounts for the poor OOS predictability of excess returns: The poor external validity of the prediction models that formal model selection criteria chose indicates model nonstationarity: the parameters of the ‘best’ prediction model change over time The following passage provides more detail about the changes in the model of excess returns over time: The discrepancy between the regression results of Period I (entire sample) and Period II (1/70-8/80) indicates the presence of model nonstationarity The sign of the regression coefficients is almost always the same across the two periods, but the magnitude often differs dramatically (with Period II generating the highest values) Pesaran and Timmerman (1995) also document an increase in predictability of U S stock returns in the 1970s (Bossaerts and Hillion, 1999, p.412) Bossaerts and Hillion cite the results reported in Pesaran and Timmerman (1995, p 1225), which also emphasise the instability in models of excess returns: Also there does not seem to be a robust forecasting model in the sense that the determinants of the predictability of stock returns in the U.S seems to have undergone important changes throughout the period under consideration The timing of the episodes where many of the regressors get included in the forecasting model seems to be linked to macroeconomic events such as the oil price shock in 1974 and the Fed’s change in its operating procedures during the 1979 to 1982 period Even Lettau and Ludvigson (2001, p 844), a study cited by the regulated businesses in support of return predictability, emphasises the changes in the performance of such models over time: Although our findings of out-of-sample predictability are particularly strong relative to those of some other studies, we caution that our results not imply forecastability in all episodes It is clear, for example, that the last five years have been marked by highly unusual stock market behavior, as prices relative to any sensible divisor have reached unprecedented levels Model instability has been a focus of phase-three research over the past decade Two other recent papers that particularly emphasise the challenges for return predictability arising from model instability are Pesaran and Timmermann (2002) and Paye and Timmermann (2006) 4.3 Data mining Findings of predictability may reflect data mining rather than genuine relationships between variables and future excess returns Data mining is a particular problem for research on return predictability 28 What is data mining? Data mining (which is also referred to as ‘data dredging’ and ‘data snooping’) may be intentional or unintentional The following is an example of unintentional data mining Suppose that twenty different econometricians are attempting to ascertain the determinants of variable Y Suppose each econometrician examines Y’s relationship to a different variable: the first tests the relationship between Y and variable X1, the second the relationship between Y and variable X2, and so forth Suppose that, in fact, Y is not related to any of the twenty variables X1, X2…X20 Nevertheless, there is a good chance that at least one of the twenty econometric tests will be a ‘false positive’ – that is, even though there is no relationship between Y and the tested variable, the relationship will be found to be statistically significant at a per cent level Suppose, then, that one econometrician finds a significant relationship and the other nineteen not Suppose, moreover, that when making decisions about what to publish, academic journals look particularly favorably on articles that purport to discover new relationships of significance As a consequence, the econometrician who finds a significant relationship is the only one that publishes his results (i.e., the nineteen econometricians who did not find a significant relationship are unable to publish their results) Then the body of academic literature on the determinants of Y will be misleading In general, data mining refers to multiple uses of a given data set – that is, it refers to the ‘mining’ of a dataset Data mining may also be intentional Suppose that an econometrician is cynically attempting to establish a relationship between variables X and Y but, in fact, there is no relationship between the two variables Suppose that the econometrician attempts to estimate the relationship by testing a large range of different model specifications (e.g different start dates, end dates, frequencies, proxies, specifications of outliers, functional forms, selections of other variables etc.) until he or she obtains a statistically significant relationship between X and Y This exemplifies intentional data mining Data mining undermines findings of statistical significance Suppose, for example, that when a dependent variable Y is regressed on X, the relationship between the two variables is found to be ‘statistically significant at a per cent level’ The standard interpretation of this claim is that the probability that there is no relationship between the two variables is no more than per cent Suppose, however, that the econometrician obtains this finding only after regressing Y on twenty different variables, but only reports the model found to be ‘statistically significant’ Then the finding of ‘statistical significance’ is undermined: it cannot be maintained that the probability that there is no relationship between the two variables is no more than per cent Verbeek (2008, pp 58-59) puts this point as follows: In general, data snooping refers to the fact that a given set of data is used more than once to choose a model specification and to test hypotheses You can imagine, for example, that, if you have a set of 20 potential regressors and you try each one of them, it is quite likely to conclude that one of them is significant, even though there is no true relationship between any of these regressors and the variable you are explaining 29 Researchers in financial economics have long recognised that data mining is a particular problem for studies on return predictability Thus in his article on market efficiency, Fama (1991, p 1577) observes that: We should also acknowledge that the apparent predictability of returns may be spurious, the result of data-dredging and chance sample-specific conditions He goes on to explain why data mining may be especially problematical in empirical research on return predictability: Inference is also clouded by an industry-level data-dredging problem With many clever researchers, on both sides of the efficiency fence, rummaging for forecasting variables, we are sure to find instances of “reliable” return predictability that are in fact spurious (Fama, 1991, p 1585) In an article entitled ‘Data-Snooping Biases in Tests of Financial Asset Pricing Models’, Lo and MacKinlay (1990, p 432) make a similar point Data mining is a particular problem for research on return predictability because – at least in part – there are a large ‘number of published studies performed on [a] single data set’: We can expect the degree of such biases to increase with the number of published studies performed on any single data set – the more scrutiny a collection of data is subject to, the more likely will interesting (spurious) patterns emerge Since stock market prices are perhaps the most studied economic quantities to date, tests of financial asset pricing models seem especially susceptible Their article argues that data mining renders ‘standard tests of significance’ invalid (p 434) Sullivan, Timmermann and White (1999) investigate the potential for data mining to produce spurious trading rules – rules that appear to identify significant predictors of future excess returns, even though, in fact, they lack ‘predictive power’: If enough trading rules are considered over time, some rules are bound by pure luck, even in a very large sample, to produce superior performance even if they not genuinely possess predictive power over asset returns (Sullivan, Timmermann and White, 1999, p 1649) The potential for data mining creates a challenge for the regulator when it attempts to evaluate econometric studies that purport to provide a basis for a conditional MRP It may be difficult, therefore, for the regulator to make an evidence-based selection of a conditional MRP model 30 Conclusion This paper explores methods for estimating the MRP for the purpose of regulatory pricing, focusing, in particular, on a comparison between conditional estimates and estimates based on historical averages It points out the connection of this question to the debate about the predictability of excess returns, and surveys three phases of the literature on the predictability debate The paper shows that the third phase might be used to support a historical estimate of the MRP According to the third-phase of research, when forecasting excess returns, it is difficult to better than a historical average; therefore the historical average can be construed as a good forward-looking estimate of the MRP The debate among researchers on predictability is, as Dimson et al (2012, p 36) put it, ‘far from settled’ But, even if it were conceded that excess returns are, to some degree, predictable from a given set of variables, the regulator faces at least three practical problems with using that set of variables to estimate a conditional MRP (1) In response to skepticism about predictability in the third phase of research, the recent literature has developed a range of models that is increasingly (i) diverse, and (ii) complex If a regulator were considering conditional models of the MRP, it would be difficult for the regulator to make an evidence-based selection of the appropriate model not only because of the diversity of touted models but also because of their increasing complexity It would also be difficult to implement many of these models (2) The third phase of research has particularly emphasised concerns about the stability of models of excess returns A number of studies have found that the values of the parameters in the models of returns tend to change over time Given the high degree of instability in models of excess returns, it is unclear how the regulator can set the MRP as a function of some specific variable In particular, it is unclear how the regulator would ascertain how much the MRP should be adjusted in response to movements in that variable (3) Apparently significant relationships between variables and excess returns may reflect data mining 31 REFERENCES Ang, Andrew and Geert Bekaert (2007), ‘Stock Return Predictability: Is it There?’, Review of Financial Studies, 20(3), pp 651-707 Armitage, Seth (2005), The Cost of Capital: Intermediate Theory (Cambridge: Cambridge University Press) Baillie, Richard and Ramon DeGennaro (1990), ‘Stock Returns and Volatility’, Journal of Financial and Quantitative Analysis 25(2), pp 203-214 Bollerslev, Tim, George Tauchen and Hao Zhou (2009), ‘Expected Stock Returns and Variance Risk Premia’, Review of Financial Studies 22(11), pp 4463-4492 Bossaerts, Peter and Pierre Hillion (1999), ‘Implementing Statistical Criteria to Select Return Forecasting Models: What Do We Learn?’, Review of Financial Studies 12(2), pp 405-28 Boudoukh, Jacob, Matthew Richardson and Robert Whitelaw (2008), ‘The Myth of LongHorizon Predictability’, Review of Financial Studies, 21(4), July, pp 1577-1605 Breen, William, Lawrence Glosten and Ravi Jagannathan (1989), ‘Economic Significance of Predictable Variations in Stock Index Returns’, Journal of Finance, 44(5), pp 1177-1189 Brooks, Chris (2008), Introductory Econometrics for Finance, 2nd ed (Cambridge, Cambridge University Press) Campbell, John (1987), ‘Stock Returns and the Term Structure’, Journal of Financial Economics 18, pp 373-99 Campbell, John (1991), ‘A Variance Decomposition for Stock Returns’, Economic Journal, Vol 101, No 405, pp 157-179 Campbell, John and John Cochrane (1999), ‘By Force of Habit: A Consumption-Based Explanation of Aggregate Stock Market Behavior’, Journal of Political Economy, 107(2), pp 205-51 Campbell, John and Ludger Hentshel (1992), ‘An Asymmetric Model of Changing Volatility in Stock Returns’, Journal of Financial Economics, 31, pp 281-318 Campbell, John and Samuel Thompson (2008), ‘Predicting Excess Stock Returns out of Sample: Can Anything Beat the Historical Average?’, Review of Financial Studies, 21(4), July, pp 1509-31 Competition Economics Group (2012), Internal Consistency of Risk Free Rate and MRP in the CAPM, March 32 Cochrane, John (2005), Asset Pricing, 2nd ed (Princeton: Princeton University Press) Cochrane, John (2008), ‘The Dog that did not Bark: A Defence of Return Predictability’, Review of Financial Studies, 21(4), pp 1533-75 Cochrane, John (2011), ‘Discount Rates’, Journal of Finance, 66(4), pp 1047-1108 Cooper, Ilan and Richard Priestley (2009), ‘Time-Varying Risk Premiums and the Output Gap’, Review of Financial Studies, 22(7), pp 2801-2833 Cornell, Bradford (1999), The Equity Risk Premium: The Long-Run Future of the Stock Market (New York: Wiley) Cuthbertson, Keith and Dirk Nitzsche (2004), Quantitative Financial Economics: Stocks, Bonds and Foreign Exchange, 2nd ed (Chichester: John Wiley & Sons) Dean, Warren and Robert Faff (2001), ‘The Intertemporal Relationship between Market Return and Variance: An Australian Perspective’, Accounting and Finance, 41, pp 169-196 Dimson, Elroy, Paul Marsh and Mike Staunton (2012), Credit Suisse Global Investment Returns Sourcebook 2012 (Zurich: Credit Suisse Research Institute) Fama, Eugene (1970), ‘Efficient Capital Markets: A Review of Theory and Empirical Work’, Journal of Finance 25(2), pp 383-417 Fama, Eugene (1991), ‘Efficient Capital Markets: II’, Journal of Finance 46(5), pp 15751617 Fama, Eugene and Kenneth French (1988), ‘Dividend Yields and Expected Stock Returns’, Journal of Financial Economics, 22, pp 3-25 Fama, Eugene and Kenneth French (1989), ‘Business Conditions and Expected Returns on Stocks and Bonds’, Journal of Financial Economics, 25, pp 23-49 Ferson, Wayne, Sergei Sarkissian and Timothy Simin (2003) ‘Spurious Regressions in Financial Economics?’, Journal of Finance, 58(4), pp 1393-1413 French, Kenneth, William Schwert and Robert Stambaugh (1987), ‘Expected Stock Returns and Volatility’, Journal of Financial Economics 19, pp 3-29 Glosten, Lawrence, Ravi Jagannathan and David Runkle (1993), ‘On the Relation between the Expected Value and the Volatility of the Nominal Excess Return on Stocks’, Journal of Finance 48(5), pp 1779-1801 Goyal, Amit and Ivo Welch (2003), ‘Predicting the Equity Premium with Dividend Ratios’, Management Science, 49(5), pp 639-54 Guo, Hui and Robert Savickas (2006), ‘Idiosyncratic Volatility, Stock Market Volatility, and Expected Stock Returns’, Journal of Business and Economic Statistics, 24(1), pp 43-56 33 Harvey, Campbell (1989), ‘Time-Varying Conditional Variances in Tests of Asset Pricing Models’, Journal of Financial Economics, 24, pp 289-317 Lettau, Martin and Sydney Ludvigson (2001), ‘Consumption, Aggregate Wealth and Expected Stock Returns’, Journal of Finance, 56(3), pp 815-849 Lettau, Martin and Sydney Ludvigson (2005), ‘Expected Returns and Expected Dividend Growth’, Journal of Financial Economics, 76, pp 583-626 Lo, Andrew and A Craig MacKinlay (1990), ‘Data-Snooping Biases in Tests of Financial Asset Pricing Models’, Review of Financial Studies 3(3), pp 431-467 McKenzie, Michael and Graham Partington (2013), ‘Report to the AER: Review of the AER’s Overall Approach’, report dated 14 February, 2013 Merton, Robert (1973), ‘An Intertemporal Capital Asset Pricing Model’, Econometrica 41(5), pp 867-87 Merton, Robert (1980), ‘On Estimating the Expected Return on the Market’, Journal of Financial Economics, 8, pp 323-61 Nelson, Daniel (1991), ‘Conditional Heteroskedasticity in Asset Returns: A New Approach’, Econometrica 59(2), pp 347-370 Paye, Bradley and Allan Timmermann (2006), ‘Instability of Return Prediction Models’, Journal of Empirical Finance 13, pp 274-315 Peirson, Graham, Rob Brown, Steve Easton, Peter Howard, Sean Pinder (2006), Business Finance, 9th ed (North Ryde: McGraw-Hill) Pesaran, Hashem and Allan Timmermann (1995), ‘Predictability of Stock Returns: Robustness and Economic Significance’, Journal of Finance 50(4), pp 1201-28 Pesaran, Hashem and Allan Timmermann (2002), ‘Market Timing and Return Prediction under Model Instability’, Journal of Empirical Finance 9, pp 495-510 Pettenuzzo, David, Allan Timmermann and Rossen Valkanov (2012), ‘Forecasting Stock Returns Under Economic Constraints’, (7 December, 2012) Available at SSRN: http://ssrn.com/abstract=2186574 or http://dx.doi.org/10.2139/ssrn.2186574 Poterba, James and Lawrence Summers (1986), ‘The Persistence of Volatility and Stock Market Fluctuations’, American Economic Review, 76(6), pp 1142-1151 Poterba, James and Lawrence Summers (1988), ‘Mean Reversion in Stock Prices’, Journal of Financial Economics 22, pp 27-59 Rapach, David, Jack Strauss and Guofu Zhou (2010), ‘Out-of-sample Equity Premium Prediction: Combination Forecasts and Links to the Real Economy’, Review of Financial Studies 23(2), pp 821-862 34 Samuelson, Paul (1965), ‘Proof that Properly Anticipated Prices Fluctuate Randomly’, Industrial Management Review 6(2), pp 41-49 Scruggs, John (1998), ‘Resolving the Puzzling Intertemporal Relation between the Market Risk Premium and Conditional Market Variance: A Two Factor Approach’, Journal of Finance, 53(2), pp 575-603 Stambaugh, Robert (1999), ‘Predictive Regressions’, Journal of Financial Econometrics, 54, pp 375-421 Sullivan, Ryan, Allan Timmermann and Halbert White (1999), ‘Data-Snooping, Technical Trading Rule Performance, and the Bootstrap’, Journal of Finance 54(5), pp 1647-1691 Timmermann, Allan (2008), ‘Elusive Return Predictability’, International Journal of Forecasting, 24 (2008), pp 1-18 Torous, Walter, Rossen Valkanov and Shu Yan (2004), ‘On Predicting Stock Returns with Nearly Integrated Explanatory Variables’, Journal of Business, 77(4), pp 937-966 Turner, Christopher, Richard Startz and Charles Nelson (1989), ‘A Markov Model of Heteroskedasticity, Risk, and Learning in the Stock Market’, Journal of Financial Economics 25, pp 3-22 Verbeek, Marno (2008), A Guide to Modern Econometrics (Chichester: Wiley) Welch, Ivo and Amit Goyal (2008), ‘A Comprehensive Look at the Empirical Performance of Equity Premium Prediction’, Review of Financial Studies, 21(4), pp 1455-1508 Whitelaw, Robert (1994), ‘Time Variations and Covariations in the Expectation and Volatility of Stock Market Returns’, Journal of Finance 49(2), pp 515-541 Zhu, Min (2013), ‘Jackknife for Bias Reduction in Predictive Regressions’, Journal of Financial Econometrics, 11(1), pp 193-220 35 ... email working. papers @accc. gov.au The papers in this series reflect the views of the individual authors The views expressed in the paper not necessarily reflect the views of the ACCC or the AER... those in phase-two studies:6 We find very little evidence of predictability at horizons greater than year in the entire 192 6- 199 4 sample period or in either the pre- 195 2 or post- 195 2 subsamples In. .. than the regulatory period, so this finding is of limited relevance for the purpose of estimating the regulatory cost of capital In his recent paper, Zhu (2013, pp 2 0 9- 11) finds that the risk- free