Ex-post economic evaluation of competition policy enforcement: A review of the literature Competition EUROPEAN COMMISSION Directorate-General for Competition E-mail: comp-publications@ec.europa.eu European Commission B-1049 Brussels Ex-post economic evaluation of competition policy enforcement: A review of the literature Fabienne Ilzkovitz and Adriaan Dierx DG Competition June 2015 Europe Direct is a service to help you find answers to your questions about the European Union Freephone number (*): 00 800 10 11 (*) The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you) LEGAL NOTICE This Study was carried out in-house by DG Competition staff This final report is a DG COMP staff paper, setting out the findings of this study It does not represent the views of the European Commission Authors: Fabienne Ilzkovitz, Principal Advisor Adriaan Dierx, Principal Administrator Editing and assistance: Antonio Bosisio Karin Hunin More information on the European Union is available on the Internet (http://www.europa.eu) Luxembourg: Publications Office of the European Union, 2015 Catalogue number: KD-02-15-397-EN-N ISBN 978-92-79-48350-9 doi: 10.2763/83196 © European Union, 2015 Reproduction is authorised provided the source is acknowledged Ex-post economic evaluation of competition policy enforcement: A review of the literature Table of contents List of tables, graphs and boxes Executive Summary 10 Introduction 30 Part I: Evaluation: Definition, objectives and practice 31 Definition and scope 31 1.1 Definition 31 1.1.1 Difference between ex-ante and ex-post evaluation 31 1.1.2 Direct and indirect economic effects of competition policy .32 1.1.3 Distinction between the effects of competition and those of competition policy 33 1.2 Scope .33 Objectives 35 2.1 Improve the effectiveness of competition policy decisions and the enforcement practice of the CAs 36 2.2 Improve the quality of competition law 36 2.3 Set internal priorities 37 2.4 Defend legitimacy and improve advocacy .37 2.5 Other reasons 38 Practice of ex-post evaluations .39 3.1 Preparation of the ex-post evaluation project 39 3.1.1 Define the main objective and the subject of the evaluation 40 3.1.2 Select the methods to be used 41 3.1.3 Choose the evaluation team: using insiders, outsiders or both? 43 3.2 Execution of the ex-post evaluation project 44 3.2.1 Collect the data 44 3.2.2 Refine the methodologies and use methodologies understandable by policy-makers .44 3.3 Exploitation of the results of the ex-post evaluation projects .45 3.3.1 Check the robustness of the methodologies and the results 45 3.3.2 Draw policy conclusions 45 3.3.3 Consider disclosure of the results 45 Experience of other Competition Authorities 46 Part II: Microeconomic impact of competition policy 50 Methodologies 50 1.1 Overview of qualitative and quantitative methods .50 1.2 Qualitative methods 51 1.2.1 Court judgements .51 1.2.2 Surveys and peer reviews 51 1.3 Quantitative methods .51 1.3.1 Event studies .52 1.3.2 Estimation and simulation of structural models including a fully specified demand side 52 1.3.3 Reduced form estimation (including cross-sectional and panel data analyses) 53 1.3.4 Quasi-experimental methods (including Difference-in-Differences) 54 1.4 Mixed methods 55 1.4.1 Case studies 55 1.4.2 Market studies 55 1.4.3 Meta-retrospectives .55 1.5 A summary of the pros and cons of the different methods 55 1.6 Key methodological issues 57 1.6.1 Counterfactual 57 1.6.2 Selection bias 57 1.6.3 Deterrent effects .59 Microeconomic impact of competition policy 60 2.1 Merger control .60 2.1.1 Microeconomic analysis of merger decisions 60 (i) (ii) (iii) (iv) (v) (vi) Surveys and peer reviews 60 Event studies 61 Estimation and simulation of structural models 62 Reduced form estimation 63 Quasi-experimental methods 65 Overview 65 2.1.2 Evaluation of the impact of merger regulations and policies 67 (i) (ii) (iii) (iv) 2.2 Surveys and peer reviews 67 Event studies 68 Meta-retrospectives 69 Overview 70 Cartel and antitrust policy 72 2.2.1 Evaluation of the effectiveness of antitrust policy enforcement 72 (i) (ii) (iii) (iv) Descriptive analysis 72 Event studies 72 Surveys and peer reviews 72 Overview 73 2.2.2 Evaluation of the effectiveness of cartel policy enforcement 73 (i) (ii) (iii) 2.3 Meta-retrospectives 74 Deterrent effects of cartel policy enforcement 75 Overview 76 Areas for further research 78 Part III: Aggregate economic effects of competition policy 80 Benefits of competition policy for consumers .80 1.1 Methodology .80 1.2 Results 84 Macroeconomic impact of competition policy .86 2.1 Analytical framework .86 2.1.1 Effects of competition policy .87 2.1.2 Effects of competition 87 2.1.3 An integrated framework .88 2.2 Indicators 89 2.2.1 Indicators of the strength of competition policy 89 (i) (ii) (iii) Input indicators 89 Output indicators 94 Indicators combining input and output variables 96 2.2.2 A critical view of the indicators of the strength of competition policy 99 (i) (ii) (iii) Pros and cons of the different indicators 99 Correlation between different indicators .100 Determinants of the perceived effectiveness of competition policy enforcement 103 2.2.3 Indicators of the intensity of competition .106 2.3 Macro-econometric modelling of the effects of competition policy 108 2.3.1 Impact of competition policy on competition 109 2.3.2 Impact of competition policy on macroeconomic performances .110 2.4 Macro-econometric modelling of the effects of competition .115 2.4.1 The mark-up and business dynamism channels (allocative efficiency) 115 2.4.2 The management channel (productive efficiency) 119 2.4.3 The innovation and TFP channel (dynamic efficiency) .121 2.5 Areas for further research 125 References 128 Annexes 140 List of tables, graphs and boxes Tables Table I.1 Main differences between ex-ante and ex-post evaluations 31 Table I.2 Typology of ex-post economic evaluations of competition policy 35 Table I.3 A short checklist of key issues to be tackled in an evaluation project: 39 Table I.4 Subject of evaluation according to the main objective pursued by the CAs 40 Table I.5 Illustration: Recent and on-going ex-post evaluations in DG Competition 41 Table I.6 Methodologies for the ex-post economic evaluation of competition policy 42 Table I.7 Comparison of the ex-post evaluation activities in different CAs 47 Table II.1 Pros and cons of different methods for ex-post economic evaluation 56 Table II.2 Evaluation of merger decisions by competition authorities 66 Table II.3 Evaluation of the impact of merger policies and regulations 71 Table II.4 Evaluation of the impact of antitrust policies and regulations 73 Table II.5 Evaluation of the impact of cartel policy enforcement 77 Table III.1 Overview of assumptions used by CAs 81 a) Cartel cases 81 b) Abuse of dominance cases 82 c) Merger cases 82 Table III.2 Estimates of annual customer savings (expr in % of GDP x10-2) 85 Table III.3 Typology of indicators used to measure the strength of competition policy 89 Table III.4 Input indicators 90 Table III.5 Output indicators 94 Table III.6 Indicators combining input and output variables 97 Table III.7 Strengths and weaknesses of competition policy indicators 99 Table III.8 Correlation between the OECD, GCR and WEF indicators 101 Table III.9 Ranking of countries according to the four surveys 102 Table III.10 Selected studies on the determinants of the perceived effectiveness of competition policy 103 Table III.11 Measures of competition 106 Table III.12 Selected studies on the impact of the strength of competition policy on competition 109 Table III.13 Selected studies on the macroeconomic impact of competition policy 110 Table III.14 Selected studies on the mark-up and business dynamism channels 116 Table III.15 Selected studies on the management channel 119 Table III.16 Selected studies on the innovation channel 122 Table A.1 Performances of countries according to four surveys 140 Table A.2 Grouping of countries according to their performances 141 Table A.3 Correlation between the OECD, GCR and WEF indicators 141 Graphs Graph I.1 The policy evaluation cycle 32 Graph I.2 Economic effects of competition policy enforcement 33 Graph II.1 Difference-in-Differences method 54 Graph III.1 Macroeconomic impact of competition policy 88 Graph III.2 Business Growth in Europe and the United States 118 Graph III.3 Management practice scores in manufacturing 120 Graph III.4 The link between competition and management performances 121 Boxes Box I.1: Reasons for performing ex-post evaluations by CAs since 2000: Results of the 2012 OECD survey 36 Box II.1: Counterfactual impact evaluation methods 58 Box III.1 Guiding principles and methodology suggested by the OECD to calculate the customer savings 83 Box III.2: Definition of the OECD Competition and Law Indicators 93