Statistical methods for financial engineering

While many financial engineering books are available, the statistical aspects behind the implementation of stochastic models used in the field are often overlooked or restricted to a few well-known cases Statistical Methods for Financial Engineering guides current and future practitioners on implementing the most useful stochastic models used in financial engineering After introducing properties of univariate and multivariate models for asset dynamics as well as estimation techniques, the book discusses limits of the Black-Scholes model, statistical tests to verify some of its assumptions, and the challenges of dynamic hedging in discrete time It then covers the estimation of risk and performance measures, the foundations of spot interest rate modeling, Lévy processes and their financial applications, the properties and parameter estimation of GARCH models, and the importance of dependence models in hedge fund replication and other applications It concludes with the topic of filtering and its financial applications K12677 Rémillard This self-contained book offers a basic presentation of stochastic models and addresses issues related to their implementation in the financial industry Each chapter introduces powerful and practical statistical tools necessary to implement the models The author not only shows how to estimate parameters efficiently, but he also demonstrates, whenever possible, how to test the validity of the proposed models Throughout the text, examples using MATLAB® illustrate the application of the techniques to solve real-world financial problems MATLAB and R programs are available on the author’s website Statistical Methods for Financial Engineering Finance STATISTICAL METHODS FOR FINANCIAL ENGINEERING STATISTICAL METHODS FOR FINANCIAL ENGINEERING BRUNO RÉMILLARD MATLAB® is a trademark of The MathWorks, Inc and is used with permission The MathWorks does not warrant the accuracy of the text or exercises in this book This book’s use or discussion of MATLAB® software or related products does not constitute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2013 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S Government works Version Date: 20130214 International Standard Book Number-13: 978-1-4398-5695-6 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint Except as permitted under U.S Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers For permission to photocopy or use material electronically from this work, please access www.copyright com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400 CCC is a not-for-profit organization that provides licenses and registration for a variety of users For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com MATLAB® is a trademark of The MathWorks, Inc and is used with permission The MathWorks does not warrant the accuracy of the text or exercises in this book This book’s use or discussion of MATLAB® software or related products does not constitute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2013 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S Government works Printed on acid-free paper Version Date: 20130214 International Standard Book Number-13: 978-1-4398-5694-9 (Hardback) This book contains information obtained from authentic and highly regarded sources Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint Except as permitted under U.S Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers For permission to photocopy or use material electronically from this work, please access www.copyright com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400 CCC is a not-for-profit organization that provides licenses and registration for a variety of users For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe Library of Congress Cataloging-in-Publication Data Remillard, Bruno Statistical methods for financial engineering / Bruno Remillard pages cm Includes bibliographical references and index ISBN 978-1-4398-5694-9 (hardcover : alk paper) Financial engineering Statistical methods Finance Statistical methods I Title HG176.7.R46 2013 332.01’5195 dc23 Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com 2012050917 Contents Preface xxi List of Figures xxv List of Tables xxix Black-Scholes Model Summary 1.1 The Black-Scholes Model 1.2 Dynamic Model for an Asset 1.2.1 Stock Exchange Data 1.2.2 Continuous Time Models 1.2.3 Joint Distribution of Returns 1.2.4 Simulation of a Geometric Brownian Motion 1.2.5 Joint Law of Prices 1.3 Estimation of Parameters 1.4 Estimation Errors 1.4.1 Estimation of Parameters for Apple 1.5 Black-Scholes Formula 1.5.1 European Call Option 1.5.1.1 Put-Call Parity 1.5.1.2 Early Exercise of an American Call Option 1.5.2 Partial Differential Equation for Option Values 1.5.3 Option Value as an Expectation 1.5.3.1 Equivalent Martingale Measures and Pricing of Options 1.5.4 Dividends 1.5.4.1 Continuously Paid Dividends 1.6 Greeks 1.6.1 Greeks for a European Call Option 1.6.2 Implied Distribution 1.6.3 Error on the Option Value 1.6.4 Implied Volatility 1.6.4.1 Problems with Implied Volatility 1.7 Estimation of Greeks using the Broadie-Glasserman Methodologies 1 2 4 5 9 10 10 11 11 12 13 13 14 15 16 16 19 20 20 v vi Contents 1.7.1 Pathwise Method 1.7.2 Likelihood Ratio Method 1.7.3 Discussion 1.8 Suggested Reading 1.9 Exercises 1.10 Assignment Questions 1.A Justification of the Black-Scholes 1.B Martingales 1.C Proof of the Results 1.C.1 Proof of Proposition 1.3.1 1.C.2 Proof of Proposition 1.4.1 1.C.3 Proof of Proposition 1.6.1 Bibliography Equation 21 23 23 24 24 27 27 28 29 29 30 30 30 Multivariate Black-Scholes Model Summary 2.1 Black-Scholes Model for Several Assets 2.1.1 Representation of a Multivariate Brownian Motion 2.1.2 Simulation of Correlated Geometric Brownian Motions 2.1.3 Volatility Vector 2.1.4 Joint Distribution of the Returns 2.2 Estimation of Parameters 2.2.1 Explicit Method 2.2.2 Numerical Method 2.3 Estimation Errors 2.3.1 Parametrization with the Correlation Matrix 2.3.2 Parametrization with the Volatility Vector 2.3.3 Estimation of Parameters for Apple and Microsoft 2.4 Evaluation of Options on Several Assets 2.4.1 Partial Differential Equation for Option Values 2.4.2 Option Value as an Expectation 2.4.2.1 Vanilla Options 2.4.3 Exchange Option 2.4.4 Quanto Options 2.5 Greeks 2.5.1 Error on the Option Value 2.5.2 Extension of the Broadie-Glasserman Methodologies for Options on Several Assets 2.6 Suggested Reading 2.7 Exercises 2.8 Assignment Questions 2.A Auxiliary Result 2.A.1 Evaluation of E eaZ N (b + cZ) 2.B Proofs of the Results 2.B.1 Proof of Proposition 2.1.1 33 33 33 34 34 35 35 36 36 37 37 38 38 40 41 41 42 43 43 44 47 47 48 50 51 53 54 54 54 54 Contents 2.B.2 Proof of Proposition 2.2.1 2.B.3 Proof of Proposition 2.3.1 2.B.4 Proof of Proposition 2.3.2 2.B.5 Proof of Proposition 2.4.1 2.B.6 Proof of Proposition 2.4.2 2.B.7 Proof of Proposition 2.5.1 2.B.8 Proof of Proposition 2.5.3 Bibliography vii 55 56 56 57 59 59 59 61 Discussion of the Black-Scholes Model Summary 3.1 Critiques of the Model 3.1.1 Independence 3.1.2 Distribution of Returns and Goodness-of-Fit Tests of Normality 3.1.3 Volatility Smile 3.1.4 Transaction Costs 3.2 Some Extensions of the Black-Scholes Model 3.2.1 Time-Dependent Coefficients 3.2.1.1 Extended Black-Scholes Formula 3.2.2 Diffusion Processes 3.3 Discrete Time Hedging 3.3.1 Discrete Delta Hedging 3.4 Optimal Quadratic Mean Hedging 3.4.1 Offline Computations 3.4.2 Optimal Solution of the Hedging Problem 3.4.3 Relationship with Martingales 3.4.3.1 Market Price vs Theoretical Price 3.4.4 Markovian Models 3.4.5 Application to Geometric Random Walks 3.4.5.1 Illustrations 3.4.6 Incomplete Markovian Models 3.4.7 Limiting Behavior 3.5 Suggested Reading 3.6 Exercises 3.7 Assignment Questions 3.A Tests of Serial Independence 3.B Goodness-of-Fit Tests 3.B.1 Cram´er-von Mises Test 3.B.1.1 Algorithms for Approximating the P -Value 3.B.2 Lilliefors Test 3.C Density Estimation 3.C.1 Examples of Kernels 3.D Limiting Behavior of the Delta Hedging Strategy 3.E Optimal Hedging for the Binomial Tree 63 63 63 63 66 68 68 69 69 70 70 72 73 74 74 75 76 76 77 77 79 83 89 89 90 92 93 94 95 95 96 96 97 97 98 viii Contents 3.F A Useful Result Bibliography Measures of Risk and Performance Summary 4.1 Measures of Risk 4.1.1 Portfolio Model 4.1.2 VaR 4.1.3 Expected Shortfall 4.1.4 Coherent Measures of Risk 4.1.4.1 Comments 4.1.5 Coherent Measures of Risk with Respect to a Stochastic Order 4.1.5.1 Simple Order 4.1.5.2 Hazard Rate Order 4.2 Estimation of Measures of Risk by Monte Carlo Methods 4.2.1 Methodology 4.2.2 Nonparametric Estimation of the Distribution Function 4.2.2.1 Precision of the Estimation of the Distribution Function 4.2.3 Nonparametric Estimation of the VaR 4.2.3.1 Uniform Estimation of Quantiles 4.2.4 Estimation of Expected Shortfall 4.2.5 Advantages and Disadvantages of the Monte Carlo Methodology 4.3 Measures of Risk and the Delta-Gamma Approximation 4.3.1 Delta-Gamma Approximation 4.3.2 Delta-Gamma-Normal Approximation 4.3.3 Moment Generating Function and Characteristic Function of Q 4.3.4 Partial Monte Carlo Method 4.3.4.1 Advantages and Disadvantages of the Methodology 4.3.5 Edgeworth and Cornish-Fisher Expansions 4.3.5.1 Edgeworth Expansion for the Distribution Function 4.3.5.2 Advantages and Disadvantages of the Edgeworth Expansion 4.3.5.3 Cornish-Fisher Expansion 4.3.5.4 Advantages and Disadvantages of the CornishFisher Expansion 4.3.6 Saddlepoint Approximation 4.3.6.1 Approximation of the Density 4.3.6.2 Approximation of the Distribution Function 100 100 103 103 103 103 104 104 105 106 107 107 107 108 109 109 109 111 113 114 116 116 117 117 118 119 120 120 120 121 121 122 122 123 124 Contents ix 4.3.6.3 Advantages and Disadvantages of the Methodology 4.3.7 Inversion of the Characteristic Function 4.3.7.1 Davies Approximation 4.3.7.2 Implementation 4.4 Performance Measures 4.4.1 Axiomatic Approach of Cherny-Madan 4.4.2 The Sharpe Ratio 4.4.3 The Sortino Ratio 4.4.4 The Omega Ratio 4.4.4.1 Relationship with Expectiles 4.4.4.2 Gaussian Case and the Sharpe Ratio 4.4.4.3 Relationship with Stochastic Dominance ¯ 4.4.4.4 Estimation of Omega and G 4.5 Suggested Reading 4.6 Exercises 4.7 Assignment Questions 4.A Brownian Bridge 4.B Quantiles 4.C Mean Excess Function 4.C.1 Estimation of the Mean Excess Function 4.D Bootstrap Methodology 4.D.1 Bootstrap Algorithm 4.E Simulation of QF,a,b 4.F Saddlepoint Approximation of a Continuous Distribution Function 4.G Complex Numbers in MATLAB 4.H Gil-Pelaez Formula 4.I Proofs of the Results 4.I.1 Proof of Proposition 4.1.1 4.I.2 Proof of Proposition 4.1.3 4.I.3 Proof of Proposition 4.2.1 4.I.4 Proof of Proposition 4.2.2 4.I.5 Proof of Proposition 4.3.1 4.I.6 Proof of Proposition 4.4.1 4.I.7 Proof of Proposition 4.4.2 4.I.8 Proof of Proposition 4.4.4 Bibliography Modeling Interest Rates Summary 5.1 Introduction 5.1.1 Vasicek Result 5.2 Vasicek Model 5.2.1 Ornstein-Uhlenbeck Processes 124 125 125 125 126 126 127 127 128 128 129 130 130 131 131 134 134 135 135 136 136 136 137 137 138 139 139 139 140 141 141 142 143 143 144 144 147 147 147 147 148 149 Estimation of Parameters with tanh(x) = ⎛ ⎜ ⎜ J =⎜ ⎜ ⎝ B.3.5 ex −e−x ex +e−x 0 0 eθ 0 0 0 441 The associated Jacobian matrix is then given by 0 e θ4 0 0 − tanh2 (θ5 ) ⎞ ⎛ ⎟ ⎜ ⎟ ⎜ ⎟=⎜ ⎟ ⎜ ⎠ ⎝ 0 0 0 0 σ1 0 0 0 σ2 0 0 − ρ2 ⎞ ⎟ ⎟ ⎟ ⎟ ⎠ Bias and Consistency Let θˆn be an estimator of θ evaluated from a sample of size n The bias of this estimator is defined by E(θˆn ) − θ The estimator is said to be unbiased if its bias is zero, i.e., E(θˆn ) = θ It is said to be asymptotically unbiased if lim E(θˆn ) = θ Finally, the estimator θˆn is said to be consistent if it converges n→∞ in probability to θ Remark B.3.4 If an (θˆn − θ) sistent estimator of θ N (0, 1), where an → ∞, then θˆn is a con- One of the most famous results involving convergence in law is the central limit theorem It is presented next B.4 Central Limit Theorem for Independent Observations The famous central limit theorem basically gives the asymptotic behavior of the estimation error of the mean of a random vector by its empirical analog Theorem B.4.1 (Central Limit Theorem) Suppose that the random vectors X1 , , Xn , with values in Rd , are independent and identically distributed, ¯ n = n Xi with mean√μ and covariance matrix Σ Set X i=1 n ¯ n − μ) Then n (X Nd (0, Σ) In particular, if V is not singular, then √ −1/2 ¯ n − μ) nΣ (X Nd (0, I), and ¯ n − μ) ¯ n − μ) Σ−1 (X n(X χ2 (d) Remark B.4.1 In practice, Σ is unknown and it is estimated by Sn , the socalled sampling covariance matrix, defined by Sn = n−1 n ¯ n )(Xi − X ¯n) (Xi − X i=1 442 Statistical Methods for Financial Engineering Sn is a consistent estimator of Σ Also, if Σ is not singular, then √ −1/2 ¯ − μ) nSn (X Nd (0, I), and ¯ − μ) ¯ − μ) S −1 (X n(X n B.4.1 χ2 (d) Consistency of the Empirical Mean Suppose that the univariate observations X1 , , Xn are independent and ¯ n = n Xi conhave the same distribution If the mean μ exists, then X i=1 n ¯ verges almost surely to μ As a result, Xn is a consistent estimator of μ In addition, if the second moment is finite, then the error made by estimating ¯ n is asymptotically Gaussian More precisely, according to the central μ by X √ ¯ limit theorem (Theorem B.4.1), n(X N (0, σ2 ), where σ2 is the n − μ) variance of Xi √ Set sn = Sn , i.e., sn is the empirical standard deviation B.4.2 Consistency of the Empirical Coefficients of Skewness and Kurtosis Proposition B.4.1 Suppose that the univariate observations X1 , , Xn of X are independent and identically distributed, and assume that the moment of order exists The estimators γˆn,1 and γˆn,2 of the skewness and kurtosis are respectively defined by γˆn,1 γˆn,2 = = n n n i=1 n i=1 ¯n Xi − X sn ¯n Xi − X sn , (B.7) (B.8) If the moment of order exists, then √ √ √ √ ¯ n − μ), n (sn − σ), n (ˆ n (X γn,1 − γ1 ), n (ˆ γn,2 − γ2 ) N4 (0, V ), Estimation of Parameters 443 where the covariance matrix V is defined by V11 = V14 = V23 = V33 = V34 = V44 = σ2 γ1 , V13 = σ γ2 − − γ12 , σ2 (γ2 − 1), σ (μ5 − 2γ1 γ2 − 4γ1 ) , V22 = σ σ μ5 − γ1 − γ1 γ2 , V24 = μ6 + γ2 − 2γ22 − 4γ12 , 2 2 35 μ6 − 3γ1 μ5 + γ12 γ2 − 6γ2 + γ12 + 9, 4 3 μ7 − γ1 μ6 − μ5 (2γ2 + 3) + 3γ1 γ22 + γ1 γ2 + 6γ13 + 12γ1 , 2 μ8 − 8γ1 μ5 − 4μ6 γ2 + 16γ12 (γ2 + 1) + γ22 (4γ2 − 1), σ2 , with μj = E V12 = X1 −μ σ j , j ∈ {1, , 8} V can be estimated by the covariance matrix Vˆn of the pseudo-observations Wn,i , i.e., Vˆn,jk = n−1 n j, k ∈ {1, , d}, Wn,ij Wn,ik , i=1 where Wni = (Wn,i1 , Wn,i2 , Wn,i3 , Wn,i4 ) , with and en,i = Wn,i1 = Wn,i3 = Wn,i4 = xi −¯ x , sx Wn,i2 = sx (e − 1), n,i e3n,i − γˆn,1 − γˆn,1 (e2n,i − 1) − 3en,i , e4n,i − γˆn,2 − 2ˆ γn,2 (e2n,i − 1) − 4ˆ γn,1 en,i , i ∈ {1, , n} In particular, Vˆn,33 = n−1 and Vˆn,44 = sx en,i , n−1 n i=1 e3i − γˆn,1 − γˆn,1 (e2n,i − 1) − 3en,i n γn,2 (e2n,i − 1) − 4ˆ γn,1 en,i e4ni − γˆ2 − 2ˆ 2 i=1 In the Gaussian case, γ1 = and γ2 = 3, so the covariance matrix V reduces to the diagonal matrix ⎞ ⎛ σ 0 ⎜ σ2 /2 0 ⎟ ⎟ (B.9) V =⎜ ⎝ 0 ⎠ 0 24 444 Statistical Methods for Financial Engineering Remark B.4.2 The MATLAB functions skewness and kurtosis can be used to estimate the skewness and the kurtosis PROOF Set εi = (Xi − μ)/σ It follows from the central limit theorem (Theorem B.4.1) that for all j ∈ {1, , 8}, Zj,n = √ n n n εji − μj Zj ∼ N (0, μ2j − μ2j ), i=1 where μj = E εji In addition, the covariance between Zj and Zk is given by μj+k − μj μk , j, k ∈ {1, , 4}, μ1 = 0, μ2 = Using the delta method (Theorem B.3.1), we have √ ¯ n − μ) n(X √ n(ˆ γn,1 − γ1 ) √ n(ˆ γn,2 − γ2 ) σ Z1 , √ n(sn − σ) σ Z2 , G1 = Z3 − γ1 Z2 − 3Z1 , G2 = Z4 − 2γ2 Z2 − 4γ1 Z1 The estimation of V by the pseudo-observations Wn,i is a consequence of the last representation Next, by hypothesis, Var(Z1 ) = 1, Var(Z2 ) = γ2 − 1, Cov(Z1 , Z2 ) = γ1 , and Cov(Z1 , Z3 ) = γ2 We can then deduce that Cov(Z1 , G2 ) = γ2 − − γ12 , = μ5 − γ1 − γ1 γ2 , 2 = μ5 − 2γ1 γ2 − 4γ1 , Cov(Z2 , G2 ) = μ6 + γ2 − 2γ22 − 4γ12 Cov(Z1 , G1 ) Cov(Z2 , G1 ) Furthermore, Var(G1 ) = Var(G2 ) = Cov(G1 , G2 ) = 35 μ6 − 3γ1 μ5 + γ12 γ2 − 6γ2 + γ12 + 9, 4 μ8 − 8γ1 μ5 − 4μ6 γ2 + 16γ12 (γ2 + 1) + γ22 (4γ2 − 1), 3 μ7 − γ1 μ6 − μ5 (2γ2 + 3) + 3γ1 γ22 + γ1 γ2 + 6γ13 + 12γ1 2 In the Gaussian case, γ1 = γ2 − = μ5 = μ7 = 0, μ6 = 15, and μ8 = 105 Hence V is diagonal, with V11 = σ2 , V22 = σ /2, V33 = Var(G1 ) = 6, and V44 = Var(G2 ) = 24 Estimation of Parameters B.4.3 445 Confidence Intervals I Using the central limit theorem, one can also build confidence intervals In ¯ n −μ √ fact, since X is approximately distributed as a standard Gaussian variate sn n when n is large enough, it follows that √ ¯ n − μ| ≤ 1.96sn / n) ≈ P (|Z| ≤ 1.96) = 0.95 P (|X The interpretation of the confidence interval is the following: In 95% of case, ¯ n − 2sn/√n, X ¯ n + 2sn /√n] will contain the true value μ Howthe interval [X ever, for a given sample, one does not know if μ belongs to the confidence interval One just predicts that it does, and 95% of the predictions should be true! B.4.4 Confidence Ellipsoids When the parameter is multidimensional, i.e., θ ∈ Rd , one can define confidence regions which are ellipsoids In fact, if χ2d,α is the quantile of order − α of a chi-square distribution with d degree of freedom, then according to Remark B.4.1, ¯ n − μ) Vn−1 (X ¯ n − μ) ≤ χ2d,α ≈ − α P n(X It is called a confidence ellipsoid since E = {μ ∈ Rd ; (a − μ) V −1 (a − μ) ≤ b} is an ellipsoid In fact, since V can be written as V = M ΔM , where Δ is a diagonal matrix and M M = I, we have ⎫ ⎧ d ⎬ ⎨ zj ≤b E = M z + a; ⎭ ⎩ Δjj j=1 Since M is a rotation matrix, E is composed of a rotation of an ellipsoid centered at the origin, followed by a translation B.4.5 Confidence Intervals II It sometimes happens that the central limit theorem does not apply, which implies that the estimation error is not Gaussian Nevertheless, in some situations, one can still define confidence intervals To this end, suppose that Θ, where Θ has distribution function H Then an (θˆn − θ) H −1 (1 − α/2) ˆ H −1 (α/2) θˆn − , θn − an an is a confidence interval of level 100(1 − α)%, since P θ+ H −1 (α/2) H −1 (1 − α/2) ≤ θˆn ≤ θ + an an ≈ H H −1 (1 − α/2) − H H −1 (α/2) = − α 446 Statistical Methods for Financial Engineering Remark B.4.3 When H is the distribution function of a standard Gaussian variate, then one recovers the usual confidence interval, since −H −1 (α/2) = −(−zα/2) = zα/2 and H −1 (1 − α/2) = zα/2 B.5 Precision of Maximum Likelihood Estimator for Serially Independent Observations Suppose that X1 , , Xn are independent, with density fθ Then the loglikelihood is given by n L(θ) = ln{fθ (xi )} i=1 Proposition B.5.1 Under weak conditions, see, e.g., Serfling [1980], the estimator θˆn obtained from the maximum likelihood principle is consistent and √ ˆ n (θn − θ) Np (0, V ), where V is a non-negative definite matrix satisfying Iθ V Iθ = Iθ , and Iθ is the Fisher information matrix given, for j, k ∈ {1, , p}, by (Iθ )jk = ∂ ln{fθ (x)} ∂θj ∂ ln{fθ (x)} fθ (x)dx ∂θk Then V is the Moore-Penrose pseudo-inverse of Iθ Note that the associated MATLAB function is pinv Also, if Iθ is invertible, then V = Iθ−1 B.5.1 Estimation of Fisher Information Matrix In practice, instead of maximizing the likelihood, one often minimizes the negative of the log-likelihood Therefore, one could estimate Iθ by Iˆn = n (yi − y¯)(yi − y¯) , (B.10) i=1 with yi = −∇θ ln{fθˆn (xi )} = − ∇θ fθˆn (xi ) fθˆn (xi ) , i ∈ {1 , n}, where ∇fθ is the column vector with components ∂θj fθ (x), j ∈ {1, , p} Note that one should have y¯ ≈ The gradient can be evaluated exactly It can Estimation of Parameters 447 also be approximated numerically by using, for example, the MATLAB function N umJacobian Another approach used in practice is to estimate Iθ by using the Hessian matrix of the −log-likelihood function Hn (matrix formed by the second derivatives), i.e., for ≤ j, k ≤ p, one has n (Hn )jk = − ∂θj ∂θk fθˆn (xi ) i=1 fθˆn (xi ) n + i=1 so Hn = y¯y¯ + Iˆn − n n n ∂θj fθˆn (xi ) ∂θk fθˆn (xi ) fθˆn (xi ) ∂θj ∂θk fθˆn (xi ) i=1 fθˆn (xi ) fθˆn (xi ) , As a result, Hn /n is close to Iˆn defined by (B.10), and it converges in probability to Iθ One cannot tell in advance which method of estimating the limiting covariance is more precise However, in some cases, the Hessian matrix might not be positive definite or even non-negative definite, so (B.10) should be the choice for estimating the Fisher information Note that in any mathematical packages, the Hessian matrix is one of the possible outputs of the minimizing functions (like fmincon and fminunc in MATLAB) Example B.5.1 (Gaussian Data (Continued)) Recall that in that case, ∇(μ,σ) ln{f (x; μ, σ)} = (x − μ) (x − μ)2 , − σ σ σ As a result, the Fisher information matrix is I(μ,σ) = E σ2 Z(Z − 1) Z2 Z(Z − 1) (Z − 1)2 ∼ N (0, 1) Since E(Z ) = and E(Z ) = 3, one obtains σ2 −1 = In addition, Hence V = I(μ,σ) that I(μ,σ) = σ12 σ /2 2 σn2 ) = (n − 1)σ /n ≈ σ√ , when n is large enough One E(ˆ μn ) = μ and E(ˆ n(¯ xn − μ) may conclude from Proposition B.5.1 that √ N2 (0, V ), where n(sn − σ) σ2 V = In particular, σ2 /2 where Z = X−μ σ zα/2 μ=x ¯n ± sn √ , n zα/2 σ = sn ± sn √ 2n are confidence intervals of level − α for μ and σ respectively (but not simultaneously) 448 Statistical Methods for Financial Engineering Remark B.5.1 The convergence result could have been obtained directly using Propositions B.4.1 and B.3.1, i.e., by using the central limit theorem and the delta method Example B.5.2 Here is an example where the covariance matrix V is singular For the Pareto distribution function F (x) = − (x/σ)−1/ξ , x ≥ σ, ξ > 0, one has σ ˆn = m = min(X1 , , Xn ) and ξˆn = n1 ni=1 ln(Xi /m) It √ ˆ √ follows that n (ξn − ξ) N (0, ξ ), while n (ˆ σn − σ) In this case, ξ2 V = 0 B.6 Convergence in Probability and the Central Limit Theorem for Serially Dependent Observations In many applications, instead of having independent observations, the latter are serially dependent, i.e., Xi can depend on X1 , , Xi−1 In such a case, one needs another version of the central limit theorem See Durrett [1996] Theorem B.6.1 Suppose that a sequence of random d-dimensional vectors ¯ n = n Xi X1 , , Xn is stationary and ergodic, with mean μ Then X i=1 n converges in probability to μ Suppose in addition that each Xi is a martingale difference, E(Xi |X1 , , Xi−1 ) = μ Then, if the moment of order √ i.e., ¯ n − μ) exists, n (X Nd (0, Σ), with Σ = E (Xi − μ)(Xi − μ) Remark B.6.1 In practice, Σ is unknown and it is estimated by Σn , the sampling covariance matrix defined by Σn = n n ¯ n )(Xi − X ¯n) (Xi − X i=1 Moreover, Σn is a consistent estimator of Σ B.7 Precision of Maximum Likelihood Estimator for Serially Dependent Observations Suppose that the data X1 , , Xn are observation from an ergodic stationary time series Assume that the conditional densities of Xi given X1 , , Xi−1 , denoted fi,θ exist and also depend on an unknown parameter θ ∈ Rp Then, using the multiplication formula (A.19), the joint density Estimation of Parameters 449 of X2 , Xn given X1 is n fθ (x2 , , xn |x1 ) = fi,θ (xi , , x1 ) i=2 It follows that if L is the log-likelihood, to estimate θ one can maximize n ln{fi,θ (xi , , x1 , θ)} L(θ) = i=2 Therefore, if θˆn in the interior of Ω, then one must have θˆn such that ∇L(θˆn ) = 0, where n n ∇θ fi,θˆ(xi , , x1 ) ˆ ˆ = ∇L(θn ) = i (θn ) f ˆn (xi , , x1 ) i, θ i=2 i=1 In many cases, one can then show that the maximum likelihood estimator is consistent; one can also evaluate the error of estimation Proposition B.7.1 Under weak assumptions, √ n (θˆn − θ) Np (0, V ), where V is a non-negative definite matrix satisfying Iθ V Iθ = Iθ , and Iθ is the Fisher information matrix defined by Iθ = E latter can be estimated by Iˆθ = n where yi = i (θ) i (θ) The n (yi − y¯)(yi − y¯) , i=1 ˆ or by Hn /n, where Hn is the Hessian matrix of −L(θˆn ) i (θ), Example B.7.1 Assume that the data (Xi )i≥1 satisfy Xi − μ = φ(Xi−1 − μ) + σεi , where |φ| < 1, σ > and the innovations εt are independent, with a standard Gaussian distribution Then, since the conditional distribution of Xi given X1 , , Xi−1 is Gaussian, with mean μ+ φ(Xi−1 − μ) and variance σ2 , setting θ = (μ, φ, σ), one has ln{fi (Xi , Xi−1 , θ)} = − ln √ 2πσ − {Xi − μ − φ(Xi−1 − μ)} 2σ2 450 Statistical Methods for Financial Engineering Consequently, ∂μ ln{fi(Xi , Xi−1 , θ)} ∂φ ln{fi(Xi , Xi−1 , θ)} = = (1 − φ) {Xi − μ − φ(Xi−1 − μ)} /σ2 , (Xi−1 − μ) {Xi − μ − φ(Xi−1 − μ)} /σ2 , ∂φ ln{fi(Xi , Xi−1 , θ)} = − Then i (θ) {Xi − μ − φ(Xi−1 − μ)}2 + σ σ3 = (1 − φ)εi , εi (Xi−1 − μ), ε2i − /σ, leading to ⎞ ⎛ 0 (1 − φ)2 ⎝ σ Iμ,φ,σ = 0 ⎠ 1−φ2 σ 0 As a result, ⎛ −1 Iμ,φ,σ In addition, by solving μ ˆn = φˆn = σ ˆn2 = =⎝ σ2 (1−φ)2 0 n ˆ i=2 i (θ) x ¯n + − φ2 = 0, one obtains φ(Xn − μ ˆn ) − (X1 − μ ˆn ) , n n ˆn )(Xi − i=2 (Xi−1 − μ n (X ˆ n )2 i−1 − μ i=2 n μ ˆn ) , n ˆn − φˆn (Xi−1 − μ ˆn ) Xi − μ i=2 Therefore, one could set μ ˆn = x ¯n , φˆn = n ⎞ 0 ⎠ σ /2 n i=2 n xn )(Xi −¯ xn ) i=2 (Xi−1 −¯ n x)2 i=2 (Xi−1 −¯ , and σ ˆn2 = Xi − x ¯n − φˆn (Xi−1 − x ¯n ) By Theorem B.6.1, these estimators are consistent Furthermore, one deduces from Theorem B.7.1 that √ √ √ −1 n (¯ xn − μ), n (φˆn − φ), n (ˆ σn − σ) N3 0, Iμ,φ,σ √ √ xn − μ)/ˆ σn Z1 , n (φˆn − φ)/ − φˆ2n Z2 and It follows that n (1 − φˆn )(¯ √ √ n (ˆ σn − σ)/(ˆ σn / 2) Z3 , where Z1 , Z2 and Z3 are independent standard Gaussian variables B.8 Method of Moments It is sometimes impossibly difficult to use the maximum likelihood principle to estimate parameters For example, it can be hard or too computationally Estimation of Parameters 451 intensive to evaluate the log-likelihood function Another methodology which is often practical and easy to implement is the method of moments, when the unknown parameters can be expressed in terms of the first moments The idea is simply to estimate the moments by the sampling moments and then recover an estimation of the parameters by inversion For example, if the expectation n ¯n = Xi is an unbiased estimator of μ If the variance σ μ exists, then X n i=1 ¯n ) = σ /n, so X ¯ n is a consistent estimator of μ exists, then V ar(X More generally, suppose that the unknown p-dimensional parameter θ can be expressed as g(μ1 , , μp ), where μj = E(X j ), j ∈ {1, , p} Then, set μ1 , , μ ˆp ), where μ ˆn,j is an estimator of μj For example, one could θˆn = g(ˆ take n xj , j ∈ {1, , p}, μ ˆn,j = n i=1 i since these estimators are consistent if μ2j < ∞ Using the delta method (Proposition B.3.1), one may conclude that √ ˆ n(θn − θ) ∇g(μ1 , , μp )Z, where Z ∼ Np (0, V ), with Vjk = μj+k − μj μk , and (∇g(μ1 , , μp ))jk = ∂gj (μ1 , ,μp ) , ∂μj j, k ∈ {1, , p} If the random variables X1 , , Xn are independent and identically distributed, with mean μ and variance σ , and if the 4th moment exists, then using the central limit theorem (Theorem B.4.1), one has √ xn − μ) √ n(¯ N2 (0, Σ), n(s2n − σ ) where Σ= σ2 γ1 σ γ1 σ σ (γ2 − 1) , and γ1 and γ2 are respectively the skewness and the kurtosis coefficients defined in Proposition √ A.5.1 Note that γ2 = in the Gaussian case In addition, n(¯ xn − μ) ˜ where one also has √ N2 (0, Σ), n(sn − σ) ˜ = σ2 Σ γ1 /2 γ1 /2 (γ2 − 1)/4 Example B.8.1 Suppose that X1 , , Xn are independent observations of X ∼ Gamma(α, β), α, β > It follows that x ¯ and s2 are unbiased estimators of the mean and variance of X respectively Since E(X) = αβ and V ar(X) = αβ , one can use the method of moments to obtain βˆ = s2 /¯ x and 2αβ −α In addition, since α ˆ = (¯ x/s)2 It then follows that ∇g = αβ2 −β β E(X ) = α(α + 1)(α + 2)β and E(X ) = α(α + 1)(α + 2)(α + 3)β , then 452 Statistical Methods for Financial Engineering 2β As a result, and γ2 = + α6 Hence, Σ = αβ 2β 2(α + 3)β √ one may conclude that n((ˆ α − α), (βˆ − β)) N2 (0, A), where γ1 = √2 α A = ∇gΣ(∇g) = 2α(α + 1) −2β(α + 1) −2β(α + 1) (2 + 3/α)β Example B.8.2 (Student Innovations) Consider the following model: Xi = μ + σ εi , i ∈ {1, , n}, where the εi , i ∈ {1, , n}, are independent and identically distributed, with εi ∼ T (ν) Since the density depends on the gamma function, there will be no explicit solution using the maximum likelihood principle However, assuming ν > and using the method of moments, one has E(Xi ) = μ, μ2 = Var(Xi ) = ν σ ν−2 , γ1 = 0, and γ2 = 3ν−6 ν−4 , where the skewness and kurtosis coefficients γ1 −6 and γ2 are defined by (A.9) and (A.10) Hence ν = 4γ and σ = μ2 2γγ22−3 γ2 −3 4ˆ γ −6 γ ˆ n,2 n,2 ¯n , μ ˆ2 = s2n , νˆn = γˆn,2 ˆn2 = s2n 2ˆγn,2 As a result, μ ˆn = x −3 and σ −3 It then follows √ from Proposition B.4.1 and the delta method (Theorem B.3.1) that if ν > 8, n (ˆ νn − ν) N 0, σν2 , where σν2 = 36 μ8 − 4μ6 γ2 + γ22 (4γ2 − 1) , (γ2 − 3)4 where μ6 and μ8 are the standardized 6-th and 8-th moments of the Student 15(ν−2)2 and distribution According to formula (A.12), it follows that μ6 = (ν−4)(ν−6) μ8 = 105(ν−2)3 (ν−4)(ν−6)(ν−8) σν2 = B.9 Hence, (ν − 4) (ν − 2)2 (ν − ν + 23 ν − 20) (ν − 8) (ν − 6) (B.11) Combining the Maximum Likelihood Method and the Method of Moments It might happen that one wants to estimate some parameters with a method of moments, while using the maximum likelihood method to estimate the rest of the parameters So suppose that θ = (θ1 , θ2 ) , where θ2 = g(α), with E{h(Xi )|Xi−1 , , X1 } = α, for some square integrable vecn tor function h It follows that one can estimate α by α ˆ n = n1 i=1 h(Xi ) and θˆn,2 = g (α ˆn ) Using the central limit theorem (Theorem √ B.6.1) together with αn − α) N (0, Σ), the delta method (Theorem B.3.1), we obtain that n (ˆ Estimation of Parameters 453 with Σ = Cov{h(Xi )}, which can be estimated by ˆ= Σ n−1 n {h(Xi ) − α ˆ n } {h(Xi ) − α ˆn} , (B.12) i=1 √ αn ) √1n ni=1 h(Xi )+oP (1) Here g˙ is the Jacobian while n θˆn,2 − θ2 = g˙ (ˆ matrix of the transformation α → θ2 = g(α) The estimation of θ1 is then defined by θˆn,1 = arg minθ1 L(θ1 ), where n ln fi Xi , , X1 , θ1 , θˆn,2 L(θ1 ) = − (B.13) i=1 Here we use the same notations as in Section B.7 Using the central limit theorem (Theorem B.6.1) and the previous convergence results, we obtain the following proposition √ Proposition B.9.1 Under weak assumptions, n(θˆn −θ) Np (0, V ), where V11 V12 ˆ g˙ (α ˆn ) Σ ˆn ), Vˆ11 = Iˆ−1 , , with V22 estimated by Vˆ22 = g˙ (α V = V12 V22 where Iˆ is the estimation of the Fisher information (as if θ2 were known), i.e., it can be estimated by Iˆ = n−1 n i θˆn i θˆn , i=1 with i (θ) = i (θ1 , θ2 ) = −∇θ1 fi (Xi , , X1 , θ1 , θ2 ), or by Iˆ = Hn /n, where Hn is the Hessian of the log-likelihood function (B.13) Finally, V12 can be estimated by Vˆ12 = Vˆ11 B.10 n−1 n i ˆn} θˆn {h(Xi ) − α ˆn ) g˙ (α i=1 M-estimators An estimation method which increases in popularity is the method of Mestimators Given a function ψ(x, θ) and a sample X1 , , Xn , one sets n θˆn = arg ˜ θ∈Ω i=1 ˜ ψ(Xi , θ) For example, the choice ψ(x, θ) = |x − θ| leads to the median, while choosing ψ(x, θ) = (x − θ)2 leads to the mean One can also choose functions which are asymmetric with respect to the origin 454 B.11 Statistical Methods for Financial Engineering Suggested Reading For a general reference on estimation, see Serfling [1980] See also Berndt et al [1974] for estimation of parameters in dependent sequences B.12 Exercises Exercise B.1 The MATLAB file DataCauchy contains observations from a Cauchy distribution (see Appendix A.6.16) with density f (x) = σπ(1 + (x − μ)2 /σ ) Assume that the data are independent We want to estimate μ and σ > The log-likelihood function to maximize is n ln + (xi − μ)2 )/σ L(μ, σ) = −n ln(σ) − i=1 (a) Using the MATLAB function EstCauchy, find μ ˆ and σ ˆ Note that the observations have been simulated with μ = 10 and σ = 5! (b) Extend the function EstCauchy so that the covariance matrix between the estimators is an output Use the exact gradients to estimate the Fisher information Bibliography E K Berndt, B H Hall, R E Hall, and J A Hausman Estimation and inference in nonlinear structural models Annals of Economics and Social Measurement, pages 653–665, 1974 R Durrett Probability: Theory and Examples Duxbury Press, Belmont, CA, second edition, 1996 R J Serfling Approximation Theorems of Mathematical Statistics John Wiley & Sons Inc., New York, 1980 Wiley Series in Probability and Mathematical Statistics While many financial engineering books are available, the statistical aspects behind the implementation of stochastic models used in the field are often overlooked or restricted to a few well-known cases Statistical Methods for Financial Engineering guides current and future practitioners on implementing the most useful stochastic models used in financial engineering After introducing properties of univariate and multivariate models for asset dynamics as well as estimation techniques, the book discusses limits of the Black-Scholes model, statistical tests to verify some of its assumptions, and the challenges of dynamic hedging in discrete time It then covers the estimation of risk and performance measures, the foundations of spot interest rate modeling, Lévy processes and their financial applications, the properties and parameter estimation of GARCH models, and the importance of dependence models in hedge fund replication and other applications It concludes with the topic of filtering and its financial applications K12677 Rémillard This self-contained book offers a basic presentation of stochastic models and addresses issues related to their implementation in the financial industry Each chapter introduces powerful and practical statistical tools necessary to implement the models The author not only shows how to estimate parameters efficiently, but he also demonstrates, whenever possible, how to test the validity of the proposed models Throughout the text, examples using MATLAB® illustrate the application of the techniques to solve real-world financial problems MATLAB and R programs are available on the author’s website Statistical Methods for Financial Engineering Finance .. .STATISTICAL METHODS FOR FINANCIAL ENGINEERING STATISTICAL METHODS FOR FINANCIAL ENGINEERING BRUNO RÉMILLARD MATLAB® is a trademark of The... are used only for identification and explanation without intent to infringe Library of Congress Cataloging-in-Publication Data Remillard, Bruno Statistical methods for financial engineering /... references and index ISBN 978-1-4398-5694-9 (hardcover : alk paper) Financial engineering Statistical methods Finance Statistical methods I Title HG176.7.R46 2013 332.01’5195 dc23 Visit the Taylor