The evaluation and optimization of trading strategies

Table of Contents Praise Title Page Copyright Page Foreword Preface THERE AND BACK AGAIN COMPUTING THE INVESTMENT INDUSTRY TRADING STRATEGY DEVELOPMENT TOOLS THE RISE OF ADVANCED MATHEMATICAL CONCEPTS IN TRADING TRADING MEETS HIGHER EDUCATION Acknowledgments Introduction CHAPTER - On Trading Strategies WHY THIS BOOK WAS WRITTEN WHO WILL BENEFIT FROM THIS BOOK? THE GOALS OF THIS BOOK THE LAY OF THE LAND CHAPTER - The Systematic Trading Edge DISCRETIONARY TRADING RAISING THE BAR VERIFICATION QUANTIFICATION RISK AND REWARD THE PERFORMANCE PROFILE OBJECTIVITY CONSISTENCY EXTENSIBILITY THE BENEFITS OF THE HISTORICAL SIMULATION POSITIVE EXPECTANCY THE LIKELIHOOD OF FUTURE PROFIT THE PERFORMANCE PROFILE PROPER CAPITALIZATION A MEASURE OF REAL-TIME TRADING PERFORMANCE THE BENEFITS OF OPTIMIZATION THE BENEFITS OF THE WALK-FORWARD ANALYSIS THE ADVANTAGES OF A THOROUGH UNDERSTANDING CONFIDENCE STRATEGY REFINEMENT CHAPTER - The Trading Strategy Development Process TWO PHILOSOPHICAL APPROACHES TO STRATEGY DEVELOPMENT AN OVERVIEW OF THE TRADING STRATEGY DESIGN PROCESS CHAPTER - The Strategy Development Platform THE SCRIPTING LANGUAGE DIAGNOSTICS REPORTING OPTIMIZATION THE OBJECTIVE FUNCTION SPEED AUTOMATION WALK-FORWARD ANALYSIS PORTFOLIO ANALYSIS IN CONCLUSION CHAPTER - The Elements of Strategy Design THE THREE PRINCIPAL COMPONENTS OF A STRATEGY AN OVERVIEW OF A TYPICAL TRADING STRATEGY A TRADE EQUALS AN ENTRY AND AN EXIT THE MANAGEMENT OF RISK THE MANAGEMENT OF PROFIT POSITION SIZING ADVANCED STRATEGIES SUMMARY CHAPTER - The Historical Simulation THE ESSENTIAL REPORTS THE IMPORTANCE OF ACCURACY SOFTWARE LIMITATIONS REALISTIC ASSUMPTIONS LIMIT MOVES MAJOR EVENTS AND DATES HISTORICAL DATA STOCK PRICES CASH MARKETS FUTURES MARKETS THE CONTINUOUS CONTRACT THE PERPETUAL CONTRACT ADJUSTED CONTINUOUS CONTRACTS THE SIZE OF THE TEST WINDOW HOW MANY TRADES? STABILITY DEGREES OF FREEDOM FREQUENCY OF TRADING TYPES OF MARKETS EFFICIENT MARKETS THE LIFE CYCLE OF A TRADING STRATEGY WINDOW SIZE AND MODEL LIFE CHAPTER - Formulation and Specification FORMULATE THE TRADING STRATEGY SPECIFICATION—TRANSLATE THE IDEA INTO A TESTABLE STRATEGY MAKE A VAGUE IDEA PRECISE CHAPTER - Preliminary Testing VERIFICATION OF CALCULATIONS AND TRADES THEORETICAL EXPECTATIONS PRELIMINARY PROFITABILITY THE MULTIMARKET AND MULTIPERIOD TEST CHAPTER - Search and Judgment SEARCH METHODS ADVANCED SEARCH METHODS GENERAL PROBLEMS WITH SEARCH METHODS THE OBJECTIVE FUNCTION A REVIEW OF A VARIETY OF EVALUATION METHODS MULTIPLE EVALUATION TYPES CHAPTER 10 - Optimization OPTIMIZATION CONTRA OVERFITTING A SIMPLE OPTIMIZATION THE OPTIMIZATION FRAMEWORK A MULTIMARKET AND MULTIPERIOD OPTIMIZATION THE EVALUATION OF THE OPTIMIZATION THE ROBUST TRADING STRATEGY THE STATISTICALLY SIGNIFICANT OPTIMIZATION PROFILE THE DISTRIBUTION OF THE OPTIMIZATION PROFILE THE SHAPE OF THE OPTIMIZATION PROFILE HOW DOES THE STRATEGY RESPOND TO OPTIMIZATION? DOES THE STRATEGY DESERVE FURTHER DEVELOPMENT? CHAPTER 11 - Walk-Forward Analysis IS THE TRADING STRATEGY ROBUST? ROBUSTNESS AND WALK-FORWARD EFFICIENCY THE CURE FOR OVERFITTING A MORE RELIABLE MEASURE OF RISK AND RETURN ASSESSING THE IMPACT OF MARKET CHANGES THE BEST PARAMETER SET FOR TRADING THE THEORY OF RELEVANT DATA SHIFTING MARKETS THE VARIETIES OF MARKET CONDITIONS THE WALK-FORWARD THE ROLE OF THE WALK-FORWARD SETTING UP A WALK-FORWARD AN EXAMPLE OF A WALK-FORWARD TEST THE WALK-FORWARD ANALYSIS IS THE STRATEGY ROBUST? WHAT RATE OF PROFIT SHOULD WE EXPECT? WHAT IS THE RISK? WALK-FORWARD ANALYSIS AND THE PORTFOLIO CHAPTER 12 - The Evaluation of Performance THE TRADING STRATEGY AS AN INVESTMENT THE DIMENSION OF RISK COMPARE THE STRATEGY TO THE ALTERNATIVES MAXIMUM DRAWDOWN AND TRADING RISK RISK ADJUSTED RETURN REWARD TO RISK RATIO MODEL EFFICIENCY CONSISTENCY PATTERNS OF PROFIT AND LOSS CHAPTER 13 - The Many Faces of Overfitting WHAT IS OVERFITTING? THE ABUSE OF HINDSIGHT THE CASE OF THE OVERFIT FORECASTING MODEL THE CASE OF THE OVERFIT TRADING MODEL THE SYMPTOMS OF AN OVERFIT TRADING MODEL THE CAUSES OF OVERFITTING CHAPTER 14 - Trading the Strategy THE MENTAL ASPECTS OF TRADING RETURN ON INVESTMENT MAXIMUM RISK REAL-TIME AND EVALUATION PERFORMANCE COMPARING THE EVALUATION AND TRADE PROFILE UNDERSTANDING THE TEST PROFILE PERFORMANCE QUIRKS IN CONCLUSION Notes Index Additional Praise for The Evaluation and Optimization of Trading Strategies “The decisive step in system trading is the determination of the reliability and robustness of your system This greatly expanded and very accessible new edition of Bob’s classic presents a thorough yet easy to apply and timetested methodology to accurately make this determination This makes it possible to form a realistic idea of how your system should perform in the future and increases the likelihood of lasting real-time trading profit.” —Murray Ruggerio, Vice President of Research and Development for TradersStudio; Contributing Editor Futures Magazine “Bob Pardo has re-written his own excellent book, and made it even better It contains a basic stepby-step guide to building a trading system, along with an introduction to advanced system building concepts and tools Simply avoiding the problems and errors Bob warns about can help lead to real time success A must read for anyone interested in a systematic approach to trading.” —Michael Tepper, Atlas Capital Management, Inc “A significant contribution that will present, explain, clarify, and illustrate Algo Trading strategies and how to properly test/optimize trading models, thoroughly researched and perceptive It will give the reader very practical and seasoned insight into the world of Algo Trading Bob Pardo brings a very lucid approach to a very esoteric subject; a welcome departure from most texts It is both accessible and rigorous, which is quite rare.” —Bruce J Serra, Vice President, Institutional Sales, MF Global Inc Founded in 1807, John Wiley & Sons is the oldest independent publishing company in the United States With offices in North America, Europe, Australia and Asia, Wiley is globally committed to developing and marketing print and electronic products and services for our customers’ professional and personal knowledge and understanding The Wiley Trading series features books by traders who have survived the market’s ever-changing temperament and have prospered—some by reinventing systems, others by getting back to basics Whether a novice trader, professional or somewhere in between, these books will provide the advice and strategies needed to prosper today and well into the future For a list of available titles, please visit our Web site at www.WileyFinance.com NASDAQ Net profit evaluation New York Stock Exchange Objective function: evaluation method example of, in Walk-Forward Analysis genetic optimization algorithm importance of optimization function optimization space parameter set particle swarm optimization (PSO) search methods search parameters selecting for optimization framework Sharpe Ratio speed of processing strategy parameters Objectivity: definition of value of, for trading Omega Research (custom programming) Opening and closing range slippage Optimal f (fixed fractional trading) Optimization: versus curve-fitting definition of degrees of freedom distribution of profile drawdown, maximum (MDD) errors evaluation example of, in Walk-Forward Analysis framework function global maximum historical trading performances model parameter combinations moving average crossover system multimarket and multiperiod overfitting overparameterization overscanning pitfalls postoptimization (out-of-sample) profile reoptimization robust trading strategy shape of profile simple, example simulated annealing (SA) method trading strategy and Walk-Forward Analysis window Optimization space dimensions of enlightened or informed selectivity hill climbing, multipoint search peaks and valleys search for Orders: buy and sell(see also entry and exit strategies) buy Market If Touched (MIT) orders buy price limit orders buy stop calculations and testing countertrending systems dollar profit target Good Till Canceled (GTC) Market on Close (MOC) Market on Open (MOO) market order opening gap overnight risk price (or limit) orders profit target sell Market If Touched (MIT) orders sell price limit orders sell stop slippage Stop Close Only (SCO) stop-loss target trailing dollar profit stop trailing volatility profit stop volatility profit target Out-of-sample testing Overcapitalization Overfitting: abuse of hindsight big fish in small pond syndrome causes of cure for versus curve-fitting degrees of freedom entry and exit filters forecasting model, example versus optimization spiky performance profile strategy design symptoms of, in trading model trading model, example Walk-Forward Analysis Overparameterization Overscanning Parameters: constant convergence crossover definition of distribution in optimization framework enlightened or informed selectivity genetic algorithms (GAs) grid search historical price data mutation of sets optimization framework particle swarm optimization (PSO) periodic reoptimization population prioritized step search reoptimization robustness scan range, in framework selection through generations set, best for trading Sharpe Ratio simulated annealing (SA) method testing top Walk-Forward Analysis Particle swarm optimization (PSO) Patterns of profit and loss Perfect profit (PP) Performance: evaluation market changes overfitting peak period postoptimization preliminary testing profile quirks real-time trading smooth profile, in pivot chart spike, in pivot chart statistical rigor summary summary, example time-sliced trading trading strategy underperformance Pessimistic return on margin (PROM) Phantom trades Pickens, T Boone Portfolio: maximum drawdown (MDD) measurement of risk risk management Walk-Forward Analysis Position sizing: advanced strategies algorithms, derived from gambling literature entry and exit strategies importance in trading strategy optimal f (fixed fractional trading) overnight, open reversal versus nonreversal rules scaling into and out of a position too large versus too small volatility adjusted Postoptimization (out-of-sample) Prediction Company, The Price and trade slippage Price data, historical Price (or limit) orders Price shock Prioritized step search Profile: distribution of the optimization profit and risk shape of the optimization statistically significant optimization test, understanding Profit See also Profit, perfect (PP) dollar profit target management of patterns of, and loss perfect (PP) porfolio risk management potential price order real-time trading risk, interrelated with risk-adjusted rate of return (RAR) targets trailing dollar profit stop trailing stop, the trailing volatility profit stop volatility profit target Walk-Forward Analysis windfall Profit, perfect: correlation coefficient, formula for, of perfect profit and equity curve (CECPP) definition of table of daily data Programming: analyzing C code, example diagnostics “if” statement interview process pseudocoding specifications scripting language testing Psychology of traders Quantification and trading advantages Quote vendors, real-time Real-time performance Real-time trading: consistency evaluation and trade profile versus historical simulation optimization overfitting and losses performance profile profits return on investment signals and Walk-Forward Analysis Renaissance Technologies Reoptimization Reports: equity curve, the graphic display historical bar-by-bar basis interval-by-interval basis performance summary trade-by-trade basis time-sliced trade list, the year-by-year basis Required capital (RC) Returns: annualized rate of potential of reliable measure of, and return risk-adjusted rate of return (RAR) Reward to risk ratio (RRR) Risk: -adjusted rate of return dimension of entry and exit signals future trading management maximum maximum drawdown (MDD) maximum potential moving target net profit evaluaton overnight risk portfolio profit, interrelated with reliable measurements required capital (RC) risk stop slippage stop stop-loss signals strategy design target orders three broad categories trade risk trailing dollar profit stop trailing volatility profit stop volatility risk stop Robustness: four features of versus profitability Rounding issues Rule of Ten Rules: -adjusted position sizing diagnostics optimization Martingale strategy optimal f (fixed fractional trading) position sizing sell (or short) specifications for programming trading Scaling into and out of a position Scholes, Myron Schwager, Jack, Market Wizards Scripting language: EasyLanguage, example “if” statement Metastock, example pseudocoding TradersStudio, example wizard Search: advanced methods evaluation method global maximum grid search hill climbing, multipoint search hill climbing search algorithms local maximum methods optimization space parameter particle swarm optimization (PSO) peaks and valleys prioritized step search problems, general simulated annealing (SA) Selectivity, enlightened or informed Self-doubt, and negative effects on trading Sell Market If Touched (MIT) orders Sell price limit orders Sell stop orders Sharpe Ratio Shaw, David Shifting markets Signals: filters for entry and exit price chart for testing real-time stop-loss Simons, Jim Simulated annealing (SA) optimization method Simulation: definition of historical Sizing See Position sizing Slippage Software: advances in automated trading, advantages of CQG (custom programming) curve, closeness of fit versus predictive capacity limitations Omega Research (custom programming) phantom trades price orders reports rounding issues tick math Specifications: ideas, vague vs precise sample interview Standard and Poor’s 500 Futures Startup overhead Statistics: correlation coefficient degrees of freedom measurements, standard modeling procedures rigor of robust sample size standard error in sample size test window Stop-loss strategy Stop Close Only (SCO) Strategy See Trading strategy Strategy design See Trading strategy design Strategy development See Trading strategy development Symmetrical trading strategy Target order: Technology See also Software automated trading, advantages of correlation coefficient, closing prices detecting bad trading ideas diagnostics growth of market opportunities (inefficiencies) objective function optimization framework reporting shifting markets simple optimization, example software limitations specifying rules, example trading strategies, large-scale versus scientific and empirical method Walk-Forward Analysis Templates Testing: back-testing computerized historical simulation multimarket and multiperiod out-of-sample performance period, length of platforms used for development preliminary procedures, benefits of results software trading strategy Test window: degrees of freedom scan ranges size of trading, frequency of Thorpe, Professor Edward Tick math Time-period-by-time-period analysis Time-sliced trading performance Trade-by-trade report Trade, definition of Trade list, the Traders: common mistakes confidence discretionary effects of emotions on versus individual investor interview with programmer maximum drawdown maximum runup portfolio analysis professional psychology of systematic value of objectivity TradersStudio Trade size: consistency historical sample TradeStation Trading: dicretionary futures optimization performance, peak risk and margin Trading models: overfitting, example pessimistic return on margin (PROM) statistical procedures symptoms of overfit Trading profile: evaluation of real-time trading expectations, theoretical Trading strategy: adaptation to markets adhering to system alternatives automated, benefits of benefits of confidence consistency of definition of development of distribution drawdown, maximum (MDD), and risk examples of expectations, theoretical expenses extensibility of formulation historical simulation improving life cycle of market conditions, unseen market contraction market opportunities (inefficiencies) mental aspects of optimization framework optimization profile parameters, multiple pessimistic return on margin (PROM) pitfalls of optimization poor portfolio maximum drawdown portfolio risk management profit and risk measurements reasons for reversal versus nonreversal rules real-time risk robust versus profitability robustness rules of rules for entry and exit search methods, advanced shifting markets, adapting to signals specifications stop-loss (SSL) superior alternatives testing trades, number of trailing stop trend-following unsuccessful validity value of consistency value of objectivity Walk-Forward Analysis Trading strategy design Trading strategy development: abuse of hindsight back-testing eight stages entry and exit formulation philosophical approaches rules of specifications scientific method testing platforms three major components Trading systems, classic Trading volume, global Trading window, size of Trailing dollar profit stop Trailing stop Trailing volatility profit stop Trend changes Turtle Trading Strategy (TTS) Undercapitalization Underperformance Values: optimization robustness of trading strategy Verification: calculations and trades definition of preliminary testing trading advantage Vince, Ralph Volatility: -adjusted position sizing cyclic component futures market high markets risk stop Walk-Forward Analysis Walk-Forward Analysis (WFA): automation cure for overfitting degrees of freedom empirical approach to trading example of life cycle of trading strategy markets objective function out-of-sample testing parameter set for trading peak performance postoptimization trading process, two-step real-time trading relevant data, theory of reoptimization, pre-emptive risk and return, reliable measures rolling optimization startup overhead test time-period-by-time-period window, test Walk-Forward Efficiency (WFE) Williams, Larry Windfall profit Window Year-by-year basis report ... OPTIMIZATION THE ROBUST TRADING STRATEGY THE STATISTICALLY SIGNIFICANT OPTIMIZATION PROFILE THE DISTRIBUTION OF THE OPTIMIZATION PROFILE THE SHAPE OF THE OPTIMIZATION PROFILE HOW DOES THE STRATEGY... there is another rather pernicious trend The real-time quote software vendors typically restrict the use of their software to their data Worse, they restrict the use of their data to their software... COMPARING THE EVALUATION AND TRADE PROFILE UNDERSTANDING THE TEST PROFILE PERFORMANCE QUIRKS IN CONCLUSION Notes Index Additional Praise for The Evaluation and Optimization of Trading Strategies “The