Martin w n (ed), spears w (ed) foundations of(bookfi)

Contents Introduction Worth), N Martin and William M Spears Overcoming Fitness Barriers in Multi-Modal Search Spaces Martin J Oates and David Come Niches in NK-Landscapes 27 Keith E Mathia, Larry J Eshelman, and J David Schaffer New Methods for Tunable, Random Landscapes 47 R E Smith and J E Smith Analysis of Recombinative Algorithms on a Non-Separable Building-Block Problem 69 Richard A Watson Direct Statistical Estimation of GA Landscape Properties 91 Colin R Reeves Comparing Population Mean Curves 109 B Naudts and I Landrieu Local Performance of the ((/(I, () -ES in a Noisy Environment 127 Dirk V Arnold and Hans-Georg Beyer Recursive Conditional Scheme Theorem, Convergence and Population Sizing in Genetic Algorithms 143 Riccardo Poli Towards a Theory of Strong Overgeneral Classifiers 165 Tim Kovacs Evolutionary Optimization through PAC Learning 185 Forbes J Burkowski Continuous Dynamical System Models of Steady-State Genetic Algorithms 209 Alden H Wright and Jonathan E Rowe Mutation-Selection Algorithm: A Large Deviation Approach 227 Paul Albuquerque and Christian Mazza The Equilibrium and Transient Behavior of Mutation and Recombination 241 William M Spears The Mixing Rate of Different Crossover Operators 261 Adam Priigel-Bennett Dynamic Parameter Control in Simple Evolutionary Algorithms 275 Stefan Droste, Thomas Jansen, and lngo Wegener Local Search and High Precision Gray Codes: Convergence Results and Neighborhoods 295 Darrell Whitle); Laura Barbulescu, and Jean-Paul Watson Burden and Benefits of Redundancy 313 Karsten Weicker and Nicole Weicker A u t h o r I n d e x 3 K e y W o r d I n d e x 3 g 00 FOGA 2000 ] ]]] ]] Introduction The 2000 Foundations of Genetic Algorithms (FOGA-6) workshop was the sixth biennial meeting in this series of workshops From the beginning, FOGA was conceived as a way of exploring and focusing on theoretical issues related to genetic algorithms (GAs) It has hence expanded to include the general field of evolutionary computation (EC), including evolution strategies (ES), evolutionary programming (EP), genetic programming (GP), and other population-based search techniques or evolutionary algorithms (EAs) FOGA now especially encourages submissions from members of other communities, such as mathematicians, physicists, population geneticists, and evolutionary biologists, in the hope of providing radically novel theoretical approaches to the analysis of evolutionary computation One of the strengths of the FOGA format is the emphasis on having a small relaxed workshop with very high quality presentations To provide a pleasant and relaxing atmosphere, FOG A-6 was held in the charming city of Charlottesville, VA To provide the quality, submissions went through a double-review process, conducted by highly qualified reviewers Of the 30 submissions, 17 were accepted for presentation and are presented in this volume Hence, the quality of the papers in this volume is considerably higher than the quality of papers generally encountered in workshops FOG A-6 also had two invited talks The first was given by David H Wood of the University of Delaware Entitled, Can You Use a Population Size o/ a Million Million Million, David's excellent talk concentrated on the connections between DNA and evolutionary computation, providing a provocative way to start the workshop Later in the workshop, Kenneth A De Jong of George Mason University gave an extremely useful outline of where we are with respect to evolutionary computation theory and where we need to go, in his talk entitled, Future Research Directions One common problem with the empirical methodology often used in the EA community occurs when the EA is carefully tuned to outperform some other algorithm on a few ad hoc problems Unfortunately, the results of such studies typically have only weak predictive value regarding the performance of EAs on new problems A better methodology is to identify characteristics of problems (e.g., epistasis, deception, multimodality) that affect Introduction EA performance, and to then use test-problem generators to produce random instances of such problems, with those characteristics We are pleased to present a FOGA volume containing a large number of papers that focus on the issue of problem characteristics and how they affect EA performance D.V Arnold and H.-G Beyer (Local Performance of the (p/#1, A)-ES in a Noisy Environment) examine the characteristic of noise and show how this affects the performance of multiparent evolution strategies R.A Watson (Analysis of Recombinative Algorithms on a Non-Separable Building-Block Problem) examines an interesting class (even if somewhat restricted) of problems which have non-separable building blocks and compares the performance of GAs with a recombinative hill-climber K.D Mathias, L.J Eshelman, and J.D Schaffer (Niches in NK-Landscapes) provide an in-depth comparison of GAs to other algorithms on NK-Landscape problems showing areas of superior GA performance R.E Smith and J.E Smith (New Methods for Tunable, Random Landscapes) further generalize the class of NK-Landscape problems by introducing new parameters - the number of epistatic partitions P, a relative scale S of lower- and higher-order effects in the partitions, and the correlation R between lower- and higher-order effects in the partitions Some papers address issues pertaining to more arbitrary landscapes D Whitley, L Barbulescu, and J.-P Watson (Local Search and High Precision Gray Codes: Convergence Results and Neighborhoods) show how the neighborhood structure of landscapes is affected by the use of different coding mechanisms, such as Gray and Binary codes C.R Reeves (Direct Statistical Estimation of GA Landscape Properties) gives techniques for providing direct statistical estimates of the number of attractors that exist for the GA population, in the hopes that this will provide a measure of GA difficulty M.J Oates and D Corne (Overcoming Fitness Barriers in Multi-Modal Search Spaces) show that EAs have certain performance features that appear over a range of different problems Finally, D Naudts and I Landrieu ( Comparing Population Mean Curves) point out that it is often difficult to compare EA performance over different problems, since different problems have different fitness ranges In response they provide a renormalization that allows one to compare population mean curves across very different problems Other papers in this volume concentrate more on the dynamics of the algorithms per se, or on components of those algorithms For example, A.H Wright and J.E Rowe (Continuous Dynamical System Models of Steady-State Genetic Algorithms) construct discrete-time and continuous-time models of steady-state evolutionary algorithms, examining their fixed points and their asymptotic stability R Poli (Recursive Conditional Schema Theorem, Convergence and Population Sizing in Genetic Algorithms) extends traditional schema analyses in order to predict with a known probability whether the number of instances of a schema at the next generation will be above a given threshold P Albuquerque and C Mazza (Mutation-Selection Algorithm: A Large Deviation Approach) provide a mathematical analysis of the convergence of an EA-like algorithm composed of Boltzmann selection and mutation, based on the probabilistic theory of large deviations S Droste, T Jansen, and I Wegener (Dynamic Parameter Control in Simple Evolutionary Algorithms) examine methods of dynamic parameter control and rigorously prove that such methods can greatly speed up optimization for simple (1 + 1) evolutionary algorithms W.M Spears (The Equilibrium and Transient Behavior of Mutation and Recombination) analyzes the transient behavior of mutation and recombination in the absence of selection, tying the more conventional schema analyses with.the theory of recombination distributions Introduction Finally, in a related paper, A Priigel-Bennett (The Mixing Rate of Different Crossover Operators) also examines recombination in the absence of selection, showing how different recombination operators affect the rate of mixing in a population This volume is fortunate to have papers that address issues and concepts not commonly found in FOGA proceedings The first, by K Weicker and N Weicker (Burden and Benefits of Redundancy), explores how different techniques for introducing redundancy into a representation affects schema processing, mutation, recombination, and performance F.J Burbowski (Evolutionary Optimization Through PA C Learning) introduces a novel population-based algorithm referred to as the 'Rising Tide Algorithm,' which is then analyzed using techniques from the PAC learning community The goal here is to show that evolutionary optimization techniques can fall under the analytically rich environment of PAC learning Finally, T Kovacs ( Towards a Theory of Strong Overgeneral Classifiers) discusses the issues of overgeneralization in traditional learning classifier systems - these issues also affect traditional EAs that attempt to learn rule sets, Lisp expressions, and finite-state automata The only other classifier system paper in a FOGA proceedings was in the first FOGA workshop in 1990 All in all, we believe the papers in this volume exemplify the strengths of FOGA - the exploitation of previous techniques and ideas, merged with the exploration of novel views and methods of analysis We hope to see FOGA continue for many further generations! Worthy N Martin University of Virginia William M Spears Naval Research Laboratory Ode Beethoven It -~ J al i le b Men - i schen - i? I Illl II Illl II Overcoming Fitness Barriers in Multi-Modal Search Spaces Martin J Oates BT Labs, Adastral Park, Martlesham Heath, Suffolk, England, IP5 3RE David Corne Dept of Computer Science, University of Reading, Reading, RG6 6AY Abstract In order to test the suitability of an evolutionary algorithm designed for real-world application, thorough parameter testing is needed to establish parameter sensitivity, solution quality reliability, and associated issues One approach is to produce 'performance profiles', which display performance measures against a variety of parameter settings Interesting and robust features have recently been observed in performance profiles of an evolutionary algorithm applied to a real world problem, which have also been observed in the performance profiles of several other problems, under a wide variety of conditions These features are essentially the existence of several peaks and troughs, indicating a range of locally optimal mutation rates in terms of (a measure of) convergence time An explanation of these features is proposed, which involves the identification of three phases of search behaviour, where each phase is identified with an interval of mutation rates for non-adaptive evolutionary algorithms These phases repeat cyclically as mutation rate is increased, and the onsets of certain phases seem to coincide with the availability of certain types of mutation event We briefly discuss future directions and possible implications for these observations INTRODUCTION The demands of real-world optimization problems provide the evolutionary algorithm researcher with several challenges One of the key challenges is that industry needs to feel confident about the speed, reliability, and robustness of EA-based methods [ 1,4,5,8] In particular, these issues must be addressed on a case by case basis in respect of tailored Martin J Oates and David C o m e EA-based approaches to specific problems A standard way to address these issues is, of course, to empirically test the performance of a chosen tailored EA against a suite of realistic problems and over a wide range of parameter and/or strategy settings Certainly, there are several applications where such a thorough analysis is not strictly necessary However, where the EA is designed for use in near-real time applications and/or is expected to perform within a given 'quality of service' constraint, substantial testing and validation of the algorithm is certainly required An example of a problem of this type, called the Adaptive Distributed Database Management Problem (ADDMP), is reported in [ 13,14,16] In order to provide suitably thorough evaluation of the performance of EAs on the ADDMP, substantial experiments have been run to generate performance profiles A performance profile is a plot of 'mean evaluations exploited' (the z axis) over a grid defining combinations of population size and mutation rate (the x and y axes) See Figure for an example, with several others in [13,14,16] 'Mean evaluations exploited' is essentially a measure of convergence time - that is, the time taken (in terms of number of evaluations) for the EA to first find the best solution it happens to find in a single trial run However we not call it 'convergence time', since it does not correspond, for example, to fixation of the entire population at a particular fitness value It is recognised that this measure is only of real significance if its variation is low The choice of mean evaluations exploited as a performance measure is guided by the industrial need for speed The alternative measure would of course be 'best-fitness found', but we also need to carefully consider the speed of finding the solution With reference also to the standard mean-fitness plot, an evaluations-exploited performance profile indicates not only whether adequate fitness can be delivered within the time limit at certain parameter settings, but whether or not we can often expect good solutions well before the time limit this is of course important and exploitable in near real-time applications - A single (x,y,z) point in a performance profile corresponds to the mean evaluations exploited (z) over 50 (unless otherwise stated) trial runs with mutation rate set to x and population size set to y An entire performance profile typically contains several hundred such points, An important feature associated with a performance profile is the time-limit (again, in terms of number of evaluations) given to individual trial runs A performance profile with a time limit of 20,000 evaluations, for example, consumes in total around half a billion evaluations Although a very time consuming enterprise, plotting performance profiles for the ADDMP has yielded some interesting features which has prompted further investigation As discussed in [13], the original aim has been served in that performance profiles of the ADDMP reveal suitably wide regions of parameter space in which the EA delivers solutions with reliable speed and quality This initial finding has been sufficiently convincing, for example, to enable maintained funding for further study towards adoption of this EA for live applications (this work is currently at the demonstrator stage) Beyond these basic issues, however, performance profiles on the ADDMP have yielded robust and unexpected features, which have consistently appeared in other problems which have now been explored The original naive expectation was that the profile would essentially reveal a 'well' with its lowest points (corresponding to fast convergence to good solutions) corresponding to ideal parameter choices What was unexpected was that Overcoming Fitness Barriers in Multi-Modal Search Spaces beyond this well (towards the r i g h t - higher mutation rates) there seemed to be an additional well, corresponding to locally good but higher mutation rates yielding fast convergence Essentially, we expected profiles to be similar in structure to the area between mutation rate = and the second peak to the right in Figure 1; however, instead we tended to find further local structure beyond this second peak Hence, the performance profile of the ADDMP seemed to reveal two locally optimal mutation rates in terms of fast and reliable convergence to good solutions Concerned that this may simply have been an artefact of the chosen EA and the chosen test problems, several further performance profiles were generated which used different time limits, quite different EA designs, and different test problems These studies revealed that the multimodality of the performance profile seemed to be a general feature of evolutionary search [16-19] Recently, we have looked further into the multimodal features in the performance profiles of a range of standard test problems, and looked into the positions of the features with respect to the variation in the evaluations exploited measure, and also mean fitness This has yielded the suggestion that there are identifiable phases of search behaviour which change and repeat as we increase the mutation rate, and that an understanding of these phases could underlay an understanding of the multimodality in performance profiles Note that these phases are not associated with intervals of time in a single trial run, but with intervals of parameter space Hence a single run of a particular (non-adaptive) EA operates in a particular phase In this article, we describe these observations of phase-based behaviour in association with multimodal performance profiles, considering a range of test problems In particular, we explore a possible explanation of the phase behaviour in terms of the frequencies of particular mutation events available as we change the mutation rate In section we describe some background and preliminary studies in more detail, setting the stage for the explorations in this article Section then describes the main test problem we focus on, Watson et al's H-IFF problem [21 ], and describes the phase-based behaviour exhibited by H-IFF performance profiles In section we set out a simple model to explain phase onsets in terms of the frequencies with which certain specific types of mutation event become available as we increase the mutation rate The model is investigated with respect to the H-IFF performance profile and found to have some explanatory power, whilst actual fitness distributions are explored in section Section then investigates whether similar effects occur on other problems, namely Kauffman NK landscapes [6] and the tuneable Royal Staircase problem [11,12], and explores the explanatory power of the 'mutation-event' based explanation on these problems A discussion and conclusions appear in sections and respectively PRELIMINARY OBSERVATION BEHAVIOUR I N P E R F O R M A N C E OF CYCLIC PROFILES PHASE In recent studies of the performance profile of the ADDMP [ 13,14], Watson et al's H-IFF problem [21], Kauffman NK landscapes [6] and the tuneable Royal Staircase problem [11], as well as simple MAX-ONES, a cyclic tri-phase behaviour has been observed [ 18,19], where phases correspond to intervals on the mutation rate axis The phases were characterised in terms of three key features: evaluations exploited, its variation, and mean Martin J Oates and David Come fitness In what has been called Phase A, evaluations exploited rises as its variation decreases, while mean fitness gradually rises This seems to be a 'discovery' phase, within which, as mutation rate rises, the EA is able to increasingly exploit a greater frequency of useful mutations becoming available to it In Phase B, evaluations exploited falls, while its variation stays low, and mean fitness remains steady This seems to be a tuning phase, wherein the increasing frequency with which useful mutations are becoming available serves to enable the EA to converge more quickly This is followed, however, by Phase C, where evaluations exploited starts to rise again, and its variation becomes quite high In this phase, it seems that the EA has broken through a 'fitness barrier', aided by the sudden availability of mutation events (eg: a significant number of two-gene mutations) which were unavailable in previous phases The end of Phase C corresponds with the onset of a new Phase A in which the newly available mutation events are beginning to deliver an improved of a new Phase A, during which the EA makes increasing use of the mutations newly available to (over the previous Phase A) fitness more and more reliably Depending strongly on the problem at hand, these phases can be seen to repeat cyclically Figure 2, described later in more detail, provides an example of this behaviour, which was reported on for H-IFF in [18] and for other uni- and multi-modal search spaces in [19] Whilst these publications voiced some tentative ideas to explain the phase onsets and their positions, no analysis nor detailed explanation was offered Our current hypothesis is essentially that these phases demonstrate that the number of 'k-gene' mutations that can be usefully exploited remains constant over certain bands of mutation rate Hence, as the mutation rate is increased within Phase B, for example, search simply needs to proceed until a certain number of k-gene mutations have occurred (k=l for the first Phase B, for the second Phase B, and so on) So, the total number of evaluations used will fall as the mutation rate increases According to this hypothesis, the onset of Phase C represents a mutation rate at which k+ 1-gene mutations are becoming available in numbers significant enough to be exploited towards, at first unreliably, delivering a better final fitness value The next Phase A begins when the new best fitness begins to be found with a significant reliability, and becomes increasingly so as mutation rate is further increased In this paper, we analyse the data from the experiments reported in [18, 19] in closer detail, and consider the expected and used numbers of specific types of mutation event Next, we begin by looking more closely at the H-IFF performance profile THE H-IFF PERFORMANCE PROFILE Watson et al's Hierarchical If and only If problem (H-IFF) [21,22] was devised to explore the performance of search strategies employing crossover operators to find and combine 'building blocks' of a decomposable, but potentially contradictory nature The fitness of a potential solution to this problem is defined to be the sum of weighted, aligned blocks of either contiguous l's or O's and can be described by : , f(B) IBI + f(BL) + f(BR), f(BL) + f(BR), if IBI = if(IBI > 1) and (Vi {bi=0} orVi {bi= 1}), otherwise O v e r c o m i n g Fitness Barriers in Multi-Modal Search Spaces where B is a block of bits, {bl, b2 b,}, IBI is the size of the block=n, bi is the ith element of B, and BL and BR are the left and right halves of B (i.e BL = {b~ b,n }, BR = {b,r2+~ bn} n must be an integer power of This produces a search landscape in which global optima exist, one as a string of all Is, the other of all 0's However a single mutation from either of these positions produces a much lower fitness Secondary optima exist at strings of 32 contiguous O's followed by 32 contiguous l's (for a binary string of length 64) and vice versa Again, further suboptima occur at 16 contiguous O's followed by 48 contiguous l's etc Watson showed that hillclimbing performs extremely badly on this problem [22] To establish a performance profile for a simple evolutionary search technique on this problem, a set of tests were run using a simple EA (described shortly) over a range of population sizes (20 through 500) and mutation rates (le-7 rising exponentially through to 0.83), noting the fitness of the best solution found, and the number of evaluations taken to first find it out of a limit of million evaluations Each trial was repeated 50 times and the mean number of evaluations used is shown in Figure This clearly shows a multimodal performance profile, particularly at lower population sizes, and is an extension of the number of features of the profile first seen in [17] in which a clear tri-modal profile was first published on the H-IFF problem with an evaluation limit of only 20,000 evaluations Previous studies of various instances of the ADDMP and One Max problem [ 15,16] (also limited to only 20,000 evaluations) had shown only bi-modal profiles Unless otherwise stated, all EAs used within this paper are steady state; employing one point crossover [5] at a probability of 1.0; single, three-way tournament selection [4] (where the resulting child automatically replaces the poorest member of the tournament); and 'per gene' New Random Allele (NRA) mutation at a fixed rate throughout the run of million evaluations (NRA mutation is used for consistency with earlier studies on the ADDMP, where a symbolic k-ary representation is used rather than a binary one) Mutation rates varied from E-7 through to 0.83 usually doubling every points creating up to 93 sampled rates over orders of magnitude All experiments are repeated 50 times with the same parameter values but with different, randomly generated initial populations Further experiments with Generational, Elitist, Breeder strategies [9] and Uniform crossover [20] are also yielding similar results Figure 2, shows detailed results on the H-IFF problem at a population size of 20, superimposing plots of mean evaluations used and its co-efficient of variation (standard deviation over the 50 runs divided by the mean) Figure plots the 'total mutations used' (being the product of the mutation rate, the mean number of evaluations used and the chromosome length) and mean fitness, the mutation axis here being a factor of times more detailed than in Figure These clearly show a multi-peaked performance profile with peaks in the number of evaluations used occurring at mutation rates of around 1.6 E6, 1.6 E-3, 5.2 E-2 and 2.1 E-1 These results seemed to indicate that the dynamics of the performance profile yield a repeating three-phase structure, with the phases characterised in terms of the combined behaviour of mean number of evaluations exploited, its variation, and mean fitness of best solution found, as the mutation rate increases In Phase A, evaluations exploited rises with decreasing variation, while mean fitness also rises This seems to be a 'Delivery' phase, in which the rise in mutation rate is gradually 326 Karsten Weicker and Nicole Weicker 28 items 28 items 0.7 24800 dec(xler (best)' decoder (best) w/out rec - 0.6 0.5 24700 0.4 9t~- 24600 0.3 0.2 24500 0.1 decoder (best) decoder (best) w/out rec - 24400 250 500 750 1000 significance for rec - significance for no rec 500 generation 750 500 750 1000 250 250 -• 0 "~ _~ ~ -2 s,g nsiigciaficaenfc:rf:o :e cc t 1000 250 500 generation 750 1000 F i g u r e Left" fitness comparison of a GA with recombination and a GA without recombination, both using the best fit decoder The t-tests show that the difference in performance of the two algorithms is not significant Right: comparison of the diversity in each population of the same experiments The t-tests show the higher diversity of the algorithm with recombination to be significant for most generations The average entropy per locus is defined for the multiset of individuals P = {i(i) E {0, 1} ~ [1 < i < n} with length as /:/(p) = l [ ~-'~ H ( S k ( P ) ) , k where the bits of the different loci are cumulated in the multisets Bk(P) = {I~ i) I < i